Quantcast

FPGrowth job got stuck when running fpGrowth.generateTopKFrequentPatterns

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

FPGrowth job got stuck when running fpGrowth.generateTopKFrequentPatterns

kayhan
Trying to mine frequent patterns of a dataset with 8000 transactions and 193 attributes using mahout's parallel-frequent-pattern-mining algorithm. I run it in map-reduce mode on a cluster of 10 machines on windows7 and I use cygwin.

When I run it, in reduce step, it shows completion of 100%, however it stucks there. No error or any other indication, but it waits there for hours and does not finish working.
The details of the job shows that, it can not finish the part 'Processing FPTree: Bottom Up FP Growth > reduce'. For example the part 'Writing Top K patterns for: 338 > reduce' is finished in seconds.
I post sample image of the gui that shows running tasks;



Also a part of the tasktracker log of the slave that is responsible for the job is posted below;


2012-04-10 16:07:11,051 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201204101532_0003_r_000014_1 1.0% Processing FPTree: Bottom Up FP Growth > reduce
2012-04-10 16:07:20,084 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201204101532_0003_r_000014_1 1.0% Processing FPTree: Bottom Up FP Growth > reduce
2012-04-10 16:07:32,127 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201204101532_0003_r_000014_1 1.0% Processing FPTree: Bottom Up FP Growth > reduce
2012-04-10 16:07:41,160 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201204101532_0003_r_000014_1 1.0% Processing FPTree: Bottom Up FP Growth > reduce
2012-04-10 16:07:50,192 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201204101532_0003_r_000014_1 1.0% Processing FPTree: Bottom Up FP Growth > reduce
2012-04-10 16:08:02,235 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201204101532_0003_r_000014_1 1.0% Processing FPTree: Bottom Up FP Growth > reduce
2012-04-10 16:08:11,268 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201204101532_0003_r_000014_1 1.0% Processing FPTree: Bottom Up FP Growth > reduce


If have any suggestions would be glad.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: FPGrowth job got stuck when running fpGrowth.generateTopKFrequentPatterns

Jay Vyas
Do you have counters for the reducers , and if so , are they continuing to
update ?

On Tue, Apr 10, 2012 at 8:45 AM, kayhan <[hidden email]> wrote:

> Trying to mine frequent patterns of a dataset with 8000 transactions and
> 193
> attributes using mahout's parallel-frequent-pattern-mining algorithm. I run
> it in map-reduce mode on a cluster of 10 machines on windows7 and I use
> cygwin.
>
> When I run it, in reduce step, it shows completion of 100%, however it
> stucks there. No error or any other indication, but it waits there for
> hours
> and does not finish working.
> The details of the job shows that, it can not finish the part 'Processing
> FPTree: Bottom Up FP Growth > reduce'. For example the part 'Writing Top K
> patterns for: 338 > reduce' is finished in seconds.
> I post sample image of the gui that shows running tasks;
>
> http://hadoop-common.472056.n3.nabble.com/file/n3899658/Ads%C4%B1z2.png
>
> Also a part of the tasktracker log of the slave that is responsible for the
> job is posted below;
>
> /
> 2012-04-10 16:07:11,051 INFO org.apache.hadoop.mapred.TaskTracker:
> attempt_201204101532_0003_r_000014_1 1.0% Processing FPTree: Bottom Up FP
> Growth > reduce
> 2012-04-10 16:07:20,084 INFO org.apache.hadoop.mapred.TaskTracker:
> attempt_201204101532_0003_r_000014_1 1.0% Processing FPTree: Bottom Up FP
> Growth > reduce
> 2012-04-10 16:07:32,127 INFO org.apache.hadoop.mapred.TaskTracker:
> attempt_201204101532_0003_r_000014_1 1.0% Processing FPTree: Bottom Up FP
> Growth > reduce
> 2012-04-10 16:07:41,160 INFO org.apache.hadoop.mapred.TaskTracker:
> attempt_201204101532_0003_r_000014_1 1.0% Processing FPTree: Bottom Up FP
> Growth > reduce
> 2012-04-10 16:07:50,192 INFO org.apache.hadoop.mapred.TaskTracker:
> attempt_201204101532_0003_r_000014_1 1.0% Processing FPTree: Bottom Up FP
> Growth > reduce
> 2012-04-10 16:08:02,235 INFO org.apache.hadoop.mapred.TaskTracker:
> attempt_201204101532_0003_r_000014_1 1.0% Processing FPTree: Bottom Up FP
> Growth > reduce
> 2012-04-10 16:08:11,268 INFO org.apache.hadoop.mapred.TaskTracker:
> attempt_201204101532_0003_r_000014_1 1.0% Processing FPTree: Bottom Up FP
> Growth > reduce
> /
>
> If have any suggestions would be glad.
>
>
> --
> View this message in context:
> http://hadoop-common.472056.n3.nabble.com/FPGrowth-job-got-stuck-when-running-fpGrowth-generateTopKFrequentPatterns-tp3899658p3899658.html
> Sent from the Users mailing list archive at Nabble.com.
>



--
Jay Vyas
MMSB/UCHC
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: FPGrowth job got stuck when running fpGrowth.generateTopKFrequentPatterns

kayhan
Yes, I have and they are zero now, no further update on them.

On 10 Nis 2012, at 18:29, "Jay Vyas [via Hadoop Common]"<[hidden email]> wrote:

Do you have counters for the reducers , and if so , are they continuing to
update ?

On Tue, Apr 10, 2012 at 8:45 AM, kayhan <[hidden email]> wrote:

> Trying to mine frequent patterns of a dataset with 8000 transactions and
> 193
> attributes using mahout's parallel-frequent-pattern-mining algorithm. I run
> it in map-reduce mode on a cluster of 10 machines on windows7 and I use
> cygwin.
>
> When I run it, in reduce step, it shows completion of 100%, however it
> stucks there. No error or any other indication, but it waits there for
> hours
> and does not finish working.
> The details of the job shows that, it can not finish the part 'Processing
> FPTree: Bottom Up FP Growth > reduce'. For example the part 'Writing Top K
> patterns for: 338 > reduce' is finished in seconds.
> I post sample image of the gui that shows running tasks;
>
> http://hadoop-common.472056.n3.nabble.com/file/n3899658/Ads%C4%B1z2.png
>
> Also a part of the tasktracker log of the slave that is responsible for the
> job is posted below;
>
> /
> 2012-04-10 16:07:11,051 INFO org.apache.hadoop.mapred.TaskTracker:
> attempt_201204101532_0003_r_000014_1 1.0% Processing FPTree: Bottom Up FP
> Growth > reduce
> 2012-04-10 16:07:20,084 INFO org.apache.hadoop.mapred.TaskTracker:
> attempt_201204101532_0003_r_000014_1 1.0% Processing FPTree: Bottom Up FP
> Growth > reduce
> 2012-04-10 16:07:32,127 INFO org.apache.hadoop.mapred.TaskTracker:
> attempt_201204101532_0003_r_000014_1 1.0% Processing FPTree: Bottom Up FP
> Growth > reduce
> 2012-04-10 16:07:41,160 INFO org.apache.hadoop.mapred.TaskTracker:
> attempt_201204101532_0003_r_000014_1 1.0% Processing FPTree: Bottom Up FP
> Growth > reduce
> 2012-04-10 16:07:50,192 INFO org.apache.hadoop.mapred.TaskTracker:
> attempt_201204101532_0003_r_000014_1 1.0% Processing FPTree: Bottom Up FP
> Growth > reduce
> 2012-04-10 16:08:02,235 INFO org.apache.hadoop.mapred.TaskTracker:
> attempt_201204101532_0003_r_000014_1 1.0% Processing FPTree: Bottom Up FP
> Growth > reduce
> 2012-04-10 16:08:11,268 INFO org.apache.hadoop.mapred.TaskTracker:
> attempt_201204101532_0003_r_000014_1 1.0% Processing FPTree: Bottom Up FP
> Growth > reduce
> /
>
> If have any suggestions would be glad.
>
>
> --
> View this message in context:
> http://hadoop-common.472056.n3.nabble.com/FPGrowth-job-got-stuck-when-running-fpGrowth-generateTopKFrequentPatterns-tp3899658p3899658.html
> Sent from the Users mailing list archive at Nabble.com.
>


--
Jay Vyas
MMSB/UCHC



To unsubscribe from FPGrowth job got stuck when running fpGrowth.generateTopKFrequentPatterns, click here.
NAML
Loading...