Hadoop Application Report from WebUI

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Hadoop Application Report from WebUI

Hilmi Egemen Ciritoğlu
Hi all,

I can see following informations on hadoop yarn web-ui report(8088) for each mappers that I run.

Status of mappers has shown like:
hdfs://c7-master:9000/user/egemen/datasets/db/year1993.txt:1207959552+134217728

What does these mean 1207959552+134217728 ? or 0+134217728. First number is multiplication of block size with some number but I have no idea about second number.

Thanks a lot,

Regards,
Egemen




---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Screen Shot 2017-06-09 at 15.30.39.png (279K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Hadoop Application Report from WebUI

Ravi Prakash-2
Hi Hilmi!

I'm not sure, but maybe the offset in the block at which the task started processing?

Ravi

On Fri, Jun 9, 2017 at 7:43 AM, Hilmi Egemen Ciritoğlu <[hidden email]> wrote:
Hi all,

I can see following informations on hadoop yarn web-ui report(8088) for each mappers that I run.

Status of mappers has shown like:
hdfs://c7-master:9000/user/egemen/datasets/db/year1993.txt:1207959552+134217728

What does these mean 1207959552+134217728 ? or 0+134217728. First number is multiplication of block size with some number but I have no idea about second number.

Thanks a lot,

Regards,
Egemen




---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Hadoop Application Report from WebUI

Erik Krogen
The first number is the offset in the file which the task will start reading from, the next is the length it will read (starting from that offset).

On Fri, Jun 9, 2017 at 10:14 AM, Ravi Prakash <[hidden email]> wrote:
Hi Hilmi!

I'm not sure, but maybe the offset in the block at which the task started processing?

Ravi

On Fri, Jun 9, 2017 at 7:43 AM, Hilmi Egemen Ciritoğlu <[hidden email]> wrote:
Hi all,

I can see following informations on hadoop yarn web-ui report(8088) for each mappers that I run.

Status of mappers has shown like:
hdfs://c7-master:9000/user/egemen/datasets/db/year1993.txt:1207959552+134217728

What does these mean 1207959552+134217728 ? or 0+134217728. First number is multiplication of block size with some number but I have no idea about second number.

Thanks a lot,

Regards,
Egemen




---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Hadoop Application Report from WebUI

Hilmi Egemen Ciritoğlu
Hi guys,

Thanks a lot for your reply.

I have doubts about second number because it's always fixed 134217728(128 MB). 
According to your answer, when I inspect all mappers I can see unprocessed blocks 
But in the meantime I know all data has been processed from the result I got. 

Are you really sure second number is length it will read ?

Regards,
Egemen

2017-06-09 18:24 GMT+01:00 Erik Krogen <[hidden email]>:
The first number is the offset in the file which the task will start reading from, the next is the length it will read (starting from that offset).

On Fri, Jun 9, 2017 at 10:14 AM, Ravi Prakash <[hidden email]> wrote:
Hi Hilmi!

I'm not sure, but maybe the offset in the block at which the task started processing?

Ravi

On Fri, Jun 9, 2017 at 7:43 AM, Hilmi Egemen Ciritoğlu <[hidden email]> wrote:
Hi all,

I can see following informations on hadoop yarn web-ui report(8088) for each mappers that I run.

Status of mappers has shown like:
hdfs://c7-master:9000/user/egemen/datasets/db/year1993.txt:1207959552+134217728

What does these mean 1207959552+134217728 ? or 0+134217728. First number is multiplication of block size with some number but I have no idea about second number.

Thanks a lot,

Regards,
Egemen




---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]



Loading...