Why does MapReduce read and process the input split line by line? Can not I upload the entire file to memory and process it?

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Why does MapReduce read and process the input split line by line? Can not I upload the entire file to memory and process it?

Daegyu Han
Hi all,

Why does MapReduce handle input split files one line at a time?

Or is it not fast enough to read all the input split (ex: 128MB) and
put the data in memory and process the data (string) in memory?

If you are processing one row at a time, as in the current approach,
is not the Java application making line-by-line read system calls to
the operating system?

Best Regards,
Daegyu

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]