Using compression support available in native libraries

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Using compression support available in native libraries

Krishna Kishore Bonagiri
Hi,

  I would like to use compression support available in native libraries. I have tried to google but couldn't know how can I use the LZ4 compression algorithm for compressing the data I am writing to the HDFS files I am writing from my C++ code?  

Is it possible to get the data written in compression mode automatically without having to compress it in our code? Like, select some compression option while opening or creating an HDFS file from our code, and just write to it using hdfsWrite() and can it underneath take care of compressing it before writing to the disk/file system?


Thanks,
Kishore