[EXT] [PATCH 1/4] compressdev: add LZ4 algorithm support
Akhil Goyal
gakhil at marvell.com
Tue Jan 31 07:29:17 CET 2023
> -----Original Message-----
> From: Michael Baum <michaelba at nvidia.com>
> Sent: Tuesday, January 31, 2023 2:07 AM
> To: Akhil Goyal <gakhil at marvell.com>; dev at dpdk.org; Mahipal Challa
> <mchalla at marvell.com>; Fan Zhang <fanzhang.oss at gmail.com>; Ashish Gupta
> <ashishg at marvell.com>
> Cc: Matan Azrad <matan at nvidia.com>; Fiona Trahe <fiona.trahe at intel.com>;
> NBU-Contact-Thomas Monjalon (EXTERNAL) <thomas at monjalon.net>
> Subject: RE: [EXT] [PATCH 1/4] compressdev: add LZ4 algorithm support
>
> On Mon, Jan 30, 2023 at 20:35 PM Akhil Goyal <gakhil at marvell.com> wrote:
> >
> > > +/**
> > > + * Block checksum flag.
> > > + * If this flag is set, each data block will be followed by a 4-bytes checksum,
> > > + * calculated by using the xxHash-32 algorithm on the raw (compressed)
> data
> > > + * block. The intention is to detect data corruption (storage or transmission
> > > + * errors) immediately, before decoding. Block checksum usage is optional.
> > > + */
> > > +#define RTE_COMP_LZ4_FLAG_BLOCK_CHECKSUM (1 << 4)
> > > +
> > > +/**
> > > + * Block Independence flag.
> > > + * If this flag is set to 1, blocks are independent.
> > > + * If this flag is set to 0, each block depends on previous ones (up to LZ4
> > > + * window size, which is 64 KB). In such case, it is necessary to decode all
> > > + * blocks in sequence.
> > > + * Block dependency improves compression ratio, especially for small
> blocks.
> > > On
> > > + * the other hand, it makes random access or multi-threaded decoding
> > > impossible.
> > > + */
> > > +#define RTE_COMP_LZ4_FLAG_BLOCK_INDEPENDENCE (1 << 5)
> >
> > Why did you start with 4th and 5th bit of the flags? Why not first two bits?
>
> I didn't choose the values by myself, I took them from LZ4 standard:
> https://github.com/lz4/lz4/blob/dev/doc/lz4_Frame_format.md#frame-descriptor
Ok, Are we defining and making the frame descriptor inside library?
Also, why all the bits not added?
I believe these are just flags which PMD would use to make the frame descriptors.
If so, we should start with 0th bit and let the PMD handle bit shifting.
More information about the dev
mailing list