[dpdk-dev] [RFC] Compression API in DPDK

[Verma, Shally]

> -----Original Message-----
> From: Trahe, Fiona [mailto:fiona.trahe at intel.com]
> Sent: 07 November 2017 16:54
> To: Verma, Shally <Shally.Verma at cavium.com>; dev at dpdk.org
> Cc: Athreya, Narayana Prasad <NarayanaPrasad.Athreya at cavium.com>;
> Challa, Mahipal <Mahipal.Challa at cavium.com>; Trahe, Fiona
> <fiona.trahe at intel.com>
> Subject: RE: [dpdk-dev] [RFC] Compression API in DPDK
> 
> Hi Shally,
> 
> ///snip///
> > [Shally] Ok. Then, just to confirm my understanding here. You mean PMD
> can figure out amount of
> > available space in dst mbuf by calling rte_pktmbuf_data_len() on each of its
> segment?
> [Fiona] exactly.
> 
> ///snip///
> > > > > > > +      * This indicates the buffer size and should be
> > > > > > > +      * set a little larger than the expected max source buffer size.
> > > > > > > +      * if the output of static compression doesn't fit in the
> > > > > > > +      * intermediate buffer dynamic compression may not be
> possible,
> > > > > > > +      * in this case the accelerator may revert back to static
> > > compression.
> > [Shally] > > > > +      * in this case the accelerator may revert back to static
> compression.> > > > > +      */
> > Can you elaborate more on this? This looks to me as decision made during
> enqueue_burst() processing.
> > If yes and If application has chosen specific Huffman code i.e.
> RTE_COMP_DYNAMIC or
> > RTE_COMP_FIXED in rte_comp_compress_xform, then how this would
> work?
> [Fiona] yes, it would have to revert back on the enqueue. The compressed
> data would still conform to deflate standard, so any decompressor would be
> able to inflate it. The ratio would not be as good as hoped for but it would be
> the best the compression engine could do with the resources it has.
> 
[Shally] Ok. However, I'm not sure how to use Intermediate bufs here as it is not requirement for us for this purpose. 
So, it looks like It is very device specific requirement where some may not need it. So, I would suggest that API should propose a way to indicate if it's a requirement for specific device so that app can input it at config time. May be feature flag or capability.

Thanks
Shally

> ///snip///
> > [Shally] Sure. So just to align here. Except few questions posted above on
> this RFC (such as Dynamic Vs
> > Static or dst mbuf parsing), following (and any other) will further be
> covered as part of 'RFC doc'
> > discussion
> > - Hash support
> > - RTE_COMPDEV_FF_MULTI_PKT_CHECKSUM
> [Fiona] Agreed.