[dpdk-dev] [RFC] Compression API in DPDK

[Verma, Shally]

Hi Fiona

Could you give some expected timeframe for next comp API spec patch?

Thanks
Shally

> -----Original Message-----
> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Verma, Shally
> Sent: 10 November 2017 17:35
> To: Trahe, Fiona <fiona.trahe at intel.com>; dev at dpdk.org
> Cc: Athreya, Narayana Prasad <NarayanaPrasad.Athreya at cavium.com>;
> Challa, Mahipal <Mahipal.Challa at cavium.com>
> Subject: Re: [dpdk-dev] [RFC] Compression API in DPDK
> 
> [This sender failed our fraud detection checks and may not be who they
> appear to be. Learn about spoofing at http://aka.ms/LearnAboutSpoofing]
> 
> > -----Original Message-----
> > From: Trahe, Fiona [mailto:fiona.trahe at intel.com]
> > Sent: 07 November 2017 16:54
> > To: Verma, Shally <Shally.Verma at cavium.com>; dev at dpdk.org
> > Cc: Athreya, Narayana Prasad <NarayanaPrasad.Athreya at cavium.com>;
> > Challa, Mahipal <Mahipal.Challa at cavium.com>; Trahe, Fiona
> > <fiona.trahe at intel.com>
> > Subject: RE: [dpdk-dev] [RFC] Compression API in DPDK
> >
> > Hi Shally,
> >
> > ///snip///
> > > [Shally] Ok. Then, just to confirm my understanding here. You mean PMD
> > can figure out amount of
> > > available space in dst mbuf by calling rte_pktmbuf_data_len() on each of
> its
> > segment?
> > [Fiona] exactly.
> >
> > ///snip///
> > > > > > > > +      * This indicates the buffer size and should be
> > > > > > > > +      * set a little larger than the expected max source buffer
> size.
> > > > > > > > +      * if the output of static compression doesn't fit in the
> > > > > > > > +      * intermediate buffer dynamic compression may not be
> > possible,
> > > > > > > > +      * in this case the accelerator may revert back to static
> > > > compression.
> > > [Shally] > > > > +      * in this case the accelerator may revert back to static
> > compression.> > > > > +      */
> > > Can you elaborate more on this? This looks to me as decision made during
> > enqueue_burst() processing.
> > > If yes and If application has chosen specific Huffman code i.e.
> > RTE_COMP_DYNAMIC or
> > > RTE_COMP_FIXED in rte_comp_compress_xform, then how this would
> > work?
> > [Fiona] yes, it would have to revert back on the enqueue. The compressed
> > data would still conform to deflate standard, so any decompressor would
> be
> > able to inflate it. The ratio would not be as good as hoped for but it would
> be
> > the best the compression engine could do with the resources it has.
> >
> [Shally] Ok. However, I'm not sure how to use Intermediate bufs here as it is
> not requirement for us for this purpose.
> So, it looks like It is very device specific requirement where some may not
> need it. So, I would suggest that API should propose a way to indicate if it's a
> requirement for specific device so that app can input it at config time. May be
> feature flag or capability.
> 
> Thanks
> Shally
> 
> > ///snip///
> > > [Shally] Sure. So just to align here. Except few questions posted above on
> > this RFC (such as Dynamic Vs
> > > Static or dst mbuf parsing), following (and any other) will further be
> > covered as part of 'RFC doc'
> > > discussion
> > > - Hash support
> > > - RTE_COMPDEV_FF_MULTI_PKT_CHECKSUM
> > [Fiona] Agreed.