[dpdk-dev] [RFC] Compression API in DPDK

[Trahe, Fiona]

Hi Shally,
I'm aiming to get a v2 of the RFC out this week.
Regards,
Fiona

> -----Original Message-----
> From: Verma, Shally [mailto:Shally.Verma at cavium.com]
> Sent: Monday, November 20, 2017 6:44 AM
> To: Verma, Shally <Shally.Verma at cavium.com>; Trahe, Fiona <fiona.trahe at intel.com>; dev at dpdk.org
> Cc: Athreya, Narayana Prasad <NarayanaPrasad.Athreya at cavium.com>; Challa, Mahipal
> <Mahipal.Challa at cavium.com>
> Subject: RE: [dpdk-dev] [RFC] Compression API in DPDK
> 
> Hi Fiona
> 
> Could you give some expected timeframe for next comp API spec patch?
> 
> Thanks
> Shally
> 
> > -----Original Message-----
> > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Verma, Shally
> > Sent: 10 November 2017 17:35
> > To: Trahe, Fiona <fiona.trahe at intel.com>; dev at dpdk.org
> > Cc: Athreya, Narayana Prasad <NarayanaPrasad.Athreya at cavium.com>;
> > Challa, Mahipal <Mahipal.Challa at cavium.com>
> > Subject: Re: [dpdk-dev] [RFC] Compression API in DPDK
> >
> > [This sender failed our fraud detection checks and may not be who they
> > appear to be. Learn about spoofing at http://aka.ms/LearnAboutSpoofing]
> >
> > > -----Original Message-----
> > > From: Trahe, Fiona [mailto:fiona.trahe at intel.com]
> > > Sent: 07 November 2017 16:54
> > > To: Verma, Shally <Shally.Verma at cavium.com>; dev at dpdk.org
> > > Cc: Athreya, Narayana Prasad <NarayanaPrasad.Athreya at cavium.com>;
> > > Challa, Mahipal <Mahipal.Challa at cavium.com>; Trahe, Fiona
> > > <fiona.trahe at intel.com>
> > > Subject: RE: [dpdk-dev] [RFC] Compression API in DPDK
> > >
> > > Hi Shally,
> > >
> > > ///snip///
> > > > [Shally] Ok. Then, just to confirm my understanding here. You mean PMD
> > > can figure out amount of
> > > > available space in dst mbuf by calling rte_pktmbuf_data_len() on each of
> > its
> > > segment?
> > > [Fiona] exactly.
> > >
> > > ///snip///
> > > > > > > > > +      * This indicates the buffer size and should be
> > > > > > > > > +      * set a little larger than the expected max source buffer
> > size.
> > > > > > > > > +      * if the output of static compression doesn't fit in the
> > > > > > > > > +      * intermediate buffer dynamic compression may not be
> > > possible,
> > > > > > > > > +      * in this case the accelerator may revert back to static
> > > > > compression.
> > > > [Shally] > > > > +      * in this case the accelerator may revert back to static
> > > compression.> > > > > +      */
> > > > Can you elaborate more on this? This looks to me as decision made during
> > > enqueue_burst() processing.
> > > > If yes and If application has chosen specific Huffman code i.e.
> > > RTE_COMP_DYNAMIC or
> > > > RTE_COMP_FIXED in rte_comp_compress_xform, then how this would
> > > work?
> > > [Fiona] yes, it would have to revert back on the enqueue. The compressed
> > > data would still conform to deflate standard, so any decompressor would
> > be
> > > able to inflate it. The ratio would not be as good as hoped for but it would
> > be
> > > the best the compression engine could do with the resources it has.
> > >
> > [Shally] Ok. However, I'm not sure how to use Intermediate bufs here as it is
> > not requirement for us for this purpose.
> > So, it looks like It is very device specific requirement where some may not
> > need it. So, I would suggest that API should propose a way to indicate if it's a
> > requirement for specific device so that app can input it at config time. May be
> > feature flag or capability.
> >
> > Thanks
> > Shally
> >
> > > ///snip///
> > > > [Shally] Sure. So just to align here. Except few questions posted above on
> > > this RFC (such as Dynamic Vs
> > > > Static or dst mbuf parsing), following (and any other) will further be
> > > covered as part of 'RFC doc'
> > > > discussion
> > > > - Hash support
> > > > - RTE_COMPDEV_FF_MULTI_PKT_CHECKSUM
> > > [Fiona] Agreed.