[dpdk-users] i40e vPMD fix out of order Rx read issue

[Zhang, Qi Z]

+ Xiaolong

I think we can merge them to dpdk-next-net-intel early since 19.11 cycle is started.

> -----Original Message-----
> From: Gavin Hu (Arm Technology China) [mailto:Gavin.Hu at arm.com]
> Sent: Wednesday, August 14, 2019 11:20 AM
> To: Zhang, Qi Z <qi.z.zhang at intel.com>; Richardson, Bruce
> <bruce.richardson at intel.com>; Ananyev, Konstantin
> <konstantin.ananyev at intel.com>
> Cc: users at dpdk.org; Honnappa Nagarahalli
> <Honnappa.Nagarahalli at arm.com>; Phil Yang (Arm Technology China)
> <Phil.Yang at arm.com>; Ruifeng Wang (Arm Technology China)
> <Ruifeng.Wang at arm.com>
> Subject: RE: i40e vPMD fix out of order Rx read issue
> 
> Hi Qi,
> 
> > -----Original Message-----
> > From: Zhang, Qi Z <qi.z.zhang at intel.com>
> > Sent: Saturday, July 27, 2019 9:33 AM
> > To: Gavin Hu (Arm Technology China) <Gavin.Hu at arm.com>; Richardson,
> > Bruce <bruce.richardson at intel.com>; Ananyev, Konstantin
> > <konstantin.ananyev at intel.com>
> > Cc: users at dpdk.org; Honnappa Nagarahalli
> > <Honnappa.Nagarahalli at arm.com>; Phil Yang (Arm Technology China)
> > <Phil.Yang at arm.com>; Ruifeng Wang (Arm Technology China)
> > <Ruifeng.Wang at arm.com>
> > Subject: RE: i40e vPMD fix out of order Rx read issue
> >
> >
> >
> > > -----Original Message-----
> > > From: Gavin Hu (Arm Technology China) [mailto:Gavin.Hu at arm.com]
> > > Sent: Friday, July 26, 2019 6:01 PM
> > > To: Zhang, Qi Z <qi.z.zhang at intel.com>; Richardson, Bruce
> > > <bruce.richardson at intel.com>; Ananyev, Konstantin
> > > <konstantin.ananyev at intel.com>
> > > Cc: users at dpdk.org; Honnappa Nagarahalli
> > > <Honnappa.Nagarahalli at arm.com>; Phil Yang (Arm Technology China)
> > > <Phil.Yang at arm.com>; Ruifeng Wang (Arm Technology China)
> > > <Ruifeng.Wang at arm.com>
> > > Subject: RE: i40e vPMD fix out of order Rx read issue
> > >
> > > Hi Qi,
> > >
> > > Thanks for your explanation!
> > > I did some testing and found the barriers caused a big drop in
> > > RFC2544 NDR performance on aarch64, how about it on X86?
> > For x86, I think we don't have performance drop, the memory barrier
> > just change the compiler's behavior to avoid generate out of order
> > read instructions, and x86 guarantee no out of read execution, so it
> > does not add new instructions that cost CPU cycles.
> >
> > > Is it possible to count DD bits in a way of surviving across the
> > > out-of-order descriptors reading?
> >
> > I think it is possible, but this will impact performance on x86, but
> > for aarch64, you can try out to see if that benefit and do proper
> > optimization on related vPMD implementation.
> The patches for aarch64 are in community review, and performances gain
> were measured both for ixgbe and i40e vPMD. Very appreciate if you can
> further review and comment.
> Here are the patch links:
> IXGBE vPMD:
> http://patches.dpdk.org/patch/57649/
> http://patches.dpdk.org/patch/57650/
> I40e vPMD:
> http://patches.dpdk.org/cover/57651/
> http://patches.dpdk.org/patch/57652/



> > >
> > > Best Regards,
> > > Gavin
> > >
> > > > -----Original Message-----
> > > > From: Zhang, Qi Z <qi.z.zhang at intel.com>
> > > > Sent: Thursday, July 25, 2019 8:11 PM
> > > > To: Gavin Hu (Arm Technology China) <Gavin.Hu at arm.com>;
> > > > Richardson, Bruce <bruce.richardson at intel.com>; Ananyev,
> > > > Konstantin <konstantin.ananyev at intel.com>
> > > > Cc: users at dpdk.org; Honnappa Nagarahalli
> > > > <Honnappa.Nagarahalli at arm.com>; Phil Yang (Arm Technology China)
> > > > <Phil.Yang at arm.com>
> > > > Subject: RE: i40e vPMD fix out of order Rx read issue
> > > >
> > > > Hi Gavin:
> > > >
> > > > in vPMD, we read 4 or 8 packets as batch, we count DD bits for
> > > > packet received, but not check the if they are continues or not,
> > > > we assume it should always be 1000, 1100, 1110, 1111 ....(take
> > > > batch size is 4 as
> > > > example) while the out of order read instruction generated by
> > > > compiler will cause driver to get un-continues DD bits, like 1011,
> > > > the descriptor on the hole actually is invalid since when it is
> > > > read , descriptor is not write back yet, but we still process this
> > > > as 1110, it cause
> > an
> > > corrupted mbuf returned.
> > > >
> > > > hope this is helpful.
> > > >
> > > > Regards
> > > > Qi
> > > >
> > > > > -----Original Message-----
> > > > > From: Gavin Hu (Arm Technology China) [mailto:Gavin.Hu at arm.com]
> > > > > Sent: Thursday, July 25, 2019 5:57 PM
> > > > > To: Zhang, Qi Z <qi.z.zhang at intel.com>; Richardson, Bruce
> > > > > <bruce.richardson at intel.com>; Ananyev, Konstantin
> > > > > <konstantin.ananyev at intel.com>
> > > > > Cc: users at dpdk.org; Honnappa Nagarahalli
> > > > > <Honnappa.Nagarahalli at arm.com>; Phil Yang (Arm Technology China)
> > > > > <Phil.Yang at arm.com>
> > > > > Subject: i40e vPMD fix out of order Rx read issue
> > > > >
> > > > > Hi Qi,
> > > > >
> > > > > I am working on optimizing the i40e vPMD on aarch64 and I see
> > > > > this patch relevant.
> > > > > Could you illuminate what issue this patch was fixing?
> > > > > I understand the PMD works at the driver layer, for delivery of
> > > > > L2
> > packets.
> > > > > It does not own the responsibility to keep order(the
> > > > > responsibility lies with
> > > > the
> > > > > protocol stack, like TCP)?
> > > > >
> > > > > http://patches.dpdk.org/patch/16665/
> > > > >
> > > > > Best regards,
> > > > > Gavin
> > > > >
> > > > > IMPORTANT NOTICE: The contents of this email and any attachments
> > > > > are confidential and may also be privileged. If you are not the
> > > > > intended recipient, please notify the sender immediately and do
> > > > > not disclose the contents to any other person, use it for any
> > > > > purpose, or store or copy the information in any medium. Thank you.
> > > IMPORTANT NOTICE: The contents of this email and any attachments are
> > > confidential and may also be privileged. If you are not the intended
> > recipient,
> > > please notify the sender immediately and do not disclose the
> > > contents to
> > any
> > > other person, use it for any purpose, or store or copy the
> > > information in any medium. Thank you.
> IMPORTANT NOTICE: The contents of this email and any attachments are
> confidential and may also be privileged. If you are not the intended recipient,
> please notify the sender immediately and do not disclose the contents to any
> other person, use it for any purpose, or store or copy the information in any
> medium. Thank you.