[dpdk-users] Unable to merge packets using GRO feature

Hu, Jiayu jiayu.hu at intel.com
Tue Aug 29 03:51:15 CEST 2017


Hi Wisam,

In the picture of your experiment topology, I guess NIC B in the machine B is connected
with another interface physically. If so, the packets that tcpdump captures are from the
physical link. However, packets after GROed are multi-segment large packets, which
are larger than MTU. I am not sure if you have make other configurations to enable
these large packets to pass the physical link, and there is jumbo frame size limit for different
NICs. So would you please to check this? BTW, you can use the VM to test this GRO feature,
since large packets can always pass to the VM.

Thanks,
Jiayu

> -----Original Message-----
> From: Wisam Monther [mailto:wisamm at mellanox.com]
> Sent: Monday, August 28, 2017 3:37 PM
> To: Hu, Jiayu <jiayu.hu at intel.com>
> Cc: users at dpdk.org; Raslan Darawsheh <rasland at mellanox.com>; Shahaf
> Shuler <shahafs at mellanox.com>
> Subject: RE: Unable to merge packets using GRO feature
> 
> Hi Jiayu,
> 
> I'm sorry for bothering you, but could you conform that the feature is
> working probably
> Because, what I ever  did, I couldn't get the merged packets.
> 
> BRs,
> Wisam Jaddo
> 
> -----Original Message-----
> From: Wisam Monther
> Sent: Thursday, August 24, 2017 9:15 AM
> To: 'Hu, Jiayu'
> Cc: users at dpdk.org; Raslan Darawsheh; Shahaf Shuler
> Subject: RE: Unable to merge packets using GRO feature
> 
> Hi,
> 
> I'm using Mellanox NICs, and it is supporting parse packet types.
> 
> Best regards,
> Wisam Jaddo
> 
> -----Original Message-----
> From: Hu, Jiayu [mailto:jiayu.hu at intel.com]
> Sent: Thursday, August 24, 2017 8:47 AM
> To: Wisam Monther
> Cc: users at dpdk.org; Raslan Darawsheh; Shahaf Shuler
> Subject: RE: Unable to merge packets using GRO feature
> 
> Hi,
> 
> Can you tell me what's the NIC type of the GRO-enabled port?
> 
> Since GRO library uses mbuf->packet_type to parse packet headers,
> applications need to fill this value before calling GRO reassembly APIs.
> Otherwise, the GRO can't work correctly.
> 
> In csum forwarding engine of testpmd, packet_type is filled by NIC drivers.
> The csum forwarding engine won't set this value. So if your NIC doesn’t
> support to parse packet types, the value of packet_type is 0 and GRO can't
> work correctly.
> 
> BRs,
> Jiayu
> 
> > -----Original Message-----
> > From: Wisam Monther [mailto:wisamm at mellanox.com]
> > Sent: Tuesday, August 22, 2017 9:25 PM
> > To: Hu, Jiayu <jiayu.hu at intel.com>
> > Cc: users at dpdk.org; Raslan Darawsheh <rasland at mellanox.com>; Shahaf
> > Shuler <shahafs at mellanox.com>
> > Subject: RE: Unable to merge packets using GRO feature
> >
> > Yes it is,
> > The fragmented packets comes from port1 / NIC b from machine A to port
> > 1 in NIC A for machine b So it's received on the port '1', which is
> > configured gro active on this port.
> >
> > Best regards,
> > Wisam Jaddo
> >
> > -----Original Message-----
> > From: Hu, Jiayu [mailto:jiayu.hu at intel.com]
> > Sent: Tuesday, August 22, 2017 4:21 PM
> > To: Wisam Monther
> > Cc: users at dpdk.org; Raslan Darawsheh; Shahaf Shuler
> > Subject: RE: Unable to merge packets using GRO feature
> >
> > Hi,
> >
> > > -----Original Message-----
> > > From: Wisam Monther [mailto:wisamm at mellanox.com]
> > > Sent: Tuesday, August 22, 2017 7:07 PM
> > > To: Hu, Jiayu <jiayu.hu at intel.com>
> > > Cc: users at dpdk.org; Raslan Darawsheh <rasland at mellanox.com>;
> Shahaf
> > > Shuler <shahafs at mellanox.com>
> > > Subject: RE: Unable to merge packets using GRO feature
> > >
> > > Hey Jiayu,
> > >
> > > Thank you for your reply.
> > > I tried what you said with the csum at fwd mode.
> > > Even so the GRO didn't works fine.
> > >
> > > I even tested with a new methodology.
> > > Two machines with two different nic for each.
> > > The methodology that I used to test it is described in the attached file.
> > >
> > > What I did from gro side:
> > > """
> > > testpmd>gro on 1
> >
> > Does the port number of NIC A in machine B is '1'? When you enable GRO
> > for port '1', Testpmd only tries to merge packets received from port '1'.
> >
> > BRs,
> > Jiayu
> >
> > > Testpmd>set fwd csum
> > > Testpmd>start
> > > """
> > > And the packet with correct dst mac.
> > >
> > > Best regards,
> > > Wisam Jaddo
> > > -----Original Message-----
> > > From: Jiayu Hu [mailto:jiayu.hu at intel.com]
> > > Sent: Monday, August 21, 2017 12:14 PM
> > > To: Wisam Monther
> > > Cc: Thomas Monjalon; users at dpdk.org; Raslan Darawsheh; Shahaf Shuler
> > > Subject: Re: Unable to merge packets using GRO feature
> > >
> > > Hi,
> > >
> > > On Mon, Aug 21, 2017 at 07:25:23AM +0000, Wisam Monther wrote:
> > > > Hello Guys,
> > > >
> > > >
> > > >
> > > > I hope this finds you well, I’m trying to test the GRO feature.
> > > > But I’m stuck with this scenario.
> > > >
> > > > As you know, GRO is only support TCP_IPV4 packet until now.
> > > >
> > > > So I’m trying to test the basic functionality of the feature, as following:
> > > >
> > > > Start testpmd:
> > > >
> > > > “””
> > > >
> > > > ./x86_64-native-linuxapp-gcc/build/app/test-pmd/testpmd -n 4  -w
> > > > 00:0a.0  -w
> > > > 00:09.0 --  --burst=64 --mbcache=512 --portmask 0xf -i  --txd=512
> > > > --rxd=512
> > > > --nb-cores=9  --rxq=2 --txq=2 --txqflags=0
> > > >
> > > > “””
> > > >
> > > >
> > > >
> > > > Then enable GRO at the two ports:
> > > >
> > > > “””
> > > >
> > > > Testpmd>gro on 0
> > > >
> > > > Testpmd>gro on 1
> > >
> > > When use GRO in testpmd, there are following things to notice:
> > >
> > > 1. In testpmd, GRO is supported by csum forwarding engine.
> > > Therefore, please use 'set fwd csum' to switch forwarding engine.
> > >
> > > 2. By default, csum forwarding engine compulsorily changes ethernet
> > > addresses. So please make sure that MAC addresses are correct.
> > >
> > > 3. When enable GRO for port0, csum forwarding engine will merge
> > > packets received from port0. If there are no packets from port1 to
> > > port0, you don't need to enable GRO for port1.
> > >
> > > 4. GRO library doesn't re-calculate checksums for merged packets. If
> > > you want merged packets have correct checksum, please select HW IP
> > > and HW TCP checksum calculation for the port which the merged
> > > packets are transmitted to in csum forwarding engine.
> > > This is because the merged packets are multi-segment mbufs, but csum
> > > forwarding engine doesn't support to calculate checksums for
> > > multi-segment mbufs in SW. So we need to select HW checksum
> offloading.
> > >
> > > e.g. If data flow is "packets -> port0 -> port1", commands used in
> testpmd:
> > > 	gro on port0
> > > 	set fwd csum
> > > 	csum set ip hw port1
> > > 	csum set tcp hw port1
> > >
> > >
> > > Besides, you need to make sure that your PMD driver doesn't use
> > > vector TX function, since vector function doesn't support checksum
> offloading.
> > >
> > > >
> > > > “””
> > > >
> > > >
> > > >
> > > > And trying to send TCP_IPV4 fragmented packet “packet with length
> > > > 1500 fragmented to three packets of 500”
> > > >
> > > > “””
> > > >
> > > > p=Ether(src=get_if_hwaddr('ens10'), dst=
> > > > '24:8A:07:88:26:6B')/IP()/TCP()
> > > >
> > > > p.add_payload('F'*(1500 - len(p)))
> > > >
> > > > frags=fragment(p,fragsize=500)
> > > >
> > > > for fragment in frags:
> > > >
> > > >      sendp(fragment, iface='ens10')
> > > >
> > > > “””
> > > >
> > > >
> > > >
> > > > But the testpmd forward the packets as it is, “ doesn’t do any merge”
> > > >
> > > >
> > > >
> > > > Tcpdump at the TG side,
> > > >
> > > > The sending fragmets using ens10:
> > > >
> > > > #tcpdump –I ens10 –vvven
> > > >
> > > > 15:45:29.083514 24:8a:07:88:26:5b > 24:8a:07:88:26:6b, ethertype
> > > > IPv4 (0x0800), length 538: (tos 0x0, ttl 64, id 1, offset 0, flags
> > > > [+], proto Options (0), length 524)
> > > >
> > > >     127.0.0.1 > 127.0.0.1:  ip-proto-0 504
> > > >
> > > > 15:45:29.115266 24:8a:07:88:26:5b > 24:8a:07:88:26:6b, ethertype
> > > > IPv4 (0x0800), length 538: (tos 0x0, ttl 64, id 1, offset 504,
> > > > flags [+], proto Options (0), length 524)
> > > >
> > > >     127.0.0.1 > 127.0.0.1: ip-proto-0
> > > >
> > > > 15:45:29.147258 24:8a:07:88:26:5b > 24:8a:07:88:26:6b, ethertype
> > > > IPv4 (0x0800), length 492: (tos 0x0, ttl 64, id 1, offset 1008,
> > > > flags [none], proto Options (0), length 478)
> > > >
> > > >     127.0.0.1 > 127.0.0.1: ip-proto-0
> > > >
> > > >
> > > >
> > > > #tcpdump -i ens9 –vvven  /// here will be received the forwarded
> > > > packets from
> > > > testpmd:
> > > >
> > > > 15:45:29.083996 24:8a:07:88:26:5b > 24:8a:07:88:26:6b, ethertype
> > > > IPv4 (0x0800), length 538: (tos 0x0, ttl 64, id 1, offset 0, flags
> > > > [+], proto Options (0), length 524)
> > > >
> > > >     127.0.0.1 > 127.0.0.1:  ip-proto-0 504
> > > >
> > > > 15:45:29.115425 24:8a:07:88:26:5b > 24:8a:07:88:26:6b, ethertype
> > > > IPv4 (0x0800), length 538: (tos 0x0, ttl 64, id 1, offset 504,
> > > > flags [+], proto Options (0), length 524)
> > > >
> > > >     127.0.0.1 > 127.0.0.1: ip-proto-0
> > > >
> > > > 15:45:29.147492 24:8a:07:88:26:5b > 24:8a:07:88:26:6b, ethertype
> > > > IPv4 (0x0800), length 492: (tos 0x0, ttl 64, id 1, offset 1008,
> > > > flags [none], proto Options (0), length 478)
> > > >
> > > >     127.0.0.1 > 127.0.0.1: ip-proto-0
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > Am I doing something wrong?! Or it is a bug.
> > > >
> > > > è As you see the tcpdump shows the offset of each fragment, and
> > > > testpmd prints L4_FRAG, so the both are recognizing that this is a
> > > fragmented packet.
> > >
> > > GRO library merges TSOed/GSOed packets, whose IP IDs and TCP
> > sequences
> > > are both consecutive. If input packets have same IP IDs, no packets
> > > will be merged.
> > >
> > > BTW, you can use iperf to test GRO feature.
> > >
> > > Best Regards,
> > > Jiayu
> > >
> > > >
> > > >
> > > >
> > > > Best regards,
> > > >
> > > > Wisam Jaddo
> > > >
> > > >
> > > >


More information about the users mailing list