[dpdk-dev] [PATCH v9 0/6] Support TCP/IPv4, VxLAN, and GRE GSO in DPDK

Ferruh Yigit ferruh.yigit at intel.com
Sat Oct 7 01:32:33 CEST 2017


On 10/5/2017 9:36 PM, Mark Kavanagh wrote:
> Generic Segmentation Offload (GSO) is a SW technique to split large
> packets into small ones. Akin to TSO, GSO enables applications to
> operate on large packets, thus reducing per-packet processing overhead.
> 
> To enable more flexibility to applications, DPDK GSO is implemented
> as a standalone library. Applications explicitly use the GSO library
> to segment packets. This patch adds GSO support to DPDK for specific
> packet types: specifically, TCP/IPv4, VxLAN, and GRE.
> 
> The first patch introduces the GSO API framework. The second patch
> adds GSO support for TCP/IPv4 packets (containing an optional VLAN
> tag). The third patch adds GSO support for VxLAN packets that contain
> outer IPv4, and inner TCP/IPv4 headers (plus optional inner and/or 
> outer VLAN tags). The fourth patch adds GSO support for GRE packets
> that contain outer IPv4, and inner TCP/IPv4 headers (with optional 
> outer VLAN tag). The fifth patch in the series enables TCP/IPv4, VxLAN,
> and GRE GSO in testpmd's checksum forwarding engine. The final patch
> in the series adds GSO documentation to the programmer's guide.
> 
> Performance Testing
> ===================
> The performance of TCP/IPv4 GSO on a 10Gbps link is demonstrated using
> iperf. Setup for the test is described as follows:
> 
> a. Connect 2 x 10Gbps physical ports (P0, P1), which are in the same
>    machine, together physically.
> b. Launch testpmd with P0 and a vhost-user port, and use csum
>    forwarding engine with "retry".
> c. Select IP and TCP HW checksum calculation for P0; select TCP HW
>    checksum calculation for vhost-user port.
> d. Launch a VM with csum and tso offloading enabled.
> e. Run iperf-client on virtio-net port in the VM to send TCP packets.
>    With enabling csum and tso, the VM can send large TCP/IPv4 packets
>    (mss is up to 64KB).
> f. P1 is assigned to linux kernel and enabled kernel GRO. Run
>    iperf-server on P1.
> 
> We conduct three iperf tests:
> 
> test-1: enable GSO for P0 in testpmd, and set max GSO segment length
>     to 1518B. Run two iperf-client in the VM.
> test-2: enable TSO for P0 in testpmd, and set TSO segsz to 1518B. Run
>     two iperf-client in the VM.
> test-3: disable GSO and TSO in testpmd. Run two iperf-client in the VM.
> 
> Throughput of the above three tests:
> 
> test-1: 9.4Gbps
> test-2: 9.5Gbps
> test-3: 3Mbps
> 
> Functional Testing
> ==================
> Unlike TCP packets, VMs can't send large VxLAN or GRE packets. The max
> length of tunneled packets from VMs is 1514B. So current experiment
> method can't be used to measure VxLAN and GRE GSO performance, but simply
> test the functionality via setting small GSO segment length (e.g. 500B).
> 
> VxLAN
> -----
> To test VxLAN GSO functionality, we use the following setup:
> 
> a. Connect 2 x 10Gbps physical ports (P0, P1), which are in the same
>    machine, together physically.
> b. Launch testpmd with P0 and a vhost-user port, and use csum forwarding
>    engine with "retry".
> c. Testpmd commands:
>     - csum parse_tunnel on "P0"
>     - csum parse_tunnel on "vhost-user port"
>     - csum set outer-ip hw "P0"
>     - csum set ip hw "P0"
>     - csum set tcp hw "P0"
>     - csum set tcp hw "vhost-user port"
>     - set port "P0" gso on
>     - set gso segsz 500
> d. Launch a VM with csum and tso offloading enabled.
> e. Create a vxlan port for the virtio-net port in the VM. Run iperf-client
>    on the VxLAN port, so TCP packets are VxLAN encapsulated. However, the
>    max packet length is 1514B.
> f. P1 is assigned to linux kernel and kernel GRO is disabled. Similarly,
>    create a VxLAN port for P1, and run iperf-server on the VxLAN port.
> 
> In testpmd, we can see the length of all packets sent from P0 is smaller
> than or equal to 500B. Additionally, the packets arriving in P1 is
> encapsulated and is smaller than or equal to 500B.
> 
> GRE
> ---
> The same process may be used to test GRE functionality, with the exception that
> the tunnel type created for both the guest's virtio-net, and the host's kernel
> interfaces is GRE:
>    `ip tunnel add <gre tunnel> mode gre remote <remote IP> local <local_ip>`
> 
> As in the VxLAN testcase, the length of packets sent from P0, and received on
> P1, is less than 500B.
> 
> Change log
> ==========
> v9:
> - fix testpmd build for i686 target
> - change log level from WARNING to DEBUG in the case of unsupported packet
>   (rte_gso_segment())
> 
> v8:
> - resolve coding style infractions (indentation).
> - centralize invalid parameter checking for rte_gso_segment() into a single
>   'if' statement.
> - don't clear PKT_TX_TCP_SEG flag for packets that don't qualify for GSO
>   on account of invalid params.
> - allow GSO for tunneled packets only via gso_ctx (by correcting 'if'
>   statement condition).
> 
> v7:
> - add RTE_GSO_SEG_SIZE_MIN macro; use this to validate gso_ctx.gso_segsz.
> - rename 'ipid_flag' member of gso_ctx to 'flag'.
> - remove mention of VLAN tags in supported packet types.
> - don't clear PKT_TX_TCP_SEG flag if GSO fails.
> - take all packet overhead into account when checking for empty packet.
> - ensure that only enabled GSO types are enacted upon (i.e. no fall-through to
>   TCP/IPv4 case from tunneled case).
> - validate user-supplied gso segsz arg against RTE_GSO_SEG_SIZE_MIN in testpmd.
> - simplify error-checking/handling for GSO failure case in testpmd csum engine.
> - use 0 instead of !RTE_GSO_IPID_FIXED in testpmd.
> 
> v6:
> - rebase to HEAD of master (i5dce9fcA)
> - remove 'l3_offset' parameter from 'update_ipv4_tcp_headers'
> 
> v5:
> - add GSO section to the programmer's guide.
> - use MF or (previously 'and') offset to check if a packet is IP
>   fragmented.
> - move 'update_header' helper functions to gso_common.h.
> - move txp/ipv4 'update_header' function to gso_tcp4.c.
> - move tunnel 'update_header' function to gso_tunnel_tcp4.c.
> - add offset parameter to 'update_header' functions.
> - combine GRE and VxLAN tunnel header update functions into a single
>   function.
> - correct typos and errors in comments/commit messages.
> 
> v4:
> - use ol_flags instead of packet_type to decide which segmentation
>   function to use.
> - use MF and offset to check if a packet is IP fragmented, instead of
>   using DF.
> - remove ETHER_CRC_LEN from gso segment payload length calculation.
> - refactor internal header update and other functions.
> - remove RTE_GSO_IPID_INCREASE.
> - add some of GSO documents.
> - set the default GSO length to 1514 and fill PKT_TX_TCP_SEG for the
>   packets sent from GSO-enabled ports in testpmd.
> v3:
> - support all IPv4 header flags, including RTE_PTYPE_(INNER_)L3_IPV4,
>   RTE_PTYPE_(INNER_)L3_IPV4_EXT and RTE_PTYPE_(INNER_)L3_IPV4_EXT_
>   UNKNOWN.
> - fill mbuf->packet_type instead of using rte_net_get_ptype() in
>   csumonly.c, since rte_net_get_ptype() doesn't support vxlan.
> - store the input packet into pkts_out inside gso_tcp4_segment() and
>   gso_tunnel_tcp4_segment() instead of rte_gso_segment(), when no GSO
>   is performed.
> - add missing incldues.
> - optimize file names, function names and function description.
> - fix one bug in testpmd.
> v2:
> - merge data segments whose data_len is less than mss into a large data
>   segment in gso_do_segment().
> - use mbuf->packet_type/l2_len/l3_len etc. instead of parsing the packet
>   header in rte_gso_segment().
> - provide IP id macros for applications to select fixed or incremental IP
>   ids.
> 
> Jiayu Hu (3):
>   gso: add Generic Segmentation Offload API framework
>   gso: add TCP/IPv4 GSO support
>   app/testpmd: enable TCP/IPv4, VxLAN and GRE GSO
> 
> Mark Kavanagh (3):
>   gso: add VxLAN GSO support
>   gso: add GRE GSO support
>   doc: add GSO programmer's guide

Hi Mark, Jiayu,

I was about the get this to next-net, but recognized same problem with
gro patch. There are uint8_t storage type usage for port_id. Port id is
now 16bits. And for testpmd you can prefer "portid_t" storage type as well.

Another thing is this patch and GRO patch conflicts, because both
touches same parts. Since Jiayu is the author of the gro patch, is it
possible to define order between these two patches and it doesn't
conflict while applying. Order doesn't matter as long as dependency
defined in cover letter of the patch.

Can you please send a new version with above two items addressed?

Thanks,
ferruh


More information about the dev mailing list