[dpdk-dev] [PATCH v4 0/5] Support TCP/IPv4, VxLAN and GRE GSO in DPDK

[Jiayu Hu]

Generic Segmentation Offload (GSO) is a SW technique to split large
packets into small ones. Akin to TSO, GSO enables applications to
operate on large packets, thus reducing per-packet processing overhead.

To enable more flexibility to applications, DPDK GSO is implemented
as a standalone library. Applications explicitly use the GSO library
to segment packets. This patch adds GSO support to DPDK for specific
packet types: specifically, TCP/IPv4, VxLAN, and GRE.

The first patch introduces the GSO API framework. The second patch
adds GSO support for TCP/IPv4 packets (containing an optional VLAN
tag). The third patch adds GSO support for VxLAN packets that contain
outer IPv4, and inner TCP/IPv4 headers (plus optional inner and/or 
outer VLAN tags). The fourth patch adds GSO support for GRE packets
that contain outer IPv4, and inner TCP/IPv4 headers (with optional 
outer VLAN tag). The last patch in the series enables TCP/IPv4, VxLAN,
and GRE GSO in testpmd's checksum forwarding engine.

The performance of TCP/IPv4 GSO on a 10Gbps link is demonstrated using
iperf. Setup for the test is described as follows:

a. Connect 2 x 10Gbps physical ports (P0, P1), which are in the same
   machine, together physically.
b. Launch testpmd with P0 and a vhost-user port, and use csum
   forwarding engine with "retry".
c. Select IP and TCP HW checksum calculation for P0; select TCP HW
   checksum calculation for vhost-user port.
d. Launch a VM with csum and tso offloading enabled.
e. Run iperf-client on virtio-net port in the VM to send TCP packets.
   With enabling csum and tso, the VM can send large TCP/IPv4 packets
   (mss is up to 64KB).
f. P1 is assigned to linux kernel and enabled kernel GRO. Run
   iperf-server on P1.

We conduct three iperf tests:

test-1: enable GSO for P0 in testpmd, and set max GSO segment length
	to 1518B. Run two iperf-client in the VM.
test-2: enable TSO for P0 in testpmd, and set TSO segsz to 1518B. Run
	two iperf-client in the VM.
test-3: disable GSO and TSO in testpmd. Run two iperf-client in the VM.

Throughput of the above three tests:

test-1: 9.4Gbps
test-2: 9.5Gbps
test-3: 3Mbps

Unlike TCP packets, VMs can't send large VxLAN or GRE packets. The max
length of tunnelled packets from VMs is 1514B. So current experiment
method can't used to measure VxLAN and GRE GSO performance, but test
the functionality via setting small GSO segment length (e.g. 500B).

To test VxLAN GSO functionality, we use the following setup:

a. Connect 2 x 10Gbps physical ports (P0, P1), which are in the same
   machine, together physically.
b. Launch testpmd with P0 and a vhost-user port, and use csum forwarding
   engine with "retry".
c. Testpmd commands:
	- csum parse-tunnel on "P0"
	- csum parse-tunnel on "vhost-user port"
	- csum set outer-ip hw "P0"
	- csum set ip hw "P0"
	- csum set tcp hw "P0"
	- csum set tcp hw "vhost-user port"
	- set port "P0" gso on
	- set gso segsz 500
d. Launch a VM with csum and tso offloading enabled.
e. Create a vxlan port for the virtio-net port in the VM. Run iperf-client
   on the VxLAN port, so TCP packets are VxLAN encapsulated. However, the
   max packet length is 1514B.
f. P1 is assigned to linux kernel and disabled kernel GRO. Similarly,
   create a VxLAN port for P1, and run iperf-server on the VxLAN port.

In testpmd, we can see the length of all packets sent from P0 is smaller
than or equal to 500B. Additionally, the packets arriving in P1 is
encapsulated and is smaller than or equal to 500B.

The experimental data of GRE GRO will be shown later, and the prog_guide
will be added later.

Change log
==========
v4:
- use ol_flags instead of packet_type to decide which segmentation
  function to use.
- use MF and offset to check if a packet is IP fragmented, instead of
  using DF.
- remove ETHER_CRC_LEN from gso segment payload length calculation.
- refactor internal header update and other functions.
- remove RTE_GSO_IPID_INCREASE.
- add some of GSO documents.
- set the default GSO length to 1514 and fill PKT_TX_TCP_SEG for the
  packets sent from GSO-enabled ports in testpmd.
v3:
- support all IPv4 header flags, including RTE_PTYPE_(INNER_)L3_IPV4,
  RTE_PTYPE_(INNER_)L3_IPV4_EXT and RTE_PTYPE_(INNER_)L3_IPV4_EXT_
  UNKNOWN.
- fill mbuf->packet_type instead of using rte_net_get_ptype() in
  csumonly.c, since rte_net_get_ptype() doesn't support vxlan.
- store the input packet into pkts_out inside gso_tcp4_segment() and
  gso_tunnel_tcp4_segment() instead of rte_gso_segment(), when no GSO
  is performed.
- add missing incldues.
- optimize file names, function names and function description.
- fix one bug in testpmd.
v2:
- merge data segments whose data_len is less than mss into a large data
  segment in gso_do_segment().
- use mbuf->packet_type/l2_len/l3_len etc. instead of parsing the packet
  header in rte_gso_segment().
- provide IP id macros for applications to select fixed or incremental IP
  ids.
- change the defination of gso_types in struct rte_gso_ctx.
- replace rte_pktmbuf_detach() with rte_pktmbuf_free().
- refactor gso_update_pkt_headers().
- change the return value of rte_gso_segment().
- remove parameter checks in rte_gso_segment().
- use rte_net_get_ptype() in app/test-pmd/csumonly.c to fill
  mbuf->packet_type.
- add a new GSO command in testpmd to show GSO configuration for ports.
- misc: fix typo and optimize function description.

Jiayu Hu (3):
  gso: add Generic Segmentation Offload API framework
  gso: add TCP/IPv4 GSO support
  app/testpmd: enable TCP/IPv4, VxLAN and GRE GSO

Mark Kavanagh (2):
  gso: add VxLAN GSO support
  gso: add GRE GSO support

 app/test-pmd/cmdline.c                      | 178 +++++++++++++++++
 app/test-pmd/config.c                       |  24 +++
 app/test-pmd/csumonly.c                     |  69 ++++++-
 app/test-pmd/testpmd.c                      |  13 ++
 app/test-pmd/testpmd.h                      |  10 +
 config/common_base                          |   5 +
 doc/api/doxy-api-index.md                   |   1 +
 doc/api/doxy-api.conf                       |   1 +
 doc/guides/rel_notes/release_17_11.rst      |  19 ++
 doc/guides/testpmd_app_ug/testpmd_funcs.rst |  41 ++++
 lib/Makefile                                |   2 +
 lib/librte_eal/common/include/rte_log.h     |   1 +
 lib/librte_gso/Makefile                     |  52 +++++
 lib/librte_gso/gso_common.c                 | 291 ++++++++++++++++++++++++++++
 lib/librte_gso/gso_common.h                 | 157 +++++++++++++++
 lib/librte_gso/gso_tcp4.c                   |  82 ++++++++
 lib/librte_gso/gso_tcp4.h                   |  76 ++++++++
 lib/librte_gso/gso_tunnel_tcp4.c            |  89 +++++++++
 lib/librte_gso/gso_tunnel_tcp4.h            |  76 ++++++++
 lib/librte_gso/rte_gso.c                    | 107 ++++++++++
 lib/librte_gso/rte_gso.h                    | 144 ++++++++++++++
 lib/librte_gso/rte_gso_version.map          |   7 +
 mk/rte.app.mk                               |   1 +
 23 files changed, 1441 insertions(+), 5 deletions(-)
 create mode 100644 lib/librte_gso/Makefile
 create mode 100644 lib/librte_gso/gso_common.c
 create mode 100644 lib/librte_gso/gso_common.h
 create mode 100644 lib/librte_gso/gso_tcp4.c
 create mode 100644 lib/librte_gso/gso_tcp4.h
 create mode 100644 lib/librte_gso/gso_tunnel_tcp4.c
 create mode 100644 lib/librte_gso/gso_tunnel_tcp4.h
 create mode 100644 lib/librte_gso/rte_gso.c
 create mode 100644 lib/librte_gso/rte_gso.h
 create mode 100644 lib/librte_gso/rte_gso_version.map

-- 
2.7.4