the packet delay is too long when I used dpdk i40e NIC sent single packet

[jiangheng]

Hi Beilei:
we used dpdk implement a user mode tcp protocol stack, and we are testing the delay of a single packet.
however, on i40e NIC(XL710), the packet delay is higher than that of the kernel. After some tests, we find that the time consuming may be the pahse when i40e sends packet.(period from rte_eth_tx_burst to packet be sent)
The same test is performed on the ixgbe(82599) NIC. the delay is lower than that in kernel.

I used dpdk-21.11 stable version. 
I tried upgrading the i40e's firmware version to 9.10, but there was no improvement.
Any suggestion?
Thank you.



The following is packet delay statistics of tcpdump:
i40e NIC:
410	10.373817	192.168.4.4	192.168.4.5	TCP	64	8877→58312 [ACK] Seq=6511 Ack=9642 Win=1048384 Len=0
    10us
411	10.373827	192.168.4.4	192.168.4.5	TCP	94	58312→8877 [PSH, ACK] Seq=9642 Ack=6511 Win=1048512 Len=36
    27us
412	10.373854	192.168.4.5	192.168.4.4	TCP	60	8877→58312 [ACK] Seq=6511 Ack=9678 Win=1048320 Len=0
    9us
413	10.373863	192.168.4.5	192.168.4.4	TCP	90	58312→8877 [PSH, ACK] Seq=9642 Ack=6511 Win=1048512 Len=36
    28us
414	10.373891	192.168.4.4	192.168.4.5	TCP	64	8877→58312 [ACK] Seq=6511 Ack=9678 Win=1048320 Len=0
    11us
415	10.373902	192.168.4.4	192.168.4.5	TCP	94	58312→8877 [PSH, ACK] Seq=9678 Ack=6511 Win=1048512 Len=36
    28us
416	10.373930	192.168.4.5	192.168.4.4	TCP	60	8877→58312 [ACK] Seq=6511 Ack=9714 Win=1048320 Len=0
    11us
417	10.373941	192.168.4.5	192.168.4.4	TCP	90	58312→8877 [PSH, ACK] Seq=9678 Ack=6511 Win=1048512 Len=36
    25us
418	10.373967	192.168.4.4	192.168.4.5	TCP	64	8877→58312 [ACK] Seq=6511 Ack=9714 Win=1048320 Len=0
    11us
419	10.373978	192.168.4.4	192.168.4.5	TCP	94	58312→8877 [PSH, ACK] Seq=9714 Ack=6511 Win=1048512 Len=36
    4us
420	10.373982	192.168.4.5	192.168.4.4	TCP	60	8877→58312 [ACK] Seq=6511 Ack=9750 Win=1048256 Len=0


ixgbe NIC:
110	13.235208	192.168.4.12	192.168.4.11	TCP	64	8877→58312 [ACK] Seq=6511 Ack=7266 Win=1047808 Len=0
    10us
111	13.235218	192.168.4.12	192.168.4.11	TCP	94	58312→8877 [PSH, ACK] Seq=7266 Ack=6511 Win=1048512 Len=36
    6us
112	13.235224	192.168.4.11	192.168.4.12	TCP	64	8877→58312 [ACK] Seq=6511 Ack=7302 Win=1047744 Len=0
    10us
113	13.235234	192.168.4.11	192.168.4.12	TCP	94	58312→8877 [PSH, ACK] Seq=7266 Ack=6511 Win=1048512 Len=36
    6us
114	13.235240	192.168.4.12	192.168.4.11	TCP	64	8877→58312 [ACK] Seq=6511 Ack=7302 Win=1047744 Len=0
    11us
115	13.235251	192.168.4.12	192.168.4.11	TCP	94	58312→8877 [PSH, ACK] Seq=7302 Ack=6511 Win=1048512 Len=36
    5us
116	13.235256	192.168.4.11	192.168.4.12	TCP	64	8877→58312 [ACK] Seq=6511 Ack=7338 Win=1047744 Len=0
    11us
117	13.235267	192.168.4.11	192.168.4.12	TCP	94	58312→8877 [PSH, ACK] Seq=7302 Ack=6511 Win=1048512 Len=36
    6us
118	13.235273	192.168.4.12	192.168.4.11	TCP	64	8877→58312 [ACK] Seq=6511 Ack=7338 Win=1047744 Len=0
    10us
119	13.235283	192.168.4.12	192.168.4.11	TCP	94	58312→8877 [PSH, ACK] Seq=7338 Ack=6511 Win=1048512 Len=36
    6us
120	13.235289	192.168.4.11	192.168.4.12	TCP	64	8877→58312 [ACK] Seq=6511 Ack=7374 Win=1047680 Len=0