[dpdk-users] Performance Problem of DPDK pkt-gen

[Wiles, Keith]

>Dear all,
>
>I am using an server with 4 cpus (4 x 8 core CPUs with HT) and NICs (X520).
>
>When I use pkt-gen on NIC1 or NIC2, the speed of generating 64Byte is
>14Mpps.
>
>If I generating both on NIC1 and NIC2, the speed of  generating 64Byte on
>both are more than  13Mpps.
>
>However, I use same configuration PC (with DPDK L2FWD) to bridge NIC1 and
>NIC2, so I can generate packet on NIC1 and receive these packets on NIC2 in
>pkt-gen. The speed of generating is decreased to 4Mpps and the receive rate
>is 3Mpps.

I am not sure how you configured the second PC for L2FWD, but I suspect the L2FWD is having to receive packets on socket 0 and send the packets on socket 1, this means the QPI bus gets involved here. Is this the case?

>
>
>I want to know why generating speed is slower than the situation without
>the bridge of NIC1 and NIC2?  how to solve these problem?
>
>The detailed information is as following.
>
>sudo sysctl vm.nr_hugepages=4096
>echo 1024 | sudo tee
>/sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
>echo 1024 | sudo tee
>/sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages
>echo 1024 | sudo tee
>/sys/devices/system/node/node2/hugepages/hugepages-2048kB/nr_hugepages
>echo 1024 | sudo tee
>/sys/devices/system/node/node3/hugepages/hugepages-2048kB/nr_hugepages
>
>
>sudo mkdir -p /dev/hugepages
>sudo mount -t hugetlbfs nodev /dev/hugepages
>
>
>sudo dpdk-2.2.0/tools/dpdk_nic_bind.py --status
>sudo modprobe uio
>sudo insmod dpdk-2.2.0/build/kmod/igb_uio.ko
>
>sudo dpdk-2.2.0/tools/dpdk_nic_bind.py -b igb_uio 04:00.0 04:00.1
>sudo dpdk-2.2.0/tools/dpdk_nic_bind.py --status
>
>cd pktgen-2.9.12/
>
>sudo app/build/pktgen -c 0x1f -n 3 --proc-type auto --socket-mem
>128,128,128,128 -- -P -m "[1:3].0, [2:4].1" -f test/set_seq.pkt
>
>I tried to change -m, but, sometime there is no packet generated by
>pkt-gen.
>
>
>The core map is :
>
>EAL: Detected lcore 0 as core 0 on socket 0
>EAL: Detected lcore 1 as core 0 on socket 1
>EAL: Detected lcore 2 as core 0 on socket 2
>EAL: Detected lcore 3 as core 0 on socket 3
>EAL: Detected lcore 4 as core 1 on socket 0
>EAL: Detected lcore 5 as core 1 on socket 1
>EAL: Detected lcore 6 as core 1 on socket 2
>EAL: Detected lcore 7 as core 1 on socket 3
>EAL: Detected lcore 8 as core 2 on socket 0
>EAL: Detected lcore 9 as core 2 on socket 1
>EAL: Detected lcore 10 as core 2 on socket 2
>EAL: Detected lcore 11 as core 2 on socket 3
>EAL: Detected lcore 12 as core 3 on socket 0
>EAL: Detected lcore 13 as core 3 on socket 1
>EAL: Detected lcore 14 as core 3 on socket 2
>EAL: Detected lcore 15 as core 3 on socket 3
>EAL: Detected lcore 16 as core 4 on socket 0
>EAL: Detected lcore 17 as core 4 on socket 1
>EAL: Detected lcore 18 as core 4 on socket 2
>EAL: Detected lcore 19 as core 4 on socket 3
>EAL: Detected lcore 20 as core 5 on socket 0
>EAL: Detected lcore 21 as core 5 on socket 1
>EAL: Detected lcore 22 as core 5 on socket 2
>EAL: Detected lcore 23 as core 5 on socket 3
>EAL: Detected lcore 24 as core 6 on socket 0
>EAL: Detected lcore 25 as core 6 on socket 1
>EAL: Detected lcore 26 as core 6 on socket 2
>EAL: Detected lcore 27 as core 6 on socket 3
>EAL: Detected lcore 28 as core 7 on socket 0
>EAL: Detected lcore 29 as core 7 on socket 1
>EAL: Detected lcore 30 as core 7 on socket 2
>EAL: Detected lcore 31 as core 7 on socket 3
>EAL: Detected lcore 32 as core 0 on socket 0
>EAL: Detected lcore 33 as core 0 on socket 1
>EAL: Detected lcore 34 as core 0 on socket 2
>EAL: Detected lcore 35 as core 0 on socket 3
>EAL: Detected lcore 36 as core 1 on socket 0
>EAL: Detected lcore 37 as core 1 on socket 1
>EAL: Detected lcore 38 as core 1 on socket 2
>EAL: Detected lcore 39 as core 1 on socket 3
>EAL: Detected lcore 40 as core 2 on socket 0
>EAL: Detected lcore 41 as core 2 on socket 1
>EAL: Detected lcore 42 as core 2 on socket 2
>EAL: Detected lcore 43 as core 2 on socket 3
>EAL: Detected lcore 44 as core 3 on socket 0
>EAL: Detected lcore 45 as core 3 on socket 1
>EAL: Detected lcore 46 as core 3 on socket 2
>EAL: Detected lcore 47 as core 3 on socket 3
>EAL: Detected lcore 48 as core 4 on socket 0
>EAL: Detected lcore 49 as core 4 on socket 1
>EAL: Detected lcore 50 as core 4 on socket 2
>EAL: Detected lcore 51 as core 4 on socket 3
>EAL: Detected lcore 52 as core 5 on socket 0
>EAL: Detected lcore 53 as core 5 on socket 1
>EAL: Detected lcore 54 as core 5 on socket 2
>EAL: Detected lcore 55 as core 5 on socket 3
>EAL: Detected lcore 56 as core 6 on socket 0
>EAL: Detected lcore 57 as core 6 on socket 1
>EAL: Detected lcore 58 as core 6 on socket 2
>EAL: Detected lcore 59 as core 6 on socket 3
>EAL: Detected lcore 60 as core 7 on socket 0
>EAL: Detected lcore 61 as core 7 on socket 1
>EAL: Detected lcore 62 as core 7 on socket 2
>EAL: Detected lcore 63 as core 7 on socket 3
>
>
>
>
>
>
>
>
>-- 
>Regards,
>
>Royce Niu
>


Regards,
Keith