<div dir="ltr">On AWS EC2 I configure: - c5n.metal instance<div>- Stock AWS AMI Ubuntu 20.04 LTS ($ cat /proc/version</div><div>Linux version 5.4.0-1045-aws) - Apply the vfio pci driver from <a href="https://github.com/amzn/am">https://github.com/amzn/am</a> zn-drivers/tree/master/userspace/dpdk/enav2-vfio-patch - Using DPDK code from head of main branch. I built using ninja/meson - Add a 2nd ENA enabled NIC - Setup 2Mb huge pages, and bound the 2nd NIC with dpdk-devbind I run test-pmd with this command: $ sudo ./dpdk-testpmd --huge-dir=/mnt/huge -c 0xf -n 4 -- -i EAL: Detected CPU lcores: 72 EAL: Detected NUMA nodes: 2 EAL: Detected static linkage of DPDK EAL: Multi-process socket /var/run/dpdk/rte/mp_socket EAL: Selected IOVA mode 'VA' EAL: No available 1048576 kB hugepages reported EAL: VFIO support initialized EAL: Using IOMMU type 1 (Type 1) EAL: Ignore mapping IO port bar(3) EAL: Probe PCI driver: net_ena (1d0f:ec20) device: 0000:7f:00.0 (socket 0) TELEMETRY: No legacy callbacks, legacy socket not created Interactive-mode selected Error picking flow transfer proxy for port 0: Function not implemented - ignore testpmd: create a new mbuf pool <mb_pool_0>: n=171456, size=2176, socket=0 testpmd: preferred mempool ops selected: ring_mp_mc Warning! port-topology=paired and odd forward ports number, the last port will pair with itself. Configuring Port 0 (socket 0) Port 0: 16:9A:11:4A:5D:61 Checking link statuses... Done Error during enabling promiscuous mode for port 0: Operation not supported - ignore Key point: I try to enable RSS hash offload for UDP or TCP. I get the following error shown in bold: testpmd> port config all rss udp Port 0 modified RSS hash function based on hardware support,requested:0x20820 configured:0x820 ena_rss_hash_set(): Setting RSS hash fields is not supported ena_rss_hash_update(): Failed to set RSS hash Configuration of RSS hash at ethernet port 0 failed with error (95): Operation not supported. testpmd> port config all rss tcp Port 0 modified RSS hash function based on hardware support,requested:0x10410 configured:0x410 ena_rss_hash_set(): Setting RSS hash fields is not supported ena_rss_hash_update(): Failed to set RSS hash Configuration of RSS hash at ethernet port 0 failed with error (95): Operation not supported. </div><div>If I run ethtool on the NIC before putting it down state and DPDK binding to vfio-pci I see an indication that there is a hash key, and RSS Hash works: ubuntu$ ethtool -x ens6 RX flow hash indirection table for ens6 with 32 RX ring(s): 0: 0 1 2 3 4 5 6 7 8: 8 9 10 11 12 13 14 15 16: 16 17 18 19 20 21 22 23 24: 24 25 26 27 28 29 30 31 32: 0 1 2 3 4 5 6 7 40: 8 9 10 11 12 13 14 15 48: 16 17 18 19 20 21 22 23 56: 24 25 26 27 28 29 30 31 64: 0 1 2 3 4 5 6 7 72: 8 9 10 11 12 13 14 15 80: 16 17 18 19 20 21 22 23 88: 24 25 26 27 28 29 30 31 96: 0 1 2 3 4 5 6 7 104: 8 9 10 11 12 13 14 15 112: 16 17 18 19 20 21 22 23 120: 24 25 26 27 28 29 30 31 RSS hash key: 80:d3:0f:c4:1d:8a:48:9e:7f:1d:18:91:d3:12:cb:38:87:fc:33:49:18:30:e4:1d:19:8f:9d:fd:c3:94:fe:f3:b2:f0:7f:e9:e9:9d:2d:af RSS hash function: toeplitz: on xor: off crc32: off What gives? Do I need to modify a constant somewhere in DPDK? At the risk of expanding scope, I am really trying to run eRPC which runs on top of DPDK. It fails in dpdk_init with a similar but slightly different error shown in bold in which the putative valid value is different than above: $ sudo ./hello_server <a href="http://172.31.77.76:31850">172.31.77.76:31850</a> 20:536566 INFOR: eRPC Nexus: Launching 0 background threads. 20:536582 INFOR: eRPC Nexus: Launching session management thread on core 53. 20:536776 INFOR: eRPC Nexus: Created with management UDP port 31850, hostname 172.31.77.76. 20:536832 INFOR: DPDK transport for Rpc 0 initializing DPDK EAL. EAL: Detected CPU lcores: 72 EAL: Detected NUMA nodes: 2 EAL: Auto-detected process type: PRIMARY EAL: Detected static linkage of DPDK EAL: Multi-process socket /var/run/dpdk/rte/mp_socket EAL: Selected IOVA mode 'VA' EAL: No available 1048576 kB hugepages reported EAL: VFIO support initialized EAL: Using IOMMU type 1 (Type 1) EAL: Ignore mapping IO port bar(3) EAL: Probe PCI driver: net_ena (1d0f:ec20) device: 0000:7f:00.0 (socket 0) TELEMETRY: No legacy callbacks, legacy socket not created 21:428883 WARNG: Running as primary DPDK process. eRPC DPDK daemon is not running. 21:428893 INFOR: DPDK transport for Rpc 0 got QP 0 21:428927 INFOR: Initializing port 0 with driver net_ena Ethdev port_id=0 invalid rss_hf: 0x20820, valid value: 0xc30 terminate called after throwing an instance of 'std::runtime_error' what(): Ethdev configuration error: Invalid argument Aborted </div></div>