<div dir="ltr">Hello, We recently ran into an issue with our product when working with an X550 nic with stable dpdk-23.11. We observed that all the incoming traffic was directed only into a single queue. The issue became evident after displaying the RSS reta which was fully zeroed after device init, thus directing all traffic to rxq0. Moreover, RSS hash key did not seem to be correctly initialized. Manually setting the reta afterwards was enough to balance the incoming traffic between our queues, which convinced me that the issue here was simply a matter of correctly initializing the device on port start. Looking into the pmd's code, I couldn't see any RSS configuration done vf side at device startup, at least not in<code> </code>ixgbevf_dev_rx_init(). I've seen ixgbe_dev_mq_rx_configure() was called during the pf's init and configured RSS to be handled by vf if sriov was on, but this isn't enough to fully configure RSS for the vf. (see code here: <a href="https://git.dpdk.org/dpdk/tree/drivers/net/ixgbe/ixgbe_rxtx.c#n4644" target="_blank">https://git.dpdk.org/dpdk/tree/drivers/net/ixgbe/ixgbe_rxtx.c#n4644</a> ) I have also observed that all different models of nics using ixgbe did not handle RSS in the same way. For example, for nics of the 82599 series, it is written in their datasheet that, for IOV mode: "— Note that RSS is not supported in IOV mode since there is only a single RSS hash function in the hardware." On the contrary, x550 nics have special registers to handle RSS in VF, like VFRSSRK and VFRETA, for example. I believe the RSS config not being initialized for X550 nics might come from a slight misunderstanding on this part. Therefore, I can suggest a patch to add a call to ixgbe_rss_configure() somewhere in ixgbevf_dev_rx_init() specifically for this model of nic. Despite this function being named ixgbe_xxx instead of ixgbevf_xxx, it will do correct initialization for RSS in vf mode because all functions to get RSS-related registers such as ixgbe_reta_reg_get() or ixgbe_reta_size_get() will check if the device is in vf or pf mode and fetch the appropriate registers. Here is a way to reproduce, on an X550 card: <div> </div><div>Here are the nics I am using: </div>0000:08:00.0 ntfp1 ac:1f:6b:57:57:74 ixgbe 1x2.5 GT/s PCIe 1x2.5 GT/s PCIe Intel Corporation Ethernet Connection X553 10 GbE SFP+ 0000:08:00.1 ntfp2 ac:1f:6b:57:57:75 ixgbe 1x2.5 GT/s PCIe 1x2.5 GT/s PCIe Intel Corporation Ethernet Connection X553 10 GbE SFP+ 0000:08:10.0 eth0 d2:c4:fc:c5:c3:05 ixgbevf 1x2.5 GT/s PCIe 0xUnknown Intel Corporation X553 Virtual Function 0000:08:10.2 eth1 e2:a8:68:09:20:29 ixgbevf 1x2.5 GT/s PCIe 0xUnknown Intel Corporation X553 Virtual Function 1) Starting up dpdk-testpmd: sudo dpdk-hugepages.py --setup 2G; dpdk-devbind --bind=vfio-pci 0000:08:10.0 dpdk-devbind --bind=vfio-pci 0000:08:10.2 dpdk-testpmd -a 0000:08:10.0 -a 0000:08:10.2 -- -i --rxq=2 --txq=2 --coremask=0xff0 --total-num-mbufs=250000 EAL: Detected CPU lcores: 12 EAL: Detected NUMA nodes: 1 EAL: Detected static linkage of DPDK EAL: Multi-process socket /var/run/dpdk/rte/mp_socket EAL: Selected IOVA mode 'VA' EAL: VFIO support initialized EAL: Using IOMMU type 1 (Type 1) EAL: Probe PCI driver: net_ixgbe_vf (8086:15c5) device: 0000:08:10.0 (socket -1) EAL: Probe PCI driver: net_ixgbe_vf (8086:15c5) device: 0000:08:10.2 (socket -1) Interactive-mode selected previous number of forwarding cores 1 - changed to number of configured cores 8 Warning: NUMA should be configured manually by using --port-numa-config and --ring-numa-config parameters along with --numa. testpmd: create a new mbuf pool <mb_pool_0>: n=250000, size=2176, socket=0 testpmd: preferred mempool ops selected: ring_mp_mc Configuring Port 0 (socket 0) Port 0: 02:09:C0:9E:09:75 Configuring Port 1 (socket 0) Port 1: 02:09:C0:76:6D:4B Checking link statuses... Done testpmd> 2) Display port info: testpmd> show port info 0 ********************* Infos for port 0 ********************* MAC address: 02:09:C0:B8:68:2F Device name: 0000:08:10.0 Driver name: net_ixgbe_vf Firmware-version: not available Devargs: Connect to socket: 0 memory allocation on the socket: 0 Link status: down Link speed: None Link duplex: half-duplex Autoneg status: On MTU: 1500 Promiscuous mode: disabled Allmulticast mode: disabled Maximum number of MAC addresses: 128 Maximum number of MAC addresses of hash filtering: 4096 VLAN offload: strip off, filter off, extend off, qinq strip off Hash key size in bytes: 40 Redirection table size: 64 Supported RSS offload flow types: ipv4 ipv4-tcp ipv4-udp ipv6 ipv6-tcp ipv6-udp ipv6-ex ipv6-tcp-ex ipv6-udp-ex Minimum size of RX buffer: 1024 Maximum configurable length of RX packet: 9728 Maximum configurable size of LRO aggregated packet: 0 Maximum number of VMDq pools: 64 Current number of RX queues: 2 Max possible RX queues: 4 Max possible number of RXDs per queue: 4096 Min possible number of RXDs per queue: 32 RXDs number alignment: 8 Current number of TX queues: 2 Max possible TX queues: 4 Max possible number of TXDs per queue: 4096 Min possible number of TXDs per queue: 32 TXDs number alignment: 8 Max segment number per packet: 40 Max segment number per MTU/TSO: 40 Device capabilities: 0x0( ) Device error handling mode: passive Device private info: none 3) Display RSS conf: testpmd> show port 0 rss-hash ixgbe_dev_rss_hash_conf_get(): <log>: rss is enabled RSS functions: ipv4 ipv4-tcp ipv6 ipv6-tcp testpmd> show port 0 rss-hash key ixgbe_dev_rss_hash_conf_get(): <log>: rss is enabled RSS functions: ipv4 ipv4-tcp ipv6 ipv6-tcp RSS key: 88F1A05B9FFCD601333EB3FF4176AE8836B36D67D4013A4B75F25806D17078D08C1EF6A69FF29A78 testpmd> show port 0 rss reta [UINT16]: show port <port_id> rss reta <size> <mask0[,mask1]*> testpmd> show port 0 rss-hash algorithm ixgbe_dev_rss_hash_conf_get(): <log>: rss is enabled RSS algorithm: default testpmd> show port 0 rss reta 64 (0xfff) RSS RETA configuration: hash index=0, queue=0 RSS RETA configuration: hash index=1, queue=0 RSS RETA configuration: hash index=2, queue=0 RSS RETA configuration: hash index=3, queue=0 RSS RETA configuration: hash index=4, queue=0 RSS RETA configuration: hash index=5, queue=0 RSS RETA configuration: hash index=6, queue=0 RSS RETA configuration: hash index=7, queue=0 RSS RETA configuration: hash index=8, queue=0 RSS RETA configuration: hash index=9, queue=0 RSS RETA configuration: hash index=10, queue=0 RSS RETA configuration: hash index=11, queue=0 Here you can see the reta is full of 0s, which is causing me the performance issues. <div>The log appearing above "ixgbe_dev_rss_hash_conf_get(): <log>: rss is enabled" is a custom log I added in ixgbe_dev_rss_hash_conf_get(), just after the check that ixgbe_rss_enabled() didn't return 0 (see <a href="https://git.dpdk.org/dpdk/tree/drivers/net/ixgbe/ixgbe_rxtx.c#n3670" target="_blank">https://git.dpdk.org/dpdk/tree/drivers/net/ixgbe/ixgbe_rxtx.c#n3670</a> ).</div><div>I also had occurrences where the RSS key and algorithm weren't set. In these cases, test-pmd would tell me that RSS is disabled, even though it was enabled and simply not configured. After adding my suggested patch, the key set was the default key and the reta was correctly initialized (alternating 1s and 0s).</div></div>