[DPDK/ethdev Bug 1789] vmxnet3_dev_start fails in vmxnet3_v4_rss_configure() for single queue in ESX8.0 causing application to exit

[Kevin Traynor]

fyi - vmxnet3 maintainer Jochen

On 19/09/2025 21:15, bugzilla at dpdk.org wrote:
> https://bugs.dpdk.org/show_bug.cgi?id=1789
> 
>             Bug ID: 1789
>            Summary: vmxnet3_dev_start fails in vmxnet3_v4_rss_configure()
>                     for single queue in ESX8.0 causing application to exit
>            Product: DPDK
>            Version: 19.11
>           Hardware: All
>                 OS: All
>             Status: UNCONFIRMED
>           Severity: normal
>           Priority: Normal
>          Component: ethdev
>           Assignee: dev at dpdk.org
>           Reporter: amiyaranjan.mohakud at gmail.com
>   Target Milestone: ---
> 
> DPDK Version: 19.11
> ESX version: ESX8.0
> 
> We faced an exact similar issue as mentioned in
> https://mails.dpdk.org/archives/users/2025-April/008236.html. 
> 
> Context Details:
> -----------------
> In ESXi 8.0 setup with dpdk-19.11, when the HW compatibility version is ESX8.0
> and with single queue pair configuration, the RSS gets enabled. As a result,
> vmxnet3_v4_rss_configure() gets called and fails while writing/reading from
> BAR1 hw register. It looks to be some compatibility issue with dpdk-19.11 and
> ESX8.0.
> 
>         if (VMXNET3_VERSION_GE_4(hw) &&                                         
>             dev->data->dev_conf.rxmode.mq_mode == ETH_MQ_RX_RSS) {              
>                 /* Check for additional RSS  */                                 
>                 ret = vmxnet3_v4_rss_configure(dev);                            
>                 if (ret != VMXNET3_SUCCESS) {                                   
>                         PMD_INIT_LOG(ERR, "Failed to configure v4 RSS");        
>                         return ret;                                             
>                 }                                                               
>         }  
> 
> As a result, it causes the application to exit.
> 
> Proposed fix:
> I have a plausible fix for this. The fix is to disable RSS for such a case (
> the fix should go only in 19.11). During dev_configure(),  we can set the
> mq_mode  to ETH_MQ_RX_NONE if the num_rx_queues is 1. This would avoid calling
> of vmxnet3_v4_rss_configure().
> 
> 
>         /* Disabling RSS for single queue pair */                               
>         if (dev->data->nb_rx_queues == 1) {                                     
>                 dev->data->dev_conf.rxmode.mq_mode = ETH_MQ_RX_NONE;            
>                 PMD_INIT_LOG(ERR, "WARN: Disabling RSS for single Rx queue");   
>         }
> 
> 
> Pls verify and let me know if it can be upstreamed. Once confirmed/approved, If
> required, I can generate a main stream patch for it.
>