[dpdk-users] [dpdk-dev] TX unable to enqueue packets to NIC due to no free TX descriptor

Stephen Hemminger stephen at networkplumber.org
Sat Jan 12 00:37:09 CET 2019


On Fri, 11 Jan 2019 22:10:39 +0000
"Soni, Shivam" <shivsoni at amazon.com> wrote:

> Hi All,
> 
> We are trying to debug and fix an issue. After the deployment, in few of the hosts we see an issue where TX is unable to enqueue packets to NIC. On rebouncing or restarting our packet processor daemon, issue gets resolved.
> 
> We are using IntelDPDK version 17.11.4 and i40e drivers.
> 
> On looking into driver’s code, we found that whenever the issue is happening the value for nb_tx_free is ‘0’. And then it tries to free the buffer by calling function ‘i40e_tx_free_bufs’.
> 
> This method returns early as the buffer its trying to free says it hasn’t finished transmitting yet. The method returns at this if condition:
> 
> /* check DD bits on threshold descriptor */
> if ((txq->tx_ring[txq->tx_next_dd].cmd_type_offset_bsz &
>                 rte_cpu_to_le_64(I40E_TXD_QW1_DTYPE_MASK)) !=
>                 rte_cpu_to_le_64(I40E_TX_DESC_DTYPE_DESC_DONE)) {
> return 0;
> }
> 
> Hence nb_tx_free remains 0.
> 
> Our tx descriptor count is 1024.
> 
> How can we fix this issue.  Can someone help us out here please

Use bigger mbuf pool.  For safety the mbuf pool has to be big enough
for Nports * (NRxd + NTxd) + NCore * (mbuf_pool_cache_size + burst_size)

Each NIC might get full receive ring and full transmit ring
and each active core might be processing a burst of packets and have
free buffers sitting in the mbuf pool cache. This doesn't account for additional
mbuf's created if doing things like reassembly, encryption, re-encapsulation, or compression

Anything smaller and your application is relying on statistical averages
to never see resource exhaustion; overcommitment


More information about the users mailing list