[dpdk-dev] net/i40e: latency issue due fix interrupt throttling setting in PF

Hanoch Haim (hhaim) hhaim at cisco.com
Tue Nov 28 14:14:03 CET 2017


Hi Jingjing,

1.       The issue is with DPDK PF.

2.       The rate is high ~10gb, one DP core, one latency core.

3.       The fix is here

/* Bind all RX queues to allocated MSIX interrupt */
                for (i = 0; i < nb_queue; i++) {
                                val = (msix_vect << I40E_QINT_RQCTL_MSIX_INDX_SHIFT) |
            #ifdef TREX_PATCH
                                                I40E_QINT_RQCTL_ITR_INDX_MASK |                                                    << low latency 11b = NoITR
            #else
                                itr_idx << I40E_QINT_RQCTL_ITR_INDX_SHIFT |                                 << high spkies
            #endif

             I40E_WRITE_REG(hw, I40E_QINT_RQCTL(base_queue + i), val);

The Interrupt Throttling ITR is configure using a different setting using a different register here :


4.       The ITR_INTERVAL is 32 usec and it affect a different PF register

#define I40E_ITR_INDEX_DEFAULT          0
#define I40E_ITR_INDEX_NONE             3
#define I40E_QUEUE_ITR_INTERVAL_DEFAULT 32 /* 32 us */


5.       My question is why the VF configuration affects PF INT_INTERVAL ? Can I remove my patch and fix this latency issue in the different way?

Thanks,
Hanoh

From: Wu, Jingjing [mailto:jingjing.wu at intel.com]
Sent: Tuesday, November 28, 2017 2:50 PM
To: Hanoch Haim (hhaim); dev at dpdk.org
Subject: RE: [dpdk-dev] net/i40e: latency issue due fix interrupt throttling setting in PF

Hi, Hanoch

Are you talking about i40 VF's latency? And you are using DPDK PF as host driver?

In this case, we are setting the Interrupt Throttling (ITR) to be maximum. That is to say, if the packet rate is very slow , the receive descriptor is written back  when ITR timeout, otherwise it is written back when cache line is full (4 descriptors/packets). I think that's why you saw the latency is varying.

If we change the ITR to minor, then huge number of interrupts will coming to core which impact performance.


Thanks
Jingjing

From: Hanoch Haim (hhaim) [mailto:hhaim at cisco.com]
Sent: Friday, November 24, 2017 7:25 PM
To: dev at dpdk.org<mailto:dev at dpdk.org>
Cc: Wu, Jingjing <jingjing.wu at intel.com<mailto:jingjing.wu at intel.com>>
Subject: RE: [dpdk-dev] net/i40e: latency issue due fix interrupt throttling setting in PF

Re-sending

Hanoh

From: Hanoch Haim (hhaim)
Sent: Monday, November 20, 2017 5:19 PM
To: dev at dpdk.org<mailto:dev at dpdk.org>
Cc: Wu, Jingjing (jingjing.wu at intel.com<mailto:jingjing.wu at intel.com>); Hanoch Haim (hhaim)
Subject: [dpdk-dev] net/i40e: latency issue due fix interrupt throttling setting in PF

Hi All,
While integrating dpdk17.11 into TRex latest code a new latency issue is observed  (i40e is very sensitive because it has very good resolution due to Qos configuration).
git bitsec found the following commit.
With this commit we observe high spikes of Rx latency (~40usec) vs (~8usec).
Any idea why?
I can send how to reproduce this, it is very simple.

cfd662d22e7bddb4ba41dbd1384f8497f38c2b4e is the first bad commit
commit cfd662d22e7bddb4ba41dbd1384f8497f38c2b4e
Author: Jingjing Wu <jingjing.wu at intel.com<mailto:jingjing.wu at intel.com>>
Date:   Thu Aug 24 09:57:51 2017 +0800

    net/i40e: fix interrupt throttling setting in PF

    As no matter the PF host driver is DPDK or other kernel drivers,
    they are sharing the same virtchnnl interfaces to communicate to VFs.
    To follow the generic interface, DPDK PF need to set Interrupt
    Throttling (ITR) index according to the rxitr_idx from virtchnnl
    instead of ITR_NONE.

    Fixes: 6d59e4ea74a6 ("net/i40e: change version number to support Linux VF")
    Cc: stable at dpdk.org<mailto:stable at dpdk.org>

    Signed-off-by: Jingjing Wu <jingjing.wu at intel.com<mailto:jingjing.wu at intel.com>>



Thanks,
Hanoh



More information about the dev mailing list