[dpdk-dev] i40e: problem with rx packet drops not accounted in statistics

Arnon Warshavsky arnon at qwilt.com
Sun Oct 25 19:51:00 CET 2015


Hi Helin

I would like to add my input for this as well.
I encountered the same issue, and as you suggested I updated to the latest
fw and changed rx and tx ring sizes to 1024.
Drop counters still do not increment as they should.

I Inject 10mpps into an x710 nic  (a 4 ports card, 10mpps on each port)
read by a simple rx-only dpdk app.
I read 10mpps from the in-packets counter ,  not getting any drop counters
incrementing , while my application counts only 8 mpps per port that are
actually arriving to the app.
Running the same on x520 I get 8 mpps from the in-packets counter and 2
mpps from dropped packets as it should.
Something seems to be broken in the error/discard accounting.

/Arnon


On Fri, Oct 23, 2015 at 3:42 AM, Zhang, Helin <helin.zhang at intel.com> wrote:

> Hi Martin
>
> Could you help to try bigger size of rx/tx ring, but not the default sizes?
> For example, could you help to try 1024 for RX ring size, and 512 or 1024
> for TX ring size.
>
> In addition, please make sure you are using the latest version of NIC
> firmware.
>
> Regards,
> Helin
>
> > -----Original Message-----
> > From: Martin Weiser [mailto:martin.weiser at allegro-packets.com]
> > Sent: Thursday, October 22, 2015 3:59 PM
> > To: Zhang, Helin
> > Cc: dev at dpdk.org
> > Subject: Re: i40e: problem with rx packet drops not accounted in
> statistics
> >
> > Hi Helin,
> >
> > good to know that there is work being done on that issue.
> > By performance problem I mean that theses packet discards start to
> appear at
> > low bandwidths where I would not expect any packets to be dropped. On the
> > same system we can reach higher bandwidths using ixgbe NICs without
> loosing a
> > single packet so seeing packets being lost at only ~5GBit/s and ~1.5Mpps
> on a
> > 40Gb adapter worries me a bit.
> >
> > Best regards,
> > Martin
> >
> >
> > On 22.10.15 02:16, Zhang, Helin wrote:
> > > Hi Martin
> > >
> > > Yes, we have a developer working on it now, and hopefully he will have
> > something soon later on this fix.
> > > But what do you mean the performance problem? Did you mean the
> > performance number is not good as expected, or else?
> > >
> > > Regards,
> > > Helin
> > >
> > >> -----Original Message-----
> > >> From: Martin Weiser [mailto:martin.weiser at allegro-packets.com]
> > >> Sent: Wednesday, October 21, 2015 4:44 PM
> > >> To: Zhang, Helin
> > >> Cc: dev at dpdk.org
> > >> Subject: Re: i40e: problem with rx packet drops not accounted in
> > >> statistics
> > >>
> > >> Hi Helin,
> > >>
> > >> any news on this issue? By the way this is not just a problem with
> > >> statistics for us but also a performance problem since these packet
> > >> discards start appearing at a relatively low bandwidth (~5GBit/s and
> > ~1.5Mpps).
> > >>
> > >> Best regards,
> > >> Martin
> > >>
> > >> On 10.09.15 03:09, Zhang, Helin wrote:
> > >>> Hi Martin
> > >>>
> > >>> Yes, the statistics issue has been reported several times recently.
> > >>> We will check the issue and try to fix it or get a workaround soon.
> > >>> Thank you
> > >> very much!
> > >>> Regards,
> > >>> Helin
> > >>>
> > >>>> -----Original Message-----
> > >>>> From: Martin Weiser [mailto:martin.weiser at allegro-packets.com]
> > >>>> Sent: Wednesday, September 9, 2015 7:58 PM
> > >>>> To: Zhang, Helin
> > >>>> Cc: dev at dpdk.org
> > >>>> Subject: i40e: problem with rx packet drops not accounted in
> > >>>> statistics
> > >>>>
> > >>>> Hi Helin,
> > >>>>
> > >>>> in one of our test setups involving i40e adapters we are
> > >>>> experiencing packet drops which are not reflected in the interfaces
> > statistics.
> > >>>> The call to rte_eth_stats_get suggests that all packets were
> > >>>> properly received but the total number of packets received through
> > >>>> rte_eth_rx_burst is less than the ipackets counter.
> > >>>> When for example running the l2fwd application (l2fwd -c 0xfe -n 4
> > >>>> -- -p
> > >>>> 0x3) and having driver debug messages enabled the following output
> > >>>> is generated for the interface in question:
> > >>>>
> > >>>> ...
> > >>>> PMD: i40e_update_vsi_stats(): ***************** VSI[6] stats start
> > >>>> *******************
> > >>>> PMD: i40e_update_vsi_stats(): rx_bytes:            242624340000
> > >>>> PMD: i40e_update_vsi_stats(): rx_unicast:          167790000
> > >>>> PMD: i40e_update_vsi_stats(): rx_multicast:        0
> > >>>> PMD: i40e_update_vsi_stats(): rx_broadcast:        0
> > >>>> PMD: i40e_update_vsi_stats(): rx_discards:         1192557
> > >>>> PMD: i40e_update_vsi_stats(): rx_unknown_protocol: 0
> > >>>> PMD: i40e_update_vsi_stats(): tx_bytes:            0
> > >>>> PMD: i40e_update_vsi_stats(): tx_unicast:          0
> > >>>> PMD: i40e_update_vsi_stats(): tx_multicast:        0
> > >>>> PMD: i40e_update_vsi_stats(): tx_broadcast:        0
> > >>>> PMD: i40e_update_vsi_stats(): tx_discards:         0
> > >>>> PMD: i40e_update_vsi_stats(): tx_errors:           0
> > >>>> PMD: i40e_update_vsi_stats(): ***************** VSI[6] stats end
> > >>>> *******************
> > >>>> PMD: i40e_dev_stats_get(): ***************** PF stats start
> > >>>> *******************
> > >>>> PMD: i40e_dev_stats_get(): rx_bytes:            242624340000
> > >>>> PMD: i40e_dev_stats_get(): rx_unicast:          167790000
> > >>>> PMD: i40e_dev_stats_get(): rx_multicast:        0
> > >>>> PMD: i40e_dev_stats_get(): rx_broadcast:        0
> > >>>> PMD: i40e_dev_stats_get(): rx_discards:         0
> > >>>> PMD: i40e_dev_stats_get(): rx_unknown_protocol: 167790000
> > >>>> PMD: i40e_dev_stats_get(): tx_bytes:            0
> > >>>> PMD: i40e_dev_stats_get(): tx_unicast:          0
> > >>>> PMD: i40e_dev_stats_get(): tx_multicast:        0
> > >>>> PMD: i40e_dev_stats_get(): tx_broadcast:        0
> > >>>> PMD: i40e_dev_stats_get(): tx_discards:         0
> > >>>> PMD: i40e_dev_stats_get(): tx_errors:           0
> > >>>> PMD: i40e_dev_stats_get(): tx_dropped_link_down:     0
> > >>>> PMD: i40e_dev_stats_get(): crc_errors:               0
> > >>>> PMD: i40e_dev_stats_get(): illegal_bytes:            0
> > >>>> PMD: i40e_dev_stats_get(): error_bytes:              0
> > >>>> PMD: i40e_dev_stats_get(): mac_local_faults:         1
> > >>>> PMD: i40e_dev_stats_get(): mac_remote_faults:        1
> > >>>> PMD: i40e_dev_stats_get(): rx_length_errors:         0
> > >>>> PMD: i40e_dev_stats_get(): link_xon_rx:              0
> > >>>> PMD: i40e_dev_stats_get(): link_xoff_rx:             0
> > >>>> PMD: i40e_dev_stats_get(): priority_xon_rx[0]:      0
> > >>>> PMD: i40e_dev_stats_get(): priority_xoff_rx[0]:     0
> > >>>> PMD: i40e_dev_stats_get(): priority_xon_rx[1]:      0
> > >>>> PMD: i40e_dev_stats_get(): priority_xoff_rx[1]:     0
> > >>>> PMD: i40e_dev_stats_get(): priority_xon_rx[2]:      0
> > >>>> PMD: i40e_dev_stats_get(): priority_xoff_rx[2]:     0
> > >>>> PMD: i40e_dev_stats_get(): priority_xon_rx[3]:      0
> > >>>> PMD: i40e_dev_stats_get(): priority_xoff_rx[3]:     0
> > >>>> PMD: i40e_dev_stats_get(): priority_xon_rx[4]:      0
> > >>>> PMD: i40e_dev_stats_get(): priority_xoff_rx[4]:     0
> > >>>> PMD: i40e_dev_stats_get(): priority_xon_rx[5]:      0
> > >>>> PMD: i40e_dev_stats_get(): priority_xoff_rx[5]:     0
> > >>>> PMD: i40e_dev_stats_get(): priority_xon_rx[6]:      0
> > >>>> PMD: i40e_dev_stats_get(): priority_xoff_rx[6]:     0
> > >>>> PMD: i40e_dev_stats_get(): priority_xon_rx[7]:      0
> > >>>> PMD: i40e_dev_stats_get(): priority_xoff_rx[7]:     0
> > >>>> PMD: i40e_dev_stats_get(): link_xon_tx:              0
> > >>>> PMD: i40e_dev_stats_get(): link_xoff_tx:             0
> > >>>> PMD: i40e_dev_stats_get(): priority_xon_tx[0]:      0
> > >>>> PMD: i40e_dev_stats_get(): priority_xoff_tx[0]:     0
> > >>>> PMD: i40e_dev_stats_get(): priority_xon_2_xoff[0]:  0
> > >>>> PMD: i40e_dev_stats_get(): priority_xon_tx[1]:      0
> > >>>> PMD: i40e_dev_stats_get(): priority_xoff_tx[1]:     0
> > >>>> PMD: i40e_dev_stats_get(): priority_xon_2_xoff[1]:  0
> > >>>> PMD: i40e_dev_stats_get(): priority_xon_tx[2]:      0
> > >>>> PMD: i40e_dev_stats_get(): priority_xoff_tx[2]:     0
> > >>>> PMD: i40e_dev_stats_get(): priority_xon_2_xoff[2]:  0
> > >>>> PMD: i40e_dev_stats_get(): priority_xon_tx[3]:      0
> > >>>> PMD: i40e_dev_stats_get(): priority_xoff_tx[3]:     0
> > >>>> PMD: i40e_dev_stats_get(): priority_xon_2_xoff[3]:  0
> > >>>> PMD: i40e_dev_stats_get(): priority_xon_tx[4]:      0
> > >>>> PMD: i40e_dev_stats_get(): priority_xoff_tx[4]:     0
> > >>>> PMD: i40e_dev_stats_get(): priority_xon_2_xoff[4]:  0
> > >>>> PMD: i40e_dev_stats_get(): priority_xon_tx[5]:      0
> > >>>> PMD: i40e_dev_stats_get(): priority_xoff_tx[5]:     0
> > >>>> PMD: i40e_dev_stats_get(): priority_xon_2_xoff[5]:  0
> > >>>> PMD: i40e_dev_stats_get(): priority_xon_tx[6]:      0
> > >>>> PMD: i40e_dev_stats_get(): priority_xoff_tx[6]:     0
> > >>>> PMD: i40e_dev_stats_get(): priority_xon_2_xoff[6]:  0
> > >>>> PMD: i40e_dev_stats_get(): priority_xon_tx[7]:      0
> > >>>> PMD: i40e_dev_stats_get(): priority_xoff_tx[7]:     0
> > >>>> PMD: i40e_dev_stats_get(): priority_xon_2_xoff[7]:  0
> > >>>> PMD: i40e_dev_stats_get(): rx_size_64:               0
> > >>>> PMD: i40e_dev_stats_get(): rx_size_127:              0
> > >>>> PMD: i40e_dev_stats_get(): rx_size_255:              0
> > >>>> PMD: i40e_dev_stats_get(): rx_size_511:              0
> > >>>> PMD: i40e_dev_stats_get(): rx_size_1023:             0
> > >>>> PMD: i40e_dev_stats_get(): rx_size_1522:             167790000
> > >>>> PMD: i40e_dev_stats_get(): rx_size_big:              0
> > >>>> PMD: i40e_dev_stats_get(): rx_undersize:             0
> > >>>> PMD: i40e_dev_stats_get(): rx_fragments:             0
> > >>>> PMD: i40e_dev_stats_get(): rx_oversize:              0
> > >>>> PMD: i40e_dev_stats_get(): rx_jabber:                0
> > >>>> PMD: i40e_dev_stats_get(): tx_size_64:               0
> > >>>> PMD: i40e_dev_stats_get(): tx_size_127:              0
> > >>>> PMD: i40e_dev_stats_get(): tx_size_255:              0
> > >>>> PMD: i40e_dev_stats_get(): tx_size_511:              0
> > >>>> PMD: i40e_dev_stats_get(): tx_size_1023:             0
> > >>>> PMD: i40e_dev_stats_get(): tx_size_1522:             0
> > >>>> PMD: i40e_dev_stats_get(): tx_size_big:              0
> > >>>> PMD: i40e_dev_stats_get(): mac_short_packet_dropped: 0
> > >>>> PMD: i40e_dev_stats_get(): checksum_error:           0
> > >>>> PMD: i40e_dev_stats_get(): fdir_match:               0
> > >>>> PMD: i40e_dev_stats_get(): ***************** PF stats end
> > >>>> ********************
> > >>>> ...
> > >>>>
> > >>>> The count for rx_unicast is exactly the number of packets we would
> > >>>> have expected and the count for rx_discards in the VSI stats is
> > >>>> exactly the number of packets we are missing.
> > >>>> The question is why this number shows up only in the VSI stats and
> > >>>> not in the PF stats and of course why the packets which were
> > >>>> obviously discarded are still counted in the rx_unicast stats.
> > >>>> This test was performed using DPDK 2.1 and the firmware of the
> > >>>> XL710 is the latest one (FW 4.40 API 1.4 NVM 04.05.03).
> > >>>> Do you have an idea what might be going on?
> > >>>>
> > >>>> Best regards,
> > >>>> Martin
> > >>>>
> > >>>>
> >
>
>


-- 

*Arnon Warshavsky*
*Qwilt | work: +972-72-2221634 | mobile: +972-50-8583058 | arnon at qwilt.com
<arnon at qwilt.com>*


More information about the dev mailing list