[dpdk-dev] Question: Can't make pcap and refcnt to match

[Mats Liljegren]

I have had stability problems when using pcap in my little
application. My application is a simple benchmark applications that is
trying to see how much data I can send and receive.

It has one lcore per NIC, where each lcore handles transmit and
receive. On the hardware, I make a loopback between two NICs, so the
NICs are in practice paired. I currently use 4 NICs and therefore 4
lcores. Port 0 sends to port 1 and vice versa. Port 2 send to port 3
and vice versa. One pair is using DPDK hardware driver against a dual
i350 NIC. The other pair is using pcap against two of the four
on-board NICs.

When enabling everything saying "DEBUG" in its name in the .config
file, I get the following error:

PMD: rte_eth_dev_config_restore: port 1: MAC address array not supported
PMD: rte_eth_promiscuous_disable: Function not supported
PMD: rte_eth_allmulticast_disable: Function not supported
Speed: 10000 Mbps, full duplex
Port 1 up and running.
PMD: e1000_put_hw_semaphore_generic(): e1000_put_hw_semaphore_generic
PANIC in rte_mbuf_sanity_check():
bad ref cnt
PANIC in rte_mbuf_sanity_check():
bad ref cnt
PMD: e1000_release_phy_82575(): e1000_release_phy_82575
PMD: e1000_release_swfw_sync_82575(): e1000_release_swfw_sync_82575
PMD: e1000_get_hw_semaphore_generic(): e1000_get_hw_semaphore_generic
PMD: eth_igb_rx_queue_setup(): sw_ring=0x7fff776eefc0
hw_ring=0x7fff76830480 dma_addr=0x464630480

PMD: e1000_put_hw_semaphore_generic(): e1000_put_hw_semaphore_generic
PMD: To improve 1G driver performance, consider setting the TX WTHRESH
value to 4, 8, or 16.
PMD: eth_igb_tx_queue_setup(): sw_ring=0x7fff776ece40
hw_ring=0x7fff76840500 dma_addr=0x464640500

PMD: eth_igb_start(): >>
PMD: e1000_read_phy_reg_82580(): e1000_read_phy_reg_82580
PMD: e1000_acquire_phy_82575(): e1000_acquire_phy_82575
PMD: e1000_acquire_swfw_sync_82575(): e1000_acquire_swfw_sync_82575
PMD: e1000_get_hw_semaphore_generic(): e1000_get_hw_semaphore_generic
PMD: e1000_get_cfg_done_82575(): e1000_get_cfg_done_82575
PMD: e1000_put_hw_semaphore_generic(): e1000_put_hw_semaphore_generic
PMD: e1000_read_phy_reg_mdic(): e1000_read_phy_reg_mdic
9: [/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7ffff72a89cd]]
8: [/lib/x86_64-linux-gnu/libpthread.so.0(+0x7f6e) [0x7ffff757df6e]]
7: [/home/mlil/dpdk-demo/build/enea-demo(eal_thread_loop+0x1b9) [0x492669]]
6: [/home/mlil/dpdk-demo/build/enea-demo() [0x4150bc]]
5: [/home/mlil/dpdk-demo/build/enea-demo() [0x414d0b]]
4: [/home/mlil/dpdk-demo/build/enea-demo() [0x4116ef]]
3: [/home/mlil/dpdk-demo/build/enea-demo(rte_mbuf_sanity_check+0xa7) [0x484707]]
2: [/home/mlil/dpdk-demo/build/enea-demo(__rte_panic+0xc1) [0x40f788]]
1: [/home/mlil/dpdk-demo/build/enea-demo(rte_dump_stack+0x18) [0x493f68]]
PMD: e1000_release_phy_82575(): e1000_release_phy_82575
PMD: e1000_release_swfw_sync_82575(): e1000_release_swfw_sync_82575
PMD: e1000_get_hw_semaphore_generic(): e1000_get_hw_semaphore_generic

I checked the source code for pcap, and in the file rte_eth_pcap.c,
function eth_pcap_rx(), I make the following observation:

It pre-allocates a number of mbufs (64 to be exact). It then fills
these mbufs with data and returns them. The pre-allocation seems to
only be done once, and then they are re-used.

This confuses me. How does this work when more than 64 packets are
requested? I see no safety checks for this.

Aren't application supposed to call rte_pktmbuf_free() on the returned
mbufs? If so, the pre-allocated mbufs will have been free'd as far as
I can see and can therefore not be re-used.

What am I missing here?

Regards
Mats