Implementing a simple TAP PMD to dpdk-vhost structure

[Nicolson Ken (ニコルソン ケン)]

Hi David,

Hmm, maybe the issue is at my end - to configure QEMU, I followed the tutorial at https://www.redhat.com/en/blog/hands-vhost-user-warm-welcome-dpdk

> Are you seeing the vhost port getting initialised in testpmd output?

Yes, I get a lot of VHOST_CONFIG messages, ending with:
VHOST_CONFIG: (/tmp/sock0) virtio is now ready for processing.
Rx csum will be done in SW, may impact performance.

***
UPDATE: I've been reading more of the manual while doing the troubleshooting below, and I think I've found a major issue while checking the Guest OS.
According to https://doc.dpdk.org/guides-21.11/linux_gsg/linux_drivers.html

dmesg | tail
...
[ 1297.875090] vfio-pci: probe of 0000:31:00.0 failed with error -22

I get the above in the Guest when trying to do devbind, and I also get this on both Host and Guest:

cat /boot/config-$(uname -r) | grep NOIOMMU
CONFIG_VFIO_NOIOMMU=y

Should that actually be "N"? Does "is not set" equal to no IOMMU? I should follow the grubby settings given on that RedHat page, I think. If that is off, then I would guess that that could very well be a source of all my issues.

Back to the previous contents:
***

Now, I try your example - in another terminal on the Host, I'll use "ping -v -I dtap0 -6 fe80::5054:ff:fe01:7d00", which is the address of the interface created in the Guest OS by QEMU (hmm, my <mac address=> seems to have been ignored), after I do "sudo ip link set enp9s0 up" there.

First, ping output (if I don't use "-I dtap0", nothing happens):
$ ping -v -I dtap0 -6 fe80::5054:ff:fe01:7d00
ping: Warning: source address might be selected on device other than: dtap0
PING fe80::5054:ff:fe01:7d00(fe80::5054:ff:fe01:7d00) from :: dtap0: 56 data bytes
From fe80::20d4:2dff:fe67:3768%dtap0 icmp_seq=1 Destination unreachable: Address unreachable
From fe80::20d4:2dff:fe67:3768%dtap0 icmp_seq=2 Destination unreachable: Address unreachable
From fe80::20d4:2dff:fe67:3768%dtap0 icmp_seq=3 Destination unreachable: Address unreachable
^C
--- fe80::5054:ff:fe01:7d00 ping statistics ---
4 packets transmitted, 0 received, +3 errors, 100% packet loss, time 3055ms

testpmd> set verbose 3
Change verbose level from 0 to 3
testpmd> start
io packet forwarding - ports=2 - cores=1 - streams=2 - NUMA support enabled, MP allocation mode: native
Logical Core 1 (socket 0) forwards packets on 2 streams:
  RX P=0/Q=0 (socket 0) -> TX P=1/Q=0 (socket 0) peer=02:00:00:00:00:01
  RX P=1/Q=0 (socket 0) -> TX P=0/Q=0 (socket 0) peer=02:00:00:00:00:00

Then for each for the 6 attempted pings I see:

port 0/queue 0: received 1 packets
  src=22:D4:2D:67:37:68 - dst=33:33:FF:01:7D:00 - pool=mb_pool_0 - type=0x86dd - length=86 - nb_segs=1 - hw ptype: L2_ETHER L3_IPV6  - sw ptype: L2_ETHER L3_IPV6  - l2_len=14 - l3_len=40 - Receive queue=0x0
  ol_flags: RTE_MBUF_F_RX_L4_CKSUM_UNKNOWN RTE_MBUF_F_RX_IP_CKSUM_UNKNOWN RTE_MBUF_F_RX_OUTER_L4_CKSUM_UNKNOWN 
port 1/queue 0: sent 1 packets
  src=22:D4:2D:67:37:68 - dst=33:33:FF:01:7D:00 - pool=mb_pool_0 - type=0x86dd - length=86 - nb_segs=1 - hw ptype: L2_ETHER L3_IPV6  - sw ptype: L2_ETHER L3_IPV6  - l2_len=14 - l3_len=40 - Send queue=0x0
  ol_flags: RTE_MBUF_F_TX_L4_NO_CKSUM

Thanks for the help,
Ken

-----Original Message-----
From: David Marchand <david.marchand at redhat.com> 
Sent: Wednesday, September 6, 2023 5:41 PM
To: Nicolson Ken (ニコルソン ケン) <ken.nicolson at jp.panasonic.com>
Cc: users at dpdk.org; Maxime Coquelin <maxime.coquelin at redhat.com>; Xia, Chenbo <chenbo.xia at intel.com>
Subject: Re: Implementing a simple TAP PMD to dpdk-vhost structure

On Wed, Sep 6, 2023 at 9:53 AM Nicolson Ken (ニコルソン ケン)
<ken.nicolson at jp.panasonic.com> wrote:
> > Alternatively, did you consider using testpmd with the vhost pmd instead ?
>
> I've tried that before, but as far as I can see from net/vhost/rte_eth_host.c it uses rte_vost_enqueue/dequeue_burst() to basically act as a loopback for the Guest OS. I use:
>
> $ sudo dpdk-testpmd -l 0-3 -n 4 --vdev 'net_tap0' --vdev 
> 'net_vhost1,iface=/tmp/sock0,client=1' -- -i
>
> But if I feed data in using "tcpreplay -I dtap0 ...", "show port stats all" shows everything going into the TAP but nothing is forward to vhost.

Well, pinging from a dtap0 netdev in the host to a virtio-net netdev in a guest works me.

testpmd> set verbose 3
Change verbose level from 0 to 3
testpmd> start
io packet forwarding - ports=2 - cores=1 - streams=2 - NUMA support enabled, MP allocation mode: native Logical Core 1 (socket 1) forwards packets on 2 streams:
  RX P=0/Q=0 (socket 0) -> TX P=1/Q=0 (socket 0) peer=02:00:00:00:00:01
  RX P=1/Q=0 (socket 0) -> TX P=0/Q=0 (socket 0) peer=02:00:00:00:00:00

port 0/queue 0: received 16 packets
  src=26:9B:E2:29:7E:C6 - dst=33:33:00:00:00:16 - pool=mb_pool_0 - type=0x86dd - length=90 - nb_segs=1 - hw ptype: L2_ETHER L3_IPV6_EXT
- sw ptype: L2_ETHER L3_IPV6_EXT  - l2_len=14 - l3_len=48 - Receive
queue=0x0
  ol_flags: RTE_MBUF_F_RX_L4_CKSUM_UNKNOWN RTE_MBUF_F_RX_IP_CKSUM_UNKNOWN RTE_MBUF_F_RX_OUTER_L4_CKSUM_UNKNOWN
  src=26:9B:E2:29:7E:C6 - dst=33:33:00:00:00:16 - pool=mb_pool_0 - type=0x86dd - length=90 - nb_segs=1 - hw ptype: L2_ETHER L3_IPV6_EXT
- sw ptype: L2_ETHER L3_IPV6_EXT  - l2_len=14 - l3_len=48 - Receive
queue=0x0
  ol_flags: RTE_MBUF_F_RX_L4_CKSUM_UNKNOWN RTE_MBUF_F_RX_IP_CKSUM_UNKNOWN RTE_MBUF_F_RX_OUTER_L4_CKSUM_UNKNOWN
  src=26:9B:E2:29:7E:C6 - dst=33:33:FF:29:7E:C6 - pool=mb_pool_0 - type=0x86dd - length=86 - nb_segs=1 - hw ptype: L2_ETHER L3_IPV6  - sw
ptype: L2_ETHER L3_IPV6  - l2_len=14 - l3_len=40 - Receive queue=0x0
  ol_flags: RTE_MBUF_F_RX_L4_CKSUM_UNKNOWN RTE_MBUF_F_RX_IP_CKSUM_UNKNOWN RTE_MBUF_F_RX_OUTER_L4_CKSUM_UNKNOWN


Are you seeing the vhost port getting initialised in testpmd output?
How are you sure that nothing is forwarded?


--
David Marchand