[dpdk-dev] virtio UIO / PMD issues in default Ubuntu Cloud Images

[Matthew Hall]

Another problem regarding virtio-net-pmd. When I tried using virtio-net-pmd, 
it compiles fine but then hits a weird error also during EAL init process:

EAL: open shared lib /vagrant/external/virtio-net-pmd/librte_pmd_virtio.so
EAL: /vagrant/external/virtio-net-pmd/librte_pmd_virtio.so: undefined symbol: per_lcore__lcore_id

It doesn't seem to have a link dependency against any DPDK library that might 
contain such a symbol, either:

$ ldd librte_pmd_virtio.so
    linux-vdso.so.1 =>  (0x00007fffd61fc000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fa2d971f000)
    /lib64/ld-linux-x86-64.so.2 (0x00007fa2d9d00000)

$ nm librte_pmd_virtio.so | fgrep -i lcore
                 U per_lcore__lcore_id
00000000000014ee t rte_lcore_id

man nm says this means: "U" The symbol is undefined.

At present I am using the common setup, static DPDK w/ COMBINE_LIBS. The 
directions don't state that a shared lib DPDK is required, and if it is 
required, that increases complexity and reduces performance so it'd be better 
not to be forced to require this unless there's a good reason.

Instead I think it would be good if this driver didn't have to be external so 
it would just work right by default as part of the normal DPDK... right now it 
seems like it's a lot harder to set it up than it really needs to be.

Another weird issue... when I tried to compile a DPDK shared lib using clang I 
got this really, really weird error:

/usr/bin/ld: test: hidden symbol `mknod' in /usr/lib/x86_64-linux-gnu/libc_nonshared.a(mknod.oS) is referenced by DSO
/usr/bin/ld: final link failed: Bad value

Did anybody ever see something this before?

Thanks,
Matthew.

On Mon, Oct 13, 2014 at 10:45:23PM -0700, Matthew Hall wrote:
> Hello,
> 
> I was working to get my open source project running in a VirtualBox Vagrant VM 
> powered by an Ubuntu Cloud image the last few days to make my project and DPDK 
> more developer friendly with a prebuilt environment. During this I was fixing 
> the ugly hardcodings I'd used to hack it together on a desktop.
> 
> In the process I found a few things I was confused about.
> 
> 1) Ubuntu doesn't include the UIO module by default in the Cloud image... only 
> the virtio-net. I was wondering if we had anybody in good contact with their 
> kernel group to lobby for inclusion of the UIO modules in the stock Cloud 
> kernel, without having to grab linux-generic and linux-headers-generic first.
> 
> 2) The directions for activating virtio-net based interfaces seem out-of-date. 
> They refer to the PMD as rte_virtio_net_pmd, when the PMD calls itself 
> rte_virtio_pmd in my copy of DPDK 1.7.1 w/ my clang compilation patches added.
> 
> I am getting some odd errors when I'm trying to load my app:
> 
> EAL: PCI device 0000:00:08.0 on NUMA socket -1
> EAL:   probe driver: 1af4:1000 rte_virtio_pmd
> EAL:   0000:00:08.0 not managed by UIO driver, skipping
> 
> 3) If I try to bind the device to the igb_uio driver even though it seems like 
> the wrong thing to do, just for testing, this is what happens (NOTE: unbound 
> the 00:08.0 device from the kernel to show this):
> 
> $ sudo tools/dpdk_nic_bind.py --status
> 
> Network devices using DPDK-compatible driver
> ============================================
> <none>
> 
> Network devices using kernel driver
> ===================================
> 0000:00:03.0 'Virtio network device' if= drv=virtio-pci unused=igb_uio
> 
> Other network devices
> =====================
> 0000:00:08.0 'Virtio network device' unused=igb_uio
> 
> $ sudo tools/dpdk_nic_bind.py -b igb_uio 00:08.0
> Error: bind failed for 0000:00:08.0 - Cannot bind to driver igb_uio
> 
> vagrant at vagrant-ubuntu-trusty-64:/vagrant/external/dpdk$ dmesg | tail
> [ 1766.445609] igb_uio: Use MSIX interrupt by default
> [ 1824.602075] igb_uio: probe of 0000:00:08.0 failed with error -2
> [ 1824.602742] igb_uio: probe of 0000:00:08.0 failed with error -2
> 
> 4) I found some old email threads asking about this:
> 
> http://comments.gmane.org/gmane.comp.networking.dpdk.devel/1357
> (there are some others as well but this seemed closest)
> 
> But the only thing present in that thread seemed to be irritated replies which 
> didn't really explain the different virtio PMD's out there and how they work, 
> and didn't explain which ones were in-tree and out-of-tree either.
> 
> So let me ask this again, when somebody wrote "virtio-net or virtio-net + uio 
> (QEMU, VirtualBox)" into the supported page (http://dpdk.org/doc/nics), who 
> tested this to prove it worked? How did they get it to work on VirtualBox? The 
> last reply stating "you have the source code" didn't really explain how they 
> proved this stuff ever worked in the first place.
> 
> Thanks,
> Matthew.