[dpdk-dev] Deadlock when start virtio_user + vhost_kernel

[Tiwei Bie]

On Wed, Sep 12, 2018 at 11:47:20AM +0800, Jason Wang wrote:
> Hi:
> 
> Try to launch virtio_user + vhost_kernel with: testpmd
> --vdev=virtio_user0,path=/dev/vhost-net -- -i
> 
> It seems we get a deadlock on
> rte_rwlock_read_lock(&mcfg->memory_hotplug_lock)

Yes, you're right. There is a deadlock here.
FYI, it can be fixed by below patch:
http://patches.dpdk.org/patch/44290/

> 
> calltrace:
> 
> Thread 1 "testpmd" received signal SIGINT, Interrupt.
> rte_memseg_contig_walk (func=func at entry=0x555555a5e630 <add_memory_region>,
> arg=arg at entry=0x7fffffffcec0)
>     at /home/devel/git/dpdk/lib/librte_eal/common/eal_common_memory.c:469
> 469 rte_rwlock_read_lock(&mcfg->memory_hotplug_lock);
> (gdb) bt
> #0  rte_memseg_contig_walk (func=func at entry=0x555555a5e630
> <add_memory_region>, arg=arg at entry=0x7fffffffcec0)
>     at /home/devel/git/dpdk/lib/librte_eal/common/eal_common_memory.c:469
> #1  0x0000555555a5e9b1 in prepare_vhost_memory_kernel () at
> /home/devel/git/dpdk/drivers/net/virtio/virtio_user/vhost_kernel.c:118
> #2  vhost_kernel_ioctl (dev=0x7ffbf5fb3300, req=<optimized out>,
> arg=<optimized out>)
>     at
> /home/devel/git/dpdk/drivers/net/virtio/virtio_user/vhost_kernel.c:190
> #3  0x0000555555a5f211 in virtio_user_mem_event_cb (type=<optimized out>,
> addr=<optimized out>, len=<optimized out>, arg=0x7ffbf5fb3300)
>     at
> /home/devel/git/dpdk/drivers/net/virtio/virtio_user/virtio_user_dev.c:297
> #4  0x000055555574814b in eal_memalloc_mem_event_notify
> (event=event at entry=RTE_MEM_EVENT_ALLOC, start=start at entry=0x7ffbf6000000,
>     len=len at entry=94371840) at
> /home/devel/git/dpdk/lib/librte_eal/common/eal_common_memalloc.c:248
> #5  0x00005555557563f6 in try_expand_heap_primary (contig=false, bound=0,
> align=64, flags=0, socket=0, elt_size=0, pg_sz=<optimized out>,
>     heap=0x7ffff7ff667c) at
> /home/devel/git/dpdk/lib/librte_eal/common/malloc_heap.c:344
> #6  try_expand_heap (heap=heap at entry=0x7ffff7ff667c, pg_sz=<optimized out>,
> elt_size=elt_size at entry=92403968, socket=socket at entry=0,
>     flags=flags at entry=0, align=align at entry=64, bound=0, contig=false) at
> /home/devel/git/dpdk/lib/librte_eal/common/malloc_heap.c:426
> #7  0x0000555555756928 in alloc_more_mem_on_socket
> (heap=heap at entry=0x7ffff7ff667c, size=size at entry=92403968,
> socket=socket at entry=0,
>     flags=flags at entry=0, align=align at entry=64, bound=bound at entry=0,
> contig=false) at
> /home/devel/git/dpdk/lib/librte_eal/common/malloc_heap.c:554
> #8  0x0000555555756e37 in heap_alloc_on_socket (contig=false, bound=0,
> align=64, flags=0, socket=0, size=92403968, type=<optimized out>)
>     at /home/devel/git/dpdk/lib/librte_eal/common/malloc_heap.c:590
> #9  malloc_heap_alloc (type=<optimized out>, size=92403968,
> socket_arg=<optimized out>, flags=0, align=<optimized out>, bound=0,
> contig=false)
>     at /home/devel/git/dpdk/lib/librte_eal/common/malloc_heap.c:626
> #10 0x0000555555753fc1 in rte_zmalloc () at
> /home/devel/git/dpdk/lib/librte_eal/common/rte_malloc.c:74
> #11 0x00005555556192f9 in init_port () at
> /home/devel/git/dpdk/app/test-pmd/testpmd.c:2645
> #12 main (argc=4, argv=0x7fffffffdb18) at
> /home/devel/git/dpdk/app/test-pmd/testpmd.c:2734
> 
> And I also get this warning:
> 
> vhost_kernel_ioctl(): VHOST_SET_OWNER failed: Device or resource busy

Since below commit:
bce7e9050f9b ("net/virtio-user: fix start with kernel vhost")
https://github.com/DPDK/dpdk/commit/bce7e9050f9b

The vhost SET_OWNER will be done in virtio_user_start_device()
unconditionally. It caused above harmless but annoying warning.

> 
> Both looks like a bug that needs to be fixed.
> 
> Thanks
>