[dpdk-dev] Deadlock when start virtio_user + vhost_kernel

[Jason Wang]

On 2018年09月12日 12:27, Tiwei Bie wrote:
> On Wed, Sep 12, 2018 at 11:47:20AM +0800, Jason Wang wrote:
>> Hi:
>>
>> Try to launch virtio_user + vhost_kernel with: testpmd
>> --vdev=virtio_user0,path=/dev/vhost-net -- -i
>>
>> It seems we get a deadlock on
>> rte_rwlock_read_lock(&mcfg->memory_hotplug_lock)
> Yes, you're right. There is a deadlock here.
> FYI, it can be fixed by below patch:
> http://patches.dpdk.org/patch/44290/

Thanks for the pointer, this fixes the deadlock.

>
>> calltrace:
>>
>> Thread 1 "testpmd" received signal SIGINT, Interrupt.
>> rte_memseg_contig_walk (func=func at entry=0x555555a5e630 <add_memory_region>,
>> arg=arg at entry=0x7fffffffcec0)
>>      at /home/devel/git/dpdk/lib/librte_eal/common/eal_common_memory.c:469
>> 469 rte_rwlock_read_lock(&mcfg->memory_hotplug_lock);
>> (gdb) bt
>> #0  rte_memseg_contig_walk (func=func at entry=0x555555a5e630
>> <add_memory_region>, arg=arg at entry=0x7fffffffcec0)
>>      at /home/devel/git/dpdk/lib/librte_eal/common/eal_common_memory.c:469
>> #1  0x0000555555a5e9b1 in prepare_vhost_memory_kernel () at
>> /home/devel/git/dpdk/drivers/net/virtio/virtio_user/vhost_kernel.c:118
>> #2  vhost_kernel_ioctl (dev=0x7ffbf5fb3300, req=<optimized out>,
>> arg=<optimized out>)
>>      at
>> /home/devel/git/dpdk/drivers/net/virtio/virtio_user/vhost_kernel.c:190
>> #3  0x0000555555a5f211 in virtio_user_mem_event_cb (type=<optimized out>,
>> addr=<optimized out>, len=<optimized out>, arg=0x7ffbf5fb3300)
>>      at
>> /home/devel/git/dpdk/drivers/net/virtio/virtio_user/virtio_user_dev.c:297
>> #4  0x000055555574814b in eal_memalloc_mem_event_notify
>> (event=event at entry=RTE_MEM_EVENT_ALLOC, start=start at entry=0x7ffbf6000000,
>>      len=len at entry=94371840) at
>> /home/devel/git/dpdk/lib/librte_eal/common/eal_common_memalloc.c:248
>> #5  0x00005555557563f6 in try_expand_heap_primary (contig=false, bound=0,
>> align=64, flags=0, socket=0, elt_size=0, pg_sz=<optimized out>,
>>      heap=0x7ffff7ff667c) at
>> /home/devel/git/dpdk/lib/librte_eal/common/malloc_heap.c:344
>> #6  try_expand_heap (heap=heap at entry=0x7ffff7ff667c, pg_sz=<optimized out>,
>> elt_size=elt_size at entry=92403968, socket=socket at entry=0,
>>      flags=flags at entry=0, align=align at entry=64, bound=0, contig=false) at
>> /home/devel/git/dpdk/lib/librte_eal/common/malloc_heap.c:426
>> #7  0x0000555555756928 in alloc_more_mem_on_socket
>> (heap=heap at entry=0x7ffff7ff667c, size=size at entry=92403968,
>> socket=socket at entry=0,
>>      flags=flags at entry=0, align=align at entry=64, bound=bound at entry=0,
>> contig=false) at
>> /home/devel/git/dpdk/lib/librte_eal/common/malloc_heap.c:554
>> #8  0x0000555555756e37 in heap_alloc_on_socket (contig=false, bound=0,
>> align=64, flags=0, socket=0, size=92403968, type=<optimized out>)
>>      at /home/devel/git/dpdk/lib/librte_eal/common/malloc_heap.c:590
>> #9  malloc_heap_alloc (type=<optimized out>, size=92403968,
>> socket_arg=<optimized out>, flags=0, align=<optimized out>, bound=0,
>> contig=false)
>>      at /home/devel/git/dpdk/lib/librte_eal/common/malloc_heap.c:626
>> #10 0x0000555555753fc1 in rte_zmalloc () at
>> /home/devel/git/dpdk/lib/librte_eal/common/rte_malloc.c:74
>> #11 0x00005555556192f9 in init_port () at
>> /home/devel/git/dpdk/app/test-pmd/testpmd.c:2645
>> #12 main (argc=4, argv=0x7fffffffdb18) at
>> /home/devel/git/dpdk/app/test-pmd/testpmd.c:2734
>>
>> And I also get this warning:
>>
>> vhost_kernel_ioctl(): VHOST_SET_OWNER failed: Device or resource busy
> Since below commit:
> bce7e9050f9b ("net/virtio-user: fix start with kernel vhost")
> https://github.com/DPDK/dpdk/commit/bce7e9050f9b
>
> The vhost SET_OWNER will be done in virtio_user_start_device()
> unconditionally. It caused above harmless but annoying warning.

Ok, but it's better to remove it anyway in the future.

Thanks

>> Both looks like a bug that needs to be fixed.
>>
>> Thanks
>>