[dpdk-dev] [PATCH v5 resend 07/12] virtio: resolve for control queue

[Yuanhan Liu]

On Mon, Oct 12, 2015 at 10:58:17PM +0200, Steffen Bauch wrote:
> On 10/12/2015 10:39 AM, Yuanhan Liu wrote:
> >Hi,
> >
> >I just recognized that this dead loop is the same one that I have
> >experienced (see
> >http://dpdk.org/ml/archives/dev/2015-October/024737.html for
> >reference). Just applying the changes in this patch (only 07/12)
> >will not fix the dead loop at least in my setup.
> >Try to enable CONFIG_RTE_LIBRTE_VIRTIO_DEBUG_INIT, and dump more log?
> I enabled the additional debug output. First try was without any
> additional changes in master, but it blocked also. Second try was
> with
> 
> [dpdk-dev] [PATCH v6 06/13] virtio: read virtio_net_config correctly
> 
> applied, but same result.
> 
> If you want to recreate my setup, just follow instructions in
> 
> http://dpdk.org/ml/archives/dev/2015-October/024737.html
> 
> 
> vagrant at vagrant-ubuntu-vivid-64:~/dpdk$ git status
> On branch master
> Your branch is up-to-date with 'origin/master'.
> Changes not staged for commit:
>   (use "git add <file>..." to update what will be committed)
>   (use "git checkout -- <file>..." to discard changes in working directory)
> 
>     modified:   config/defconfig_x86_64-native-linuxapp-gcc
> 
> ..

Don't have clear clue there. But you could try Huawei's solution first.
It's likely that it will fix your problem.

If not, would you please try to reproduce it with qemu (you were using
virtualbox, right)?  And then dump the whoe command line here so that I
can try to reproduce and debug it on my side. Sorry that I don't use
virtualbox, as well as vagrant.

	--yliu

> 
> vagrant at vagrant-ubuntu-vivid-64:~/dpdk/x86_64-native-linuxapp-gcc/app$
> sudo ./testpmd -b 0000:00:03.0 -c 3 -n 1 -- -i
> EAL: Detected lcore 0 as core 0 on socket 0
> EAL: Detected lcore 1 as core 1 on socket 0
> EAL: Support maximum 128 logical core(s) by configuration.
> EAL: Detected 2 lcore(s)
> EAL: VFIO modules not all loaded, skip VFIO support...
> EAL: Setting up physically contiguous memory...
> EAL: Ask a virtual area of 0x400000 bytes
> EAL: Virtual area found at 0x7f2a3a800000 (size = 0x400000)
> EAL: Ask a virtual area of 0xe000000 bytes
> EAL: Virtual area found at 0x7f2a2c600000 (size = 0xe000000)
> EAL: Ask a virtual area of 0x30c00000 bytes
> EAL: Virtual area found at 0x7f29fb800000 (size = 0x30c00000)
> EAL: Ask a virtual area of 0x400000 bytes
> EAL: Virtual area found at 0x7f29fb200000 (size = 0x400000)
> EAL: Ask a virtual area of 0xa00000 bytes
> EAL: Virtual area found at 0x7f29fa600000 (size = 0xa00000)
> EAL: Ask a virtual area of 0x200000 bytes
> EAL: Virtual area found at 0x7f29fa200000 (size = 0x200000)
> EAL: Requesting 512 pages of size 2MB from socket 0
> EAL: TSC frequency is ~2198491 KHz
> EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using
> unreliable clock cycles !
> EAL: Master lcore 0 is ready (tid=3c9938c0;cpuset=[0])
> EAL: lcore 1 is ready (tid=fa1ff700;cpuset=[1])
> EAL: PCI device 0000:00:03.0 on NUMA socket -1
> EAL:   probe driver: 1af4:1000 rte_virtio_pmd
> EAL:   Device is blacklisted, not initializing
> EAL: PCI device 0000:00:08.0 on NUMA socket -1
> EAL:   probe driver: 1af4:1000 rte_virtio_pmd
> PMD: parse_sysfs_value(): parse_sysfs_value(): cannot open sysfs
> value /sys/bus/pci/devices/0000:00:08.0/uio/uio0/portio/port0/size
> PMD: virtio_resource_init_by_uio(): virtio_resource_init_by_uio():
> cannot parse size
> PMD: virtio_resource_init_by_ioports(): PCI Port IO found
> start=0xd040 with size=0x20
> PMD: virtio_negotiate_features(): guest_features before negotiate = cf8020
> PMD: virtio_negotiate_features(): host_features before negotiate = 410fdda3
> PMD: virtio_negotiate_features(): features after negotiate = f8020
> PMD: eth_virtio_dev_init(): PORT MAC: 08:00:27:CC:DE:CD
> PMD: eth_virtio_dev_init(): VIRTIO_NET_F_MQ is not supported
> PMD: virtio_dev_cq_queue_setup():  >>
> PMD: virtio_dev_queue_setup(): selecting queue: 2
> PMD: virtio_dev_queue_setup(): vq_size: 16 nb_desc:0
> PMD: virtio_dev_queue_setup(): vring_size: 4228, rounded_vring_size: 8192
> PMD: virtio_dev_queue_setup(): vq->vq_ring_mem:      0x67b54000
> PMD: virtio_dev_queue_setup(): vq->vq_ring_virt_mem: 0x7f29fb354000
> PMD: eth_virtio_dev_init(): config->max_virtqueue_pairs=1
> PMD: eth_virtio_dev_init(): config->status=1
> PMD: eth_virtio_dev_init(): PORT MAC: 08:00:27:CC:DE:CD
> PMD: eth_virtio_dev_init(): hw->max_rx_queues=1 hw->max_tx_queues=1
> PMD: eth_virtio_dev_init(): port 0 vendorID=0x1af4 deviceID=0x1000
> PMD: virtio_dev_vring_start():  >>
> EAL: PCI device 0000:00:09.0 on NUMA socket -1
> EAL:   probe driver: 1af4:1000 rte_virtio_pmd
> PMD: parse_sysfs_value(): parse_sysfs_value(): cannot open sysfs
> value /sys/bus/pci/devices/0000:00:09.0/uio/uio1/portio/port0/size
> PMD: virtio_resource_init_by_uio(): virtio_resource_init_by_uio():
> cannot parse size
> PMD: virtio_resource_init_by_ioports(): PCI Port IO found
> start=0xd060 with size=0x20
> PMD: virtio_negotiate_features(): guest_features before negotiate = cf8020
> PMD: virtio_negotiate_features(): host_features before negotiate = 410fdda3
> PMD: virtio_negotiate_features(): features after negotiate = f8020
> PMD: eth_virtio_dev_init(): PORT MAC: 08:00:27:07:D3:F5
> PMD: eth_virtio_dev_init(): VIRTIO_NET_F_MQ is not supported
> PMD: virtio_dev_cq_queue_setup():  >>
> PMD: virtio_dev_queue_setup(): selecting queue: 2
> PMD: virtio_dev_queue_setup(): vq_size: 16 nb_desc:0
> PMD: virtio_dev_queue_setup(): vring_size: 4228, rounded_vring_size: 8192
> PMD: virtio_dev_queue_setup(): vq->vq_ring_mem:      0x67b50000
> PMD: virtio_dev_queue_setup(): vq->vq_ring_virt_mem: 0x7f29fb350000
> PMD: eth_virtio_dev_init(): config->max_virtqueue_pairs=1
> PMD: eth_virtio_dev_init(): config->status=1
> PMD: eth_virtio_dev_init(): PORT MAC: 08:00:27:07:D3:F5
> PMD: eth_virtio_dev_init(): hw->max_rx_queues=1 hw->max_tx_queues=1
> PMD: eth_virtio_dev_init(): port 1 vendorID=0x1af4 deviceID=0x1000
> PMD: virtio_dev_vring_start():  >>
> Interactive-mode selected
> Configuring Port 0 (socket 0)
> PMD: virtio_dev_configure(): configure
> PMD: virtio_dev_tx_queue_setup():  >>
> PMD: virtio_dev_queue_setup(): selecting queue: 1
> PMD: virtio_dev_queue_setup(): vq_size: 256 nb_desc:512
> PMD: virtio_dev_queue_setup(): vring_size: 10244, rounded_vring_size: 12288
> PMD: virtio_dev_queue_setup(): vq->vq_ring_mem:      0x5fb000
> PMD: virtio_dev_queue_setup(): vq->vq_ring_virt_mem: 0x7f2a3a9fb000
> PMD: virtio_dev_rx_queue_setup():  >>
> PMD: virtio_dev_queue_setup(): selecting queue: 0
> PMD: virtio_dev_queue_setup(): vq_size: 256 nb_desc:128
> PMD: virtio_dev_queue_setup(): vring_size: 10244, rounded_vring_size: 12288
> PMD: virtio_dev_queue_setup(): vq->vq_ring_mem:      0x5f6000
> PMD: virtio_dev_queue_setup(): vq->vq_ring_virt_mem: 0x7f2a3a9f6000
> PMD: virtio_dev_link_update(): Get link status from hw
> PMD: virtio_dev_link_update(): Port 0 is up
> PMD: virtio_dev_rxtx_start():  >>
> PMD: virtio_dev_vring_start():  >>
> PMD: virtio_dev_vring_start(): Allocated 256 bufs
> PMD: virtio_dev_vring_start():  >>
> PMD: virtio_dev_start(): nb_queues=1
> PMD: virtio_dev_start(): Notified backend at initialization
> PMD: virtio_send_command(): vq->vq_desc_head_idx = 0, status = 255,
> vq->hw->cvq = 0x7f29fb3567c0 vq = 0x7f29fb3567c0
> PMD: virtio_send_command(): vq->vq_queue_index = 2
> 
> [ ... blocking]
> 
> 
> Results after patch was applied:
> 
> vagrant at vagrant-ubuntu-vivid-64:~/dpdk/x86_64-native-linuxapp-gcc/app$
> sudo ./testpmd -b 0000:00:03.0 -c 3 -n 1 -- -iEAL: Detected lcore 0
> as core 0 on socket 0
> EAL: Detected lcore 1 as core 1 on socket 0
> EAL: Support maximum 128 logical core(s) by configuration.
> EAL: Detected 2 lcore(s)
> EAL: VFIO modules not all loaded, skip VFIO support...
> EAL: Setting up physically contiguous memory...
> EAL: Ask a virtual area of 0x400000 bytes
> EAL: Virtual area found at 0x7f2c8e000000 (size = 0x400000)
> EAL: Ask a virtual area of 0xe000000 bytes
> EAL: Virtual area found at 0x7f2c7fe00000 (size = 0xe000000)
> EAL: Ask a virtual area of 0x30c00000 bytes
> EAL: Virtual area found at 0x7f2c4f000000 (size = 0x30c00000)
> EAL: Ask a virtual area of 0x400000 bytes
> EAL: Virtual area found at 0x7f2c4ea00000 (size = 0x400000)
> EAL: Ask a virtual area of 0xa00000 bytes
> EAL: Virtual area found at 0x7f2c4de00000 (size = 0xa00000)
> EAL: Ask a virtual area of 0x200000 bytes
> EAL: Virtual area found at 0x7f2c4da00000 (size = 0x200000)
> EAL: Requesting 512 pages of size 2MB from socket 0
> EAL: TSC frequency is ~2198491 KHz
> EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using
> unreliable clock cycles !
> EAL: Master lcore 0 is ready (tid=904928c0;cpuset=[0])
> EAL: lcore 1 is ready (tid=4d9ff700;cpuset=[1])
> EAL: PCI device 0000:00:03.0 on NUMA socket -1
> EAL:   probe driver: 1af4:1000 rte_virtio_pmd
> EAL:   Device is blacklisted, not initializing
> EAL: PCI device 0000:00:08.0 on NUMA socket -1
> EAL:   probe driver: 1af4:1000 rte_virtio_pmd
> PMD: parse_sysfs_value(): parse_sysfs_value(): cannot open sysfs
> value /sys/bus/pci/devices/0000:00:08.0/uio/uio0/portio/port0/size
> PMD: virtio_resource_init_by_uio(): virtio_resource_init_by_uio():
> cannot parse size
> PMD: virtio_resource_init_by_ioports(): PCI Port IO found
> start=0xd040 with size=0x20
> PMD: virtio_negotiate_features(): guest_features before negotiate = cf8020
> PMD: virtio_negotiate_features(): host_features before negotiate = 410fdda3
> PMD: virtio_negotiate_features(): features after negotiate = f8020
> PMD: eth_virtio_dev_init(): PORT MAC: 08:00:27:CC:DE:CD
> PMD: eth_virtio_dev_init(): VIRTIO_NET_F_MQ is not supported
> PMD: virtio_dev_cq_queue_setup():  >>
> PMD: virtio_dev_queue_setup(): selecting queue: 2
> PMD: virtio_dev_queue_setup(): vq_size: 16 nb_desc:0
> PMD: virtio_dev_queue_setup(): vring_size: 4228, rounded_vring_size: 8192
> PMD: virtio_dev_queue_setup(): vq->vq_ring_mem:      0x67b54000
> PMD: virtio_dev_queue_setup(): vq->vq_ring_virt_mem: 0x7f2c4eb54000
> PMD: eth_virtio_dev_init(): config->max_virtqueue_pairs=1
> PMD: eth_virtio_dev_init(): config->status=1
> PMD: eth_virtio_dev_init(): PORT MAC: 08:00:27:CC:DE:CD
> PMD: eth_virtio_dev_init(): hw->max_rx_queues=1 hw->max_tx_queues=1
> PMD: eth_virtio_dev_init(): port 0 vendorID=0x1af4 deviceID=0x1000
> PMD: virtio_dev_vring_start():  >>
> EAL: PCI device 0000:00:09.0 on NUMA socket -1
> EAL:   probe driver: 1af4:1000 rte_virtio_pmd
> PMD: parse_sysfs_value(): parse_sysfs_value(): cannot open sysfs
> value /sys/bus/pci/devices/0000:00:09.0/uio/uio1/portio/port0/size
> PMD: virtio_resource_init_by_uio(): virtio_resource_init_by_uio():
> cannot parse size
> PMD: virtio_resource_init_by_ioports(): PCI Port IO found
> start=0xd060 with size=0x20
> PMD: virtio_negotiate_features(): guest_features before negotiate = cf8020
> PMD: virtio_negotiate_features(): host_features before negotiate = 410fdda3
> PMD: virtio_negotiate_features(): features after negotiate = f8020
> PMD: eth_virtio_dev_init(): PORT MAC: 08:00:27:07:D3:F5
> PMD: eth_virtio_dev_init(): VIRTIO_NET_F_MQ is not supported
> PMD: virtio_dev_cq_queue_setup():  >>
> PMD: virtio_dev_queue_setup(): selecting queue: 2
> PMD: virtio_dev_queue_setup(): vq_size: 16 nb_desc:0
> PMD: virtio_dev_queue_setup(): vring_size: 4228, rounded_vring_size: 8192
> PMD: virtio_dev_queue_setup(): vq->vq_ring_mem:      0x67b50000
> PMD: virtio_dev_queue_setup(): vq->vq_ring_virt_mem: 0x7f2c4eb50000
> PMD: eth_virtio_dev_init(): config->max_virtqueue_pairs=1
> PMD: eth_virtio_dev_init(): config->status=1
> PMD: eth_virtio_dev_init(): PORT MAC: 08:00:27:07:D3:F5
> PMD: eth_virtio_dev_init(): hw->max_rx_queues=1 hw->max_tx_queues=1
> PMD: eth_virtio_dev_init(): port 1 vendorID=0x1af4 deviceID=0x1000
> PMD: virtio_dev_vring_start():  >>
> Interactive-mode selected
> Configuring Port 0 (socket 0)
> PMD: virtio_dev_configure(): configure
> PMD: virtio_dev_tx_queue_setup():  >>
> PMD: virtio_dev_queue_setup(): selecting queue: 1
> PMD: virtio_dev_queue_setup(): vq_size: 256 nb_desc:512
> PMD: virtio_dev_queue_setup(): vring_size: 10244, rounded_vring_size: 12288
> PMD: virtio_dev_queue_setup(): vq->vq_ring_mem:      0x5fb000
> PMD: virtio_dev_queue_setup(): vq->vq_ring_virt_mem: 0x7f2c8e1fb000
> PMD: virtio_dev_rx_queue_setup():  >>
> PMD: virtio_dev_queue_setup(): selecting queue: 0
> PMD: virtio_dev_queue_setup(): vq_size: 256 nb_desc:128
> PMD: virtio_dev_queue_setup(): vring_size: 10244, rounded_vring_size: 12288
> PMD: virtio_dev_queue_setup(): vq->vq_ring_mem:      0x5f6000
> PMD: virtio_dev_queue_setup(): vq->vq_ring_virt_mem: 0x7f2c8e1f6000
> PMD: virtio_dev_link_update(): Get link status from hw
> PMD: virtio_dev_link_update(): Port 0 is up
> PMD: virtio_dev_rxtx_start():  >>
> PMD: virtio_dev_vring_start():  >>
> PMD: virtio_dev_vring_start(): Allocated 256 bufs
> PMD: virtio_dev_vring_start():  >>
> PMD: virtio_dev_start(): nb_queues=1
> PMD: virtio_dev_start(): Notified backend at initialization
> PMD: virtio_send_command(): vq->vq_desc_head_idx = 0, status = 255,
> vq->hw->cvq = 0x7f2c4eb567c0 vq = 0x7f2c4eb567c0
> PMD: virtio_send_command(): vq->vq_queue_index = 2
> 
> [... blocking]