[dpdk-dev] [Bug 448] Post-copy-Live migration with 8 vhost queues succeeds with warning from QEMU

[bugzilla at dpdk.org]

https://bugs.dpdk.org/show_bug.cgi?id=448

            Bug ID: 448
           Summary: Post-copy-Live migration with 8 vhost queues succeeds
                    with warning from QEMU
           Product: DPDK
           Version: 20.02
          Hardware: x86
                OS: Linux
            Status: UNCONFIRMED
          Severity: minor
          Priority: Normal
         Component: vhost/virtio
          Assignee: dev at dpdk.org
          Reporter: sunil.pai.g at intel.com
  Target Milestone: ---

Overview:
Even though Post-copy Live migration succeeds, QEMU reports the following
warning with 8 vhost queues:
qemu-system-x86_64: vhost_user_set_mem_table_postcopy: Received unexpected msg
type.Expected 5 received 30
qemu-system-x86_64: vhost_set_mem_table failed: Resource temporarily
unavailable (11)
qemu-system-x86_64: unable to start vhost net: 11: falling back on userspace
virtio

Steps to Reproduce:
1.download and build dpdk for x86_64-native-linuxapp-gcc with
CONFIG_RTE_LIBRTE_VHOST_POSTCOPY=y on both source and destination host.

2.Set 8192 2MB huge pages on both destination and source host

3.Run DPDK's Testpmd application on source:
./testpmd -m 512 --file-prefix=src -l 0,2 -n 4 --vdev
'net_vhost0,iface=/tmp/vu-src,queues=8' -- --portmask=1 -i --rxq=8 --txq=8
--nb-cores=1 --eth-peer=0,52:54:00:11:22:12 --no-mlockall 

4.Run DPDK's Testpmd application on destination:
./testpmd -m 512 --file-prefix=dst -l 0,2 -n 4 --vdev
'net_vhost0,iface=/tmp/vu-dst,queues=8,postcopy-support=1' -- --portmask=1 -i
--rxq=8 --txq=8 --nb-cores=1 --eth-peer=0,52:54:00:11:22:12 --no-mlockall

5.Run QEMU on src host:
export VM_NAME=vhost-vm
export GUEST_MEM=4096M
export VM_PATH=~/Downloads/grpc-2.qcow2
export qemu_queues=8
export qemu_vectors=18
export QEMU=~/qemu-latest/qemu/x86_64-softmmu/qemu-system-x86_64

taskset -c 5-8 $QEMU -name $vhost-vm -cpu host -enable-kvm \
   -m $GUEST_MEM -drive file=$VM_PATH --nographic \
   -numa node,memdev=mem -mem-prealloc -smp sockets=1,cores=4 \
   -object
memory-backend-file,id=mem,size=$GUEST_MEM,mem-path=/dev/hugepages,share=on \
   -chardev socket,id=char0,path=/tmp/vu-src \
   -netdev
type=vhost-user,id=mynet1,chardev=char0,vhostforce,queues=$qemu_queues \
   -device
virtio-net-pci,mac=00:00:00:00:00:01,netdev=mynet1,mq=on,mrg_rxbuf=on,rx_queue_size=1024,tx_queue_size=1024,vectors=$qemu_vectors
\
   -monitor telnet::1234,server,nowait \
   -net user,hostfwd=tcp::10023-:22 -net nic &

6.Run QEMU on dst host:
export VM_NAME=vhost-vm
export GUEST_MEM=4096M
export VM_PATH=~/Downloads/grpc-2.qcow2
export qemu_queues=8
export qemu_vectors=18
export QEMU=~/qemu-latest/qemu/x86_64-softmmu/qemu-system-x86_64

taskset -c 5-8 $QEMU -name $VM_NAME -cpu host -enable-kvm \
   -m $GUEST_MEM -drive file=$VM_PATH --nographic \
   -numa node,memdev=mem -mem-prealloc -smp sockets=1,cores=4 \
   -object
memory-backend-file,id=mem,size=$GUEST_MEM,mem-path=/dev/hugepages,share=on \
   -chardev socket,id=char0,path=/tmp/vu-src \
   -netdev
type=vhost-user,id=mynet1,chardev=char0,vhostforce,queues=$qemu_queues \
   -device
virtio-net-pci,mac=00:00:00:00:00:01,netdev=mynet1,mq=on,mrg_rxbuf=on,rx_queue_size=1024,tx_queue_size=1024,vectors=$qemu_vectors
\
   -monitor telnet::1234,server,nowait \
   -incoming tcp:0:4444 \
   -net user,hostfwd=tcp::10023-:22 -net nic &

7.In both testpmd prompts, start flooding the virtio-net device:
testpmd> set fwd txonly
testpmd> start

8.In destination's Qemu monitor, enable postcopy:
(qemu) migrate_set_capability postcopy-ram on

9.In source's Qemu monitor, enable postcopy and launch migration:
(qemu) migrate_set_capability postcopy-ram on
(qemu) migrate -d tcp:<dst host ip>:4444
(qemu) migrate_start_postcopy


Actual Results:
Even though migration succeeds, on dst host following message is seen on
console:
qemu-system-x86_64: vhost_user_set_mem_table_postcopy: Received unexpected msg
type.Expected 5 received 30
qemu-system-x86_64: vhost_set_mem_table failed: Resource temporarily
unavailable (11)
qemu-system-x86_64: unable to start vhost net: 11: falling back on userspace
virtio

Expected Results:
No Warning or error messages from QEMU

Version information:
QEMU-version: master branch [v5.0.0-rc1]
DPDK-version: master branch [v20.02]

Environment Information:
  Operating System: Ubuntu 18.04.4 LTS
            Kernel: Linux 4.15.0-74-generic
      Architecture: x86-64

Additional information:
This error is not seen when number of vhost queues is set to 1

-- 
You are receiving this mail because:
You are the assignee for the bug.