<div dir="ltr">I am running a VM inside a VMWARE server (vSphere). My goal it to set up DPDK with two HW ports, and set up a virtio_user to interact with the kernel stack. In another app I have it working but instead of virtio_user I am running KNI, it works in IOVA-PA mode. I am looking to replace the KNI. When I try to set up virtio_user port as in the doc: <a href="https://doc.dpdk.org/guides/howto/virtio_user_as_exception_path.html#virtio-user-as-exception-path">https://doc.dpdk.org/guides/howto/virtio_user_as_exception_path.html#virtio-user-as-exception-path</a> I get a error it can't run in PA mode. When I try to run as VA mode from a parameter, I get the following errors: EAL: lib.eal log level changed from info to debug EAL: Detected lcore 0 as core 0 on socket 0 EAL: Detected lcore 1 as core 0 on socket 0 EAL: Support maximum 128 logical core(s) by configuration. EAL: Detected 2 lcore(s) EAL: Detected 1 NUMA nodes EAL: Checking presence of .so 'librte_eal.so.21.3' EAL: Checking presence of .so 'librte_eal.so.21' EAL: Checking presence of .so 'librte_eal.so' EAL: Detected static linkage of DPDK EAL: Ask a virtual area of 0x7000 bytes EAL: Virtual area found at 0x100000000 (size = 0x7000) EAL: Multi-process socket /var/run/dpdk/rte/mp_socket EAL: DPAA Bus not present. Skipping. EAL: VFIO PCI modules not loaded EAL: Selected IOVA mode 'VA' EAL: Probing VFIO support... EAL: IOMMU type 1 (Type 1) is supported EAL: IOMMU type 7 (sPAPR) is not supported EAL: IOMMU type 8 (No-IOMMU) is supported EAL: VFIO support initialized EAL: Ask a virtual area of 0x5b000 bytes EAL: Virtual area found at 0x100007000 (size = 0x5b000) EAL: Setting up physically contiguous memory... EAL: Setting maximum number of open files to 1048576 EAL: Detected memory type: socket_id:0 hugepage_sz:1073741824 EAL: Creating 2 segment lists: n_segs:128 socket_id:0 hugepage_sz:1073741824 EAL: Ask a virtual area of 0x2000 bytes EAL: Virtual area found at 0x100062000 (size = 0x2000) EAL: Memseg list allocated at socket 0, page size 0x100000kB EAL: Ask a virtual area of 0x2000000000 bytes EAL: Virtual area found at 0x140000000 (size = 0x2000000000) EAL: VA reserved for memseg list at 0x140000000, size 2000000000 EAL: Ask a virtual area of 0x2000 bytes EAL: Virtual area found at 0x2140000000 (size = 0x2000) EAL: Memseg list allocated at socket 0, page size 0x100000kB EAL: Ask a virtual area of 0x2000000000 bytes EAL: Virtual area found at 0x2180000000 (size = 0x2000000000) EAL: VA reserved for memseg list at 0x2180000000, size 2000000000 EAL: TSC frequency is ~2350000 KHz EAL: Main lcore 0 is ready (tid=7f8ad790ec00;cpuset=[0]) EAL: lcore 1 is ready (tid=7f8ad6907400;cpuset=[1]) EAL: Trying to obtain current memory policy. EAL: Setting policy MPOL_PREFERRED for socket 0 EAL: Restoring previous memory policy: 0 EAL: request: mp_malloc_sync EAL: Heap on socket 0 was expanded by 1024MB EAL: PCI device 0000:0b:00.0 on NUMA socket -1 EAL: probe driver: 15ad:7b0 net_vmxnet3 EAL: Expecting 'PA' IOVA mode but current mode is 'VA', not initializing EAL: Requested device 0000:0b:00.0 cannot be used EAL: PCI device 0000:13:00.0 on NUMA socket -1 EAL: probe driver: 15ad:7b0 net_vmxnet3 EAL: Expecting 'PA' IOVA mode but current mode is 'VA', not initializing EAL: Requested device 0000:13:00.0 cannot be used EAL: Bus (pci) probe failed. EAL: lib.telemetry log level changed from disabled to warning EAL: Error - exiting with code: 1 Cause: Error: number of ports must be even EAL: request: mp_malloc_sync EAL: Heap on socket 0 was shrunk by 1024MB For some reason the HW ports won't setup. From what I understand net_vmxnet3 should work with VA mode. I enabled I/OMUU for the VM. The weird thing even when enabled, I still have the enable_unsafe_noiommu_mode flag on. And because it's on the this: dev_iova_mode = pci_device_iova_mode(dr, dev); return PA mode, and it fails. When I disable it by modifying /sys/module/vfio/parameters/enable_unsafe_noiommu_mode, I get another error. The error is that it doesn't find a suitable IOMMU type: Just putting the relevant message: EAL: Heap on socket 0 was expanded by 1024MB EAL: PCI device 0000:0b:00.0 on NUMA socket -1 EAL: probe driver: 15ad:7b0 net_vmxnet3 EAL: Set IOMMU type 1 (Type 1) failed, error 19 (No such device) EAL: Set IOMMU type 7 (sPAPR) failed, error 19 (No such device) EAL: Set IOMMU type 8 (No-IOMMU) failed, error 19 (No such device) EAL: 0000:0b:00.0 failed to select IOMMU type EAL: Requested device 0000:0b:00.0 cannot be used EAL: PCI device 0000:13:00.0 on NUMA socket -1 EAL: probe driver: 15ad:7b0 net_vmxnet3 EAL: Set IOMMU type 1 (Type 1) failed, error 19 (No such device) EAL: Set IOMMU type 7 (sPAPR) failed, error 19 (No such device) EAL: Set IOMMU type 8 (No-IOMMU) failed, error 19 (No such device) EAL: 0000:13:00.0 failed to select IOMMU type EAL: Requested device 0000:13:00.0 cannot be used I can see where it fails: const struct vfio_iommu_type * vfio_set_iommu_type(int vfio_container_fd) { unsigned idx; for (idx = 0; idx < RTE_DIM(iommu_types); idx++) { const struct vfio_iommu_type *t = &iommu_types[idx]; int ret = ioctl(vfio_container_fd, VFIO_SET_IOMMU, t->type_id); if (!ret) { RTE_LOG(INFO, EAL, "Using IOMMU type %d (%s)\n", t->type_id, t->name); return t; } /* not an error, there may be more supported IOMMU types */ RTE_LOG(DEBUG, EAL, "Set IOMMU type %d (%s) failed, error " "%i (%s)\n", t->type_id, t->name, errno, strerror(errno)); } /* if we didn't find a suitable IOMMU type, fail */ return NULL; } The ioctl returns -1. I tried many things. Enabling/disabling IOMMU. Modifying the enable_unsafe_noiommu_mode flag. Running in PA mode. Any help will be much appreciated. </div>