<div dir="ltr">Hi, I am using a Vmware host (ESXI 7.0) where I have IOMMU enabled. I am using virtio_user ports as an exception path so I am trying to use IOVA as VA. Just as an example, I am trying to run the l2fw example application with --iova-mode=va. Although I shows there is support for IOMMU, it fails to negotiate the IOMMU type, here is the output of: ./dpdk-l2fwd --iova-mode=va --log-level=eal,8 EAL: lib.eal log level changed from info to debug EAL: Detected lcore 0 as core 0 on socket 0 EAL: Detected lcore 1 as core 0 on socket 0 EAL: Support maximum 128 logical core(s) by configuration. EAL: Detected 2 lcore(s) EAL: Detected 1 NUMA nodes EAL: Checking presence of .so 'librte_eal.so.21.3' EAL: Checking presence of .so 'librte_eal.so.21' EAL: Checking presence of .so 'librte_eal.so' EAL: Detected static linkage of DPDK EAL: Ask a virtual area of 0x7000 bytes EAL: Virtual area found at 0x100000000 (size = 0x7000) EAL: Multi-process socket /var/run/dpdk/rte/mp_socket EAL: DPAA Bus not present. Skipping. EAL: VFIO PCI modules not loaded EAL: Selected IOVA mode 'VA' EAL: Probing VFIO support... EAL: IOMMU type 1 (Type 1) is supported EAL: IOMMU type 7 (sPAPR) is not supported EAL: IOMMU type 8 (No-IOMMU) is not supported EAL: VFIO support initialized EAL: Ask a virtual area of 0x5b000 bytes EAL: Virtual area found at 0x100007000 (size = 0x5b000) EAL: Setting up physically contiguous memory... EAL: Setting maximum number of open files to 1048576 EAL: Detected memory type: socket_id:0 hugepage_sz:1073741824 EAL: Creating 2 segment lists: n_segs:128 socket_id:0 hugepage_sz:1073741824 EAL: Ask a virtual area of 0x2000 bytes EAL: Virtual area found at 0x100062000 (size = 0x2000) EAL: Memseg list allocated at socket 0, page size 0x100000kB EAL: Ask a virtual area of 0x2000000000 bytes EAL: Virtual area found at 0x140000000 (size = 0x2000000000) EAL: VA reserved for memseg list at 0x140000000, size 2000000000 EAL: Ask a virtual area of 0x2000 bytes EAL: Virtual area found at 0x2140000000 (size = 0x2000) EAL: Memseg list allocated at socket 0, page size 0x100000kB EAL: Ask a virtual area of 0x2000000000 bytes EAL: Virtual area found at 0x2180000000 (size = 0x2000000000) EAL: VA reserved for memseg list at 0x2180000000, size 2000000000 EAL: TSC frequency is ~2350000 KHz EAL: Main lcore 0 is ready (tid=7f3f3a1e5c00;cpuset=[0]) EAL: lcore 1 is ready (tid=7f3f391de400;cpuset=[1]) EAL: Trying to obtain current memory policy. EAL: Setting policy MPOL_PREFERRED for socket 0 EAL: Restoring previous memory policy: 0 EAL: request: mp_malloc_sync EAL: Heap on socket 0 was expanded by 1024MB EAL: PCI device 0000:03:00.0 on NUMA socket -1 EAL: probe driver: 15ad:7b0 net_vmxnet3 EAL: Not managed by a supported kernel driver, skipped EAL: PCI device 0000:0b:00.0 on NUMA socket -1 EAL: probe driver: 15ad:7b0 net_vmxnet3 EAL: Set IOMMU type 1 (Type 1) failed, error 19 (No such device) EAL: Set IOMMU type 7 (sPAPR) failed, error 19 (No such device) EAL: Set IOMMU type 8 (No-IOMMU) failed, error 19 (No such device) EAL: 0000:0b:00.0 failed to select IOMMU type EAL: Requested device 0000:0b:00.0 cannot be used EAL: PCI device 0000:13:00.0 on NUMA socket -1 EAL: probe driver: 15ad:7b0 net_vmxnet3 EAL: Set IOMMU type 1 (Type 1) failed, error 19 (No such device) EAL: Set IOMMU type 7 (sPAPR) failed, error 19 (No such device) EAL: Set IOMMU type 8 (No-IOMMU) failed, error 19 (No such device) EAL: 0000:13:00.0 failed to select IOMMU type EAL: Requested device 0000:13:00.0 cannot be used EAL: lib.telemetry log level changed from disabled to warning MAC updating enabled EAL: Error - exiting with code: 1 Cause: No Ethernet ports - bye EAL: request: mp_malloc_sync EAL: Heap on socket 0 was shrunk by 1024MB The negotiation for IOMMU type happens in rte_vfio_setup_device(), it calls the function: set_iommu_type() in the following snippet: /* * pick an IOMMU type and set up DMA mappings for container * * needs to be done only once, only when first group is * assigned to a container and only in primary process. * Note this can happen several times with the hotplug * functionality. */ if (internal_conf->process_type == RTE_PROC_PRIMARY && vfio_cfg->vfio_active_groups == 1 && vfio_group_device_count(vfio_group_fd) == 0) { const struct vfio_iommu_type *t; /* select an IOMMU type which we will be using */ t = vfio_set_iommu_type(vfio_container_fd); if (!t) { RTE_LOG(ERR, EAL, "%s failed to select IOMMU type\n", dev_addr); close(vfio_group_fd); rte_vfio_clear_group(vfio_group_fd); return -1; } <div> </div><div>In there there is a ioctl call: unsigned idx; for (idx = 0; idx < RTE_DIM(iommu_types); idx++) { const struct vfio_iommu_type *t = &iommu_types[idx]; int ret = ioctl(vfio_container_fd, VFIO_SET_IOMMU, t->type_id); which fails.</div><div> </div><div>Its also weird for me the when using ./usertool/dpdk-devbind.py --status I gett the following output: Network devices using DPDK-compatible driver ============================================ 0000:0b:00.0 'VMXNET3 Ethernet Controller 07b0' drv=vfio-pci unused=vmxnet3 0000:13:00.0 'VMXNET3 Ethernet Controller 07b0' drv=vfio-pci unused=vmxnet3 Network devices using kernel driver =================================== 0000:03:00.0 'VMXNET3 Ethernet Controller 07b0' if=eth0 drv=vmxnet3 unused=vfio-pci *Active* <div> Meaning the driver that is connected is vfio-pci. So why is it probing VMXNET3? Does anyone have any experience in running IOVA as VA in VMWARE host? </div></div></div>