[dpdk-users] Issue with ipsec-secgw sample application on VM using Intel QAT device (pass-through mode)
Chinmaya Dwibedy
ckdwibedy at gmail.com
Wed Jun 15 14:01:50 CEST 2016
Hi All,
I have created two VM instances (Fedora release 20 (Heisenbug) using
openstack. The kernel version of the VM : 3.12.9-301.fc20.x86_64. The
compute node (real system has CentOS 7.2, kernel version:
3.10.0-327.18.2.el7.x86_64)
has two Intel QAT devices.
[root at localhost ~(keystone_admin)]# lspci -vd:0435
83:00.0 Co-processor: Intel Corporation DH895XCC Series QAT
Subsystem: Intel Corporation Device 35c5
Physical Slot: 0-1
Flags: bus master, fast devsel, latency 0, IRQ 35
Memory at 387fffd00000 (64-bit, prefetchable) [size=512K]
Memory at c8200000 (64-bit, non-prefetchable) [size=256K]
Memory at c8240000 (64-bit, non-prefetchable) [size=256K]
Capabilities: [b0] MSI: Enable- Count=1/1 Maskable+ 64bit+
Capabilities: [60] MSI-X: Enable+ Count=33 Masked-
Capabilities: [6c] Power Management version 3
Capabilities: [74] Express Endpoint, MSI 00
Capabilities: [100] Advanced Error Reporting
Capabilities: [138] Alternative Routing-ID Interpretation (ARI)
Capabilities: [140] Single Root I/O Virtualization (SR-IOV)
Kernel driver in use: vfio-pci
88:00.0 Co-processor: Intel Corporation DH895XCC Series QAT
Subsystem: Intel Corporation Device 35c5
Physical Slot: 0-2
Flags: bus master, fast devsel, latency 0, IRQ 39
Memory at 387fffe00000 (64-bit, prefetchable) [size=512K]
Memory at c8000000 (64-bit, non-prefetchable) [size=256K]
Memory at c8040000 (64-bit, non-prefetchable) [size=256K]
Capabilities: [b0] MSI: Enable- Count=1/1 Maskable+ 64bit+
Capabilities: [60] MSI-X: Enable+ Count=33 Masked-
Capabilities: [6c] Power Management version 3
Capabilities: [74] Express Endpoint, MSI 00
Capabilities: [100] Advanced Error Reporting
Capabilities: [138] Alternative Routing-ID Interpretation (ARI)
Capabilities: [140] Single Root I/O Virtualization (SR-IOV)
Kernel driver in use: vfio-pci
[root at localhost ~(keystone_admin)]#
I have configured both the QAT devices to be pass-through devices. So
that, each VM will have exclusive access to an Intel QAT card. On VMs, I
configured, build and installed latest Intel driver (qatmux.l.2.6.0-60)
(downloaded from
https://01.org/packet-processing/intel%C2%AE-quickassist-technology-drivers-and-patches).
Started
the driver and checked via #service qat_service status and found that, it
detects 1 acceleration device(s) in the system.
[root at vpn-c openssl-async]# service qat_service status
There is 1 acceleration device(s) in the system:
icp_dev0 - type=dh895xcc, inst_id=0, node_id=0, bdf=00:05:0, #accel=6,
#engines=12, state=up
[root at vpn-c openssl-async]#
Also the output lspci on VM as follows
[root at vpn-c openssl-async]# lspci -nn | grep 0435
00:05.0 Co-processor [0b40]: Intel Corporation Coleto Creek PCIe Endpoint
[8086:0435]
[root at vpn-c openssl-async]#
I have configured “intel_iommu=off” in boot arguments (VMs).
[root at vpn-c dpdk-2.2.0]# cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-3.12.9-301.fc20.x86_64
root=UUID=ab47cbc9-68ee-403c-96a5-184e68238e65 ro console=ttyS0,115200n8
intel_iommu=off
[root at vpn-c dpdk-2.2.0]#
I run the ipsec-secgw sample application (
http://dpdk.org/browse/dpdk/tree/examples/ipsec-secgw) on VM with -cdev QAT
(e.g., #./examples/ipsec-secgw/build/ipsec-secgw -l 0,1 -n 4 -- -p 0x3 -P
--config="(0,0,0), (1,0,1)" --cdev QAT --ep0) . I found that,
rte_cryptodev_count_devtype() returns zero (i.e., no QAT crypto device
found).
Can anyone please suggest what might be the issue and the way to resolve
this ? Also kindly let me know using QAT device (passed through mode) by
DPDK application is possible or not. Thank you in advance for your time and
help. please feel free to let me know if additional information is needed.
Here go the console logs.
EAL: Master lcore 0 is ready (tid=e110a940;cpuset=[0])
EAL: lcore 1 is ready (tid=2f0b7700;cpuset=[1])
EAL: PCI device 0000:00:03.0 on NUMA socket -1
EAL: probe driver: 1af4:1000 rte_virtio_pmd
PMD: parse_sysfs_value(): parse_sysfs_value(): cannot open sysfs value
/sys/bus/pci/devices/0000:00:03.0/uio/uio1/portio/port0/size
PMD: virtio_resource_init_by_uio(): virtio_resource_init_by_uio(): cannot
parse size
PMD: virtio_resource_init_by_ioports(): PCI Port IO found start=0xc060 with
size=0x20
PMD: virtio_negotiate_features(): guest_features before negotiate = cf8020
PMD: virtio_negotiate_features(): host_features before negotiate = 799fffe3
PMD: virtio_negotiate_features(): features after negotiate = 8f8020
PMD: eth_virtio_dev_init(): PORT MAC: FA:16:3E:BB:8A:DC
PMD: eth_virtio_dev_init(): VIRTIO_NET_F_MQ is not supported
PMD: virtio_dev_cq_queue_setup(): >>
PMD: virtio_dev_queue_setup(): selecting queue: 2
PMD: virtio_dev_queue_setup(): vq_size: 64 nb_desc:0
PMD: virtio_dev_queue_setup(): vring_size: 4612, rounded_vring_size: 8192
PMD: virtio_dev_queue_setup(): vq->vq_ring_mem: 0x219143000
PMD: virtio_dev_queue_setup(): vq->vq_ring_virt_mem: 0x7fcf30f43000
PMD: eth_virtio_dev_init(): config->max_virtqueue_pairs=1
PMD: eth_virtio_dev_init(): config->status=1
PMD: eth_virtio_dev_init(): PORT MAC: FA:16:3E:BB:8A:DC
PMD: eth_virtio_dev_init(): hw->max_rx_queues=1 hw->max_tx_queues=1
PMD: eth_virtio_dev_init(): port 0 vendorID=0x1af4 deviceID=0x1000
PMD: virtio_dev_vring_start(): >>
EAL: PCI device 0000:00:06.0 on NUMA socket -1
EAL: probe driver: 8086:10fb rte_ixgbe_pmd
EAL: PCI memory mapped at 0x7fd0dee00000
EAL: PCI memory mapped at 0x7fd0def00000
PMD: eth_ixgbe_dev_init(): MAC: 2, PHY: 15, SFP+: 5
PMD: eth_ixgbe_dev_init(): port 1 vendorID=0x8086 deviceID=0x10fb
Promiscuous mode selected
endpoint 0
Found 0 QAT devices
PANIC in cryptodevs_init():
Not enough QAT devices detected, need 2 (1 per core), found 0
5: [./examples/ipsec-secgw/build/ipsec-secgw() [0x42ebf5]]
4: [/lib64/libc.so.6(__libc_start_main+0xf5) [0x7fd0dfe49d65]]
3: [./examples/ipsec-secgw/build/ipsec-secgw(main+0xd75) [0x42d695]]
2: [./examples/ipsec-secgw/build/ipsec-secgw(__rte_panic+0xc9) [0x42733e]]
1: [./examples/ipsec-secgw/build/ipsec-secgw(rte_dump_stack+0x1a)
[0x49561a]]
./run.sh: line 5: 8202 Aborted (core dumped)
./examples/ipsec-secgw/build/ipsec-secgw -l 0,1 -n 4 -- -p 0x3 -P
--config="(0,0,0), (1,0,1)" --cdev QAT --ep0
Regards,
Chinmaya
More information about the users
mailing list