[dpdk-users] Issue with ipsec-secgw sample application on VM using Intel QAT device (pass-through mode)

Chinmaya Dwibedy ckdwibedy at gmail.com
Wed Jun 15 14:01:50 CEST 2016


Hi All,



I have created two VM instances  (Fedora release 20 (Heisenbug) using
openstack. The kernel version of the VM : 3.12.9-301.fc20.x86_64. The
compute node (real system has CentOS 7.2, kernel version:
3.10.0-327.18.2.el7.x86_64)
has two Intel QAT devices.



[root at localhost ~(keystone_admin)]# lspci -vd:0435

83:00.0 Co-processor: Intel Corporation DH895XCC Series QAT

        Subsystem: Intel Corporation Device 35c5

        Physical Slot: 0-1

        Flags: bus master, fast devsel, latency 0, IRQ 35

        Memory at 387fffd00000 (64-bit, prefetchable) [size=512K]

        Memory at c8200000 (64-bit, non-prefetchable) [size=256K]

        Memory at c8240000 (64-bit, non-prefetchable) [size=256K]

        Capabilities: [b0] MSI: Enable- Count=1/1 Maskable+ 64bit+

        Capabilities: [60] MSI-X: Enable+ Count=33 Masked-

        Capabilities: [6c] Power Management version 3

        Capabilities: [74] Express Endpoint, MSI 00

        Capabilities: [100] Advanced Error Reporting

        Capabilities: [138] Alternative Routing-ID Interpretation (ARI)

        Capabilities: [140] Single Root I/O Virtualization (SR-IOV)

        Kernel driver in use: vfio-pci



88:00.0 Co-processor: Intel Corporation DH895XCC Series QAT

        Subsystem: Intel Corporation Device 35c5

        Physical Slot: 0-2

        Flags: bus master, fast devsel, latency 0, IRQ 39

        Memory at 387fffe00000 (64-bit, prefetchable) [size=512K]

        Memory at c8000000 (64-bit, non-prefetchable) [size=256K]

        Memory at c8040000 (64-bit, non-prefetchable) [size=256K]

        Capabilities: [b0] MSI: Enable- Count=1/1 Maskable+ 64bit+

        Capabilities: [60] MSI-X: Enable+ Count=33 Masked-

        Capabilities: [6c] Power Management version 3

        Capabilities: [74] Express Endpoint, MSI 00

        Capabilities: [100] Advanced Error Reporting

        Capabilities: [138] Alternative Routing-ID Interpretation (ARI)

        Capabilities: [140] Single Root I/O Virtualization (SR-IOV)

        Kernel driver in use: vfio-pci



[root at localhost ~(keystone_admin)]#



I have configured both the QAT devices to be pass-through devices.  So
that, each VM will have exclusive access to an Intel QAT card. On VMs, I
configured, build  and   installed latest Intel driver (qatmux.l.2.6.0-60)
(downloaded from
https://01.org/packet-processing/intel%C2%AE-quickassist-technology-drivers-and-patches).
Started
the driver and checked via #service qat_service status and found that, it
detects 1 acceleration device(s) in the system.


[root at vpn-c openssl-async]# service qat_service status

There is 1 acceleration device(s) in the system:

 icp_dev0 - type=dh895xcc, inst_id=0, node_id=0,  bdf=00:05:0, #accel=6,
#engines=12, state=up

[root at vpn-c openssl-async]#



Also the output lspci on VM as follows

[root at vpn-c openssl-async]# lspci -nn | grep 0435

00:05.0 Co-processor [0b40]: Intel Corporation Coleto Creek PCIe Endpoint
[8086:0435]

[root at vpn-c openssl-async]#



I have configured “intel_iommu=off” in boot arguments (VMs).


[root at vpn-c dpdk-2.2.0]# cat /proc/cmdline

BOOT_IMAGE=/boot/vmlinuz-3.12.9-301.fc20.x86_64
root=UUID=ab47cbc9-68ee-403c-96a5-184e68238e65 ro console=ttyS0,115200n8
intel_iommu=off

[root at vpn-c dpdk-2.2.0]#


I run the  ipsec-secgw sample application (
http://dpdk.org/browse/dpdk/tree/examples/ipsec-secgw) on VM with -cdev QAT
(e.g., #./examples/ipsec-secgw/build/ipsec-secgw  -l 0,1 -n 4  -- -p 0x3 -P
--config="(0,0,0), (1,0,1)" --cdev QAT --ep0) . I found that,
rte_cryptodev_count_devtype() returns zero (i.e., no QAT crypto device
found).


Can anyone please suggest what might be the issue and the way to resolve
this ?  Also kindly let me know  using QAT device (passed through mode) by
DPDK application is possible or not. Thank you in advance for your time and
help. please feel free to let me know if additional information is needed.


Here go the console logs.



EAL: Master lcore 0 is ready (tid=e110a940;cpuset=[0])

EAL: lcore 1 is ready (tid=2f0b7700;cpuset=[1])

EAL: PCI device 0000:00:03.0 on NUMA socket -1

EAL:   probe driver: 1af4:1000 rte_virtio_pmd

PMD: parse_sysfs_value(): parse_sysfs_value(): cannot open sysfs value
/sys/bus/pci/devices/0000:00:03.0/uio/uio1/portio/port0/size

PMD: virtio_resource_init_by_uio(): virtio_resource_init_by_uio(): cannot
parse size

PMD: virtio_resource_init_by_ioports(): PCI Port IO found start=0xc060 with
size=0x20

PMD: virtio_negotiate_features(): guest_features before negotiate = cf8020

PMD: virtio_negotiate_features(): host_features before negotiate = 799fffe3

PMD: virtio_negotiate_features(): features after negotiate = 8f8020

PMD: eth_virtio_dev_init(): PORT MAC: FA:16:3E:BB:8A:DC

PMD: eth_virtio_dev_init(): VIRTIO_NET_F_MQ is not supported

PMD: virtio_dev_cq_queue_setup():  >>

PMD: virtio_dev_queue_setup(): selecting queue: 2

PMD: virtio_dev_queue_setup(): vq_size: 64 nb_desc:0

PMD: virtio_dev_queue_setup(): vring_size: 4612, rounded_vring_size: 8192

PMD: virtio_dev_queue_setup(): vq->vq_ring_mem:      0x219143000

PMD: virtio_dev_queue_setup(): vq->vq_ring_virt_mem: 0x7fcf30f43000

PMD: eth_virtio_dev_init(): config->max_virtqueue_pairs=1

PMD: eth_virtio_dev_init(): config->status=1

PMD: eth_virtio_dev_init(): PORT MAC: FA:16:3E:BB:8A:DC

PMD: eth_virtio_dev_init(): hw->max_rx_queues=1   hw->max_tx_queues=1

PMD: eth_virtio_dev_init(): port 0 vendorID=0x1af4 deviceID=0x1000

PMD: virtio_dev_vring_start():  >>

EAL: PCI device 0000:00:06.0 on NUMA socket -1

EAL:   probe driver: 8086:10fb rte_ixgbe_pmd

EAL:   PCI memory mapped at 0x7fd0dee00000

EAL:   PCI memory mapped at 0x7fd0def00000

PMD: eth_ixgbe_dev_init(): MAC: 2, PHY: 15, SFP+: 5

PMD: eth_ixgbe_dev_init(): port 1 vendorID=0x8086 deviceID=0x10fb

Promiscuous mode selected

endpoint 0

Found 0 QAT devices

PANIC in cryptodevs_init():

Not enough QAT devices detected, need 2 (1 per core), found 0

5: [./examples/ipsec-secgw/build/ipsec-secgw() [0x42ebf5]]

4: [/lib64/libc.so.6(__libc_start_main+0xf5) [0x7fd0dfe49d65]]

3: [./examples/ipsec-secgw/build/ipsec-secgw(main+0xd75) [0x42d695]]

2: [./examples/ipsec-secgw/build/ipsec-secgw(__rte_panic+0xc9) [0x42733e]]

1: [./examples/ipsec-secgw/build/ipsec-secgw(rte_dump_stack+0x1a)
[0x49561a]]

./run.sh: line 5:  8202 Aborted                 (core dumped)
./examples/ipsec-secgw/build/ipsec-secgw -l 0,1 -n 4 -- -p 0x3 -P
--config="(0,0,0), (1,0,1)" --cdev QAT --ep0



Regards,

Chinmaya


More information about the users mailing list