Intel QAT 8970 accel card on ARM Ampere Server

[Ruifeng Wang]

+Dharmik

Hi Dharmik,

Do you see a similar problem on your machine?

Thanks,
Ruifeng

From: Patrick Robb <probb at iol.unh.edu>
Sent: Thursday, September 21, 2023 2:28 AM
To: Juraj Linkeš <juraj.linkes at pantheon.tech>
Cc: Ruifeng Wang <Ruifeng.Wang at arm.com>; Honnappa Nagarahalli <Honnappa.Nagarahalli at arm.com>; ci at dpdk.org; nd <nd at arm.com>
Subject: Re: Intel QAT 8970 accel card on ARM Ampere Server

Hi Juraj,

Sorry for the late reply. So, yes I applied those diffs and set the QAT modules to =y in the .config file when building the custom kernel. It appears to have worked correctly. The qat module is now built into the new kernel running on the ampere server (called 5.15.82+). You can see it listed on modules.builtin and from modinfo.

probb at arm-ampere-dut:~$ modinfo qat_c62x
name:           qat_c62x
filename:       (builtin)
version:        0.6.0
description:    Intel(R) QuickAssist Technology
firmware:       qat_c62x_mmp.bin
firmware:       qat_c62x.bin
author:         Intel
license:        Dual BSD/GPL
file:           drivers/crypto/qat/qat_c62x/qat_c62x

And there is a /sys/bus/pci/drivers/c6xx/(pci address)/sriov_numvfs path now so that's good. However, when trying to create the VFs for the 3 PFs on the card, a segmentation fault was returned the first time, and on subsequent tries it hangs now. So like:

root at arm-ampere-dut:~# lspci -d:37c8
0000:03:00.0 Co-processor: Intel Corporation C62x Chipset QuickAssist Technology (rev 04)
0000:04:00.0 Co-processor: Intel Corporation C62x Chipset QuickAssist Technology (rev 04)
0000:05:00.0 Co-processor: Intel Corporation C62x Chipset QuickAssist Technology (rev 04)
root at arm-ampere-dut:~# echo 16 > /sys/bus/pci/drivers/c6xx/0000:03:00.0/sriov_numvfs

The sriov_numvfs file should be writable from root so I'm a bit perplexed. I am wondering whether it is relevant to statically build in the qat_c62x module with the kernel, vs having it be a loadable driver? What do you do?

On Mon, Sep 11, 2023 at 4:13 AM Juraj Linkeš <juraj.linkes at pantheon.tech<mailto:juraj.linkes at pantheon.tech>> wrote:
Hi Patrick,

This is good news. How does the server fare after the restart?

Juraj

On Fri, Sep 1, 2023 at 11:30 PM Patrick Robb <probb at iol.unh.edu<mailto:probb at iol.unh.edu>> wrote:
Thanks Juraj,

I did bring the system to 22.04 based on our conversation from yesterday. From there and from checking out to the new current kernel (5.15.0-82-generic) yes the diffs cleanly apply, removing the x86 dependency on the QAT kernel drivers, and then you can make the kernel, enabling the QAT driver. It looks like that all worked fine.

I didn't actually install and reboot with the custom kernel today because I don't want to do that with a production server right before the weekend, particularly with USA having a holiday on Monday. I will reboot with the custom kernel on Tuesday morning though, and then hopefully the compress/crypto testsuites on QAT will be unblocked. Thanks, the guidance is greatly appreciated.

Best,
Patrick



--

Patrick Robb

Technical Service Manager

UNH InterOperability Laboratory

21 Madbury Rd, Suite 100, Durham, NH 03824

www.iol.unh.edu<http://www.iol.unh.edu/>



[https://lh4.googleusercontent.com/7sTY8VswXadak_YT0J13osh5ockNVRX2BuYaRsKoTTpkpilBokA0WlocYHLB4q7XUgXNHka6-ns47S8R_am0sOt7MYQQ1ILQ3S-P5aezsrjp3-IsJMmMrErHWmTARNgZhpAx06n2]
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mails.dpdk.org/archives/ci/attachments/20230925/f44bd2f3/attachment.htm>