<html><head><meta http-equiv="Content-Type" content="text/html; charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">Hi Nicolas,<div class=""><br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Oct 26, 2022, at 7:11 PM, Chautru, Nicolas <<a href="mailto:nicolas.chautru@intel.com" class="">nicolas.chautru@intel.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><meta charset="UTF-8" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">Hi John,</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">General comment. I was a bit lost in the split in the commits 10 to 14.<span class="Apple-converted-space"> </span></span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">First I would have expected it to build from the first commit. I don't believe it makes sense to add 13 and 14 after the fact.<span class="Apple-converted-space"> </span></span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""></div></blockquote><div><br class=""></div><div>I first introduced this patch set in 22.07 but we had to defer due other company priorities. I had 10 to 14 in the same commit but you asked me to split it into smaller commits. Perhaps I misunderstood what you were asking. I will put 10 thru 14 back into the same commit.</div><div><br class=""></div><div><br class=""></div><br class=""><blockquote type="cite" class=""><div class=""><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">Between 10 and 11 there were a bit of confusion as well to me.</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">Like ark_bbdev_info_get is being first referred in 10 but then the implementation is in 11.</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">The ldpc decoding functions are also split between 10 and 11. As a consequence the commit 10 is a hard to review in one chunk arguably.<span class="Apple-converted-space"> </span></span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""></div></blockquote><div><br class=""></div><div>This will be addressed when I put them all in the same commit.</div><br class=""><blockquote type="cite" class=""><div class=""><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">I would suggest to consider a split of commits that may be more logical and incremental. For instance what was done recently for the acc driver just as an imperfect example.<span class="Apple-converted-space"> </span></span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">This way it would also provide more digestible chunks of code to be reviewed incrementally.<span class="Apple-converted-space"> </span></span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">I would be nice to have some doc with the first commit matching the code. Notably as I had the impression the implementation doesn't fully match your cover letter (I will add some more comments on this for 11), but unclear to me whether this is intentional or not.</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""></div></blockquote><div><br class=""></div><div><br class=""></div><div>We will address the doc to make sure it is accurate.</div><div><br class=""></div><div><br class=""></div><div>We will address your other comments in a separate response.</div><div><br class=""></div><div>Thank you</div><div>-John</div><br class=""><blockquote type="cite" class=""><div class=""><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">Thanks</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">Nic</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><blockquote type="cite" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none;" class="">-----Original Message-----<br class="">From: John Miller <<a href="mailto:john.miller@atomicrules.com" class="">john.miller@atomicrules.com</a>><br class="">Sent: Wednesday, October 26, 2022 12:46 PM<br class="">To: Chautru, Nicolas <<a href="mailto:nicolas.chautru@intel.com" class="">nicolas.chautru@intel.com</a>><br class="">Cc:<span class="Apple-converted-space"> </span><a href="mailto:dev@dpdk.org" class="">dev@dpdk.org</a>;<span class="Apple-converted-space"> </span><a href="mailto:ed.czeck@atomicrules.com" class="">ed.czeck@atomicrules.com</a>; Shepard Siegel<br class=""><<a href="mailto:shepard.siegel@atomicrules.com" class="">shepard.siegel@atomicrules.com</a>>; John Miller<br class=""><<a href="mailto:john.miller@atomicrules.com" class="">john.miller@atomicrules.com</a>><br class="">Subject: [PATCH 10/14] baseband/ark: introduce ark baseband driver<br class=""><br class="">This patch introduces the Arkville baseband device driver.<br class=""><br class="">Signed-off-by: John Miller <<a href="mailto:john.miller@atomicrules.com" class="">john.miller@atomicrules.com</a>><br class="">---<br class="">drivers/baseband/ark/ark_bbdev.c | 1127<br class="">++++++++++++++++++++++++++++++ drivers/baseband/ark/ark_bbext.h | 163<br class="">+++++<br class="">2 files changed, 1290 insertions(+)<br class="">create mode 100644 drivers/baseband/ark/ark_bbdev.c create mode 100644<br class="">drivers/baseband/ark/ark_bbext.h<br class=""><br class="">diff --git a/drivers/baseband/ark/ark_bbdev.c<br class="">b/drivers/baseband/ark/ark_bbdev.c<br class="">new file mode 100644<br class="">index 0000000000..8736d170d1<br class="">--- /dev/null<br class="">+++ b/drivers/baseband/ark/ark_bbdev.c<br class="">@@ -0,0 +1,1127 @@<br class="">+/* SPDX-License-Identifier: BSD-3-Clause<br class="">+ * Copyright(c) 2016-2021 Atomic Rules LLC */<br class="">+<br class="">+#include <sys/stat.h><br class="">+#include <dlfcn.h><br class="">+#include <unistd.h><br class="">+<br class="">+#include <rte_common.h><br class="">+#include <rte_pci.h><br class="">+#include <bus_pci_driver.h><br class="">+#include <rte_bbdev.h><br class="">+#include <rte_bbdev_pmd.h><br class="">+#include <rte_bus_pci.h><br class="">+#include <rte_devargs.h><br class="">+#include <rte_malloc.h><br class="">+#include <rte_ring.h><br class="">+<br class="">+#include "ark_common.h"<br class="">+#include "ark_bbdev_common.h"<br class="">+#include "ark_bbdev_custom.h"<br class="">+#include "ark_ddm.h"<br class="">+#include "ark_mpu.h"<br class="">+#include "ark_rqp.h"<br class="">+#include "ark_udm.h"<br class="">+#include "ark_bbext.h"<br class="">+<br class="">+#define DRIVER_NAME baseband_ark<br class="">+<br class="">+#define ARK_SYSCTRL_BASE 0x0<br class="">+#define ARK_PKTGEN_BASE 0x10000<br class="">+#define ARK_MPU_RX_BASE 0x20000<br class="">+#define ARK_UDM_BASE 0x30000<br class="">+#define ARK_MPU_TX_BASE 0x40000<br class="">+#define ARK_DDM_BASE 0x60000<br class="">+#define ARK_PKTDIR_BASE 0xa0000<br class="">+#define ARK_PKTCHKR_BASE 0x90000<br class="">+#define ARK_RCPACING_BASE 0xb0000<br class="">+#define ARK_MPU_QOFFSET 0x00100<br class="">+<br class="">+#define BB_ARK_TX_Q_FACTOR 4<br class="">+<br class="">+#define ARK_RX_META_SIZE 32<br class="">+#define ARK_RX_META_OFFSET (RTE_PKTMBUF_HEADROOM -<br class="">ARK_RX_META_SIZE)<br class="">+#define ARK_RX_MAX_NOCHAIN (RTE_MBUF_DEFAULT_DATAROOM)<br class="">+<br class="">+static_assert(sizeof(struct ark_rx_meta) == ARK_RX_META_SIZE,<br class="">+"Unexpected struct size ark_rx_meta"); static_assert(sizeof(union<br class="">+ark_tx_meta) == 8, "Unexpected struct size ark_tx_meta");<br class="">+<br class="">+static struct rte_pci_id pci_id_ark[] = {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>{RTE_PCI_DEVICE(AR_VENDOR_ID, 0x1015)},<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>{RTE_PCI_DEVICE(AR_VENDOR_ID, 0x1016)},<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>{.device_id = 0},<br class="">+};<br class="">+<br class="">+static const struct ark_dev_caps<br class="">+ark_device_caps[] = {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> SET_DEV_CAPS(0x1015, true, false),<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> SET_DEV_CAPS(0x1016, true, false),<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> {.device_id = 0,}<br class="">+};<br class="">+<br class="">+<br class="">+/* Forward declarations */<br class="">+static const struct rte_bbdev_ops ark_bbdev_pmd_ops;<br class="">+<br class="">+static int<br class="">+check_for_ext(struct ark_bbdevice *ark) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* Get the env */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>const char *dllpath = getenv("ARK_BBEXT_PATH");<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>if (dllpath == NULL) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>ARK_BBDEV_LOG(DEBUG, "EXT NO dll path specified\n");<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>return 0;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ARK_BBDEV_LOG(NOTICE, "EXT found dll path at %s\n", dllpath);<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* Open and load the .so */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark->d_handle = dlopen(dllpath, RTLD_LOCAL | RTLD_LAZY);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>if (ark->d_handle == NULL) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>ARK_BBDEV_LOG(ERR, "Could not load user extension %s\n",<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> dllpath);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>return -1;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ARK_BBDEV_LOG(DEBUG, "SUCCESS: loaded user extension %s\n",<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> dllpath);<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* Get the entry points */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark->user_ext.dev_init =<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>(void *(*)(struct rte_bbdev *, void *))<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>dlsym(ark->d_handle, "rte_pmd_ark_bbdev_init");<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark->user_ext.dev_uninit =<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>(int (*)(struct rte_bbdev *, void *))<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>dlsym(ark->d_handle, "rte_pmd_ark_dev_uninit");<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark->user_ext.dev_start =<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>(int (*)(struct rte_bbdev *, void *))<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>dlsym(ark->d_handle, "rte_pmd_ark_bbdev_start");<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark->user_ext.dev_stop =<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>(int (*)(struct rte_bbdev *, void *))<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>dlsym(ark->d_handle, "rte_pmd_ark_bbdev_stop");<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark->user_ext.dequeue_ldpc_dec =<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>(int (*)(struct rte_bbdev *,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>struct rte_bbdev_dec_op *,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>uint32_t *,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>void *))<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>dlsym(ark->d_handle,<br class="">"rte_pmd_ark_bbdev_dequeue_ldpc_dec");<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark->user_ext.enqueue_ldpc_dec =<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>(int (*)(struct rte_bbdev *,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>struct rte_bbdev_dec_op *,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>uint32_t *,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>uint8_t *,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>void *))<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>dlsym(ark->d_handle,<br class="">"rte_pmd_ark_bbdev_enqueue_ldpc_dec");<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark->user_ext.dequeue_ldpc_enc =<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>(int (*)(struct rte_bbdev *,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>struct rte_bbdev_enc_op *,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>uint32_t *,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>void *))<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>dlsym(ark->d_handle,<br class="">"rte_pmd_ark_bbdev_dequeue_ldpc_enc");<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark->user_ext.enqueue_ldpc_enc =<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>(int (*)(struct rte_bbdev *,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>struct rte_bbdev_enc_op *,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>uint32_t *,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>uint8_t *,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>void *))<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>dlsym(ark->d_handle,<br class="">"rte_pmd_ark_bbdev_enqueue_ldpc_enc");<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>return 0;<br class="">+}<br class="">+<br class="">+<br class="">+/* queue */<br class="">+struct ark_bbdev_queue {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>struct ark_bbdevice *ark_bbdev;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>struct rte_ring *active_ops; /* Ring for processed packets */<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* RX components */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* array of physical addresses of the mbuf data pointer */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>rte_iova_t *rx_paddress_q;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>struct ark_udm_t *udm;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>struct ark_mpu_t *rx_mpu;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* TX components */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>union ark_tx_meta *tx_meta_q;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>struct ark_mpu_t *tx_mpu;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>struct ark_ddm_t *ddm;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>uint32_t tx_queue_mask;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>uint32_t rx_queue_mask;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>int32_t rx_seed_index;<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>/* step 1 set with empty mbuf */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>int32_t rx_cons_index;<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>/* step 3 consumed by driver */<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* 3 indexes to the paired data rings. */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>int32_t tx_prod_index;<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>/* where to put the next one */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>int32_t tx_free_index;<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>/* local copy of tx_cons_index */<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* separate cache line -- written by FPGA -- RX announce */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>RTE_MARKER cacheline1 __rte_cache_min_aligned;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>volatile int32_t rx_prod_index; /* step 2 filled by FPGA */<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* Separate cache line -- written by FPGA -- RX completion */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>RTE_MARKER cacheline2 __rte_cache_min_aligned;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>volatile int32_t tx_cons_index; /* hw is done, can be freed */ }<br class="">+__rte_cache_aligned;<br class="">+<br class="">+<br class="">+static int<br class="">+ark_bb_hw_q_setup(struct rte_bbdev *bbdev, uint16_t q_id, uint16_t<br class="">+queue_size) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>struct ark_bbdev_queue *q = bbdev->data-<br class=""><blockquote type="cite" class="">queues[q_id].queue_private;<br class=""></blockquote>+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>rte_iova_t queue_base;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>rte_iova_t phys_addr_q_base;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>rte_iova_t phys_addr_prod_index;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>rte_iova_t phys_addr_cons_index;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>if (ark_mpu_verify(q->rx_mpu, sizeof(rte_iova_t))) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>ARK_BBDEV_LOG(ERR, "Illegal hw/sw configuration RX<br class="">queue");<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>return -1;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ARK_BBDEV_LOG(DEBUG, "ark_bb_q setup %u:%u",<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> bbdev->data->dev_id, q_id);<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* RX MPU */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>phys_addr_q_base = rte_malloc_virt2iova(q->rx_paddress_q);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* Force TX mode on MPU to match bbdev behavior */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_mpu_configure(q->rx_mpu, phys_addr_q_base, queue_size, 1);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_mpu_start(q->rx_mpu);<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* UDM */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>queue_base = rte_malloc_virt2iova(q);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>phys_addr_prod_index = queue_base +<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>offsetof(struct ark_bbdev_queue, rx_prod_index);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_udm_write_addr(q->udm, phys_addr_prod_index);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_udm_queue_enable(q->udm, 1);<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* TX MPU */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>phys_addr_q_base = rte_malloc_virt2iova(q->tx_meta_q);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_mpu_configure(q->tx_mpu, phys_addr_q_base,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> BB_ARK_TX_Q_FACTOR * queue_size, 1);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_mpu_start(q->tx_mpu);<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* DDM */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>phys_addr_cons_index = queue_base +<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>offsetof(struct ark_bbdev_queue, tx_cons_index);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_ddm_queue_setup(q->ddm, phys_addr_cons_index);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_ddm_queue_reset_stats(q->ddm);<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>return 0;<br class="">+}<br class="">+<br class="">+<br class="">+<br class="">+/* Setup a queue */<br class="">+static int<br class="">+ark_bb_q_setup(struct rte_bbdev *bbdev, uint16_t q_id,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> const struct rte_bbdev_queue_conf *queue_conf) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>struct ark_bbdev_queue *q;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>struct ark_bbdevice *ark_bb = bbdev->data->dev_private;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>const uint32_t queue_size = queue_conf->queue_size;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>const int socket_id = queue_conf->socket;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>const uint64_t pg_sz = sysconf(_SC_PAGESIZE);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>char ring_name[RTE_RING_NAMESIZE];<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* Configuration checks */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>if (!rte_is_power_of_2(queue_size)) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>ARK_BBDEV_LOG(ERR,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> "Configuration queue size"<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> " must be power of two %u",<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> queue_size);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>return -EINVAL;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>if (RTE_PKTMBUF_HEADROOM < ARK_RX_META_SIZE) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>ARK_BBDEV_LOG(ERR,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> "Error: Ark bbdev requires head room > %d bytes<br class="">(%s)",<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> ARK_RX_META_SIZE, __func__);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>return -EINVAL;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* Allocate the queue data structure. */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>q = rte_zmalloc_socket(RTE_STR(DRIVER_NAME), sizeof(*q),<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>RTE_CACHE_LINE_SIZE, queue_conf->socket);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>if (q == NULL) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>ARK_BBDEV_LOG(ERR, "Failed to allocate queue memory");<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>return -ENOMEM;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>bbdev->data->queues[q_id].queue_private = q;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>q->ark_bbdev = ark_bb;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* RING */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>snprintf(ring_name, RTE_RING_NAMESIZE, RTE_STR(DRIVER_NAME)<br class="">"%u:%u",<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>bbdev->data->dev_id, q_id);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>q->active_ops = rte_ring_create(ring_name,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>queue_size,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>queue_conf->socket,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>RING_F_SP_ENQ | RING_F_SC_DEQ);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>if (q->active_ops == NULL) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>ARK_BBDEV_LOG(ERR, "Failed to create ring");<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>goto free_all;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>q->rx_queue_mask = queue_size - 1;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>q->tx_queue_mask = (BB_ARK_TX_Q_FACTOR * queue_size) - 1;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* Each mbuf requires 2 to 4 objects, factor by BB_ARK_TX_Q_FACTOR<br class="">*/<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>q->tx_meta_q =<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>rte_zmalloc_socket("Ark_bb_txqueue meta",<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> queue_size * BB_ARK_TX_Q_FACTOR *<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> sizeof(union ark_tx_meta),<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> pg_sz,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> socket_id);<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>if (q->tx_meta_q == 0) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>ARK_BBDEV_LOG(ERR, "Failed to allocate "<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> "queue memory in %s", __func__);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>goto free_all;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>q->ddm = RTE_PTR_ADD(ark_bb->ddm.v, q_id * ARK_DDM_QOFFSET);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>q->tx_mpu = RTE_PTR_ADD(ark_bb->mputx.v, q_id *<br class="">ARK_MPU_QOFFSET);<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>q->rx_paddress_q =<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>rte_zmalloc_socket("ark_bb_rx_paddress_q",<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> queue_size * sizeof(rte_iova_t),<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> pg_sz,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> socket_id);<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>if (q->rx_paddress_q == 0) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>ARK_BBDEV_LOG(ERR,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> "Failed to allocate queue memory in %s",<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> __func__);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>goto free_all;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>q->udm = RTE_PTR_ADD(ark_bb->udm.v, q_id * ARK_UDM_QOFFSET);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>q->rx_mpu = RTE_PTR_ADD(ark_bb->mpurx.v, q_id *<br class="">ARK_MPU_QOFFSET);<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* Structure have been configured, set the hardware */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>return ark_bb_hw_q_setup(bbdev, q_id, queue_size);<br class="">+<br class="">+free_all:<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>rte_free(q->tx_meta_q);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>rte_free(q->rx_paddress_q);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>rte_free(q);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>return -EFAULT;<br class="">+}<br class="">+<br class="">+<br class="">+/* Release queue */<br class="">+static int<br class="">+ark_bb_q_release(struct rte_bbdev *bbdev, uint16_t q_id) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>struct ark_bbdev_queue *q = bbdev->data-<br class=""><blockquote type="cite" class="">queues[q_id].queue_private;<br class=""></blockquote>+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_mpu_dump(q->rx_mpu, "rx_MPU release", q_id);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_mpu_dump(q->tx_mpu, "tx_MPU release", q_id);<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>rte_ring_free(q->active_ops);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>rte_free(q->tx_meta_q);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>rte_free(q->rx_paddress_q);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>rte_free(q);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>bbdev->data->queues[q_id].queue_private = NULL;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ARK_BBDEV_LOG(DEBUG, "released device queue %u:%u",<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> bbdev->data->dev_id, q_id);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>return 0;<br class="">+}<br class="">+<br class="">+static int<br class="">+ark_bbdev_start(struct rte_bbdev *bbdev) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>struct ark_bbdevice *ark_bb = bbdev->data->dev_private;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ARK_BBDEV_LOG(DEBUG, "Starting device %u", bbdev->data-<br class=""><blockquote type="cite" class="">dev_id);<br class=""></blockquote>+<span class="Apple-tab-span" style="white-space: pre;"> </span>if (ark_bb->started)<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>return 0;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* User start hook */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>if (ark_bb->user_ext.dev_start)<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>ark_bb->user_ext.dev_start(bbdev,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> ark_bb->user_data);<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_bb->started = 1;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>if (ark_bb->start_pg)<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>ark_pktchkr_run(ark_bb->pc);<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>if (ark_bb->start_pg) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>pthread_t thread;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>/* Delay packet generator start allow the hardware to be<br class="">ready<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>* This is only used for sanity checking with internal generator<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>*/<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>if (pthread_create(&thread, NULL,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> ark_pktgen_delay_start, ark_bb->pg)) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>ARK_BBDEV_LOG(ERR, "Could not create pktgen "<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> "starter thread");<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>return -1;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>return 0;<br class="">+}<br class="">+<br class="">+<br class="">+static void<br class="">+ark_bbdev_stop(struct rte_bbdev *bbdev) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>struct ark_bbdevice *ark_bb = bbdev->data->dev_private;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ARK_BBDEV_LOG(DEBUG, "Stopping device %u", bbdev->data-<br class=""><blockquote type="cite" class="">dev_id);<br class=""></blockquote>+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>if (!ark_bb->started)<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>return;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* Stop the packet generator */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>if (ark_bb->start_pg)<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>ark_pktgen_pause(ark_bb->pg);<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* STOP RX Side */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_udm_dump_stats(ark_bb->udm.v, "Post stop");<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* Stop the packet checker if it is running */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>if (ark_bb->start_pg) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>ark_pktchkr_dump_stats(ark_bb->pc);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>ark_pktchkr_stop(ark_bb->pc);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* User stop hook */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>if (ark_bb->user_ext.dev_stop)<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>ark_bb->user_ext.dev_stop(bbdev,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> ark_bb->user_data);<br class="">+<br class="">+}<br class="">+<br class="">+<br class="">+static int<br class="">+ark_bb_q_start(struct rte_bbdev *bbdev, uint16_t q_id) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>struct ark_bbdev_queue *q = bbdev->data-<br class=""><blockquote type="cite" class="">queues[q_id].queue_private;<br class=""></blockquote>+<span class="Apple-tab-span" style="white-space: pre;"> </span>ARK_BBDEV_LOG(DEBUG, "ark_bb_q start %u:%u", bbdev->data-<br class=""><blockquote type="cite" class="">dev_id, q_id);<br class=""></blockquote>+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_ddm_queue_enable(q->ddm, 1);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_udm_queue_enable(q->udm, 1);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_mpu_start(q->tx_mpu);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_mpu_start(q->rx_mpu);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>return 0;<br class="">+}<br class="">+static int<br class="">+ark_bb_q_stop(struct rte_bbdev *bbdev, uint16_t q_id) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>struct ark_bbdev_queue *q = bbdev->data-<br class=""><blockquote type="cite" class="">queues[q_id].queue_private;<br class=""></blockquote>+<span class="Apple-tab-span" style="white-space: pre;"> </span>int cnt = 0;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ARK_BBDEV_LOG(DEBUG, "ark_bb_q stop %u:%u", bbdev->data-<br class=""><blockquote type="cite" class="">dev_id,<br class=""></blockquote>+q_id);<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>while (q->tx_cons_index != q->tx_prod_index) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>usleep(100);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>if (cnt++ > 10000) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>fprintf(stderr, "XXXX %s(%u, %u %u) %d Failured\n",<br class="">__func__, q_id,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>q->tx_cons_index, q->tx_prod_index,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>(int32_t) (q->tx_prod_index - q-<br class=""><blockquote type="cite" class="">tx_cons_index));<br class=""></blockquote>+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>return -1;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_mpu_stop(q->tx_mpu);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_mpu_stop(q->rx_mpu);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_udm_queue_enable(q->udm, 0);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_ddm_queue_enable(q->ddm, 0);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>return 0;<br class="">+}<br class="">+<br class="">+<br class="">+<br class="">+<br class="">+/*<br class="">+***************************************************************<br class="">********<br class="">+** */<br class="">+/* Common function for all enqueue and dequeue ops */ static inline<br class="">+void ark_bb_enqueue_desc_fill(struct ark_bbdev_queue *q,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>struct rte_mbuf *mbuf,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>uint16_t offset, /* Extra offset */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>uint8_t flags,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>uint32_t *meta,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>uint8_t meta_cnt /* 0, 1 or 2 */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>)<br class="">+{<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>union ark_tx_meta *tx_meta;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>int32_t tx_idx;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>uint8_t m;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* Header */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>tx_idx = q->tx_prod_index & q->tx_queue_mask;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>tx_meta = &q->tx_meta_q[tx_idx];<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>tx_meta->data_len = rte_pktmbuf_data_len(mbuf) - offset;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>tx_meta->flags = flags;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>tx_meta->meta_cnt = meta_cnt;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>tx_meta->user1 = *meta++;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>q->tx_prod_index++;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>for (m = 0; m < meta_cnt; m++) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>tx_idx = q->tx_prod_index & q->tx_queue_mask;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>tx_meta = &q->tx_meta_q[tx_idx];<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>tx_meta->usermeta0 = *meta++;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>tx_meta->usermeta1 = *meta++;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>q->tx_prod_index++;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>tx_idx = q->tx_prod_index & q->tx_queue_mask;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>tx_meta = &q->tx_meta_q[tx_idx];<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>tx_meta->physaddr = rte_mbuf_data_iova(mbuf) + offset;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>q->tx_prod_index++;<br class="">+}<br class="">+<br class="">+static inline void<br class="">+ark_bb_enqueue_segmented_pkt(struct ark_bbdev_queue *q,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> struct rte_mbuf *mbuf,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> uint16_t offset,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> uint32_t *meta, uint8_t meta_cnt) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>struct rte_mbuf *next;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>uint8_t flags = ARK_DDM_SOP;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>while (mbuf != NULL) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>next = mbuf->next;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>flags |= (next == NULL) ? ARK_DDM_EOP : 0;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>ark_bb_enqueue_desc_fill(q, mbuf, offset, flags,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>meta, meta_cnt);<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>flags &= ~ARK_DDM_SOP;<span class="Apple-tab-span" style="white-space: pre;"> </span>/* drop SOP flags */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>meta_cnt = 0;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>offset = 0;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>mbuf = next;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class="">+}<br class="">+<br class="">+static inline int<br class="">+ark_bb_enqueue_common(struct ark_bbdev_queue *q,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> struct rte_mbuf *m_in, struct rte_mbuf *m_out,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> uint16_t offset,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> uint32_t *meta, uint8_t meta_cnt) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>int32_t free_queue_space;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>int32_t rx_idx;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* TX side limit */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>free_queue_space = q->tx_queue_mask -<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>(q->tx_prod_index - q->tx_free_index);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>if (unlikely(free_queue_space < (2 + (2 * m_in->nb_segs))))<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>return 1;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* RX side limit */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>free_queue_space = q->rx_queue_mask -<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>(q->rx_seed_index - q->rx_cons_index);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>if (unlikely(free_queue_space < m_out->nb_segs))<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>return 1;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>if (unlikely(m_in->nb_segs > 1))<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>ark_bb_enqueue_segmented_pkt(q, m_in, offset, meta,<br class="">meta_cnt);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>else<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>ark_bb_enqueue_desc_fill(q, m_in, offset,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>ARK_DDM_SOP | ARK_DDM_EOP,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>meta, meta_cnt);<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* We assume that the return mubf has exactly enough segments for<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>* return data, which is 2048 bytes per segment.<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>*/<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>do {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>rx_idx = q->rx_seed_index & q->rx_queue_mask;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>q->rx_paddress_q[rx_idx] = m_out->buf_iova;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>q->rx_seed_index++;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>m_out = m_out->next;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>} while (m_out);<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>return 0;<br class="">+}<br class="">+<br class="">+static inline void<br class="">+ark_bb_enqueue_finalize(struct rte_bbdev_queue_data *q_data,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>struct ark_bbdev_queue *q,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>void **ops,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>uint16_t nb_ops, uint16_t nb)<br class="">+{<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* BBDEV global stats */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* These are not really errors, not sure why bbdev counts these. */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>q_data->queue_stats.enqueue_err_count += nb_ops - nb;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>q_data->queue_stats.enqueued_count += nb;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* Notify HW that */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>if (unlikely(nb == 0))<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>return;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_mpu_set_producer(q->tx_mpu, q->tx_prod_index);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_mpu_set_producer(q->rx_mpu, q->rx_seed_index);<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* Queue info for dequeue-side processing */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>rte_ring_enqueue_burst(q->active_ops,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> (void **)ops, nb, NULL);<br class="">+}<br class="">+<br class="">+static int<br class="">+ark_bb_dequeue_segmented(struct rte_mbuf *mbuf0,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>int32_t *prx_cons_index,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>uint16_t pkt_len<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>)<br class="">+{<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>struct rte_mbuf *mbuf;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>uint16_t data_len;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>uint16_t remaining;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>uint16_t segments = 1;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>data_len = RTE_MIN(pkt_len, RTE_MBUF_DEFAULT_DATAROOM);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>remaining = pkt_len - data_len;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>mbuf = mbuf0;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>mbuf0->data_len = data_len;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>while (remaining) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>segments += 1;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>mbuf = mbuf->next;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>if (unlikely(mbuf == 0)) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>ARK_BBDEV_LOG(CRIT, "Expected chained mbuf with<br class="">"<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> "at least %d segments for dequeue "<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> "of packet length %d",<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> segments, pkt_len);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>return 1;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>data_len = RTE_MIN(remaining,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> RTE_MBUF_DEFAULT_DATAROOM);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>remaining -= data_len;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>mbuf->data_len = data_len;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>*prx_cons_index += 1;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>if (mbuf->next != 0) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>ARK_BBDEV_LOG(CRIT, "Expected chained mbuf with "<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> "at exactly %d segments for dequeue "<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> "of packet length %d. Found %d "<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> "segments",<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> segments, pkt_len, mbuf0->nb_segs);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>return 1;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>return 0;<br class="">+}<br class="">+<br class="">+/*<br class="">+***************************************************************<br class="">********<br class="">+** */<br class="">+/* LDPC Decode ops */<br class="">+static int16_t<br class="">+ark_bb_enqueue_ldpc_dec_one_op(struct ark_bbdev_queue *q,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> struct rte_bbdev_dec_op *this_op) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>struct rte_bbdev_op_ldpc_dec *ldpc_dec_op = &this_op->ldpc_dec;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>struct rte_mbuf *m_in = ldpc_dec_op->input.data;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>struct rte_mbuf *m_out = ldpc_dec_op->hard_output.data;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>uint16_t offset = ldpc_dec_op->input.offset;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>uint32_t meta[5] = {0};<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>uint8_t meta_cnt = 0;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>if (q->ark_bbdev->user_ext.enqueue_ldpc_dec) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>if (q->ark_bbdev->user_ext.enqueue_ldpc_dec(q->ark_bbdev-<br class=""><blockquote type="cite" class="">bbdev,<br class=""></blockquote>+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> this_op,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> meta,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> &meta_cnt,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> q->ark_bbdev-<br class=""><blockquote type="cite" class="">user_data)) {<br class=""></blockquote>+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>ARK_BBDEV_LOG(ERR, "%s failed", __func__);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>return 1;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>return ark_bb_enqueue_common(q, m_in, m_out, offset, meta,<br class="">meta_cnt);<br class="">+}<br class="">+<br class="">+/* Enqueue LDPC Decode -- burst */<br class="">+static uint16_t<br class="">+ark_bb_enqueue_ldpc_dec_ops(struct rte_bbdev_queue_data *q_data,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> struct rte_bbdev_dec_op **ops, uint16_t nb_ops) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>struct ark_bbdev_queue *q = q_data->queue_private;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>unsigned int max_enq;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>uint16_t nb;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>max_enq = rte_ring_free_count(q->active_ops);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>max_enq = RTE_MIN(max_enq, nb_ops);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>for (nb = 0; nb < max_enq; nb++) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>if (ark_bb_enqueue_ldpc_dec_one_op(q, ops[nb]))<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>break;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_bb_enqueue_finalize(q_data, q, (void **)ops, nb_ops, nb);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>return nb;<br class="">+}<br class="">+<br class="">+<br class="">+/*<br class="">+***************************************************************<br class="">********<br class="">+** */<br class="">+/* Dequeue LDPC Decode -- burst */<br class="">+static uint16_t<br class="">+ark_bb_dequeue_ldpc_dec_ops(struct rte_bbdev_queue_data *q_data,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> struct rte_bbdev_dec_op **ops, uint16_t nb_ops) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>struct ark_bbdev_queue *q = q_data->queue_private;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>struct rte_mbuf *mbuf;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>struct rte_bbdev_dec_op *this_op;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>struct ark_rx_meta *meta;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>uint32_t *usermeta;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>uint16_t nb = 0;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>int32_t prod_index = q->rx_prod_index;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>int32_t cons_index = q->rx_cons_index;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>q->tx_free_index = q->tx_cons_index;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>while ((prod_index - cons_index) > 0) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>if (rte_ring_dequeue(q->active_ops, (void **)&this_op)) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>ARK_BBDEV_LOG(ERR, "%s data ready but no op!",<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> __func__);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>q_data->queue_stats.dequeue_err_count += 1;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>break;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>ops[nb] = this_op;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>mbuf = this_op->ldpc_dec.hard_output.data;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>/* META DATA embedded in headroom */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>meta = RTE_PTR_ADD(mbuf->buf_addr,<br class="">ARK_RX_META_OFFSET);<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>mbuf->pkt_len = meta->pkt_len;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>mbuf->data_len = meta->pkt_len;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>if (unlikely(meta->pkt_len > ARK_RX_MAX_NOCHAIN)) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>if (ark_bb_dequeue_segmented(mbuf, &cons_index,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> meta->pkt_len))<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>q_data->queue_stats.dequeue_err_count +=<br class="">1;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>} else if (mbuf->next != 0) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>ARK_BBDEV_LOG(CRIT, "Expected mbuf with "<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> "at exactly 1 segments for dequeue "<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> "of packet length %d. Found %d "<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> "segments",<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> meta->pkt_len, mbuf->nb_segs);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>q_data->queue_stats.dequeue_err_count += 1;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>usermeta = meta->user_meta;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>/* User's meta move from Arkville HW to bbdev OP */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>if (q->ark_bbdev->user_ext.dequeue_ldpc_dec) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>if (q->ark_bbdev->user_ext.dequeue_ldpc_dec(q-<br class=""><blockquote type="cite" class="">ark_bbdev->bbdev,<br class=""></blockquote>+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> this_op,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> usermeta,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> q-<br class=""><blockquote type="cite" class="">ark_bbdev->user_data)) {<br class=""></blockquote>+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>ARK_BBDEV_LOG(ERR, "%s failed", __func__);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>return 1;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>nb++;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>cons_index++;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>if (nb >= nb_ops)<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>break;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>q->rx_cons_index = cons_index;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* BBdev stats */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>q_data->queue_stats.dequeued_count += nb;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>return nb;<br class="">+}<br class="">+<br class="">+/**************************************************************<br class="">********<br class="">+****/<br class="">+/* Enqueue LDPC Encode */<br class="">+static int16_t<br class="">+ark_bb_enqueue_ldpc_enc_one_op(struct ark_bbdev_queue *q,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> struct rte_bbdev_enc_op *this_op) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>struct rte_bbdev_op_ldpc_enc *ldpc_enc_op = &this_op->ldpc_enc;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>struct rte_mbuf *m_in = ldpc_enc_op->input.data;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>struct rte_mbuf *m_out = ldpc_enc_op->output.data;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>uint16_t offset = ldpc_enc_op->input.offset;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>uint32_t meta[5] = {0};<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>uint8_t meta_cnt = 0;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* User's meta move from bbdev op to Arkville HW */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>if (q->ark_bbdev->user_ext.enqueue_ldpc_enc) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>if (q->ark_bbdev->user_ext.enqueue_ldpc_enc(q->ark_bbdev-<br class=""><blockquote type="cite" class="">bbdev,<br class=""></blockquote>+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> this_op,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> meta,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> &meta_cnt,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> q->ark_bbdev-<br class=""><blockquote type="cite" class="">user_data)) {<br class=""></blockquote>+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>ARK_BBDEV_LOG(ERR, "%s failed", __func__);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>return 1;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>return ark_bb_enqueue_common(q, m_in, m_out, offset, meta,<br class="">meta_cnt);<br class="">+}<br class="">+<br class="">+/* Enqueue LDPC Encode -- burst */<br class="">+static uint16_t<br class="">+ark_bb_enqueue_ldpc_enc_ops(struct rte_bbdev_queue_data *q_data,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> struct rte_bbdev_enc_op **ops, uint16_t nb_ops) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>struct ark_bbdev_queue *q = q_data->queue_private;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>unsigned int max_enq;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>uint16_t nb;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>max_enq = rte_ring_free_count(q->active_ops);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>max_enq = RTE_MIN(max_enq, nb_ops);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>for (nb = 0; nb < max_enq; nb++) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>if (ark_bb_enqueue_ldpc_enc_one_op(q, ops[nb]))<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>break;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_bb_enqueue_finalize(q_data, q, (void **)ops, nb_ops, nb);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>return nb;<br class="">+}<br class="">+<br class="">+/* Dequeue LDPC Encode -- burst */<br class="">+static uint16_t<br class="">+ark_bb_dequeue_ldpc_enc_ops(struct rte_bbdev_queue_data *q_data,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> struct rte_bbdev_enc_op **ops, uint16_t nb_ops) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>struct ark_bbdev_queue *q = q_data->queue_private;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>struct rte_mbuf *mbuf;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>struct rte_bbdev_enc_op *this_op;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>struct ark_rx_meta *meta;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>uint32_t *usermeta;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>uint16_t nb = 0;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>int32_t prod_index = q->rx_prod_index;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>int32_t cons_index = q->rx_cons_index;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>q->tx_free_index = q->tx_cons_index;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>while ((prod_index - cons_index) > 0) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>if (rte_ring_dequeue(q->active_ops, (void **)&this_op)) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>ARK_BBDEV_LOG(ERR, "%s data ready but no op!",<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> __func__);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>q_data->queue_stats.dequeue_err_count += 1;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>break;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>ops[nb] = this_op;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>mbuf = this_op->ldpc_enc.output.data;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>/* META DATA embedded in headroom */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>meta = RTE_PTR_ADD(mbuf->buf_addr,<br class="">ARK_RX_META_OFFSET);<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>mbuf->pkt_len = meta->pkt_len;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>mbuf->data_len = meta->pkt_len;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>usermeta = meta->user_meta;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>if (unlikely(meta->pkt_len > ARK_RX_MAX_NOCHAIN)) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>if (ark_bb_dequeue_segmented(mbuf, &cons_index,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> meta->pkt_len))<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>q_data->queue_stats.dequeue_err_count +=<br class="">1;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>} else if (mbuf->next != 0) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>ARK_BBDEV_LOG(CRIT, "Expected mbuf with "<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> "at exactly 1 segments for dequeue "<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> "of packet length %d. Found %d "<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> "segments",<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> meta->pkt_len, mbuf->nb_segs);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>q_data->queue_stats.dequeue_err_count += 1;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>/* User's meta move from Arkville HW to bbdev OP */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>if (q->ark_bbdev->user_ext.dequeue_ldpc_enc) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>if (q->ark_bbdev->user_ext.dequeue_ldpc_enc(q-<br class=""><blockquote type="cite" class="">ark_bbdev->bbdev,<br class=""></blockquote>+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> this_op,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> usermeta,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> q-<br class=""><blockquote type="cite" class="">ark_bbdev->user_data)) {<br class=""></blockquote>+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>ARK_BBDEV_LOG(ERR, "%s failed", __func__);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>return 1;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>nb++;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>cons_index++;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>if (nb >= nb_ops)<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>break;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>q->rx_cons_index = cons_index;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* BBdev stats */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>q_data->queue_stats.dequeued_count += nb;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>return nb;<br class="">+}<br class="">+<br class="">+<br class="">+/**************************************************************<br class="">********<br class="">+****/<br class="">+/*<br class="">+ *Initial device hardware configuration when device is opened<br class="">+ * setup the DDM, and UDM; called once per PCIE device */ static int<br class="">+ark_bb_config_device(struct ark_bbdevice *ark_bb) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>uint16_t num_q, i;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>struct ark_mpu_t *mpu;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/*<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>* Make sure that the packet director, generator and checker are in a<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>* known state<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>*/<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_bb->start_pg = 0;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_bb->pg = ark_pktgen_init(ark_bb->pktgen.v, 0, 1);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>if (ark_bb->pg == NULL)<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>return -1;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_pktgen_reset(ark_bb->pg);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_bb->pc = ark_pktchkr_init(ark_bb->pktchkr.v, 0, 1);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>if (ark_bb->pc == NULL)<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>return -1;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_pktchkr_stop(ark_bb->pc);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_bb->pd = ark_pktdir_init(ark_bb->pktdir.v);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>if (ark_bb->pd == NULL)<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>return -1;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* Verify HW */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>if (ark_udm_verify(ark_bb->udm.v))<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>return -1;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>if (ark_ddm_verify(ark_bb->ddm.v))<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>return -1;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* MPU reset */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>mpu = ark_bb->mpurx.v;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>num_q = ark_api_num_queues(mpu);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_bb->max_nb_queues = num_q;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>for (i = 0; i < num_q; i++) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>ark_mpu_reset(mpu);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>mpu = RTE_PTR_ADD(mpu, ARK_MPU_QOFFSET);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_udm_configure(ark_bb->udm.v,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> RTE_PKTMBUF_HEADROOM,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> RTE_MBUF_DEFAULT_DATAROOM);<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>mpu = ark_bb->mputx.v;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>num_q = ark_api_num_queues(mpu);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>for (i = 0; i < num_q; i++) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>ark_mpu_reset(mpu);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>mpu = RTE_PTR_ADD(mpu, ARK_MPU_QOFFSET);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_rqp_stats_reset(ark_bb->rqpacing);<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ARK_BBDEV_LOG(INFO, "packet director set to 0x%x", ark_bb-<br class=""><blockquote type="cite" class="">pkt_dir_v);<br class=""></blockquote>+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_pktdir_setup(ark_bb->pd, ark_bb->pkt_dir_v);<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>if (ark_bb->pkt_gen_args[0]) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>ARK_BBDEV_LOG(INFO, "Setting up the packet generator");<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>ark_pktgen_parse(ark_bb->pkt_gen_args);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>ark_pktgen_reset(ark_bb->pg);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>ark_pktgen_setup(ark_bb->pg);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>ark_bb->start_pg = 1;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>return 0;<br class="">+}<br class="">+<br class="">+static int<br class="">+ark_bbdev_init(struct rte_bbdev *bbdev, struct rte_pci_driver *pci_drv)<br class="">+{<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>struct ark_bbdevice *ark_bb = bbdev->data->dev_private;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>struct rte_pci_device *pci_dev = RTE_DEV_TO_PCI(bbdev->device);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>bool rqpacing = false;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>int p;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_bb->bbdev = bbdev;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>RTE_SET_USED(pci_drv);<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_bb->bar0 = (uint8_t *)pci_dev->mem_resource[0].addr;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_bb->a_bar = (uint8_t *)pci_dev->mem_resource[2].addr;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_bb->sysctrl.v = (void *)&ark_bb->bar0[ARK_SYSCTRL_BASE];<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_bb->mpurx.v = (void *)&ark_bb->bar0[ARK_MPU_RX_BASE];<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_bb->udm.v = (void *)&ark_bb->bar0[ARK_UDM_BASE];<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_bb->mputx.v = (void *)&ark_bb->bar0[ARK_MPU_TX_BASE];<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_bb->ddm.v = (void *)&ark_bb->bar0[ARK_DDM_BASE];<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_bb->pktdir.v = (void *)&ark_bb->bar0[ARK_PKTDIR_BASE];<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_bb->pktgen.v = (void *)&ark_bb->bar0[ARK_PKTGEN_BASE];<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_bb->pktchkr.v = (void *)&ark_bb->bar0[ARK_PKTCHKR_BASE];<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>p = 0;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>while (ark_device_caps[p].device_id != 0) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>if (pci_dev->id.device_id == ark_device_caps[p].device_id) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>rqpacing = ark_device_caps[p].caps.rqpacing;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>break;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>p++;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>if (rqpacing)<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>ark_bb->rqpacing =<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>(struct ark_rqpace_t *)(ark_bb->bar0 +<br class="">ARK_RCPACING_BASE);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>else<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>ark_bb->rqpacing = NULL;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* Check to see if there is an extension that we need to load */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>if (check_for_ext(ark_bb))<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>return -1;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_bb->started = 0;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ARK_BBDEV_LOG(INFO, "Sys Ctrl Const = 0x%x HW Commit_ID: %08x",<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> ark_bb->sysctrl.t32[4],<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> rte_be_to_cpu_32(ark_bb->sysctrl.t32[0x20 / 4]));<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ARK_BBDEV_LOG(INFO, "Arkville HW Commit_ID: %08x",<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> rte_be_to_cpu_32(ark_bb->sysctrl.t32[0x20 / 4]));<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* If HW sanity test fails, return an error */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>if (ark_bb->sysctrl.t32[4] != 0xcafef00d) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>ARK_BBDEV_LOG(ERR,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> "HW Sanity test has failed, expected constant"<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> " 0x%x, read 0x%x (%s)",<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> 0xcafef00d,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> ark_bb->sysctrl.t32[4], __func__);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>return -1;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>return ark_bb_config_device(ark_bb);<br class="">+}<br class="">+<br class="">+static int<br class="">+ark_bbdev_uninit(struct rte_bbdev *bbdev) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>struct ark_bbdevice *ark_bb = bbdev->data->dev_private;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>if (rte_eal_process_type() != RTE_PROC_PRIMARY)<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>return 0;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_pktgen_uninit(ark_bb->pg);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_pktchkr_uninit(ark_bb->pc);<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>return 0;<br class="">+}<br class="">+<br class="">+static int<br class="">+ark_bbdev_probe(struct rte_pci_driver *pci_drv,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>struct rte_pci_device *pci_dev)<br class="">+{<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>struct rte_bbdev *bbdev = NULL;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>char dev_name[RTE_BBDEV_NAME_MAX_LEN];<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>struct ark_bbdevice *ark_bb;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>if (pci_dev == NULL)<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>return -EINVAL;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>rte_pci_device_name(&pci_dev->addr, dev_name, sizeof(dev_name));<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* Allocate memory to be used privately by drivers */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>bbdev = rte_bbdev_allocate(pci_dev->device.name);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>if (bbdev == NULL)<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>return -ENODEV;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* allocate device private memory */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>bbdev->data->dev_private = rte_zmalloc_socket(dev_name,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>sizeof(struct ark_bbdevice),<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>RTE_CACHE_LINE_SIZE,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>pci_dev->device.numa_node);<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>if (bbdev->data->dev_private == NULL) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>ARK_BBDEV_LOG(CRIT,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>"Allocate of %zu bytes for device \"%s\"<br class="">failed",<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>sizeof(struct ark_bbdevice), dev_name);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>rte_bbdev_release(bbdev);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>return -ENOMEM;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_bb = bbdev->data->dev_private;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* Initialize ark_bb */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_bb->pkt_dir_v = 0x00110110;<br class=""></blockquote><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">There a few magic number that you may to define separately</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><blockquote type="cite" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none;" class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* Fill HW specific part of device structure */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>bbdev->device = &pci_dev->device;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>bbdev->intr_handle = NULL;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>bbdev->data->socket_id = pci_dev->device.numa_node;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>bbdev->dev_ops = &ark_bbdev_pmd_ops;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>if (pci_dev->device.devargs)<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>parse_ark_bbdev_params(pci_dev->device.devargs->args,<br class="">ark_bb);<br class="">+<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* Device specific initialization */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>if (ark_bbdev_init(bbdev, pci_drv))<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>return -EIO;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>if (ark_bbdev_start(bbdev))<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>return -EIO;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* Core operations LDPC encode amd decode */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>bbdev->enqueue_ldpc_enc_ops = ark_bb_enqueue_ldpc_enc_ops;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>bbdev->dequeue_ldpc_enc_ops = ark_bb_dequeue_ldpc_enc_ops;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>bbdev->enqueue_ldpc_dec_ops = ark_bb_enqueue_ldpc_dec_ops;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>bbdev->dequeue_ldpc_dec_ops = ark_bb_dequeue_ldpc_dec_ops;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ARK_BBDEV_LOG(DEBUG, "bbdev id = %u [%s]",<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> bbdev->data->dev_id, dev_name);<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>return 0;<br class="">+}<br class="">+<br class="">+/* Uninitialize device */<br class="">+static int<br class="">+ark_bbdev_remove(struct rte_pci_device *pci_dev) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>struct rte_bbdev *bbdev;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>int ret;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>if (pci_dev == NULL)<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>return -EINVAL;<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* Find device */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>bbdev = rte_bbdev_get_named_dev(pci_dev->device.name);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>if (bbdev == NULL) {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>ARK_BBDEV_LOG(CRIT,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>"Couldn't find HW dev \"%s\" to Uninitialize<br class="">it",<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>pci_dev->device.name);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>return -ENODEV;<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>}<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* Arkville device close */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ark_bbdev_uninit(bbdev);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>rte_free(bbdev->data->dev_private);<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>/* Close device */<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>ret = rte_bbdev_close(bbdev->data->dev_id);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>if (ret < 0)<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>ARK_BBDEV_LOG(ERR,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>"Device %i failed to close during remove: %i",<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>bbdev->data->dev_id, ret);<br class="">+<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>return rte_bbdev_release(bbdev);<br class="">+}<br class="">+<br class="">+/* Operation for the PMD */<br class="">+static const struct rte_bbdev_ops ark_bbdev_pmd_ops = {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>.info_get = ark_bbdev_info_get,<br class=""></blockquote><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">This is the one mentioned above. This is actually defined in your future commit.</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">You could start with info_get, then queue, then processing for instance.<span class="Apple-converted-space"> </span></span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><blockquote type="cite" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none;" class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>.start = ark_bbdev_start,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>.stop = ark_bbdev_stop,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>.queue_setup = ark_bb_q_setup,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>.queue_release = ark_bb_q_release,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>.queue_start = ark_bb_q_start,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>.queue_stop = ark_bb_q_stop,<br class="">+};<br class="">+<br class="">+static struct rte_pci_driver ark_bbdev_pmd_drv = {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>.probe = ark_bbdev_probe,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>.remove = ark_bbdev_remove,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>.id_table = pci_id_ark,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>.drv_flags = RTE_PCI_DRV_NEED_MAPPING<br class="">+};<br class="">+<br class="">+RTE_PMD_REGISTER_PCI(DRIVER_NAME, ark_bbdev_pmd_drv);<br class="">+RTE_PMD_REGISTER_PCI_TABLE(DRIVER_NAME, pci_id_ark);<br class="">+RTE_PMD_REGISTER_PARAM_STRING(DRIVER_NAME,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> ARK_BBDEV_PKTGEN_ARG "=<filename> "<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> ARK_BBDEV_PKTCHKR_ARG "=<filename> "<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> ARK_BBDEV_PKTDIR_ARG "=<bitmap>"<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> );<br class="">diff --git a/drivers/baseband/ark/ark_bbext.h<br class="">b/drivers/baseband/ark/ark_bbext.h<br class="">new file mode 100644<br class="">index 0000000000..2e9cc4ccf3<br class="">--- /dev/null<br class="">+++ b/drivers/baseband/ark/ark_bbext.h<br class="">@@ -0,0 +1,163 @@<br class="">+/* SPDX-License-Identifier: BSD-3-Clause<br class="">+ * Copyright (c) 2015-2018 Atomic Rules LLC */<br class="">+<br class="">+#ifndef _ARK_BBEXT_H_<br class="">+#define _ARK_BBEXT_H_<br class="">+<br class="">+#include <rte_bbdev.h><br class="">+#include <rte_bbdev_pmd.h><br class="">+<br class="">+/* The following section lists function prototypes for Arkville's<br class="">+ * baseband dynamic PMD extension. User's who create an extension<br class="">+ * must include this file and define the necessary and desired<br class="">+ * functions. Only 1 function is required for an extension,<br class="">+ * rte_pmd_ark_bbdev_init(); all other functions prototypes in this<br class="">+ * section are optional.<br class="">+ * See documentation for compiling and use of extensions.<br class="">+ */<br class="">+<br class="">+/**<br class="">+ * Extension prototype, required implementation if extensions are used.<br class="">+ * Called during device probe to initialize the user structure<br class="">+ * passed to other extension functions. This is called once for each<br class="">+ * port of the device.<br class="">+ *<br class="">+ * @param dev<br class="">+ * current device.<br class="">+ * @param a_bar<br class="">+ * access to PCIe device bar (application bar) and hence access to<br class="">+ * user's portion of FPGA.<br class="">+ * @return user_data<br class="">+ * which will be passed to other extension functions.<br class="">+ */<br class="">+void *rte_pmd_ark_bbdev_init(struct rte_bbdev *dev, void *a_bar);<br class="">+<br class="">+/**<br class="">+ * Extension prototype, optional implementation.<br class="">+ * Called during device uninit.<br class="">+ *<br class="">+ * @param dev<br class="">+ * current device.<br class="">+ * @param user_data<br class="">+ * user argument from dev_init() call.<br class="">+ */<br class="">+int rte_pmd_ark_bbdev_uninit(struct rte_bbdev *dev, void *user_data);<br class="">+<br class="">+/**<br class="">+ * Extension prototype, optional implementation.<br class="">+ * Called during rte_bbdev_start().<br class="">+ *<br class="">+ * @param dev<br class="">+ * current device.<br class="">+ * @param user_data<br class="">+ * user argument from dev_init() call.<br class="">+ * @return (0) if successful.<br class="">+ */<br class="">+int rte_pmd_ark_bbdev_start(struct rte_bbdev *dev, void *user_data);<br class="">+<br class="">+/**<br class="">+ * Extension prototype, optional implementation.<br class="">+ * Called during rte_bbdev_stop().<br class="">+ *<br class="">+ * @param dev<br class="">+ * current device.<br class="">+ * @param user_data<br class="">+ * user argument from dev_init() call.<br class="">+ * @return (0) if successful.<br class="">+ */<br class="">+int rte_pmd_ark_bbdev_stop(struct rte_bbdev *dev, void *user_data);<br class="">+<br class="">+/**<br class="">+ * Extension prototype, optional implementation.<br class="">+ * Called during rte_bbdev_dequeue_ldpc_dec_ops<br class="">+ *<br class="">+ * @param dev<br class="">+ * current device.<br class="">+ * @param user_data<br class="">+ * user argument from dev_init() call.<br class="">+ * @return (0) if successful.<br class="">+ */<br class="">+int rte_pmd_ark_bbdev_dequeue_ldpc_dec(struct rte_bbdev *dev,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> struct rte_bbdev_dec_op *this_op,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> uint32_t *usermeta,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> void *user_data);<br class="">+<br class="">+/**<br class="">+ * Extension prototype, optional implementation.<br class="">+ * Called during rte_bbdev_dequeue_ldpc_enc_ops<br class="">+ *<br class="">+ * @param dev<br class="">+ * current device.<br class="">+ * @param user_data<br class="">+ * user argument from dev_init() call.<br class="">+ * @return (0) if successful.<br class="">+ */<br class="">+int rte_pmd_ark_bbdev_dequeue_ldpc_enc(struct rte_bbdev *dev,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> struct rte_bbdev_enc_op *this_op,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> uint32_t *usermeta,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span> void *user_data);<br class="">+<br class="">+/**<br class="">+ * Extension prototype, optional implementation.<br class="">+ * Called during rte_bbdev_enqueue_ldpc_dec_ops<br class="">+ *<br class="">+ * @param dev<br class="">+ * current device.<br class="">+ * @param user_data<br class="">+ * user argument from dev_init() call.<br class="">+ * @return (0) if successful.<br class="">+ */<br class="">+int rte_pmd_ark_bbdev_enqueue_ldpc_dec(struct rte_bbdev *dev,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>struct rte_bbdev_dec_op *this_op,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>uint32_t *usermeta,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>uint8_t *meta_cnt,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>void *user_data);<br class="">+<br class="">+/**<br class="">+ * Extension prototype, optional implementation.<br class="">+ * Called during rte_bbdev_enqueue_ldpc_enc_ops<br class="">+ *<br class="">+ * @param dev<br class="">+ * current device.<br class="">+ * @param user_data<br class="">+ * user argument from dev_init() call.<br class="">+ * @return (0) if successful.<br class="">+ */<br class="">+int rte_pmd_ark_bbdev_enqueue_ldpc_enc(struct rte_bbdev *dev,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>struct rte_bbdev_enc_op *this_op,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>uint32_t *usermeta,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>uint8_t *meta_cnt,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span>void *user_data);<br class="">+<br class="">+<br class="">+struct arkbb_user_ext {<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>void *(*dev_init)(struct rte_bbdev *dev, void *abar);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>int (*dev_uninit)(struct rte_bbdev *dev, void *udata);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>int (*dev_start)(struct rte_bbdev *dev, void *udata);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>int (*dev_stop)(struct rte_bbdev *dev, void *udata);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>int (*dequeue_ldpc_dec)(struct rte_bbdev *dev,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>struct rte_bbdev_dec_op *op,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>uint32_t *v,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>void *udata);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>int (*dequeue_ldpc_enc)(struct rte_bbdev *dev,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>struct rte_bbdev_enc_op *op,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>uint32_t *v,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>void *udata);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>int (*enqueue_ldpc_dec)(struct rte_bbdev *dev,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>struct rte_bbdev_dec_op *op,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>uint32_t *v,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>uint8_t *v1,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>void *udata);<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span>int (*enqueue_ldpc_enc)(struct rte_bbdev *dev,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>struct rte_bbdev_enc_op *op,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>uint32_t *v,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>uint8_t *v1,<br class="">+<span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-tab-span" style="white-space: pre;"> </span><span class="Apple-converted-space"> </span>void *udata);<br class="">+};<br class="">+<br class="">+<br class="">+<br class="">+<br class="">+<br class="">+#endif<br class="">--<br class="">2.25.1</blockquote></div></blockquote></div><br class=""></div></body></html>