[PATCH v4 1/4] bus/cdx: introduce cdx bus

[Gupta, Nipun]

[AMD Official Use Only - General]



> -----Original Message-----
> From: Xia, Chenbo <chenbo.xia at intel.com>
> Sent: Tuesday, May 9, 2023 12:25 PM
> To: Gupta, Nipun <Nipun.Gupta at amd.com>; dev at dpdk.org;
> thomas at monjalon.net; david.marchand at redhat.com
> Cc: Yigit, Ferruh <Ferruh.Yigit at amd.com>; Anand, Harpreet
> <harpreet.anand at amd.com>; Agarwal, Nikhil <nikhil.agarwal at amd.com>
> Subject: RE: [PATCH v4 1/4] bus/cdx: introduce cdx bus
> 
> Caution: This message originated from an External Source. Use proper
> caution when opening attachments, clicking links, or responding.
> 
> 
> > -----Original Message-----
> > From: Nipun Gupta <nipun.gupta at amd.com>
> > Sent: Monday, May 8, 2023 7:18 PM
> > To: dev at dpdk.org; thomas at monjalon.net; david.marchand at redhat.com
> > Cc: ferruh.yigit at amd.com; harpreet.anand at amd.com;
> nikhil.agarwal at amd.com;
> > Nipun Gupta <nipun.gupta at amd.com>
> > Subject: [PATCH v4 1/4] bus/cdx: introduce cdx bus
> >
> > CDX bus supports multiple type of devices, which can be
> > exposed to user-space via vfio-cdx.
> >
> > vfio-cdx provides the MMIO IO_MEMORY regions as well as the
> > DMA interface for the device (IOMMU).
> >
> > This support aims to enable the DPDK to support the cdx
> > devices in user-space using VFIO interface.
> >
> > Signed-off-by: Nipun Gupta <nipun.gupta at amd.com>
> > ---
> >  MAINTAINERS                            |   5 +
> >  doc/guides/rel_notes/release_23_07.rst |   6 +
> >  drivers/bus/cdx/bus_cdx_driver.h       | 201 ++++++++++
> >  drivers/bus/cdx/cdx.c                  | 520 +++++++++++++++++++++++++
> >  drivers/bus/cdx/cdx_logs.h             |  37 ++
> >  drivers/bus/cdx/cdx_vfio.c             | 437 +++++++++++++++++++++
> >  drivers/bus/cdx/meson.build            |  13 +
> >  drivers/bus/cdx/private.h              |  49 +++
> >  drivers/bus/cdx/version.map            |  11 +
> >  drivers/bus/meson.build                |   1 +
> >  10 files changed, 1280 insertions(+)
> >  create mode 100644 drivers/bus/cdx/bus_cdx_driver.h
> >  create mode 100644 drivers/bus/cdx/cdx.c
> >  create mode 100644 drivers/bus/cdx/cdx_logs.h
> >  create mode 100644 drivers/bus/cdx/cdx_vfio.c
> >  create mode 100644 drivers/bus/cdx/meson.build
> >  create mode 100644 drivers/bus/cdx/private.h
> >  create mode 100644 drivers/bus/cdx/version.map
> >
> 
> ...
> 
> > --- /dev/null
> > +++ b/drivers/bus/cdx/cdx.c
> > @@ -0,0 +1,520 @@
> > +/* SPDX-License-Identifier: BSD-3-Clause
> > + * Copyright (C) 2022-2023, Advanced Micro Devices, Inc.
> > + */
> > +
> > +/*
> > + * Architecture Overview
> > + * =====================
> > + * CDX is a Hardware Architecture designed for AMD FPGA devices. It
> > + * consists of sophisticated mechanism for interaction between FPGA,
> > + * Firmware and the APUs (Application CPUs).
> > + *
> > + * Firmware resides on RPU (Realtime CPUs) which interacts with
> > + * the FPGA program manager and the APUs. The RPU provides memory-
> mapped
> > + * interface (RPU if) which is used to communicate with APUs.
> > + *
> > + * The diagram below shows an overview of the CDX architecture:
> > + *
> > + *          +--------------------------------------+
> > + *          |   DPDK                               |
> > + *          |                    DPDK CDX drivers  |
> > + *          |                             |        |
> > + *          |                      DPDK CDX bus    |
> > + *          |                             |        |
> > + *          +-----------------------------|--------+
> > + *                                        |
> > + *          +-----------------------------|--------+
> > + *          |    Application CPUs (APU)   |        |
> > + *          |                             |        |
> > + *          |                     VFIO CDX driver  |
> > + *          |     Linux OS                |        |
> > + *          |                      Linux CDX bus   |
> > + *          |                             |        |
> > + *          +-----------------------------|--------+
> > + *                                        |
> > + *                                        |
> > + *          +------------------------| RPU if |----+
> > + *          |                             |        |
> > + *          |                             V        |
> > + *          |          Realtime CPUs (RPU)         |
> > + *          |                                      |
> > + *          +--------------------------------------+
> > + *                                |
> > + *          +---------------------|----------------+
> > + *          |  FPGA               |                |
> > + *          |      +-----------------------+       |
> > + *          |      |           |           |       |
> > + *          | +-------+    +-------+   +-------+   |
> > + *          | | dev 1 |    | dev 2 |   | dev 3 |   |
> > + *          | +-------+    +-------+   +-------+   |
> > + *          +--------------------------------------+
> > + *
> > + * The RPU firmware extracts the device information from the loaded
> FPGA
> > + * image and implements a mechanism that allows the APU drivers to
> > + * enumerate such devices (device personality and resource details) via
> > + * a dedicated communication channel.
> 
> What is APU? CPU resources that application uses? Then why cpu resources
> that DPDK
> Uses are not part of it?

Hi Chenbo,

APU's are application processor unit and are on-chip CPU's. So in short APUs
are CPUs only on which applications like DPDK are running.

Thanks,
Nipun