[dpdk-dev] [PATCH v6 1/4] ena: Amazon ENA documentation

Jan Medala jan at semihalf.com
Mon Mar 14 20:12:18 CET 2016


Signed-off-by: Alexander Matushevsky <matua at amazon.com>
Signed-off-by: Jan Medala <jan at semihalf.com>
Signed-off-by: Jakub Palider <jpa at semihalf.com>
---
 MAINTAINERS                  |   8 ++
 doc/guides/nics/ena.rst      | 251 +++++++++++++++++++++++++++++++++++++++++++
 doc/guides/nics/index.rst    |   1 +
 doc/guides/nics/overview.rst | 116 ++++++++++----------
 4 files changed, 318 insertions(+), 58 deletions(-)
 create mode 100644 doc/guides/nics/ena.rst

diff --git a/MAINTAINERS b/MAINTAINERS
index f10b26a..d37aad6 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -260,6 +260,14 @@ Linux AF_PACKET
 M: John W. Linville <linville at tuxdriver.com>
 F: drivers/net/af_packet/
 
+Amazon ena
+M: Jan Medala <jan at semihalf.com>
+M: Jakub Palider <jpa at semihalf.com>
+M: Netanel Belgazal <netanel at amazon.com>
+M: Evgeny Schemeilin <evgenys at amazon.com>
+F: drivers/net/ena/
+F: doc/guides/nics/ena.rst
+
 Chelsio cxgbe
 M: Rahul Lakkireddy <rahul.lakkireddy at chelsio.com>
 F: drivers/net/cxgbe/
diff --git a/doc/guides/nics/ena.rst b/doc/guides/nics/ena.rst
new file mode 100644
index 0000000..9f93848
--- /dev/null
+++ b/doc/guides/nics/ena.rst
@@ -0,0 +1,251 @@
+.. BSD LICENSE
+
+    Copyright (c) 2015-2016 Amazon.com, Inc. or its affiliates.
+    All rights reserved.
+
+    Redistribution and use in source and binary forms, with or without
+    modification, are permitted provided that the following conditions
+    are met:
+
+    * Redistributions of source code must retain the above copyright
+    notice, this list of conditions and the following disclaimer.
+    * Redistributions in binary form must reproduce the above copyright
+    notice, this list of conditions and the following disclaimer in
+    the documentation and/or other materials provided with the
+    distribution.
+    * Neither the name of Amazon.com, Inc. nor the names of its
+    contributors may be used to endorse or promote products derived
+    from this software without specific prior written permission.
+
+    THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+    "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+    LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+    A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+    OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+    SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+    LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+    DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+    THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+    (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+    OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+ENA Poll Mode Driver
+====================
+
+The ENA PMD is a DPDK poll-mode driver for the Amazon Elastic
+Network Adapter (ENA) family.
+
+Overview
+--------
+
+The ENA driver exposes a lightweight management interface with a
+minimal set of memory mapped registers and an extendable command set
+through an Admin Queue.
+
+The driver supports a wide range of ENA adapters, is link-speed
+independent (i.e., the same driver is used for 10GbE, 25GbE, 40GbE,
+etc.), and it negotiates and supports an extendable feature set.
+
+ENA adapters allow high speed and low overhead Ethernet traffic
+processing by providing a dedicated Tx/Rx queue pair per CPU core.
+
+The ENA driver supports industry standard TCP/IP offload features such
+as checksum offload and TCP transmit segmentation offload (TSO).
+
+Receive-side scaling (RSS) is supported for multi-core scaling.
+
+Some of the ENA devices support a working mode called Low-latency
+Queue (LLQ), which saves several more microseconds.
+
+Management Interface
+--------------------
+
+ENA management interface is exposed by means of:
+
+* Device Registers
+* Admin Queue (AQ) and Admin Completion Queue (ACQ)
+
+ENA device memory-mapped PCIe space for registers (MMIO registers)
+are accessed only during driver initialization and are not involved
+in further normal device operation.
+
+AQ is used for submitting management commands, and the
+results/responses are reported asynchronously through ACQ.
+
+ENA introduces a very small set of management commands with room for
+vendor-specific extensions. Most of the management operations are
+framed in a generic Get/Set feature command.
+
+The following admin queue commands are supported:
+
+* Create I/O submission queue
+* Create I/O completion queue
+* Destroy I/O submission queue
+* Destroy I/O completion queue
+* Get feature
+* Set feature
+* Get statistics
+
+Refer to ``ena_admin_defs.h`` for the list of supported Get/Set Feature
+properties.
+
+Data Path Interface
+-------------------
+
+I/O operations are based on Tx and Rx Submission Queues (Tx SQ and Rx
+SQ correspondingly). Each SQ has a completion queue (CQ) associated
+with it.
+
+The SQs and CQs are implemented as descriptor rings in contiguous
+physical memory.
+
+Refer to ``ena_eth_io_defs.h`` for the detailed structure of the descriptor
+
+The driver supports multi-queue for both Tx and Rx.
+
+Configuration information
+-------------------------
+
+**DPDK Configuration Parameters**
+
+  The following configuration options are available for the ENA PMD:
+
+   * **CONFIG_RTE_LIBRTE_ENA_PMD** (default y): Enables or disables inclusion
+     of the ENA PMD driver in the DPDK compilation.
+
+
+   * **CONFIG_RTE_LIBRTE_ENA_DEBUG_INIT** (default y): Enables or disables debug
+     logging of device initialization within the ENA PMD driver.
+
+   * **CONFIG_RTE_LIBRTE_ENA_DEBUG_RX** (default n): Enables or disables debug
+     logging of RX logic within the ENA PMD driver.
+
+   * **CONFIG_RTE_LIBRTE_ENA_DEBUG_TX** (default n): Enables or disables debug
+     logging of TX logic within the ENA PMD driver.
+
+   * **CONFIG_RTE_LIBRTE_ENA_COM_DEBUG** (default n): Enables or disables debug
+     logging of low level tx/rx logic in ena_com(base) within the ENA PMD driver.
+
+**ENA Configuration Parameters**
+
+   * **Number of Queues**
+
+     This is the requested number of queues upon initialization, however, the actual
+     number of receive and transmit queues to be created will be the minimum between
+     the maximal number supported by the device and number of queues requested.
+
+   * **Size of Queues**
+
+     This is the requested size of receive/transmit queues, while the actual size
+     will be the minimum between the requested size and the maximal receive/transmit
+     supported by the device.
+
+Building DPDK
+-------------
+
+See the :ref:`DPDK Getting Started Guide for Linux <linux_gsg>` for
+instructions on how to build DPDK.
+
+By default the ENA PMD library will be built into the DPDK library.
+
+For configuring and using UIO and VFIO frameworks, please also refer :ref:`the
+documentation that comes with DPDK suite <linux_gsg>`.
+
+Supported ENA adapters
+----------------------
+
+Current ENA PMD supports the following ENA adapters including:
+
+* ``1d0f:ec20`` - ENA VF
+* ``1d0f:ec21`` - ENA VF with LLQ support
+
+Supported Operating Systems
+---------------------------
+
+Any Linux distribution fulfilling the conditions described in ``System Requirements``
+section of :ref:`the DPDK documentation <linux_gsg>` or refer to *DPDK Release Notes*.
+
+Supported features
+------------------
+
+* Jumbo frames up to 9K
+* Port Hardware Statistics
+* IPv4/TCP/UDP checksum offload
+* TSO offload
+* Multiple receive and transmit queues
+* RSS
+* Low Latency Queue for Tx
+
+Unsupported features
+--------------------
+
+The features supported by the device and not yet supported by this PMD include:
+
+* Asynchronous Event Notification Queue (AENQ)
+
+Prerequisites
+-------------
+
+#. Prepare the system as recommended by DPDK suite.  This includes environment
+   variables, hugepages configuration, tool-chains and configuration
+
+#. Insert igb_uio kernel module using the command 'modprobe igb_uio'
+
+#. Bind the intended ENA device to igb_uio module
+
+
+At this point the system should be ready to run DPDK applications. Once the
+application runs to completion, the ENA can be detached from igb_uio if necessary.
+
+Usage example
+-------------
+
+This section demonstrates how to launch **testpmd** with Amazon ENA
+devices managed by librte_pmd_ena.
+
+#. Load the kernel modules:
+
+   .. code-block:: console
+
+      modprobe uio
+      insmod ./x86_64-native-linuxapp-gcc/kmod/igb_uio.ko
+
+   .. note::
+
+      Currently Amazon ENA PMD driver depends on igb_uio user space I/O kernel module
+
+#. Mount and request huge pages:
+
+   .. code-block:: console
+
+      mount -t hugetlbfs nodev /mnt/hugepages
+      echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
+
+#. Bind UIO driver to ENA device (using provided by DPDK binding tool):
+
+   .. code-block:: console
+
+      ./tools/dpdk_nic_bind.py --bind=igb_uio 0000:02:00.1
+
+#. Start testpmd with basic parameters:
+
+   .. code-block:: console
+
+      ./x86_64-native-linuxapp-gcc/app/testpmd -c 0xf -n 4 -- -i
+
+   Example output:
+
+   .. code-block:: console
+
+      [...]
+      EAL: PCI device 0000:02:00.1 on NUMA socket -1
+      EAL:   probe driver: 1d0f:ec20 rte_ena_pmd
+      EAL:   PCI memory mapped at 0x7f9b6c400000
+      PMD: eth_ena_dev_init(): Initializing 0:2:0.1
+      Interactive-mode selected
+      Configuring Port 0 (socket 0)
+      Port 0: 00:00:00:11:00:01
+      Checking link statuses...
+      Port 0 Link Up - speed 10000 Mbps - full-duplex
+      Done
+      testpmd>
diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst
index 0b353a8..d060265 100644
--- a/doc/guides/nics/index.rst
+++ b/doc/guides/nics/index.rst
@@ -39,6 +39,7 @@ Network Interface Controller Drivers
     bnx2x
     cxgbe
     e1000em
+    ena
     enic
     fm10k
     i40e
diff --git a/doc/guides/nics/overview.rst b/doc/guides/nics/overview.rst
index 2d4f014..28237e9 100644
--- a/doc/guides/nics/overview.rst
+++ b/doc/guides/nics/overview.rst
@@ -74,74 +74,74 @@ Most of these differences are summarized below.
 
 .. table:: Features availability in networking drivers
 
-   ==================== = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
-   Feature              a b b b c e e i i i i i i i i i i f f m m m n n p r s v v v x
-                        f n n o x 1 n 4 4 4 4 g g x x x x m m l l p f u c i z i i m e
-                        p x x n g 0 i 0 0 0 0 b b g g g g 1 1 x x i p l a n e r r x n
-                        a 2 2 d b 0 c e e e e   v b b b b 0 0 4 5 p   l p g d t t n v
-                        c x x i e 0     . v v   f e e e e k k     e         a i i e i
-                        k   v n         . f f       . v v   .               t o o t r
-                        e   f g         .   .       . f f   .               a   . 3 t
-                        t               v   v       v   v   v               2   v
-                                        e   e       e   e   e                   e
-                                        c   c       c   c   c                   c
-   ==================== = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
-   link status                  X     X X                                   X
-   link status event                  X X
-   Rx interrupt                       X X X X
-   queue start/stop             X     X X X X                               X
-   MTU update                   X
-   jumbo frame                  X     X X X X
-   scattered Rx                 X     X X X X                               X
+   ==================== = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
+   Feature              a b b b c e e e i i i i i i i i i i f f m m m n n p r s v v v x
+                        f n n o x 1 n n 4 4 4 4 g g x x x x m m l l p f u c i z i i m e
+                        p x x n g 0 a i 0 0 0 0 b b g g g g 1 1 x x i p l a n e r r x n
+                        a 2 2 d b 0   c e e e e   v b b b b 0 0 4 5 p   l p g d t t n v
+                        c x x i e 0       . v v   f e e e e k k     e         a i i e i
+                        k   v n           . f f       . v v   .               t o o t r
+                        e   f g           .   .       . f f   .               a   . 3 t
+                        t                 v   v       v   v   v               2   v
+                                          e   e       e   e   e                   e
+                                          c   c       c   c   c                   c
+   ==================== = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
+   link status                  X       X X                                   X
+   link status event                    X X
+   Rx interrupt                         X X X X
+   queue start/stop             X   X   X X X X                               X
+   MTU update                   X   X
+   jumbo frame                  X   X   X X X X
+   scattered Rx                 X   X   X X X X                               X
    LRO
-   TSO                          X     X X X X
-   promiscuous mode             X     X X X X                               X
-   allmulticast mode            X     X X X X                               X
-   unicast MAC filter                 X X X X
-   multicast MAC filter               X X X X
-   RSS hash                     X     X X X X
-   RSS key update                     X X X X
-   RSS reta update                    X X X X
-   VMDq                               X X
-   SR-IOV                             X X
-   DCB                                X X
-   VLAN filter                        X X X X
-   ethertype filter                   X X
+   TSO                          X   X   X X X X
+   promiscuous mode             X       X X X X                               X
+   allmulticast mode            X       X X X X                               X
+   unicast MAC filter                   X X X X
+   multicast MAC filter                 X X X X
+   RSS hash                     X   X   X X X X
+   RSS key update                   X   X X X X
+   RSS reta update                  X   X X X X
+   VMDq                                 X X
+   SR-IOV                           X   X X
+   DCB                                  X X
+   VLAN filter                          X X X X
+   ethertype filter                     X X
    n-tuple filter
    SYN filter
-   tunnel filter                      X X
+   tunnel filter                        X X
    flexible filter
-   hash filter                        X X X X
-   flow director                      X X
-   flow control                 X     X X
+   hash filter                          X X X X
+   flow director                        X X
+   flow control                 X       X X
    rate limitation
-   traffic mirroring                  X X
-   CRC offload                  X     X   X
-   VLAN offload                 X     X   X
-   QinQ offload                       X   X
-   L3 checksum offload          X     X   X
-   L4 checksum offload          X     X   X
-   inner L3 checksum                  X   X
-   inner L4 checksum                  X   X
-   packet type parsing          X     X   X
-   timesync                           X X
-   basic stats                  X     X X X X                               X
-   extended stats                     X X X X
-   stats per queue              X                                           X
+   traffic mirroring                    X X
+   CRC offload                  X   X   X   X
+   VLAN offload                 X   X   X   X
+   QinQ offload                         X   X
+   L3 checksum offload          X   X   X   X
+   L4 checksum offload          X   X   X   X
+   inner L3 checksum                X   X   X
+   inner L4 checksum                X   X   X
+   packet type parsing          X       X   X
+   timesync                             X X
+   basic stats                  X   X   X X X X                               X
+   extended stats                   X   X X X X
+   stats per queue              X                                             X
    EEPROM dump
    registers dump
-   multiprocess aware                 X X X X
-   BSD nic_uio                  X     X X X X
-   Linux UIO                    X     X X X X
-   Linux VFIO                   X     X X X X
-   other kdrv                                                               X
+   multiprocess aware                   X X X X
+   BSD nic_uio                  X       X X X X
+   Linux UIO                    X   X   X X X X
+   Linux VFIO                   X       X X X X
+   other kdrv                                                                 X
    ARMv7
    ARMv8
    Power8
    TILE-Gx
-   x86-32                       X     X X X X
-   x86-64                       X     X X X X                               X
-   usage doc                    X                                           X
+   x86-32                       X   X   X X X X
+   x86-64                       X   X   X X X X                               X
+   usage doc                    X                                             X
    design doc
    perf doc
-   ==================== = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
+   ==================== = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
-- 
1.9.1



More information about the dev mailing list