[dpdk-dev] [PATCH v4 06/11] dma/ioat: add data path job submission functions

[Conor Walsh]

On 20/09/2021 14:36, Bruce Richardson wrote:
> On Fri, Sep 17, 2021 at 03:42:22PM +0000, Conor Walsh wrote:
>> Add data path functions for enqueuing and submitting operations to
>> IOAT devices.
>>
>> Signed-off-by: Conor Walsh <conor.walsh at intel.com>
>> Reviewed-by: Kevin Laatz <kevin.laatz at intel.com>
>> ---
>>   doc/guides/dmadevs/ioat.rst    | 54 ++++++++++++++++++++
>>   drivers/dma/ioat/ioat_dmadev.c | 92 ++++++++++++++++++++++++++++++++++
>>   2 files changed, 146 insertions(+)
>>
>> diff --git a/doc/guides/dmadevs/ioat.rst b/doc/guides/dmadevs/ioat.rst
>> index a64d67bf89..2464207e20 100644
>> --- a/doc/guides/dmadevs/ioat.rst
>> +++ b/doc/guides/dmadevs/ioat.rst
>> @@ -89,3 +89,57 @@ The following code shows how the device is configured in ``test_dmadev.c``:
>>   
>>   Once configured, the device can then be made ready for use by calling the
>>   ``rte_dma_start()`` API.
>> +
>> +Performing Data Copies
>> +~~~~~~~~~~~~~~~~~~~~~~~
>> +
>> +To perform data copies using IOAT dmadev devices, the functions
>> +``rte_dma_copy()`` and ``rte_dma_submit()`` should be used. Alternatively
>> +``rte_dma_copy()`` can be called with the ``RTE_DMA_OP_FLAG_SUBMIT`` flag
>> +set.
>> +
>> +The ``rte_dma_copy()`` function enqueues a single copy to the
>> +device ring for copying at a later point. The parameters to the function
>> +include the device ID of the desired device, the virtual DMA channel required
>> +(always 0 for IOAT), the IOVA addresses of both the source and destination
>> +buffers, the length of the data to be copied and any operation flags. The
>> +function will return the index of the enqueued job which can be use to
>> +track that operation.
>> +
>> +While the ``rte_dma_copy()`` function enqueues a copy operation on the device
>> +ring, the copy will not actually be performed until after the application calls
>> +the ``rte_dma_submit()`` function. This function informs the device hardware
>> +of the elements enqueued on the ring, and the device will begin to process them.
>> +It is expected that, for efficiency reasons, a burst of operations will be
>> +enqueued to the device via multiple enqueue calls between calls to the
>> +``rte_dma_submit()`` function. If desired you can pass the
>> +``RTE_DMA_OP_FLAG_SUBMIT`` flag when calling ``rte_dma_copy()`` and this will
>> +tell the device to perform the enqueued operation and any unperformed operations
>> +before it. The ``RTE_DMA_OP_FLAG_SUBMIT`` flag can be passed instead of calling
>> +the ``rte_dma_submit()`` function for example on the last enqueue of the burst.
>> +
>> +The following code from demonstrates how to enqueue a burst of copies to the
>> +device and start the hardware processing of them:
>> +
>> +.. code-block:: C
>> +
>> +   for (i = 0; i < BURST_SIZE; i++) {
>> +      if (rte_dma_copy(dev_id, vchan, rte_mbuf_data_iova(srcs[i]),
>> +            rte_mbuf_data_iova(dsts[i]), COPY_LEN, 0) < 0) {
>> +         PRINT_ERR("Error with rte_dma_copy for buffer %u\n", i);
>> +         return -1;
>> +      }
>> +   }
>> +   if (rte_dma_submit(dev_id, vchan) < 0) {
>> +      PRINT_ERR("Error with performing operations\n", i);
>> +      return -1;
>> +   }
>> +
>> +Filling an Area of Memory
>> +~~~~~~~~~~~~~~~~~~~~~~~~~~
>> +
>> +The driver also has support for the ``fill`` operation, where an area
>> +of memory is overwritten, or filled, with a short pattern of data.
>> +Fill operations can be performed in much the same was as copy operations
>> +described above, just using the ``rte_dma_fill()`` function rather
>> +than the ``rte_dma_copy()`` function.
> Similar to the feedback on the idxd driver, I think we need to see how much
> of this text is already present in the generic dmadev documentation and
> re-use or reference that. If it's not present, then these patches should
> add it to the common doc, not a separate driver-specific doc.
>
> /Bruce

I will work with Kevin to rewrite these to reduce the amount of 
duplication between our drivers and for future drivers in the next version.

Thanks,

Conor.