[dpdk-dev] [PATCH v2 3/6] bus: introduce device level DMA memory mapping

[Burakov, Anatoly]

On 28-Feb-19 12:14 PM, Burakov, Anatoly wrote:
> On 21-Feb-19 2:50 PM, Shahaf Shuler wrote:
>> The DPDK APIs expose 3 different modes to work with memory used for DMA:
>>
>> 1. Use the DPDK owned memory (backed by the DPDK provided hugepages).
>> This memory is allocated by the DPDK libraries, included in the DPDK
>> memory system (memseg lists) and automatically DMA mapped by the DPDK
>> layers.
>>
>> 2. Use memory allocated by the user and register to the DPDK memory
>> systems. Upon registration of memory, the DPDK layers will DMA map it
>> to all needed devices. After registration, allocation of this memory
>> will be done with rte_*malloc APIs.
>>
>> 3. Use memory allocated by the user and not registered to the DPDK memory
>> system. This is for users who wants to have tight control on this
>> memory (e.g. avoid the rte_malloc header).
>> The user should create a memory, register it through rte_extmem_register
>> API, and call DMA map function in order to register such memory to
>> the different devices.
>>
>> The scope of the patch focus on #3 above.
>>
>> Currently the only way to map external memory is through VFIO
>> (rte_vfio_dma_map). While VFIO is common, there are other vendors
>> which use different ways to map memory (e.g. Mellanox and NXP).
>>
>> The work in this patch moves the DMA mapping to vendor agnostic APIs.
>> Device level DMA map and unmap APIs were added. Implementation of those
>> APIs was done currently only for PCI devices.
>>
>> For PCI bus devices, the pci driver can expose its own map and unmap
>> functions to be used for the mapping. In case the driver doesn't provide
>> any, the memory will be mapped, if possible, to IOMMU through VFIO APIs.
>>
>> Application usage with those APIs is quite simple:
>> * allocate memory
>> * call rte_extmem_register on the memory chunk.
>> * take a device, and query its rte_device.
>> * call the device specific mapping function for this device.
>>
>> Future work will deprecate the rte_vfio_dma_map and rte_vfio_dma_unmap
>> APIs, leaving the rte device APIs as the preferred option for the user.
>>
>> Signed-off-by: Shahaf Shuler <shahafs at mellanox.com>
>> ---
> 
> <snip>
> 
>> +
>> +    if (!pdev || !pdev->driver) {
>> +        rte_errno = EINVAL;
>> +        return -rte_errno;
>> +    }
> 
> We could put a check in here to see if the memory has been registered 
> with DPDK. Just call rte_mem_virt2memseg_list(addr) - if it returns 
> NULL, the memory wasn't registered, so you can throw an error. Not sure 
> of appropriate errno in that case - ENODEV? EINVAL?

Apologies - i meant to delete that, but hit one ctrl+Z too many :(

-- 
Thanks,
Anatoly