[dpdk-dev] [PATCH] eal/vfio: share the default container in multi process
anatoly.burakov at intel.com
Fri Oct 5 10:52:28 CEST 2018
On 04-Oct-18 4:52 PM, Burakov, Anatoly wrote:
> On 03-Oct-18 1:39 PM, Darek Stojaczyk wrote:
>> So far each process in MP used to have a separate container
>> and relied on the primary process to register all memsegs.
>> Mapping external memory via rte_vfio_container_dma_map()
>> in secondary processes was broken, because the default
>> (process-local) container had no groups bound. There was
>> even no way to bind any groups to it, because the container
>> fd was deeply encapsulated within EAL.
>> This patch introduces a new SOCKET_REQ_DEFAULT_CONTAINER
>> message type for MP synchronization, makes all processes
>> within a MP party use a single default container, and hence
>> fixes rte_vfio_container_dma_map() for secondary processes.
>>> From what I checked this behavior was always the same, but
>> started to be invalid/insufficient once mapping external
>> memory was allowed.
>> Fixes: 73a639085938 ("vfio: allow to map other memory regions")
>> Cc: anatoly.burakov at intel.com
>> Cc: pawelx.wodkowski at intel.com
>> Cc: gowrishankar.m at linux.vnet.ibm.com
>> Cc: stable at dpdk.org
>> While here, fix up the comment on rte_vfio_get_container_fd().
>> This function always opens a new container, never reuses
>> an old one.
>> Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk at intel.com>
> This makes things a bit funky when we consider external memory support.
> The same address space may or may not be mapped into the process, so we
> may or may not need to map things for DMA depending on whether we're
> mapping shared memory for DMA, or not.
> The previous behavior may be counter-intuitive, but it's IMO the correct
> one - each process has its own container and performs its own DMA mappings.
Re-reading the commit message, i can see that i've misinterpreted the
commit a bit (or rather i missed the fact that we can't even map
anything for DMA in the secondaries due to container being not the same
container we use devices for).
However, my above comment regarding external memory still applies - if
we share DMA mappings between processes, it will be possible for
secondary to map something else into space occupied by external memory,
and potentially overwrite the DMA mappings.
Unfortunately, we cannot do anything about it, as that memory isn't
under our control. I will have to update my external memory patchset to
account for this and map memory for DMA only once, in whatever process
that creates the memory in the first place.
So, on the concept of this i have no objections here, as i think this
fix is necessary. I'll have to review the code some more to make sure
i'm not missing anything.
More information about the dev