[dpdk-dev] [RFC] pci: force address of mappings in secondary process

[Sergio Gonzalez Monroy]

On 11/07/2017 21:00, Stephen Hemminger wrote:
> On Tue, 11 Jul 2017 12:35:39 +0100
> Sergio Gonzalez Monroy <sergio.gonzalez.monroy at intel.com> wrote:
>
>> On 11/07/2017 02:56, Tan, Jianfeng wrote:
>>>> -----Original Message-----
>>>> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Stephen
>>>> Hemminger
>>>> Sent: Tuesday, July 11, 2017 9:13 AM
>>>> To: dev at dpdk.org
>>>> Cc: Stephen Hemminger
>>>> Subject: [dpdk-dev] [RFC] pci: force address of mappings in secondary
>>>> process
>>>>
>>>> The PCI memory resources in the secondary process should be in
>>>> the exact same location as the primary process. Otherwise
>>>> there is a risk of a stray pointer.
>>>>
>>>> Not sure if this is right, but it looks like a potential
>>>> problem.
>>>>
>>>> ---
>>>>    lib/librte_eal/common/eal_common_pci_uio.c | 2 +-
>>>>    1 file changed, 1 insertion(+), 1 deletion(-)
>>>>
>>>> diff --git a/lib/librte_eal/common/eal_common_pci_uio.c
>>>> b/lib/librte_eal/common/eal_common_pci_uio.c
>>>> index 367a6816dcb8..2156b1a436c4 100644
>>>> --- a/lib/librte_eal/common/eal_common_pci_uio.c
>>>> +++ b/lib/librte_eal/common/eal_common_pci_uio.c
>>>> @@ -77,7 +77,7 @@ pci_uio_map_secondary(struct rte_pci_device *dev)
>>>>
>>>>    			void *mapaddr = pci_map_resource(uio_res-
>>>>> maps[i].addr,
>>>>    					fd, (off_t)uio_res->maps[i].offset,
>>>> -					(size_t)uio_res->maps[i].size, 0);
>>>> +					(size_t)uio_res->maps[i].size,
>>>> MAP_FIXED);
>>>>    			/* fd is not needed in slave process, close it */
>>>>    			close(fd);
>>>>    			if (mapaddr != uio_res->maps[i].addr) {
>>>> --
>>>> 2.11.0
>>> +1 for this RFC. I also once encounter such problem, and I use the same way to solve it. The addr parameter of mmap() syscall is only a hint instead of a must even the VMA is not occupied yet.
>>>
>>> Thanks,
>>> Jianfeng
>> How do you know the VMA is not occupied?
>>
>> I think the risk here is that the dynamic linker loaded some shared
>> library in that VMA, and forcing MAP_FIXED is not a safe solution.
>> What I have observed is that Linux will return a different VMA than the
>> one hinted when there is already a mapping in the requested/hinted VMA.
>>
>> I reckon this is a similar issue as we have with the multi-process model
>> when we do not get the VMA requested for the huge-pages.
>> AFAIK we do not have a robust solution for this issue other than restart
>> the program and hope the dynamic linker does not map anything in the VMA
>> ranges that we need to map from the primary. This is also assuming that
>> the application does not allocate memory and maps things before calling
>> eal_init as it could potentially use VMA ranges that we need in the
>> secondary process.
>>
>> The proposal for new secondary process model would solve these issues:
>> http://dpdk.org/ml/archives/dev/2017-May/066147.html
>>
>> Thanks,
>> Sergio
> That proposal defeats some of the isolation of secondary process model.
> The idea is that secondary could be built separately. It is also overly
> complex and would make a somewhat fragile part of the DPDK, more difficult.
>

I did not mean/want to change the focus of this thread.
Those are valid concerns which should be discussed in its own thread.

Thanks,
Sergio