[dpdk-dev] Question on mlx5 PMD txq memory registration

[Sagi Grimberg]

>> I don't understand the security argument. Its completely private to the
>> driver. anything under librte is equivalent to an OS wrt networking, so I fail to
>> see what is the security feature your talking about.
> 
> You are correct that as a root you are able to do whatever you want on the server.
> The security I refer to is to protect against badly written code.
> 
> The fact the PMD only registers the mempools, and use the device engine to translate the VA, provide some protection.
> For example, one DPDK process will not be able to access the memory of other DPDK process *by mistake*.

Well, this is a fair argument, but without a *complete* solution for all
of dpdk peripherals, it has very little merit (if at all). A badly
written code can just as easily crash a server by passing a mbuf to
a crypto device or another network device that co-exists with mlx5.

So, while I understand the argument, I think its value is not worth the
hassle that mlx5_pmd needs to take to achieve it. Did this come from a
real requirement (from a real implementation)?

> I am not saying using the reserved lkey is not a good suggestion, and we plan to test its value.
> All I am saying is there are maybe other option to provide the same performance with the extra protection mentioned above.
> One of them can be to use indirect keys. One indirect key to represent 64b memory area, and other regular keys for the hugepages.
> The rest of the memory area can be filled with keys pointing to /dev/null.

If I understand what you are suggesting, this would trigger out-of-order
transfers on an indirect memory key just about always (each transfer can 
originate from a different hugepage and SGL resolution alone will
require a walk on the memory key context SGL list). I'm afraid this
would introduce a very bad performance scaling due to the fact that a
SGL context (klm) will need to be fetched from the ICM for essentially
every send operation.

Having said that, its just my 2 cents, if your solution works then I
don't really care. You are the one testing it...

>>> The fact running under root doesn't mean you have privileges to access
>> every physical page on the server (even if you try very hard to be aware).
>>
>> But dpdk core mbufs structs are built this way.
>>
>>> The issue here, AFAIU, is performance.
>>> We are now looking into ways to provide the same performance as if it was
>> only a single lkey, while preserving the security feature.
>>
>> Hmm, What exactly do you have in mind?
>>
>> I'm hoping that you are not referring to ODP. If you are, I think that latency
>> unpredictability would be a huge non-starter, page-faults are way too
>> expensive for dpdk users.
> 
> No ODP :).
> As all relevant DPDK memory is on top of hugepages, there is no reason to avoid registration and pinning  in advance.

Agree.