[dpdk-dev] [PATCH v2 0/3] ethdev: configure SR-IOV VF from host

[Ilya Maximets]

On 04.11.2019 21:33, Shahaf Shuler wrote:
> Monday, November 4, 2019 12:28 PM, Ilya Maximets:
>> Subject: Re: [dpdk-dev] [PATCH v2 0/3] ethdev: configure SR-IOV VF from
>> host
>>
>> On 03.11.2019 7:48, Shahaf Shuler wrote:
>>> Friday, November 1, 2019 11:33 AM, Ilya Maximets:
>>>> Subject: Re: [dpdk-dev] [PATCH v2 0/3] ethdev: configure SR-IOV VF
>>>> from host
>>>>
>>>> On 30.10.2019 22:42, Thomas Monjalon wrote:
>>>>> 30/10/2019 17:09, Ilya Maximets:
>>>>>> On 30.10.2019 16:49, Thomas Monjalon wrote:
>>>>>>> 30/10/2019 16:07, Ilya Maximets:
>>>>>>>> On 29.10.2019 19:50, Thomas Monjalon wrote:
>>>>>>>>> In a virtual environment, the network controller may have to
>>>>>>>>> configure some SR-IOV VF parameters for security reasons.
>>>>>>>>>
>>> [...]
>>>
>>>>> If we consider what Intel did, i.e. configure VF in place of
>>>>> representor for some operations, there are two drawbacks:
>>>>> - confusing that some ops apply to representor, others apply to VF
>>>>> - some ops are not possible on representor (because targetted to VF)
>>>>>
>>>>> I still feel that the addition of one single bit in the port ID is
>>>>> an elegant solution to target either the VF or its representor.
>>>>
>>>> Since we already have a confusion about what is configured when
>>>> operations are performed on a representor port we have 2 options:
>>>
>>> I don't agree we have. I don't think there is any design note or API doc that
>> says the ethdev configuration on representor should be applied on VF
>> (please share if I missed it).
>>> The fact that there are some drivers that implemented it doesn't mean it is
>> correct.
>>>
>>>> 1. Have this proposed API to configure representor itself while
>>>>       setting config to representor and configuring VF if special
>>>>       bit enabled.
>>>> 2. Reverse the logic of current proposal, i.e. always apply
>>>>       configuration to VF while working with representor and apply
>>>>       configuration to representor itself if special bit is set.
>>>>
>>>> I'd probably prefer option #2, because:
>>>> - From the OVS and OpenStack point of view, I think, we don't
>>>>      really need to configure representor itself in most cases.
>>>>      And OVS really should not know if it works with representor
>>>>      or some real port.
>>>
>>> I don't thinks OVS can be really agnostic to the fact it runs on top of
>> representors:
>>> 1. probing of representor has different command line -w
>>> <bdf>,representor=XXX
>>
>> OVS doesn't care about content of devargs. It just passes them to hotplug
>> engine without any parsing (except a single case that must be eliminated
>> with a proper device iterators, not an OVS issue).
>>
>>> 2. the whole acceleration framework based on insertion of flow rules for
>> direct forward from the VF to target entity. Rules are applied on the
>> representor and would not work if port is not such.
>>
>> OVS tries to offload rules to the netdev from which packet was received.
>> That's it.  If it succeeds - OK.  If not, OVS doesn't care.
>>
>>> 3. some multi-port devices cannot do direct fwd between its different port.
>> This is why rep has switch_id and application should query it and act upon.
>>
>> This is part of offloading engine that doesn't affect the generic code.
>> If needed, OVS could request switch_id for netdev it tries to offload rules on.
>> OVS should not know if it representor port or not. If this operation will not
>> succeed for non-representors, OVS should not care because we can't offload
>> anything for non-representors anyway.
> 
> I am not speaking on specific module in OVS if it is aware or not, rather OVS as a whole.
> There is a big code part in OVS that its sole purpose is to create flow rule on representors.
> There is no meaning for this code w/o representors and it should be used only when OVS runs on top of representors (otherwise I would expect bad perf due to the constant preparation of rte_flow rules).

I tend to disagree with that statement.  You're missing cases like
MARK+RSS actions which is the only offloading type supported now
in OVS userspace datapath.  Few more cases I can think of are ingress
policing, access control, some fast rx drop rules that could be used
directly with any HW port that supports offloading, not only representors.
I don't know if all of this is possible right now with DPDK, but these
are definitely valid use cases.

Second point is that current implementation in OVS tries to offload
to any port, and you were one of the co-authors of that code.
After re-working of offloading achitecture that was done this year it's
possible to probe support of the feature before enabling offloading
for particular port, but it's 'TODO' item that is not implemented.

> 
>>
>>> 4. representor carry the VF port id. This is how application know to which
>> VF (or vport) they associated with on their other side.
>>
>> This is just part of devargs,
> 
> It is not part of devargs, it is part of the ethdev info.
> OVS should expose such info to the user in order to help w/ the open flow rules creation.

This info already exposed via devargs that are stored in ovsdb.

> 
> i.e. part of device unique identifier.
>> Once again, OVS doesn't parse devargs and should not do that.
> 
> So how can OVS user know the mapping between representor and its VF?

User already knows that because he/she is the one who wrote devargs.
The case is that OVS should not care about that. If users doesn't
want to check devargs passed to OVS to remember which port is the
representor of which VF, he/she could use meaningful port names.

> 
>>
>>>
>>>> - It seems that most of the existing code in DPDK already works
>>>>      like this, i.e. applying configs to VF itself.  Intel drivers
>>>>      works like this and  Mellanox drivers, as Thomas said, doesn't
>>>>      have this functionality at all.
>>>
>>> As I said above, I don't think we need to refer to specific driver behavior,
>> rather the API guidelines.
>>> To me, it is a bit strange and not natural that ethdev configuration is applied
>> to different port w/o any explicit request from the application.
>>> This is why I would prefer #1 above.
>>
>> IMHO, the whole concept of representors is that representor is a way of
>> attaching same port both to VM and vSwitch/hypervisor.
>>
>> If you're looking at representors as a separate ports on a switch, well..
> 
> I think we cannot ignore the fact that both in Linux and DPDK those are netdev/ethdev and as such holds some network configuration (even though on some case it is very minimal or some random values).
> 
>> In this case, for me VF configuration looks like something that vSwitch should
>> not do at all, because it should not configure ports that doesn't attached to it.
>> It's like configuring the other side of veth pair, which is nonsense.
> 
> It seems to me we configure the remote VF on every possible solution. The suggest one discussed here just make it more explicit.
> 
>>
>>
>> BTW, I don't know a way to find out if port is a representor of something or
>> not in Linux kernel.
> 
> One example is by querying the netdev phys_port_name. representor ports will have specific strings to describe the remote end (e.g. pf0vf0) while regular port (e.g. uplink) will have pf0.
>