[dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an offload

[Andrew Rybchenko]

On 11/8/19 4:17 PM, Thomas Monjalon wrote:
> 08/11/2019 13:00, Andrew Rybchenko:
>> On 11/8/19 2:03 PM, Thomas Monjalon wrote:
>>> 08/11/2019 11:42, Andrew Rybchenko:
>>>> On 11/8/19 1:28 PM, Thomas Monjalon wrote:
>>>>> 08/11/2019 09:35, Andrew Rybchenko:
>>>>>> The problem:
>>>>>> ~~~~~~~~~~~~
>>>>>> PMD wants to know before port start if application wants to
>>>>>> to use flow MARK/FLAG in the future. It is required because:
>>>>>>
>>>>>> 1. HW may be configured in a different way to reserve resources
>>>>>>    for MARK/FLAG delivery
>>>>>>
>>>>>> 2. Datapath implementation choice may depend on it (e.g. vPMD
>>>>>>    is faster, but does not support MARK)
>>>>> Thank you for the clear problem statement.
>>>>> I agree with it. This is a real design issue.
>>>>>
>>>>>
>>>>>> Discussed solutions:
>>>>>> ~~~~~~~~~~~~~~~~~~~~
>>>> May be it is not 100% clear since below are alternatives.
>>>>
>>>>>> A. Explicit Rx offload suggested by the patch.
>>>>>>
>>>>>> B. Implicit by validation of a flow rule with MARK/FLAG actions used.
>>>>>>
>>>>>> C. Use dynamic field/flag (i.e. application registers dynamic field
>>>>>>    and/or flag and PMD uses lookup to solve the problem) plus part
>>>>>>    of (B) to discover if a feature is supported.
>>>>> The dynamic field should be registered via a new API function
>>>>> named '<feature>_init'.
>>>>> It means the application must explicit request the feature.
>>>>> I agree this is the way to go.
>>>> If I understand your statement correctly, but (C) is not ideal since it
>>>> looks global. If registered dynamic field of mbuf and is flag that
>>>> the feature should be enabled, it is a flag to all ports/devices.
>>>>
>>>>>> All solutions require changes in applications which use these
>>>>>> features. There is a deprecation notice in place which advertises
>>>>>> DEV_RX_OFFLOAD_FLOW_MARK addition, but may be it is OK to substitute
>>>>>> it with solution (B) or (C). Solution (C) requires changes since
>>>>>> it should be combined with (B) in order to understand if
>>>>>> the feature is supported.
>>>>> I don't understand.
>>>>> Application request and PMD support are two different things.
>>>>> PMD support must be via rte_flow validation on a case by case anyway.
>>>> I mean that application wants to understand if the feature is
>>>> supported. Then, it wants to enable it. In the case of (B),
>>>> if I understand the solution correctly, there is no explicit
>>>> way to enable, PMD just detects it because of discovery is done
>>>> (that's what I mean by "implicit" and it is a drawback from my
>>>> point of view, but still could be considered). (C) solves the
>>>> problem of (B).
>>>>
>>>>>> Advantages and drawbacks of solutions:
>>>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>>>> 1. The main drawback of (A) is a "duplication" since we already
>>>>>>    have a way to request flow MARK using rte_flow API.
>>>>>>    I don't fully agree that it is a duplication, but I agree
>>>>>>    that it sounds like duplication and complicates a bit flow
>>>>>>    MARK usage by applications. (B) complicates it as well.
>>>>>>
>>>>>> 2. One more drawback of the solution (A) is the necessity of
>>>>>>    similar solution for META and it eats one more offload bit.
>>>>>>    Yes, that's true and I think it is not a problem.
>>>>>>    It would make it easier for applications to find out if
>>>>>>    either MARK or META is supported.
>>>>>>
>>>>>> 3. The main advantage of the solution (A) is simplicity.
>>>>>>    It is simple for application to understand if it supported.
>>>>>>    It is simple in PMD to understand that it is required.
>>>>>>    It is simple to disable it - just reconfigure.
>>>>>>    Also it is easier to document it - just mention that
>>>>>>    the offload should be supported and enabled.
>>>>>>
>>>>>> 4. The main advantage of the solution (B) is no "duplication".
>>>>>>    I agree that it is valid argument. Solving the problem
>>>>>>    without extra entities is always nice, but unfortunately
>>>>>>    it is too complex in this case.
>>>>>>
>>>>>> 5. The main drawback of the solution (B) is the complexity.
>>>>>>    It is necessary to choose a flow rule which should be used
>>>>>>    as a criteria. It could be hardware dependent.
>>>>>>    Complex logic is require in PMD if it wants to address the
>>>>>>    problem and control MARK delivery based on validated flow
>>>>>>    rules. It adds dependency between start/stop processing and
>>>>>>    flow rules validation code.
>>>>>>    It is pretty complicated to document it.
>>>>>>
>>>>>> 6. Useless enabling of the offload in the case of solution (A)
>>>>>>    if really used flow rules do not support MARK looks like
>>>>>>    drawback as well, but easily mitigated by a combination
>>>>>>    with solution (B) and only required if the application wants
>>>>>>    to dive in the level of optimization and complexity and
>>>>>>    makes sense if application knows required flow rules in
>>>>>>    advance. So, it is not a problem in this case.
>>>>>>
>>>>>> 7. Solution (C) has drawbacks of the solution (B) for
>>>>>>    applications to understand if these features are supported,
>>>>>>    but no drawbacks in PMD, since explicit criteria is used to
>>>>>>    enable/disable (dynamic field/flag lookup).
>>>>>>
>>>>>> 8. Solution (C) is nice since it avoids "duplication".
>>>>>>
>>>>>> 9. The main drawback of the solution (C) is asymmetry.
>>>>>>    As it was discussed in the case of RX_TIMESTAMP
>>>>>>    (if I remember it correctly):
>>>>>>     - PMD advertises RX_TIMESTAMP offload capability
>>>>>>     - application enables the offload
>>>>>>     - PMD registers dynamic field for timestamp
>>>>>>    Solution (C):
>>>>>>      - PMD advertises nothing
>>>>>>      - application uses solution (B) to understand if
>>>>>>        these features are supported
>>>>>>      - application registers dynamic field/flag
>>>>>>      - PMD does lookup and solve the problem
>>>>>>    The asymmetry could be partially mitigated if RX_TIMESTAMP
>>>>>>    solution is changed to require an application to register
>>>>>>    dynamic fields and PMD to do lookup if the offload is
>>>>>>    enabled. So, the only difference will be in no offload
>>>>>>    in the case of flow MARK/FLAG and usage of complex logic
>>>>>>    to understand if it is supported or no.
>>>>>>    May be it would be really good since it will allow to
>>>>>>    have dynamic fields registered before mempool population.
>>>>>>
>>>>>> 10. Common drawback of solutions (B) and (C) is no granularity.
>>>>>>     Solution (A) may be per queue while (B) and (C) cannot be
>>>>>>     per queue. Moreover (C) looks global - for all devices.
>>>>>>     It could be really painful.
>>>>>>
>>>>>> (C) is nice, but I still vote for simplicity and
>>>>>> granularity of (A).
>>>>> I vote for clear separation of application needs and PMD support,
>>>>> by using the method C (dynamic fields).
>>>>> I agree timestamp must use the same path.
>>>>> I agree it's complicate because we don't know in advance whether
>>>>> a flow rule will be accepted, but that's the reality, config is complex.
>>>> Do you think that global nature of the (C) is acceptable?
>>> That's a good question.
>>> Maybe the feature request should be per port.
>>> In this case, we are back to solution A with a flag per port?
>> Offloads are natively per-queue as well, so (A) keeps the choice
>> between per-port vs per-queue to PMDs as usual.
>>
>>> Note that A and C will not guarantee that the offload will be possible.
>> Yes, definitely.
>>
>>> We need B (flow rule validation) anyway.
>> Strictly speaking (B) (checking flow rules before device
>> startup) is required if an application can predict flow
>> rules and wants to ensure that MARK offload will be usable.
>> Otherwise, it may be skipped.
> No no, I mean flow rule validation MUST be used anyway
> during the runtime before applying a rule.
> I agree it is hard to predict. I speak only about real rules.

OK, I see. Of course, flow rule validation is required at runtime.
I was rather concentrated on the stated problem solutions.

>>> It seems A, B, C are not alternatives but all required
>>> as pieces of a puzzle...
>> Unfortunately true in the most complex case.
>> Right now it will be A with B if required as explained above.
>> C will come a bit later when the field migrates to dynamic.
>>
>> May be it is even better if application registers dynamic
>> fields before an attempt to enable offload to be sure that
>> it will not fail because of impossibility to register
>> dynamic field (lack of space). I'm not sure, but it is not
>> not that important.
> Yes of course, lack of mbuf space is another reason for
> disabling the feature.
>
>> If we finally go way A, should we add offloads for META back?
>> I guess separate Rx and Tx are required.
> I would prefer to add it as dynamic flags.
> Why rushing on a very temporary solution while it is not a new issue?

Basically it means that we go just (B)+(C) in the case of META.
I have no strong opinion but thought that it could be better to
align the solution. Of course, we can wait with it. As I understand
META is an experimental feature.