[dpdk-dev] [RFC v2 3/5] ether: Add flow timeout support

[Alex Rosenbaum]

On Tue, Dec 26, 2017 at 5:28 AM, Zhang, Qi Z <qi.z.zhang at intel.com> wrote:
>> On Fri, Dec 22, 2017 at 11:03 AM, Zhang, Qi Z <qi.z.zhang at intel.com> wrote:
>> >> On Thu, Dec 21, 2017 at 4:35 AM, Qi Zhang <qi.z.zhang at intel.com> wrote:
>> >> > Add new APIs to support flow timeout, application is able to 1.
>> >> > Setup the time duration of a flow, the flow is expected to be
>> >> > deleted automatically when timeout.
>> >>
>> >> Can you explain how the application (OVS) is expected to use this API?
>> >> It will help to better understand the motivation here...
>> >
>> > I think the purpose of the APIs is to expose the hardware feature that
>> > support flow auto delete with a timeout.
>> > As I know, for OVS, every flow in flow table will have time duration A
>> > flow be offloaded to hardware is still required to be deleted in
>> > specific time, I think these APIs help OVS to take advantage HW
>> > feature and simplify the flow aging management
>>
>> Are you sure this will allow OVS to 'fire-and-forget' about the rule removal?
>> or will OVS anyway do rule cleanup from application tables?
>
> There is some framework design about offload flow management on OVS side.
> Since I'm not a OVS guy, I can't answer OVS specific question precisely right now,
> but the feedback I got is, it will be nice if rte_flow could support flow timeout
> I may check with some OVS expert to give further explanation.
> BTW, I think there is no harmful to add these APIs into rte_flow, since a flow timeout is quite
> generic feature to me. it may be useful even for non-OVS case in future.

I'm not a core OVS guy ether :) but adding a feature to DPDK because
it "might be nice" and not a "real benefit" does not sound like a
right approach to me. Each new feature will add extra
work/support/bugs for something which might not really used. I think
it is critical that OVS guys provide you a clear evidence how this
will help.
And we need to try and make this generic so other then OVS application
can use this. e.g. adding a re-arm to the timeout.


>> Do you know if OVS flow timers are (or can be) re-armed in different use
>> cases? e.g. extending the timeout duration if traffic is still flowing?
>
> As I know, for OVS every flow just has a fixed time duration, so "hard_timeout"
> is going for this requirement, but by following OpenFlow spec, idle_timeout is paired
> with hard_timeout so I just add it since its generic and maybe useful for future.

Yes, I also heard OF does issue a hard timeout value.
But I also understood from OVS guys that OVS will manage the timeout
internally. OVS will do traffic sampling for activity before deleting
the flow. so that the timeout value needs to be updated constantly
depending on connection state information (re you other patch exposing
the last_seen_timeout).
So the current suggestion, based on hard timeout, will not fit OVS. At
least as I understood from other OVS guys.
You can see that Kernel OVS guys did not add this to TC flower offload
support ether. I am sure they would have if it would have improved
performance.
That will be a fame if we miss the target of the feature.


>> >> Are you trying to move the aging timer from application code into the
>> PMD?
>> >> or can your HW remove/disable/inactivate a flow at certain time
>> >> semantics without software context?
>> >
>> > Yes, it for hardware feature.
>>
>> So if the hardware auto removes the hardware steering entry, what software
>> part deletes the rte_flow handle?
>> What software part triggers the application callback? from what context? will
>> locks be required?
>> How do you prevent races between application thread and the context
>> deleting/accessing the rte_flow handle?
>> I mean in cases that application wants to delete the flow before the timeout
>> expires, but actually it is same time hardware deletes it.
>
> Usually the flow auto delete is running on a separate background thread
> (an interrupt handler or a watchdog thread base on hardware capability)
> The low level driver is responsible to take care of the race condition between background and foreground flow deleting.

Please explain in details how the race is solved. Maybe a patch will
make this clearer? This is one of the main objection for this feature.
e.g.: If the application holds a rte_flow handle, and the timeout
expired, is not handled yet. Now the application will try to use the
rte_flow which state changed underneath, or even got deleted.

Besides you now require each and every low level driver to add this
same timeout expiration logic, alarm registration and race protection.
This should be done once for all PMD's.
PS: Will this callback thread clear the HW rule as well?

> For application, it should be aware

Need to make this clear in API definitions

> that the callback function is running on a separate thread, so it is also required to
> take care of race condition if it will access some data that is shared by foreground thread.

This means locks in application? or some lock-less async model?
Isn't it simpler to let the application (OVS) delete the rte_flow from
it's own thread, save locks, allow timeout updates according traffic
counter.

Alex