[dpdk-dev] [PATCH v4 1/2] eventdev: add device stop flush callback

[Van Haaren, Harry]

> From: Eads, Gage
> Sent: Tuesday, March 20, 2018 2:13 PM
> To: dev at dpdk.org
> Cc: jerin.jacob at caviumnetworks.com; Van Haaren, Harry
> <harry.van.haaren at intel.com>; hemant.agrawal at nxp.com; Richardson, Bruce
> <bruce.richardson at intel.com>; santosh.shukla at caviumnetworks.com;
> nipun.gupta at nxp.com
> Subject: [PATCH v4 1/2] eventdev: add device stop flush callback
> 
> When an event device is stopped, it drains all event queues. These events
> may contain pointers, so to prevent memory leaks eventdev now supports a
> user-provided flush callback that is called during the queue drain process.
> This callback is stored in process memory, so the callback must be
> registered by any process that may call rte_event_dev_stop().
> 
> This commit also clarifies the behavior of rte_event_dev_stop().
> 
> This follows this mailing list discussion:
> http://dpdk.org/ml/archives/dev/2018-January/087484.html
> 
> Signed-off-by: Gage Eads <gage.eads at intel.com>

<snip most of the code - looks good!>

>  /**
> - * Stop an event device. The device can be restarted with a call to
> - * rte_event_dev_start()
> + * Stop an event device.
> + *
> + * This function causes all queued events to be drained. While draining
> events
> + * out of the device, this function calls the user-provided flush callback
> + * (if one was registered) once per event.
> + *
> + * This function does not drain events from event ports; the application is
> + * responsible for flushing events from all ports before stopping the
> device.


Question about how an application is expected to correctly cleanup all the
events here. Note in particular the last part: "application is responsible
for flushing events from all ports **BEFORE** stopping the device".

Given the event device is still running, how can the application be sure it has
flushed all the events (from the dequeue side in particular)?


In order to drain all events from the ports, I was expecting the following:

// stop scheduling new events to worker cores
rte_event_dev_stop()
---> callback gets called for each event

// to dequeue events from each port, and app cleans them up?
FOR_EACH_PORT( rte_event_dev_dequeue(..., port_id, ...) )


I'd like to avoid the dequeue-each-port() approach in application, as it adds
extra burden to clean up correctly...

What if we say that dequeue() returns zero after stop() (leaving events possibly
in the port-dequeue side SW buffers), and these events which were about to
be dequeued by the worker core are also passed to the dev_stop_flush callback?