[dpdk-dev] [PATCH v2 09/28] sched: update pkt read and write API

Singh, Jasvinder jasvinder.singh at intel.com
Tue Jul 2 23:05:12 CEST 2019



> -----Original Message-----
> From: Dumitrescu, Cristian
> Sent: Tuesday, July 2, 2019 12:25 AM
> To: Singh, Jasvinder <jasvinder.singh at intel.com>; dev at dpdk.org
> Cc: Tovar, AbrahamX <abrahamx.tovar at intel.com>; Krakowiak, LukaszX
> <lukaszx.krakowiak at intel.com>
> Subject: RE: [PATCH v2 09/28] sched: update pkt read and write API
> 
> 
> 
> > -----Original Message-----
> > From: Singh, Jasvinder
> > Sent: Tuesday, June 25, 2019 4:32 PM
> > To: dev at dpdk.org
> > Cc: Dumitrescu, Cristian <cristian.dumitrescu at intel.com>; Tovar,
> > AbrahamX <abrahamx.tovar at intel.com>; Krakowiak, LukaszX
> > <lukaszx.krakowiak at intel.com>
> > Subject: [PATCH v2 09/28] sched: update pkt read and write API
> >
> > Update run time packet read and write api implementation to allow
> > configuration flexiblity for pipe traffic classes and queues, and
> > subport level configuration of the pipe parameters.
> >
> > Signed-off-by: Jasvinder Singh <jasvinder.singh at intel.com>
> > Signed-off-by: Abraham Tovar <abrahamx.tovar at intel.com>
> > Signed-off-by: Lukasz Krakowiak <lukaszx.krakowiak at intel.com>
> > ---
> >  lib/librte_sched/rte_sched.c | 32 +++++++++++++++++---------------
> > lib/librte_sched/rte_sched.h |  8 ++++----
> >  2 files changed, 21 insertions(+), 19 deletions(-)
> >
> > diff --git a/lib/librte_sched/rte_sched.c
> > b/lib/librte_sched/rte_sched.c index 1999bbfa3..cd82fd918 100644
> > --- a/lib/librte_sched/rte_sched.c
> > +++ b/lib/librte_sched/rte_sched.c
> > @@ -1433,17 +1433,15 @@ rte_sched_port_pipe_profile_add(struct
> > rte_sched_port *port,
> >
> >  static inline uint32_t
> >  rte_sched_port_qindex(struct rte_sched_port *port,
> > +	struct rte_sched_subport *s,
> >  	uint32_t subport,
> >  	uint32_t pipe,
> > -	uint32_t traffic_class,
> >  	uint32_t queue)
> >  {
> >  	return ((subport & (port->n_subports_per_port - 1)) <<
> > -			(port->n_pipes_per_subport_log2 + 4)) |
> > -			((pipe & (port->n_pipes_per_subport - 1)) << 4) |
> > -			((traffic_class &
> > -			    (RTE_SCHED_TRAFFIC_CLASSES_PER_PIPE - 1)) <<
> > 2) |
> > -			(queue &
> > (RTE_SCHED_QUEUES_PER_TRAFFIC_CLASS - 1));
> > +			(port->max_subport_pipes_log2 + 4)) |
> > +			((pipe & (s->n_subport_pipes - 1)) << 4) |
> > +			(queue & (RTE_SCHED_QUEUES_PER_PIPE - 1));
> >  }
> >
> 
> This function contains a critical bug: this patchset proposes that the number of
> pipes per subport is configurable independently for each subport; in other
> words, each subport can be configured with a different number of pipes.
> Therefore, the above logic is broken, as it assumes all subports have the same
> number of pipes. There is no longer possible to compute port-
> >max_subport_pipes_log2. Correct?

Yes, you are right. I didn't realize this issue.
 
> 
> We might need to rethink the design solution for the per-subport independent
> configuration.

One option to get around this is by computing  start_queue_offset for each subport and store that in rte_sched_subport data structure during subport configuration.

During run time, when writing pkt metadata;  add that offset to calculate queue index ;
	subport->start_queue_offset+((pipe & (s->n_pipes_per_subport - 1)) << 4) | (queue & (RTE_SCHED_QUEUES_PER_PIPE - 1));
At the same time,  subport id can be written to reserved field of the mbuf->hash.sched 

During packet read, offset value is retrieved from subport_id (from reserved field). By subtracting offset from the qindex, 
pipe, tc and queue id can determined from the remaining value.

This will allow contiguous value of the queue id at the port level.

> We also need to make sure we test this library with multiple subports per port,
> with each subport having different number of pipes. Need to do the basic uni
> test proposed earlier to trace the packet through the scheduler hierarchy up to
> the packet queue.

Yes, will add unit test for this case.
> 
> >  void
> > @@ -1453,9 +1451,9 @@ rte_sched_port_pkt_write(struct rte_sched_port
> > *port,
> >  			 uint32_t traffic_class,
> >  			 uint32_t queue, enum rte_color color)  {
> > -	uint32_t queue_id = rte_sched_port_qindex(port, subport, pipe,
> > -			traffic_class, queue);
> > -	rte_mbuf_sched_set(pkt, queue_id, traffic_class, (uint8_t)color);
> > +	struct rte_sched_subport *s = port->subports[subport];
> > +	uint32_t qindex = rte_sched_port_qindex(port, s, subport, pipe,
> > queue);
> > +	rte_mbuf_sched_set(pkt, qindex, traffic_class, (uint8_t)color);
> >  }
> >
> 
> Same comment here.
> 
> >  void
> > @@ -1464,13 +1462,17 @@ rte_sched_port_pkt_read_tree_path(struct
> > rte_sched_port *port,
> >  				  uint32_t *subport, uint32_t *pipe,
> >  				  uint32_t *traffic_class, uint32_t *queue)  {
> > -	uint32_t queue_id = rte_mbuf_sched_queue_get(pkt);
> > +	struct rte_sched_subport *s;
> > +	uint32_t qindex = rte_mbuf_sched_queue_get(pkt);
> > +	uint32_t tc_id = rte_mbuf_sched_traffic_class_get(pkt);
> > +
> > +	*subport = (qindex >> (port->max_subport_pipes_log2 + 4)) &
> > +		(port->n_subports_per_port - 1);
> >
> > -	*subport = queue_id >> (port->n_pipes_per_subport_log2 + 4);
> > -	*pipe = (queue_id >> 4) & (port->n_pipes_per_subport - 1);
> > -	*traffic_class = (queue_id >> 2) &
> > -				(RTE_SCHED_TRAFFIC_CLASSES_PER_PIPE -
> > 1);
> > -	*queue = queue_id & (RTE_SCHED_QUEUES_PER_TRAFFIC_CLASS -
> > 1);
> > +	s = port->subports[*subport];
> > +	*pipe = (qindex >> 4) & (s->n_subport_pipes - 1);
> > +	*traffic_class = tc_id;
> > +	*queue = qindex & (RTE_SCHED_QUEUES_PER_PIPE - 1);
> >  }
> >
> 
> Same comment here.
> 
> >  enum rte_color
> > diff --git a/lib/librte_sched/rte_sched.h
> > b/lib/librte_sched/rte_sched.h index 121e1f669..6a6ea84aa 100644
> > --- a/lib/librte_sched/rte_sched.h
> > +++ b/lib/librte_sched/rte_sched.h
> > @@ -421,9 +421,9 @@ rte_sched_queue_read_stats(struct rte_sched_port
> > *port,
> >   * @param pipe
> >   *   Pipe ID within subport
> >   * @param traffic_class
> > - *   Traffic class ID within pipe (0 .. 3)
> > + *   Traffic class ID within pipe (0 .. 8)
> >   * @param queue
> > - *   Queue ID within pipe traffic class (0 .. 3)
> > + *   Queue ID within pipe traffic class (0 .. 15)
> >   * @param color
> >   *   Packet color set
> >   */
> > @@ -448,9 +448,9 @@ rte_sched_port_pkt_write(struct rte_sched_port
> > *port,
> >   * @param pipe
> >   *   Pipe ID within subport
> >   * @param traffic_class
> > - *   Traffic class ID within pipe (0 .. 3)
> > + *   Traffic class ID within pipe (0 .. 8)
> >   * @param queue
> > - *   Queue ID within pipe traffic class (0 .. 3)
> > + *   Queue ID within pipe traffic class (0 .. 15)
> >   *
> >   */
> >  void
> > --
> > 2.21.0



More information about the dev mailing list