[dpdk-dev] [PATCH v4 1/3] ethdev: support API to set max LRO packet size

Matan Azrad matan at mellanox.com
Mon Nov 11 14:32:36 CET 2019



From: Ferruh Yigit
> On 11/11/2019 11:33 AM, Matan Azrad wrote:
> >
> >
> > From: Ferruh Yigit
> >> On 11/9/2019 6:20 PM, Matan Azrad wrote:
> >>> Hi
> >>>
> >>> From: Ferruh Yigit
> >>>> On 11/8/2019 11:56 AM, Matan Azrad wrote:
> >>>>>
> >>>>>
> >>>>> From: Ferruh Yigit
> >>>>>> On 11/8/2019 10:10 AM, Matan Azrad wrote:
> >>>>>>>
> >>>>>>>
> >>>>>>> From: Ferruh Yigit
> >>>>>>>> On 11/8/2019 6:54 AM, Matan Azrad wrote:
> >>>>>>>>> Hi
> >>>>>>>>>
> >>>>>>>>> From: Ferruh Yigit
> >>>>>>>>>> On 11/7/2019 12:35 PM, Dekel Peled wrote:
> >>>>>>>>>>> @@ -1266,6 +1286,18 @@ struct rte_eth_dev *
> >>>>>>>>>>>
> >>>>>>>>>> 	RTE_ETHER_MAX_LEN;
> >>>>>>>>>>>  	}
> >>>>>>>>>>>
> >>>>>>>>>>> +	/*
> >>>>>>>>>>> +	 * If LRO is enabled, check that the maximum
> aggregated
> >>>>>> packet
> >>>>>>>>>>> +	 * size is supported by the configured device.
> >>>>>>>>>>> +	 */
> >>>>>>>>>>> +	if (dev_conf->rxmode.offloads &
> >>>>>> DEV_RX_OFFLOAD_TCP_LRO) {
> >>>>>>>>>>> +		ret = check_lro_pkt_size(
> >>>>>>>>>>> +				port_id, dev_conf-
> >>>>>>>>>>> rxmode.max_lro_pkt_size,
> >>>>>>>>>>> +				dev_info.max_lro_pkt_size);
> >>>>>>>>>>> +		if (ret != 0)
> >>>>>>>>>>> +			goto rollback;
> >>>>>>>>>>> +	}
> >>>>>>>>>>> +
> >>>>>>>>>>
> >>>>>>>>>> This check forces applications that enable LRO to provide
> >>>>>>>> 'max_lro_pkt_size'
> >>>>>>>>>> config value.
> >>>>>>>>>
> >>>>>>>>> Yes.(we can break an API, we noticed it)
> >>>>>>>>
> >>>>>>>> I am not talking about API/ABI breakage, that part is OK.
> >>>>>>>> With this check, if the application requested LRO offload but
> >>>>>>>> not provided 'max_lro_pkt_size' value, device configuration will
> fail.
> >>>>>>>>
> >>>>>>> Yes
> >>>>>>>> Can there be a case application is good with whatever the PMD
> >>>>>>>> can support as max?
> >>>>>>> Yes can be - you know, we can do everything we want but it is
> >>>>>>> better to be
> >>>>>> consistent:
> >>>>>>> Due to the fact of Max rx pkt len field is mandatory for JUMBO
> >>>>>>> offload, max
> >>>>>> lro pkt len should be mandatory for LRO offload.
> >>>>>>>
> >>>>>>> So your question is actually why both, non-lro packets and LRO
> >>>>>>> packets max
> >>>>>> size are mandatory...
> >>>>>>>
> >>>>>>>
> >>>>>>> I think it should be important values for net applications
> management.
> >>>>>>> Also good for mbuf size managements.
> >>>>>>>
> >>>>>>>>>
> >>>>>>>>>> - Why it is mandatory now, how it was working before if it is
> >>>>>>>>>> mandatory value?
> >>>>>>>>>
> >>>>>>>>> It is the same as max_rx_pkt_len which is mandatory for jumbo
> >>>>>>>>> frame
> >>>>>>>> offload.
> >>>>>>>>> So now, when the user configures a LRO offload he must to set
> >>>>>>>>> max lro pkt
> >>>>>>>> len.
> >>>>>>>>> We don't want to confuse the user here with the max rx pkt len
> >>>>>>>> configurations and behaviors, they should be with same logic.
> >>>>>>>>>
> >>>>>>>>> This parameter defines well the LRO behavior.
> >>>>>>>>> Before this, each PMD took its own interpretation to what
> >>>>>>>>> should be the
> >>>>>>>> maximum size for LRO aggregated packets.
> >>>>>>>>> Now, the user must say what is his intension, and the ethdev
> >>>>>>>>> can limit it
> >>>>>>>> according to the device capability.
> >>>>>>>>> By this way, also, the PMD can organize\optimize its data-path
> >> more.
> >>>>>>>>> Also, the application can create different mempools for LRO
> >>>>>>>>> queues to
> >>>>>>>> allow bigger packet receiving for LRO traffic.
> >>>>>>>>>
> >>>>>>>>>> - What happens if PMD doesn't provide 'max_lro_pkt_size', so
> >>>>>>>>>> it is
> >>>> '0'?
> >>>>>>>>> Yes, you can see the feature description Dekel added.
> >>>>>>>>> This patch also updates all the PMDs support an LRO for non-0
> >> value.
> >>>>>>>>
> >>>>>>>> Of course I can see the updates Matan, my point is "What
> >>>>>>>> happens if PMD doesn't provide 'max_lro_pkt_size'",
> >>>>>>>> 1) There is no check for it right, so it is acceptable?
> >>>>>>>
> >>>>>>> There is check.
> >>>>>>> If the capability is 0, any non-zero configuration will fail.
> >>>>>>>
> >>>>>>>> 2) Are we making this filed mandatory to provide for PMDs, it
> >>>>>>>> is easy to make new fields mandatory for PMDs but is this
> >>>>>>>> really
> >>>> necessary?
> >>>>>>>
> >>>>>>> Yes, for consistence.
> >>>>>>>
> >>>>>>>>>
> >>>>>>>>> as same as max rx pkt len, no?
> >>>>>>>>>
> >>>>>>>>>> - What do you think setting 'max_lro_pkt_size' config value
> >>>>>>>>>> to what PMD provided if application doesn't provide it?
> >>>>>>>>> Same answers as above.
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>> If application doesn't care the value, as it has been till now,
> >>>>>>>> and not provided explicit 'max_lro_pkt_size', why not ethdev
> >>>>>>>> level use the value provided by PMD instead of failing?
> >>>>>>>
> >>>>>>> Again, same question we can ask on max rx pkt len.
> >>>>>>>
> >>>>>>> Looks like the packet size is very important value which should
> >>>>>>> be set by
> >>>>>> the application.
> >>>>>>>
> >>>>>>> Previous applications have no option to configure it, so they
> >>>>>>> haven't
> >>>>>> configure it, (probably cover it somehow) I think it is our miss
> >>>>>> to supply this info.
> >>>>>>>
> >>>>>>> Let's do it in same way as we do max rx pkt len (as this patch
> >>>>>>> main
> >> idea).
> >>>>>>> Later, we can change both to other meaning.
> >>>>>>>
> >>>>>>
> >>>>>> I think it is not a good reason to introduce a new mandatory
> >>>>>> config option for application because of 'max_rx_pkt_len' does it.
> >>>>>
> >>>>> It is mandatory only if LRO offload is configured.
> >>>>>
> >>>>>> Will it work, if:
> >>>>>> - If application doesn't provide this value, use the PMD max
> >>>>>
> >>>>> May cause a problem if the mbuf size is not enough for the PMD
> >> maximum.
> >>>>
> >>>> OK, this is what I was missing, for this case I was thinking
> >>>> max_rx_pkt_len will be used but you already explained that
> >>>> application may want to use different mempools for LRO queues.
> >>>>
> >>> So , are you agree with the idea?
> >>>
> >>>> For this case shouldn't PMDs take the 'rxmode.max_lro_pkt_size'
> >>>> into account and program the device accordingly (of course in LRO
> >>>> enabled
> >>>> case) ?
> >>>> This part seems missing and should be highlighted to other PMD
> >> maintainers.
> >>>
> >>>
> >>> Yes, you are right.
> >>> PMDs must limit the LRO aggregated packet according to the new
> >>> field, And it probably very hard for the patch introducer to
> >>> understand how to do
> >> it for each PMD.
> >>>
> >>> I think each new configuration requires other maintainers\developers
> >>> to
> >> adjust their own PMD code to the new configuration and it should be
> >> done in limited time.
> >>
> >> Agree.
> >> But experience showed that this synchronization is not as easy as it
> >> sounds, whoever changing the interface/library says other PMDs should
> >> reflect the change but most of the times other PMD maintainers not
> >> aware of it or if they do they have other priorities for the release,
> >> so the changes should be in a way to give more time to PMDs to adapt
> >> it and during this time library change shouldn't break other PMDs.
> >>
> >
> > Yes.
> >
> >>> My suggestion here:
> >>> 1. To reserve the info field and the configuration field for rc2.(if
> >>> it is critical not to break ABI for rc3) 2. To merge the ethdev
> >>> patch in the
> >> start of rc3.
> >>> 3. Request each relevant PMD to adjust its PMD to the new
> >>> configuration
> >> for the end of rc3.
> >>> 	Note: this should be small change and only for ~5 PMDs:
> >>> 		a. Introduce the info field according to the device ability.
> >>> 		b. For each LRO queue:
> >>> 			Use the LRO max size configuration instead of the
> >> current max rx pkt len configuration(looks like small condition).
> >>>
> >>> What do you think?
> >>
> >> There is already a v6 which only updates dev_info fields to have the
> >> 'max_lro_pktlen' field, the PMD updates there also looks safe, so I
> >> think we can go with it for rc2.
> >>
> >
> > Doesn’t make sense to expose the info field without the configuration.
> >
> >
> >> For the configuration part, I suggest deferring it next release,
> >> which gives more time for discussion and enough time for other PMDs to
> implement it.
> >>
> >>
> >> And related configuration, right now devices already configured to
> >> limit the packet size to 'max_rx_pkt_len', it can be an optimization
> >> to increase it to 'max_lro_pkt_len' for the queues LRO is supported,
> >> why not make this configuration more explicitly with specific API as
> >> Konstantin suggested [1], this way it only affects the applications
> >> that are interested in and the PMDs that want to support this.
> >> Current implementation is under 'rte_eth_dev_configure()' which is
> >> used by all DPDK applications and impact of changing it is much
> >> larger, also it makes mandatory for applications to provide this
> >> config option when LRO enabled, explicit API gives same result without
> making a mandatory config option.
> >>
> >> [1]
> >> int rte_eth_dev_set_max_lro(uint16_t port_id, uint32_t lro);
> >
> > Please see my answers to Konstantin regarding this topic.
> >
> >
> >
> > One more option:
> > In order to not break PMDs because of this feature:
> > 0 in the capability field means, The PMD doesn't support LRO special
> limitation so if the application configuration is not the same like
> max_rx_pkt_len the validation will fail.
> >
> 
> I don't see this is a mandatory field if the LRO is enabled, am I missing
> something?

From the application size, this is mandatory, you right exactly like max_rx_pkt_len.

> And current implementation does so by failing configure(), the
> affect to the applications is my first concern.

This is a small effect as any API change.
If an exists application wants to save its current LRO behavior, it just need to put max_lro_pkt_len=max_rx_pkt_len.
Do you think this is a big change? Why?


> Second is when application supplied the proper values but PMD is not doing
> anything without letting application anything done.
> 

PMD which doesn't change its info to value != 0, as I said, means that it doesn't support LRO queues special limited size.
When the PMD maintainers have time to support the feature, they just need to change the info value to be !=0 and to take into account the max_lro_pkt_len in configuration.

> That is why I think explicit API makes this clear and only required by
> application wants to use it.


I think that exposing a new function API is not good because it introduce a different way to limit the Rx packet size.

For regular packet - configure it in the configuration struct.
For LRO packets - use a function to do it.

It is very confusing and not intuitive from the user side.

Why not to save convention and consistent, so
max_rx_pkt_len  is mandatory (for JUMBO offload) and in the config structure 
Also the new lro conf is mandatory (for LRO offload) and in the configuration structure. 

So, all the configurations to limit the Rx packet size are in the same place and done by the same way.

> Similar can be done with following, this also doesn't require both application
> and PMD changes, wdyt?
> 
> ethdev, configure():
> if (dev_conf->rxmode.offloads & DEV_RX_OFFLOAD_TCP_LRO) {
>   if (dev_conf->rxmode.max_lro_pktlen) {
>     if (dev_info.max_lro_pktlen) {
> 	validate(rxmode.max_lro_pktlen, dev_info.max_lro_pktlen)
>     } else if (dev_info.max_rx_pktlen)
>         validate(rxmode.max_lro_pktlen, dev_info.max_rx_pktlen)
>     }
>   }
> }
> 
> 
> in PMD:
> if (LRO) {
> 	queue.max_pktlen = rxmode.max_lro_pktlen ?
> 		rxmode.max_lro_pktlen :
> 		rxmode.max_tx_pktlen;
> }

Again, my only concern here is the consistency of Rx packet size limitation mandatory for LRO and non-LRO.



More information about the dev mailing list