[dpdk-dev] [PATCH v2 01/22] ethdev: fix MTU size exceeds max rx packet length

Ferruh Yigit ferruh.yigit at intel.com
Wed Jan 13 11:25:37 CET 2021


On 1/6/2021 3:36 AM, Yang, SteveX wrote:
> Hi Lijun,
> 
> Thanks for your review. Please check my comments inline.
> Regards,
> Steve Yang.
> 
>> -----Original Message-----
>> From: oulijun <oulijun at huawei.com>
>> Sent: Wednesday, December 30, 2020 6:20 PM
>> To: Yang, SteveX <stevex.yang at intel.com>; dev at dpdk.org
>> Cc: Lu, Wenzhuo <wenzhuo.lu at intel.com>; Xing, Beilei
>> <beilei.xing at intel.com>; Iremonger, Bernard
>> <bernard.iremonger at intel.com>; asomalap at amd.com;
>> rahul.lakkireddy at chelsio.com; hemant.agrawal at nxp.com;
>> sachin.saxena at oss.nxp.com; Guo, Jia <jia.guo at intel.com>; Wang, Haiyue
>> <haiyue.wang at intel.com>; g.singh at nxp.com; xuanziyang2 at huawei.com;
>> cloud.wangxiaoyun at huawei.com; zhouguoyang at huawei.com;
>> xavier.huwei at huawei.com; humin29 at huawei.com;
>> yisen.zhuang at huawei.com; Wu, Jingjing <jingjing.wu at intel.com>; Yang,
>> Qiming <qiming.yang at intel.com>; Zhang, Qi Z <qi.z.zhang at intel.com>; Xu,
>> Rosen <rosen.xu at intel.com>; sthotton at marvell.com;
>> srinivasan at marvell.com; heinrich.kuhn at netronome.com;
>> hkalra at marvell.com; jerinj at marvell.com; ndabilpuram at marvell.com;
>> kirankumark at marvell.com; rmody at marvell.com; shshaikh at marvell.com;
>> andrew.rybchenko at oktetlabs.ru; mczekaj at marvell.com;
>> thomas at monjalon.net; Yigit, Ferruh <ferruh.yigit at intel.com>;
>> ivan.boule at 6wind.com; Ananyev, Konstantin
>> <konstantin.ananyev at intel.com>; samuel.gauthier at 6wind.com;
>> david.marchand at 6wind.com; shahafs at mellanox.com;
>> stephen at networkplumber.org; maxime.coquelin at redhat.com;
>> olivier.matz at 6wind.com; lihuisong at huawei.com; shreyansh.jain at nxp.com;
>> wei.dai at intel.com; fengchunsong at huawei.com; chenhao164 at huawei.com;
>> tangchengchang at hisilicon.com; Zhang, Helin <helin.zhang at intel.com>;
>> yanglong.wu at intel.com; xiaolong.ye at intel.com; Xu, Ting
>> <ting.xu at intel.com>; Li, Xiaoyun <xiaoyun.li at intel.com>; Wei, Dan
>> <dan.wei at intel.com>; Pei, Andy <andy.pei at intel.com>;
>> vattunuru at marvell.com; skori at marvell.com; sony.chacko at qlogic.com;
>> Richardson, Bruce <bruce.richardson at intel.com>; ivan.malov at oktetlabs.ru;
>> rad at semihalf.com; slawomir.rosek at semihalf.com;
>> kamil.rytarowski at caviumnetworks.com; Zhao1, Wei <wei.zhao1 at intel.com>;
>> Jiang, JunyuX <junyux.jiang at intel.com>; kumaras at chelsio.com;
>> girish.nandibasappa at amd.com; rolf.neugebauer at netronome.com;
>> alejandro.lucero at netronome.com
>> Subject: Re: [PATCH v2 01/22] ethdev: fix MTU size exceeds max rx packet
>> length
>>
>>
>>
>> 在 2020/12/17 17:22, Steve Yang 写道:
>>> If max rx packet length is smaller then MTU + Ether overhead, that
>>> will drop all MTU size packets.
>>>
>>> Update the MTU size according to the max rx packet and Ether overhead.
>>>
>>> Fixes: 59d0ecdbf0e1 ("ethdev: MTU accessors")
>>>
>>> Signed-off-by: Steve Yang <stevex.yang at intel.com>
>>> ---
>>>    lib/librte_ethdev/rte_ethdev.c | 21 ++++++++++++++++++---
>>>    1 file changed, 18 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/lib/librte_ethdev/rte_ethdev.c
>>> b/lib/librte_ethdev/rte_ethdev.c index 17ddacc78d..ff6a1e675f 100644
>>> --- a/lib/librte_ethdev/rte_ethdev.c
>>> +++ b/lib/librte_ethdev/rte_ethdev.c
>>> @@ -1292,6 +1292,7 @@ rte_eth_dev_configure(uint16_t port_id,
>> uint16_t nb_rx_q, uint16_t nb_tx_q,
>>>    struct rte_eth_dev *dev;
>>>    struct rte_eth_dev_info dev_info;
>>>    struct rte_eth_conf orig_conf;
>>> +uint16_t overhead_len;
>>>    int diag;
>>>    int ret;
>>>
>>> @@ -1323,6 +1324,15 @@ rte_eth_dev_configure(uint16_t port_id,
>> uint16_t nb_rx_q, uint16_t nb_tx_q,
>>>    if (ret != 0)
>>>    goto rollback;
>>>
>>> +/* Get the real Ethernet overhead length */
>>> +if (dev_info.max_mtu &&
>>> +    dev_info.max_mtu != UINT16_MAX &&
>>> +    dev_info.max_rx_pktlen &&
>>> +    dev_info.max_rx_pktlen > dev_info.max_mtu)
>>> +overhead_len = dev_info.max_rx_pktlen -
>> dev_info.max_mtu;
>>> +else
>>> +overhead_len = RTE_ETHER_HDR_LEN +
>> RTE_ETHER_CRC_LEN;
>>> +
>>>    /* If number of queues specified by application for both Rx and Tx is
>>>     * zero, use driver preferred values. This cannot be done individually
>>>     * as it is valid for either Tx or Rx (but not both) to be zero.
>>> @@ -1410,13 +1420,18 @@ rte_eth_dev_configure(uint16_t port_id,
>> uint16_t nb_rx_q, uint16_t nb_tx_q,
>>>    goto rollback;
>>>    }
>>>    } else {
>>> -if (dev_conf->rxmode.max_rx_pkt_len <
>> RTE_ETHER_MIN_LEN ||
>>> -dev_conf->rxmode.max_rx_pkt_len >
>> RTE_ETHER_MAX_LEN)
>>> +uint16_t pktlen = dev_conf->rxmode.max_rx_pkt_len;
>>> +if (pktlen < RTE_ETHER_MIN_MTU + overhead_len ||
>>> +pktlen > RTE_ETHER_MTU + overhead_len)
>>>    /* Use default value */
>>>    dev->data->dev_conf.rxmode.max_rx_pkt_len =
>>> -
>> RTE_ETHER_MAX_LEN;
>>> +RTE_ETHER_MTU +
>> overhead_len;
>>>    }
>>>
>>> +/* Scale the MTU size to adapt max_rx_pkt_len */
>>> +dev->data->mtu = dev->data->dev_conf.rxmode.max_rx_pkt_len -
>>> +overhead_len;
>>> +
>> Hi
>>     I think the dev->data->mtu should be updated after configured success. So
>> the update in this position seems unreasonable.
> 
> Good suggestion, I've checked some PMDs, and the corresponding assignments are existed within their 'dev_configure_ops'.
> For example: [bnxt_ethdev.c # 1100]
> ```
> if (rx_offloads & DEV_RX_OFFLOAD_JUMBO_FRAME) {
> eth_dev->data->mtu =
> eth_dev->data->dev_conf.rxmode.max_rx_pkt_len -
> RTE_ETHER_HDR_LEN - RTE_ETHER_CRC_LEN - VLAN_TAG_SIZE *
> BNXT_NUM_VLANS;
> bnxt_mtu_set_op(eth_dev, eth_dev->data->mtu);
> }
> ```
> Hence, I will move this 'mtu' assignment to line after '(*dev->dev_ops->dev_configure)(dev)'.

There is a 'rollback' already for the similar reason.

What do you think to store the old MTU and restore it in the rollback if needed? 
So you don't need to change where MTU set.

> Actually, it doesn't matter if it is the JUMBO frame, the 'mtu' must keep pace with 'max_rx_pkt_len' anytime.
> E.g.: if 'mtu' is 1500, and 'max_rx_pkt_len' is set to 1510 by command line, that 'mtu' must be reduced to '1510 - 18 = 1492' in 'dev_configure' phase, even though it is not a Jumbo frame.
> 

According to the API definition:
  max_rx_pkt_len;  /**< Only used if JUMBO_FRAME enabled. */

The concern was removing this check from the ethdev may break some PMDs that 
does not follow the API and use the 'max_rx_pkt_len' even if JUMBO frame offload 
set.

For this release, we can afford to break the PMDs implementing wrong and fix 
them after testing.

>> 'max_rx_pkt_len' is only used when jumbo_frame enabled, as follows:
>> struct rte_eth_rxmode {
>> .....
>> uint32_t max_rx_pkt_len;  /**< Only used if JUMBO_FRAME
>> enabled. */
>> /** Maximum allowed size of LRO aggregated packet. */
>> ....};
>>
>> So If DEV_RX_OFFLOAD_JUMBO_FRAME is set to rxmode.offload, driver
>> should configure mtu to hardware according to 'max_rx_pkt_len' and update
>> dev->data->mtu. This seems more reasonable. And some PMD drivers are
>> already doing this.
> 
>>
>> In addition, validity check for 'max_rx_pkt_len' in
>> rte_eth_dev_configure API may be error. It should be greater than
>> 'RTE_ETHER_MTU + overhead_len'.
>> Because driver must enable DEV_RX_OFFLOAD_JUMBO_FRAME when user
>> set mtu
>> with greater than 1500 by 'rte_eth_dev_set_mtu' API.
>>
> 
> I'm not sure if it is following validity check you mentioned.
>>> +if (pktlen < RTE_ETHER_MIN_MTU + overhead_len ||
>>> +pktlen > RTE_ETHER_MTU + overhead_len)
> If yes, no issue here, because the 'rte_eth_dev_configure' is only used for initializing device,
> and just gives out an initial 'max_rx_pkt_len' value by default. Once user tries to change mtu via 'set mtu',
> the 'max_rx_pkt_len' will be updated to expected value in the 'mtu_set' ops.
> Another aspect, that is the 'disable DEV_RX_OFFLOAD_JUMBO' branch, 'max_rx_pkt_len' must be limited to
> effective max value for non-Jumbo frame.
> 

In the 'disable DEV_RX_OFFLOAD_JUMBO' branch, 'max_rx_pkt_len' should be invalid 
according API doc as mentioned above, I guess Lijun refers to this.

Overall what about updating as following, in 'rte_eth_dev_configure()':

if (offloads & DEV_RX_OFFLOAD_JUMBO_FRAME) {
   // max_rx_pkt_len checks
   old_mtu = mtu
   mtu = max_rx_pkt_len - overhead_len
}

...

rollback:
  mtu = old_mtu;


>> What do you think?
>>
>> Thanks
>> Lijun Ou
>>
>>>    /*
>>>     * If LRO is enabled, check that the maximum aggregated packet
>>>     * size is supported by the configured device.
>>>



More information about the dev mailing list