<html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> </head> <body> Hi Ferruh, <div class="moz-cite-prefix">On Wed, June 7, 2023 at 2:09AM, Ferruh Yigit wrote: </div> <blockquote type="cite" cite="mid:38f4bfd4-fced-bd0b-6261-1b34cd894dda@amd.com"> <pre class="moz-quote-pre" wrap="">On 5/8/2023 2:21 AM, zhoumin wrote: </pre> <blockquote type="cite"> <pre class="moz-quote-pre" wrap="">Kindly ping. </pre> </blockquote> <pre class="moz-quote-pre" wrap=""> Hi Min, Sorry for the delay. </pre> <blockquote type="cite"> <pre class="moz-quote-pre" wrap="">Should we solve this problem by modifying PMDs? Is it suitable to maintain the RSS configurations for NIC by testpmd in order to set DCB? On Thur, Apr 27, 2023 at 4:33PM, zhoumin wrote: </pre> <blockquote type="cite"> <pre class="moz-quote-pre" wrap="">Kindly ping. Any comments or suggestions will be appreciated. Best regards Min On 2023/4/12 下午5:52, Min Zhou wrote: </pre> <blockquote type="cite"> <pre class="moz-quote-pre" wrap="">In the testpmd command, we have to stop the port firstly before configuring the DCB. However, some PMDs may execute a hardware reset during the port stop, such as ixgbe. Some kind of reset operations of PMD could clear the configurations of RSS in the hardware register. This would cause the loss of RSS configurations that were set during the testpmd initialization. As a result, I find that I cannot enable RSS and DCB at the same time in the testpmd command when using Intel 82599 NIC. </pre> </blockquote> </blockquote> </blockquote> <pre class="moz-quote-pre" wrap=""> cc'ed ixgbe maintainers too.</pre> </blockquote> OK, thanks. <blockquote type="cite" cite="mid:38f4bfd4-fced-bd0b-6261-1b34cd894dda@amd.com"> <pre class="moz-quote-pre" wrap=""> @Qiming, @Wenjun, are you testing DCB with RSS configuration? Min, Can you please give more detail on the problem, do you have in only on device reset, or is there a generic problem on enabling DCB with RSS? </pre> </blockquote> Yes, I will give more details on the problem blow. I think it is not a generic problem on enabling DCB with RSS. It depends on the implementation of "port stop" in the driver's codes. I can successfully enable DCB with RSS in the testpmd when it uses the i40e driver. It's very easy to reproduce this problem. I will demonstrate the process below: 1. First we start the testpmd program with enabling RSS using 82599 NICs: $ ./build/app/dpdk-testpmd -l 0-6 -- -i --rss-ip --rxq=6 --txq=6 --nb-cores=6 2. Then show the RSS configuration in the testpmd: testpmd> show port 0 rss-hash RSS functions: ipv4 ipv6 ipv6-ex 3. The above outputs are normal and then we try to enable DCB: testpmd> port config 0 dcb vt off 4 pfc on Please stop port 0 first 4. So we have to stop the port before enabling DCB: testpmd> port stop 0 Stopping ports... Checking link statuses... ixgbe_dev_link_update_share(): Other link thread is running now! Done 5. However, we will find that the RSS was disabled after stopping the port: testpmd> show port 0 rss-hash RSS disabled RSS is enabled in the MRQC register of 82599. The MRQC register supplied in section 8.2.3.7.12 of the Intel 82599 datasheet. I added some outputs before and after port reset in the port stop process of ixgbe driver as follows: diff --git a/drivers/net/ixgbe/base/ixgbe_82599.c b/drivers/net/ixgbe/base/ixgbe_82599.c index c6e8b7e976..b13431a43f 100644 --- a/drivers/net/ixgbe/base/ixgbe_82599.c +++ b/drivers/net/ixgbe/base/ixgbe_82599.c @@ -1053,6 +1053,7 @@ s32 ixgbe_reset_hw_82599(struct ixgbe_hw *hw) if (link_up) ctrl = IXGBE_CTRL_RST; } + printf("%s: MRQC = 0x%x\n", __func__, IXGBE_READ_REG(hw, IXGBE_MRQC)); ctrl |= IXGBE_READ_REG(hw, IXGBE_CTRL); IXGBE_WRITE_REG(hw, IXGBE_CTRL, ctrl); @@ -1065,6 +1066,7 @@ s32 ixgbe_reset_hw_82599(struct ixgbe_hw *hw) if (!(ctrl & IXGBE_CTRL_RST_MASK)) break; } + printf("%s: MRQC = 0x%x\n", __func__, IXGBE_READ_REG(hw, IXGBE_MRQC)); if (ctrl & IXGBE_CTRL_RST_MASK) { status = IXGBE_ERR_RESET_FAILED; We could find that the RSS configurations will be cleared after port reset as follows: testpmd> port stop 0 Stopping ports... ixgbe_reset_hw_82599: MRQC = 0x1a0001 ixgbe_reset_hw_82599: MRQC = 0x0 Checking link statuses... ixgbe_dev_link_update_share(): Other link thread is running now! Done However, I have not find the exact descriptions about this behavior in the 82599 datasheet. Could @Qiming, @Wenjun give me some indications ? So, if we want to get the RSS configurations by reading from the register after the port stop, we will get nothing. <span style="color: rgb(51, 51, 51); font-family: Arial, "PingFang SC", "Hiragino Sans GB", STHeiti, "Microsoft YaHei", "WenQuanYi Micro Hei", sans-serif; font-size: 14px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(242, 244, 251); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;"> <blockquote type="cite" cite="mid:38f4bfd4-fced-bd0b-6261-1b34cd894dda@amd.com"> <pre class="moz-quote-pre" wrap=""> Also can you please describe how/why removing RSS configuration (this patch) helps on the problem? </pre> </blockquote> As above said, I'm concerned whether we could reuse the RSS configurations recorded in the testpmd if the port reset cannot be avoided in the driver. So, this patch wants to reuse the RSS configurations maintained by testpmd instead of reading from the register. I'm not sure if this is a reasonable method to solve this problem. <blockquote type="cite" cite="mid:38f4bfd4-fced-bd0b-6261-1b34cd894dda@amd.com"> <pre class="moz-quote-pre" wrap=""> Thanks, ferruh </pre> </blockquote> [...] Best regards, Min </body> </html>