[dpdk-dev] i40e and RSS woes

[Vladislav Zolotarov]

On Aug 24, 2015 21:54, "Zhang, Helin" <helin.zhang at intel.com> wrote:
>
>
>
> > -----Original Message-----
> > From: Vlad Zolotarov [mailto:vladz at cloudius-systems.com]
> > Sent: Monday, August 24, 2015 11:26 AM
> > To: Zhang, Helin; Gleb Natapov
> > Cc: dev at dpdk.org
> > Subject: Re: [dpdk-dev] i40e and RSS woes
> >
> >
> >
> > On 08/24/15 20:51, Zhang, Helin wrote:
> > >
> > >> -----Original Message-----
> > >> From: Vlad Zolotarov [mailto:vladz at cloudius-systems.com]
> > >> Sent: Monday, August 24, 2015 4:14 AM
> > >> To: Zhang, Helin; Gleb Natapov; dev at dpdk.org
> > >> Subject: Re: [dpdk-dev] i40e and RSS woes
> > >>
> > >>
> > >>
> > >> On 03/05/15 07:56, Zhang, Helin wrote:
> > >>> Hi Gleb
> > >>>
> > >>> Sorry for late! I am struggling on my tasks for the following DPDK
> > >>> release these
> > >> days.
> > >>>> -----Original Message-----
> > >>>> From: Gleb Natapov [mailto:gleb at cloudius-systems.com]
> > >>>> Sent: Monday, March 2, 2015 8:56 PM
> > >>>> To: dev at dpdk.org
> > >>>> Cc: Zhang, Helin
> > >>>> Subject: Re: i40e and RSS woes
> > >>>>
> > >>>> Ping.
> > >>>>
> > >>>> On Thu, Feb 19, 2015 at 04:50:10PM +0200, Gleb Natapov wrote:
> > >>>>> CCing i40e driver author in a hope to get an answer.
> > >>>>>
> > >>>>> On Mon, Feb 16, 2015 at 03:36:54PM +0200, Gleb Natapov wrote:
> > >>>>>> I have an application that works reasonably well with ixgbe
> > >>>>>> driver, but when I try to use it with i40e I encounter various
RSS related
> > issues.
> > >>>>>>
> > >>>>>> First one is that for some reason i40e, when it builds default
> > >>>>>> reta table, round down number of queues to power of two. Why is
> > >>>>>> this? If
> > >>> It seems because of i40e queue configuration. We will check it more
> > >>> and see if it can be changed or improved later.
> > >> Helin, hi!
> > >> Sorry for bringing it back but it seems that the RSS queues number
> > >> issue (rounding it down to the nearest power of 2) still hasn't been
> > >> addressed in the master branch.
> > >>
> > >> Could u, pls., clarify what is that "i40e queue configuration" that
> > >> requires this alignment u are referring above?
> > >>
> > >>   From what i could see "num" parameter is not propagated outside the
> > >> i40e_pf_config_rss() in any form except for RSS table contents.
> > >> This means that any code that would need to know the number of Rx
> > >> queues would use the dev_data->nb_rx_queues (e.g. i40e_dev_rx_init())
> > >> and wouldn't be able to know that i40e_pf_config_rss() something
> > >> different except for scanning the RSS table in HW which is of course
not an
> > option.
> > >>
> > >> Therefore, from the first look it seems that this rounding may be
> > >> safely removed unless I've missed something.
> > > Could you help to refer to the data sheet of 'Hash Filter', 'Receive
> > > Queue Regions', it is said that '1, 2, 4, 8, 16, 32, 64' are the
supported queue
> > regions.
> > > Yes, we should support more than 64 queues per port, but for rss, it
> > > should be one of '1, 2, 4, 8, 16, 32, 64'.
> >
> > "The VSIs support 8 regions of receive queues that are aimed mainly for
the TCs.
> > The TC regions are defined per VSI by the VSIQF_TCREGION register. The
region
> > sizes (defined by the TC_SIZE fields) can be any of the following
value: 1, 2, 4, 8,
> > 16, 32, 64 as long as the total number of queues do not exceed the VSI
allocation.
> > These regions starts at the offset defined by the TC_OFFSET parameter.
> > According to the region size, the 'n' LS bits of the Queue Index from
the LUT are
> > enabled."
> >
> > I think the above says that the region sizes may only be one of the
mentioned
> > values.
> >
> > AFAIU this doesn't mean that the number or RSS queues has to be the same
> > - it may not exceed it.
> >
> > Just like it's stated in the "Outcome Queue Index" definition the final
mapping to
> > the PF index space is done using the VSILAN_QTABLE or VSILAN_QBASE
registers
> > (a.k.a. RSS indirection table).
> >
> > For instance if u have a region of size 8 u may configure 3 RSS queues
by setting
> > the following RSS table:
> > 0,1,2,0,1,2,0,1
> I tend to agree with you. Anyway, I am working on supporting more queues
per port than 64,
> and I will take this into account. If not other strong reasons, I will
change it. Thank you very much!

Great! Thanks a lot, Helin.

>
> Regards,
> Helin
>
> >
> > >
> > > Thanks,
> > > Helin
> > >
> > >> Pls., comment.
> > >>
> > >> thanks,
> > >> vlad
> > >>
> > >>>>>> I configure reta by my own using all of the queues everything
> > >>>>>> seams to be working. To add insult to injury I do not get any
> > >>>>>> errors during configuration some queues just do not receive any
traffic.
> > >>>>>>
> > >>>>>> The second problem is that for some reason i40e does not use 40
> > >>>>>> byte toeplitz hash key like any other driver, but it expects the
> > >>>>>> key to be 52 bytes. And it would have being fine (if we ignore
> > >>>>>> the fact that it contradicts MS spec), but how my high level code
> > >>>>>> suppose to know
> > >>>> that?
> > >>> Actually a rss_key_len was introduced in struct rte_eth_rss_conf
> > >>> recently. So the length should be indicated clearly. But I found the
> > >>> annotations of that structure should have been reworked. I will try
> > >>> to rework it
> > >> with clear descriptions.
> > >>>>>> And again, device configuration does not fail when wrong key
> > >>>>>> length is provided, it just uses some other key. Guys this kind
> > >>>>>> of error handling is completely unacceptable.
> > >>> If less length of key is provided, it will not be used at all, the
> > >>> default key will be
> > >> used.
> > >>> So there is no issue as you said. But we need to add more clear
> > >>> description for the structure of rte_eth_rss_conf.
> > >>>
> > >>> Thank you very much for the good comments!
> > >>>
> > >>> Regards,
> > >>> Helin
> > >>>
> > >>>>>> The last one is more of a question. Why interface to change RSS
> > >>>>>> hash function (XOR or toeplitz) is part of a filter configuration
> > >>>>>> and not rss config?
> > >>>>>>
> > >>>>>> --
> > >>>>>>                        Gleb.
> > >>>>> --
> > >>>>>                         Gleb.
> > >>>> --
> > >>>>                  Gleb.
>