[dpdk-dev] [PATCH 00/13] Introducing EAL Bus-Device-Driver Model

[Shreyansh Jain]

Hello David,

On Wednesday 07 December 2016 02:22 AM, David Marchand wrote:
> "Big patchset and a lot of things to look at.
>
> Here is a first look at it.

Thanks for comments.

>
> On Sun, Dec 4, 2016 at 11:11 AM, Shreyansh Jain <shreyansh.jain at nxp.com> wrote:
>> In continuation to the RFC posted on 17/Nov [9],
>> A series of patches is being posted which attempts to create:
>>  1. A basic bus model
>>     `- define rte_bus and associated methods/helpers
>>     `- test infrastructure to test the Bus infra
>>  2. Changes in EAL to support PCI as a bus
>>     `- a "pci" bus is registered
>>     `- existing scan/match/probe are modified to allow for bus integration
>>     `- PCI Device and Driver list, which were global entities, have been
>>        moved to rte_bus->[device/driver]_list
>>
>> I have sanity tested this patch over a XeonD X552 available with me, as
>> well as part of PoC for verifying NXP's DPAA2 PMD (being pushed out in a
>> separate series). Exhaustive testing is still pending.
>
> I saw some checkpatch issues for patch 1 (I would ignore this one) and
> 4 (please check).

Yes, for Patch 1 I too ignored the warnings.
For Patch 4, there are 3 warnings, of which first 2 were reported by 
checkpatch tool before I sent out. Surprisingly, I didn't see the third 
one when I ran the tool in my environment.

I will fix them (those which I can) in v2.

>
>
>> :: Brief about Patch Layout ::
>>
>> 0001:      Container_of patch from [3]
>
>> 0002~0003: Introducing the basic Bus model and associated test case
>> 0005:      Support insertion of device rather than addition to tail
>
> Patch 2 and 5 could be squashed.

I deliberately kept them separate. I intent to extend the Patch 5 for 
hotplugging. But, if I don't end up adding support for that in this 
series, I will merge these two.

> The constructor priority stuff seems unneeded as long as we use
> explicit reference to a global (or local, did not check) bus symbol
> rather than a runtime lookup.

I didn't understand your point here.
IMO, constructor priority (or some other way to handle this) is 
important. I faced this issue while verifying it at my end when the 
drivers were getting registered before the bus.

Can you elaborate more on '..use explicit reference to a global...'?

>
>
>> 0004:      Add scan and match callbacks for the Bus and updated test case
>
> Why do you push back the bus object in the 'scan' method ?
> This method is bus specific which means that the code "knows" the
> object registered with the callback.

This 'knows' is the grey area for me.
The bus (for example, PCI) after scanning needs to call 
rte_eal_bus_add_device() to link the device in bus's device_list.

Two options:
1. Have a global reference to "pci" bus (rte_bus) somewhere in eal_pci.c
2. Call rte_eal_get_bus() every time someone needs the reference.
3. C++ style, 'this->'.

I have taken the 3rd path. It simplifies my code to not assume a handle 
as well as not allow for reference fetch calls every now and then.

As a disadvantage: it means passing this as argument - and some cases 
maintaining it as __rte_unused.

Taking (1) or (2) is not advantageous than this approach.

> Is is that you want to have a single scan method used by multiple buses ?

Yes, but only as a use case. For example, platform devices are of 
various types - what if we have a south-bound bus over a platform bus. 
In which case, a hierarchical bus layout is possible.
But, this is far-fetched idea for now.

>
>
>> 0006:      Integrate bus scan/match with EAL, without any effective driver
>
> Hard to find a right balance in patch splittng, but patch 4 and 6 are
> linked, I would squash them into one.

Yes, it is hard and sometimes there is simply no strong rationale for 
splitting or merging. This is one of those cases.
My idea was that one patch _only_ introduces Bus services (structures, 
functions etc) and another should enable the calls to it from EAL.
In that sense, I still think 4 and 6 should remain separate, may be 
consecutive, though.

>
>
>> 0007:      rte_pci_driver->probe replaced with rte_driver->probe
>
> This patch is too big, please separate in two patches: eal changes
> then ethdev/driver changes.

I don't think that can be done. One change is incomplete without the other.

Changes to all files are only for rte_pci_driver->probe to 
rte_driver->probe movement. EAL changes is to allow 
rte_eth_dev_pci_probe function after such a change as rte_driver->probe 
has different arguments as compared to rte_pci_driver->probe. The 
patches won't compile if I split.

Am I missing something?

> I almost missed that mlx4 has been broken : you moved the drv_flags
> from the mlx4 pci driver to rte_driver.

Oh! This is indeed my mistake. I will fix this right away.

>
> Why do you push back the driver object in the 'probe' method ? (idem
> rte_bus->scan).

I am assuming you are referring to rte_driver->probe().
This is being done so that implementations (specific to drivers on a 
particular bus) can start extracting the rte_xxx_driver, if need be.

For example, for e1000/em_ethdev.c, rte_driver->probe() have been set to 
rte_eth_dev_pci_probe() which requires rte_pci_driver to work with. In 
absence of the rte_driver object, this function cannot call 
rte_pci_driver->probe (for example) for driver specific operations.

>
>
>> 0008:      Integrate probe of drivers with EAL
>
> This patch does nothing about "remove" while its title talks about it.

Yes, that is a mistake on my part. I will fix this.

>
> + What about hotplug code ? I suppose this is for later.

I am still working on it. Maybe, with the current set, if there is some 
agreement over the overall changes, I would be more confident on this 
next level of changes.

>
>
>> 0009:      Split the existing PCI probe into match and probe
>
> You don't need to expose rte_eal_pci_match_default() (and I am not
> sure what the "default" means here).
> This method should be internal to the pci bus object ?

1) Yes, rte_eal_pci_match_default isn't really a 'default' I will remove 
that.

2) Yes, there is no need to expose it.
One trivial reason I took this liberty was so that I can put my 
implementation for PCI bus in ${RTE_SDK}/drivers/bus/pci/* rather than 
current ${RTE_SDK}/lib/librte_eal/linuxapp/eal_pci.c
And similarly if there are other PCI-like implementations which would 
like to have their own bus rather than using PCI bus.

But, I can remove this (from Map as well)

>
>
>> 0010:      Make PCI probe/match work on rte_driver/device rather than
>>            rte_pci_device/rte_pci_driver
>> 0011:      Patch from Ben [8], part of series [2]
>
>
>> 0012~0013: Enable Scan/Match/probe on Bus from EAL and remove unused
>>            functions and lists
>
> Same thing as earlier in the series, you don't need to check for the bus object.
> The pci bus code can't be called if the bus was not registered and
> consequently, you are sure that the pci bus object is the pci_bus
> symbol.

Hm. Ok. I will remove this.
I was doing this for two reasons 1) somehow I found drivers getting 
registered *before* bus and for debugging that. and 2) it made me avoid 
segfaults because of my wrong code.

Anyways, it was only a debugging level change.

>
>
>
> Regards,
>


Thanks for your review. I still look forward to more discussion on 
overall approach and if there are holes which I can't see (for example, 
hotplugging).

Regards,
Shreyansh