[dpdk-dev] [RFC] DFS (DPDK Filesystem)

[Wiles, Keith]

Sent from my iPhone

> On Nov 30, 2018, at 10:01 AM, Wiles, Keith <keith.wiles at intel.com> wrote:
> 
> Plain text format is a real pain unless I time travel to the 1990’s :-(
> 
>> On Nov 30, 2018, at 7:13 AM, Zhang, Qi Z <qi.z.zhang at intel.com> wrote:
>> 
>> Hi Keith:
>> 
>> 
>>> -----Original Message-----
>>> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Wiles, Keith
>>> Sent: Tuesday, November 27, 2018 11:00 PM
>>> To: dpdk-dev <dev at dpdk.org>
>>> Cc: Richardson, Bruce <bruce.richardson at intel.com>; Yigit, Ferruh
>>> <ferruh.yigit at intel.com>; Thomas Monjalon <thomas at monjalon.net>;
>>> Ananyev, Konstantin <konstantin.ananyev at intel.com>
>>> Subject: Re: [dpdk-dev] [RFC] DFS (DPDK Filesystem)
>>> 
>>> 
>>> 
>>>> On Oct 8, 2018, at 5:42 PM, Wiles, Keith <keith.wiles at intel.com> wrote:
>>>> 
>>>> Hi Everyone,
>>>> 
>>>> I would like to request comments on DFS as I presented at the DPDK summit
>>> in Ireland.
>>>> 
>>>> As we know DPDK can be difficult to manage at run time and as DPDK
>>> becomes more dynamic we need to address how to configure and monitor
>>> DPDK and DPDK-based applications. A possible solution is to use FUSE file
>>> system support already in the Linux kernel through a DPDK library to expose a
>>> directory structure for anyone to be able to monitor and control the app by
>>> reading/writing files.
>>>> 
>>>> Primarily DPDK runs on Linux based systems, where FUSE is well supported.
>>> FUSE was added to FreeBSD 10 which appears to be the same design as Linux.
>>> Windows does not support FUSE directly from MS, but a open source design
>>> has a FUSE based set of wrapper APIs to help port FUSE based file systems.
>>>> 
>>>> Today the only way to get information out of DPDK is to use log files or a
>>> command line within DPDK to display the information. The information is not
>>> easy for an external application to collect or monitor. Using DFS any application
>>> can read and write into this virtual file system (FUSE) and retrieve information
>>> from DPDK/Application via a raw ASCII text file and/or JSON formatted text.
>>>> 
>>>> Data read from DFS can be static or dynamic e.g. version/copyright text
>>> would be static, but port stats would be dynamic and generated when the user
>>> requests the information. Writing new configuration data into DPDK can be a
>>> simple number or text or a JSON data structure. Writing simple ASCII text data
>>> is simple, but writing a long JSON or configuration file may some care, i.e. write
>>> a path to a file in DFS to the location of the configuration and use that file path
>>> for the configuration.
>>>> 
>>>> DPDK command line is getting to be very complex and long, but we can
>>> eliminate or greatly reduce the command line by using DFS to configure DPDK
>>> at startup along with the application configuration. Having DPDK start up with
>>> minimum resources and allow updating/creating a file in the DPDK file system
>>> to configure DPDK would be easier.
>>>> 
>>>> The DFS is a FUSE filesystem anchored at /dpdk in the host file system, then a
>>> directory is created under /dpdk for each DPDK instance or application using
>>> <name>-<PID> as a directory name format. Then it is up to DPDK/Application to
>>> create the directory structures, permissions and files within the directory
>>> mount point.
>>>> 
>>>> Currently I have a directory and files layout for DPDK, that needs more
>>> attention as I was focusing on FUSE then the layout as it can be changed fairly
>>> easily. I can email the working documentation for DFS if needed please send me
>>> a short email or I can post to the dev list. The docs also try to explain the design
>>> and APIs used by the developer for creating files/directories. The docs are still a
>>> work in progress and more attention needs to be done as we progress.
>>>> 
>>>> DFS is just a library and not required to be used plus it does not modify DPDK
>>> APIs or structures except for adding a few new dump routines for features that
>>> do not have one today. The file system or FUSE backend in DPDK is a free
>>> floating thread at this time, but could be affinitized to a core if required.
>>>> 
>>>> The FUSE protocol to the backend code (in DPDK) has a large number of
>>> operations, but we have simplified the interface for developers to 5-6. The
>>> interface on top of libfuse3 is very simple and currently provides most of the
>>> needed APIs for managing the files and directories. The FUSE file system with
>>> the added simpler API is dynamic and can be changed at runtime. The
>>> directory/files can be constructed via APIs at startup or anytime after startup.
>>>> 
>>>> I would like to put a repo someplace with DPDK/DFS say on GitHub to allow
>>> others to play with the design in the short term. I also have a doc file for DFS if
>>> anyone wants to learn more information and will be in the repo as well. If we
>>> agree I can then create a set of patches for DPDK.
>>> 
>>> Well the 18.11 release is out the door, this one seemed pretty big and I lot of
>>> hard work was put into this one, thanks to everyone.
>>> 
>>> 
>>> It is time to try and get some feedback on this RFC. I know the code in the repo
>>> is not finished, but I would like to get some folks to give it a try and give
>>> feedback. I would like to submit the basic DFS code with some low hanging
>>> items mostly readonly information for the 19.02 release.
>>> 
>>> Here is the repo again you can look at and play with. The code is mostly ready
>>> for upstream but still more work needs to be done. I will try to update the repo
>>> to the 18.11 release code based soon.
>>> 
>>> https://github.com/pktgen/dfs-dpdk
>>> 
>>> Thanks for giving this a review.
>> 
>> Some thoughts base on a quick test and review.
>> 
>> Overall, I think this is a very handy feature. I really hope it can be merged quickly :)
>> 
>> Below are some captures:
>> 
>> 1) when I try to test dfs with exist application (say testpmd), I need to replace rte_eal_init with dfs_eal_init, 
>> I think a better way is by using args like --dfs which can avoid the code change, but please let me know if I missed the point.
> 
> This is a good idea, let me look into what that would take. I do not want to require DFS to be linked into the image in this case so —dfs need to be a noop if not present.

I did change the code to use the —dfs option and it looks like it will work out well. Will update the repo soon with changes. 
>> 
>> 2) Is PMD allowed to use dfs to expose device specific information (private API access, utilities for diagnose purpose ...) ? if it is 
>> I think dfs could provide more standard way for PMD to add content to a sub directory , It could be 
>>    struct dfs_ops {
>>        mount(dfs_node parent...);   /* create content of in the sub directory */
>>        unmount(dfs_node parent...); /* destroy the content. */
>>    }    
>> 
>> PMD is responsible for creating a dfs_ops instance and setup the callback functions during initialization.(probe)
>> An API rte_eth_dev_get_dfs_ctx(port_id) can be used to get the context that wrapped dfs_ops and related ethdev instance. (similar for cryptodev and others )
>> So during dfs init, after we create folder for a port, we can do additional step to create a parent node for PMD then call its's dev_ops->mount, then all the device specific information will be popped in that folder
>> 
> 
> This seems like a reasonable option, it sounds like a callback from the DFS init or possible a constructor method.
> 
> PMD is responsible for creating a dfs_ops instance and setup the callback functions during initialization.(probe)
> An API rte_eth_dev_get_dfs_ctx(port_id) can be used to get the context that wrapped dfs_ops and related ethdev instance. (similar for cryptodev and others )
> So during dfs init, after we create folder for a port, we can do additional step to create a parent node for PMD then call its's dev_ops->mount, then all the device specific information will be popped in that folder
>> 3) For primary-secondary model, dfs should works well, but the missing part is there is no link between primary and secondary process, 
>> it will be nature if we can list all secondary process in a primary folder or even better if we can jump to secondary folder by some soft link and same thing in the reverse direction. 
> 
> I think this can be done, but each secondary would have its own dfs file system. Adding the links could be reasonable if we could assume the secondary processes are always up.
> 
>> 
>> Thanks
>> Qi
>> 
>>> 
>>>> 
>>>> Regards,
>>>> Keith
>>>> 
>>> 
>>> Regards,
>>> Keith
> 
> Regards,
> Keith
>