[dpdk-dev] [PATCH v2] mbuf: support dynamic fields and flags

[Wang, Haiyue]

> -----Original Message-----
> From: Stephen Hemminger [mailto:stephen at networkplumber.org]
> Sent: Wednesday, October 23, 2019 23:00
> To: Olivier Matz <olivier.matz at 6wind.com>
> Cc: Wang, Haiyue <haiyue.wang at intel.com>; Ananyev, Konstantin <konstantin.ananyev at intel.com>;
> dev at dpdk.org; Andrew Rybchenko <arybchenko at solarflare.com>; Richardson, Bruce
> <bruce.richardson at intel.com>; Jerin Jacob Kollanukkaran <jerinj at marvell.com>; Wiles, Keith
> <keith.wiles at intel.com>; Morten Brørup <mb at smartsharesystems.com>; Thomas Monjalon
> <thomas at monjalon.net>
> Subject: Re: [PATCH v2] mbuf: support dynamic fields and flags
> 
> On Wed, 23 Oct 2019 12:21:43 +0200
> Olivier Matz <olivier.matz at 6wind.com> wrote:
> 
> > On Wed, Oct 23, 2019 at 03:16:13AM +0000, Wang, Haiyue wrote:
> > > > -----Original Message-----
> > > > From: Ananyev, Konstantin
> > > > Sent: Wednesday, October 23, 2019 06:52
> > > > To: Olivier Matz <olivier.matz at 6wind.com>; dev at dpdk.org
> > > > Cc: Andrew Rybchenko <arybchenko at solarflare.com>; Richardson, Bruce <bruce.richardson at intel.com>;
> Wang,
> > > > Haiyue <haiyue.wang at intel.com>; Jerin Jacob Kollanukkaran <jerinj at marvell.com>; Wiles, Keith
> > > > <keith.wiles at intel.com>; Morten Brørup <mb at smartsharesystems.com>; Stephen Hemminger
> > > > <stephen at networkplumber.org>; Thomas Monjalon <thomas at monjalon.net>
> > > > Subject: RE: [PATCH v2] mbuf: support dynamic fields and flags
> > > >
> > > >
> > > > > Many features require to store data inside the mbuf. As the room in mbuf
> > > > > structure is limited, it is not possible to have a field for each
> > > > > feature. Also, changing fields in the mbuf structure can break the API
> > > > > or ABI.
> > > > >
> > > > > This commit addresses these issues, by enabling the dynamic registration
> > > > > of fields or flags:
> > > > >
> > > > > - a dynamic field is a named area in the rte_mbuf structure, with a
> > > > >   given size (>= 1 byte) and alignment constraint.
> > > > > - a dynamic flag is a named bit in the rte_mbuf structure.
> > > > >
> > > > > The typical use case is a PMD that registers space for an offload
> > > > > feature, when the application requests to enable this feature.  As
> > > > > the space in mbuf is limited, the space should only be reserved if it
> > > > > is going to be used (i.e when the application explicitly asks for it).
> > > > >
> > > > > The registration can be done at any moment, but it is not possible
> > > > > to unregister fields or flags for now.
> > > > >
> > > > > Signed-off-by: Olivier Matz <olivier.matz at 6wind.com>
> > > > > Acked-by: Thomas Monjalon <thomas at monjalon.net>
> > > > > ---
> > > > >
> > > > > v2
> > > > >
> > > > > * Rebase on top of master: solve conflict with Stephen's patchset
> > > > >   (packet copy)
> > > > > * Add new apis to register a dynamic field/flag at a specific place
> > > > > * Add a dump function (sugg by David)
> > > > > * Enhance field registration function to select the best offset, keeping
> > > > >   large aligned zones as much as possible (sugg by Konstantin)
> > > > > * Use a size_t and unsigned int instead of int when relevant
> > > > >   (sugg by Konstantin)
> > > > > * Use "uint64_t dynfield1[2]" in mbuf instead of 2 uint64_t fields
> > > > >   (sugg by Konstantin)
> > > > > * Remove unused argument in private function (sugg by Konstantin)
> > > > > * Fix and simplify locking (sugg by Konstantin)
> > > > > * Fix minor typo
> > > > >
> > > > > rfc -> v1
> > > > >
> > > > > * Rebase on top of master
> > > > > * Change registration API to use a structure instead of
> > > > >   variables, getting rid of #defines (Stephen's comment)
> > > > > * Update flag registration to use a similar API as fields.
> > > > > * Change max name length from 32 to 64 (sugg. by Thomas)
> > > > > * Enhance API documentation (Haiyue's and Andrew's comments)
> > > > > * Add a debug log at registration
> > > > > * Add some words in release note
> > > > > * Did some performance tests (sugg. by Andrew):
> > > > >   On my platform, reading a dynamic field takes ~3 cycles more
> > > > >   than a static field, and ~2 cycles more for writing.
> > > > >
> > > > >  app/test/test_mbuf.c                   | 145 ++++++-
> > > > >  doc/guides/rel_notes/release_19_11.rst |   7 +
> > > > >  lib/librte_mbuf/Makefile               |   2 +
> > > > >  lib/librte_mbuf/meson.build            |   6 +-
> > > > >  lib/librte_mbuf/rte_mbuf.h             |  23 +-
> > > > >  lib/librte_mbuf/rte_mbuf_dyn.c         | 548 +++++++++++++++++++++++++
> > > > >  lib/librte_mbuf/rte_mbuf_dyn.h         | 226 ++++++++++
> > > > >  lib/librte_mbuf/rte_mbuf_version.map   |   7 +
> > > > >  8 files changed, 959 insertions(+), 5 deletions(-)
> > > > >  create mode 100644 lib/librte_mbuf/rte_mbuf_dyn.c
> > > > >  create mode 100644 lib/librte_mbuf/rte_mbuf_dyn.h
> > > > >
> > > > > diff --git a/app/test/test_mbuf.c b/app/test/test_mbuf.c
> > > > > index b9c2b2500..01cafad59 100644
> > > > > --- a/app/test/test_mbuf.c
> > > > > +++ b/app/test/test_mbuf.c
> > > > > @@ -28,6 +28,7 @@
> > > > >  #include <rte_random.h>
> > > > >  #include <rte_cycles.h>
> > > > >  #include <rte_malloc.h>
> > > > > +#include <rte_mbuf_dyn.h>
> > > > >
> > >
> > > [snip]
> > > > > +int
> > > > > +rte_mbuf_dynflag_register_bitnum(const struct rte_mbuf_dynflag *params,
> > > > > +				unsigned int req)
> > > > > +{
> > > > > +	int ret;
> > > > > +
> > > > > +	if (req != UINT_MAX && req >= 64) {
> > > >
> > > > Might be better to replace 64 with something like sizeof(mbuf->ol_flags) * CHAR_BIT or so.
> > >
> > > Might introduce a new macro like kernel:
> > >
> > > /**
> > >  * FIELD_SIZEOF - get the size of a struct's field
> > >  * @t: the target struct
> > >  * @f: the target struct's field
> > >  * Return: the size of @f in the struct definition without having a
> > >  * declared instance of @t.
> > >  */
> > > #define FIELD_SIZEOF(t, f) (sizeof(((t*)0)->f))
> > >
> > > Then: FIELD_SIZEOF(rte_mbuf, ol_flags) * CHAR_BIT
> >
> > Good idea, thanks
> >
> 
> Kernel is replacing FIELD_SIZEOF with sizeof_member

Yes, but looks like in 5.5 ? 5.4 hasn't merged. ;-)

https://www.phoronix.com/scan.php?page=news_item&px=Linux-5.4-Size-Of-Member

https://patchwork.kernel.org/patch/11184583/

+/**
+ * sizeof_member(TYPE, MEMBER) - get the size of a struct's member
+ *
+ * @TYPE: the target struct
+ * @MEMBER: the target struct's member
+ *
+ * Return: the size of @MEMBER in the struct definition without having a
+ * declared instance of @TYPE.
+ */
+#define sizeof_member(TYPE, MEMBER)	(sizeof(((TYPE *)0)->MEMBER))