[dpdk-dev] mlx5 & pdump: convert HW timestamps to nanoseconds

[Slava Ovsiienko]

Hi, Patrick

ConnectX HW timestamp is the captured value of internal 64-bit counter running at the frequency,
reported in the device_frequency_khz field of struct mlx5_ifc_cmd_hca_cap_bits{}.
This structure is queried in mlx5_devx_cmd_query_hca_attr() routine.
So, with known frequency it is possible to recalculate timestamp ticks to desired units.

With best regards, Slava

> -----Original Message-----
> From: dev <dev-bounces at dpdk.org> On Behalf Of PATRICK KEROULAS
> Sent: Friday, May 22, 2020 21:43
> To: Thomas Monjalon <thomas at monjalon.net>
> Cc: dev at dpdk.org; Vivien Didelot <vivien.didelot at gmail.com>; Shahaf
> Shuler <shahafs at mellanox.com>; Raslan Darawsheh
> <rasland at mellanox.com>; Matan Azrad <matan at mellanox.com>
> Subject: Re: [dpdk-dev] mlx5 & pdump: convert HW timestamps to
> nanoseconds
> 
> > > > > mlx5 part of libibverbs includes a ts-to-ns converter which
> > > > > takes the instantaneous clock info. It's unused in dpdk so far.
> > > > > I've tested it in the device/port init routine and the result
> > > > > looks reliable. Since this approach looks very simple, compared
> > > > > to the time sync mechanism, I'm trying to integrate.
> > > > >
> > > > > The conversion should occur in the primary process (testpmd) I
> suppose.
> > > > > 1) The needed clock info derives from ethernet device. Is it possible to
> > > > >    access that struct from a rx callback?
> > > > > 2) how to attach the nanosecond to mbuf so that `pdump` catches it?
> > > > >    (workaround: copy `mbuf->udata64` in forwarded packets.)
> > > > > 3) any other idea?
> > > >
> > > > The timestamp is carried in mbuf.
> > > > Then the conversion must be done by the ethdev caller (application
> > > > or any other upper layer).
> > >
> > > What if the converter function needs a clock_info?
> > > https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgi
> > > thub.com%2Flinux-rdma%2Frdma-
> core%2Fblob%2F7af01c79e00555207dee6132d
> > >
> 72e7bfc1bb5485e%2Fproviders%2Fmlx5%2Fmlx5dv.h%23L1201&data=
> 02%7C
> > >
> 01%7Cviacheslavo%40mellanox.com%7C381f1c9dd36f4e18e9c908d7fe8001
> 3b%7
> > >
> Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C63725769806348887
> 6&s
> > >
> data=CNc%2B5dyFCeFRQn56S5NNzEfTCtnInm059wxwX5GX96E%3D&re
> served=0
> > > I'm affraid this info may change by the time the converter is called
> > > by upper layer.
> >
> > Indeed, the clock in the device is not an atomic one :) We need to
> > adjust the time conversion continuously.
> > I am not an expert of time synchronization, so I add more people Cc
> > who could help for having a precise timestamp.
> 
> Thanks Thomas.
> Not sure this is a synchronization issue. We have dedicated processes
> (linuxptp) to keep both NIC and sys clocks in sync with an external clock.
> It is "just" a matter of unit conversion.
> 
> If it has to be performed in dpdk-pdump, I would need some help to retrieve
> mlx5_clock_info from inside a secondary process. Looking at
> mlx5_read_clock(), this info is extracted from ibv_context which looks
> reachable in a primary process only (segfault, if I try in pdump).