[PATCH grout] add high-performance clock

[Morten Brørup]

> From: Robin Jarry [mailto:rjarry at redhat.com]
> Sent: Tuesday, 16 June 2026 19.13
> 
> Hey Morten,
> 
> Morten Brørup, Jun 12, 2026 at 20:45:
> > I really want a high-performance clock in Grout.
> >
> > And since the rte_rdtsc() clock is out of sync with any system wide
> > clock, e.g. CLOCK_MONOTONIC_RAW, I am making the clock private to
> > Grout, and exposing an API for other processes to read it.
> >
> > This also means that gr_clock_ns() is no longer available as a
> generic
> > utility function for other processes. (I.e. it cannot be used for
> > profiling in smoke/fib_inject.c or for random generator seeding in
> > cli/main.c, so I have fixed both of these.)
> >
> > The gr_clock_ns() function has become a client API that reads the
> > clock from the Grout process.
> >
> > So far, so good!
> >
> > Now, I'm unsure about where the new clock API and API
> "implementation"
> > should go.
> >
> > So I am asking for guidance on this.
> >
> > For now, there is only one API function, "gr_clock_ns_t
> > gr_clock_ns(struct gr_api_client *);".
> >
> > I have put it in gr_clock.h as an inline function, but is that the
> > right place?
> >
> > Should I make it non-inline and add the implementation to
> > gr_api_client_impl.h?
> >
> > And should I move the clock module stuff from gr_clock.h to gr_api.h?
> >
> > The "clock" module seems essential, and not really an "infra" module,
> > so I have put it in the main/ directory rather than under
> > modules/infra/. Is that wrong? Should I just consider the "clock"
> > module an "infra" module, and move everything there?
> 
> I think you should leave gr_clock.h as it is currently (calling
> clock_gettime(CLOCK_MONOTONIC_RAW)). Having an API message to "get
> current time" is not a good idea.
> 
> The only thing needed is to ensure that datapath workers don't call
> gr_clock_ns() directly. Instead, they should call rte_rdtsc() and the
> conversion to clock_gettime(CLOCK_MONOTONIC_RAW) compatible values
> should only be done on the API border (i.e. when writing values on the
> API socket for clients consumption). That way, gr_clock_ns() can be
> used
> transparently.
> 
> As I understand, rte_rdtsc() values are not thread dependent and can be
> compared regardless from which thread they were computed. That means
> that the only thing you need to do is sample the value of gr_clock_ns()
> and rte_rdtsc() on program startup:
> 
> 	rte_eal_init(...);
> 	...
> 	gr_clock_ns_t clock_epoch = gr_clock_ns();
> 	uint64_t tsc_epoch = rte_rdtsc();
> 
> And then, do a conversion when exporting objects that expose
> gr_clock_ns_t timestamps:
> 
> 	double factor = (double)(gr_clock_ns() - clock_epoch) /
> 			(double)(rte_rdtsc() - tsc_epoch);
> 
> 	pub->timestamp = priv->timestamp * factor;
> 
> No special API message needed. Timestamps are valid from the user point
> of view and can be compared with gr_clock_ns() values.
> 
> What do you think?

That's what I thought too, when I submitted the initial patch (Tuesday 9th).

But further experimentation showed that clock_gettime(CLOCK_MONOTONIC_RAW) and rte_rdtsc() (divided by rte_get_tsc_hz()) are not in sync, not even when the clocksource is "tsc".

They are not only offset by a different epoch (which I thought), they also drift differently (which came as a surprise to me).
On a VMware virtual machine, I saw them drifting a few seconds further apart overnight.

It seems there is no simple way to convert between these two clocks, just like there is no way to convert between CLOCK_MONOTONIC_RAW and CLOCK_MONOTONIC.

Train of thought:

For now, Grout uses the timestamps for e.g. ageing in the bridge FDB, which doesn't require a high-res clock.
I could add a low-accuracy clock based on CLOCK_MONOTONIC (which is NTP adjusted), and use that for timestamping the FDB entries.
It would be somewhat similar to my suggestion about adding support for CLOCK_REALTIME (Saturday 13th).

With such a change, CLOCK_MONOTONIC would replace CLOCK_MONOTONIC_RAW as the "common clock across all processes", and gr_clock_ns() would go back to being a wrapper around clock_gettime(), but now using CLOCK_MONOTONIC instead of CLOCK_MONOTONIC_RAW.

But only provides a solution for low-res clocks.

The root cause is that we are mixing time intervals (e.g. last seen "60 seconds ago") with timestamps (e.g. last seen "17-JUN-2026 13:18:00 CEST"). We do this, because we have to. E.g. we put a timestamp on an FDB entry, so we can calculate its age (the time interval).


Another solution would be to convert between relevant clock domains at the API border.
But we might extend the API border with shared memory for accessing very large tables, which may contain entries with Grout timestamps.
If so, the non-Grout process needs to understand Grout's clock domain, to be able to convert to its own clock domain.


The solution I had proposed - making gr_clock_ns() a client API - solves this challenge:
The API allows reading Grout's clock through gr_clock_ns(), so the non-Grout application can calculate the age of an entry by subtracting the entry's timestamp from the return value of gr_clock_ns().
And if it needs a wall clock timestamp of the entry, it can subtract the age from the return value of clock_gettime(CLOCK_REALTIME).


PS: Your understanding of rte_rdtsc() is correct. It is thread independent and can be compared across threads.