[PATCH 2/2] service: fix potential stats race-condition on MT services

[Honnappa Nagarahalli]

<snip>
> 
> This commit fixes a potential racey-add that could occur if multiple service-
> lcores were executing the same MT-safe service at the same time, with
> service statistics collection enabled.
> 
> Because multiple threads can run and execute the service, the stats values
> can have multiple writer threads, resulting in the requirement of using
> atomic addition for correctness.
> 
> Note that when a MT unsafe service is executed, a spinlock is held, so the
> stats increments are protected. This fact is used to avoid executing atomic
> add instructions when not required.
> 
> This patch causes a 1.25x increase in cycle-cost for polling a MT safe service
> when statistics are enabled. No change was seen for MT unsafe services, or
> when statistics are disabled.
> 
> Reported-by: Mattias Rönnblom <mattias.ronnblom at ericsson.com>
> Suggested-by: Honnappa Nagarahalli <Honnappa.Nagarahalli at arm.com>
> Suggested-by: Morten Brørup <mb at smartsharesystems.com>
> Signed-off-by: Harry van Haaren <harry.van.haaren at intel.com>
> 
> ---
> ---
>  lib/eal/common/rte_service.c | 10 ++++++++--
>  1 file changed, 8 insertions(+), 2 deletions(-)
> 
> diff --git a/lib/eal/common/rte_service.c b/lib/eal/common/rte_service.c
> index ef31b1f63c..f045e74ef3 100644
> --- a/lib/eal/common/rte_service.c
> +++ b/lib/eal/common/rte_service.c
> @@ -363,9 +363,15 @@ service_runner_do_callback(struct
> rte_service_spec_impl *s,
>  		uint64_t start = rte_rdtsc();
>  		s->spec.callback(userdata);
>  		uint64_t end = rte_rdtsc();
> -		s->cycles_spent += end - start;
> +		uint64_t cycles = end - start;
>  		cs->calls_per_service[service_idx]++;
> -		s->calls++;
> +		if (service_mt_safe(s)) {
> +			__atomic_fetch_add(&s->cycles_spent, cycles,
> __ATOMIC_RELAXED);
> +			__atomic_fetch_add(&s->calls, 1,
> __ATOMIC_RELAXED);
> +		} else {
> +			s->cycles_spent += cycles;
> +			s->calls++;
This is still a problem from a reader perspective. It is possible that the writes could be split while a reader is reading the stats. These need to be atomic adds.

> +		}
>  	} else
>  		s->spec.callback(userdata);
>  }
> --
> 2.32.0