<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
Hello Bruce & Harry,</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
There is an aspect we've not discussed yet.</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
DPDK is a framework. It's integrated into a network application.</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
>From the application perspective what is a ratio between "pure" application code and DPDK API ?<br>
The exact numbers differ, but it's clear that most of application code is not about DPDK.</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
Another question to consider - what is more complicated </div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
rewrite entire application from C to Rust or, while having Rust application, upgrade or even replace DPDK API ? </div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
DPDK provides a solid framework for both stability and performance. </div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
In my opinion, binding DPDK as it is today with Rust can significantly improve application design. </div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<br>
</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
Regards,</div>
<div class="elementToProof" style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
Gregory</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Van Haaren, Harry <harry.van.haaren@intel.com><br>
<b>Sent:</b> Friday, May 2, 2025 16:58<br>
<b>To:</b> Gregory Etelson <getelson@nvidia.com>; Richardson, Bruce <bruce.richardson@intel.com><br>
<b>Cc:</b> dev@dpdk.org <dev@dpdk.org>; owen.hilyard@unh.edu <owen.hilyard@unh.edu><br>
<b>Subject:</b> Re: [PATCH] rust: RFC/demo of safe API for Dpdk Eal, Eth and Rxq</font>
<div> </div>
</div>
<div class="BodyFragment"><font size="2"><span style="font-size:11pt;">
<div class="PlainText">External email: Use caution opening links or attachments<br>
<br>
<br>
> From: Etelson, Gregory<br>
> Sent: Friday, May 02, 2025 1:46 PM<br>
> To: Richardson, Bruce<br>
> Cc: Gregory Etelson; Van Haaren, Harry; dev@dpdk.org; owen.hilyard@unh.edu<br>
> Subject: Re: [PATCH] rust: RFC/demo of safe API for Dpdk Eal, Eth and Rxq<br>
><br>
> Hello Bruce,<br>
<br>
Hi All,<br>
<br>
> > Thanks for sharing. However, IMHO using EAL for thread management in rust<br>
> > is the wrong interface to expose.<br>
><br>
> EAL is a singleton object in DPDK architecture.<br>
> I see it as a hub for other resources.<br>
<br>
Yep, i tend to agree here; EAL is central to the rest of DPDK working correctly.<br>
And given EALs implementation is heavily relying on global static variables, it is<br>
certainly a "singleton" instance, yes.<br>
<br>
> Following that idea, the EAL structure can be divided to hold the<br>
> "original" resources inherited from librte_eal and new resources<br>
> introduced in Rust EAL.<br>
<br>
Here we can look from different perspectives. Should "Rust EAL" even exist?<br>
If so, why? The DPDK C APIs were designed in baremetal/linux days, where<br>
certain "best-practices" didn't exist yet, and Rust language was pre 1.0 release.<br>
<br>
Of course, certain parts of Rust API must depend on EAL being initialized.<br>
There is a logical flow to DPDK initialization, these must be kept for correct functionality.<br>
<br>
I guess I'm saying, perhaps we can do better than mirroring the concept of<br>
"DPDK EAL in C" in to "DPDK EAL in Rust".<br>
<br>
> > Instead, I believe we should be<br>
> > encouraging native rust thread management, and not exposing any DPDK<br>
> > threading APIs except those necessary to have rust threads work with DPDK,<br>
> > i.e. with an lcore ID. Many years ago when DPDK started, and in the C<br>
> > world, having DPDK as a runtime environment made sense, but times have<br>
> > changed and for Rust, there is a whole ecosystem out there already that we<br>
> > need to "play nice with", so having Rust (not DPDK) do all thread<br>
> > management is the way to go (again IMHO).<br>
> ><br>
><br>
> I'm not sure what exposed DPDK API you refer to.<br>
<br>
I think that's the point :) Perhaps the Rust application should decide how/when to<br>
create threads, and how to schedule & pin them. Not the "DPDK crate for Rust".<br>
To give a more concrete examples, lets look at Tokio (or Monoio, or Glommio, or .. )<br>
which are prominent players in the Rust ecosystem, particularly for networking workloads<br>
where request/response patterns are well served by the "async" programming model (e.g HTTP server).<br>
<br>
Lets focus on Tokio first: it is an "async runtime" (two links for future readers)<br>
<a href="https://corrode.dev/blog/async/">https://corrode.dev/blog/async/</a><br>
<a href="https://rust-lang.github.io/async-book/08_ecosystem/00_chapter.html">
https://rust-lang.github.io/async-book/08_ecosystem/00_chapter.html</a><br>
So an async runtime can run "async" Rust functions (called Futures, or Tasks when run independently..)<br>
There are lots of words/concepts, but I'll focus only on the thread creation/control aspect, given the DPDK EAL lcore context.<br>
<br>
Tokio is a work-stealing scheduler. It spawns "worker" threads, and then gives these "tasks"<br>
to various worker cores (similar to how Golang does its work-stealing scheduling). Some<br>
DPDK crate users might like this type of workflow, where e.g. RXQ polling is a task, and the<br>
"tokio runtime" figures out which worker to run it on. "Spawning" a task causes the "Future"<br>
to start executing. (technical Rust note: notice the "Send" bound on Future: <a href="https://docs.rs/tokio/latest/tokio/task/fn.spawn.html">
https://docs.rs/tokio/latest/tokio/task/fn.spawn.html</a> )<br>
<br>
Other users might prefer the "thread-per-core" and CPU pinning approach (like DPDK itself would do).<br>
Monoio and Glommio both serve these use cases (but in slightly different ways!). They both spawn threads and do CPU pinning.<br>
Monoio and Glommio say "tasks will always remain on the local thread". In Rust techie terms: "Futures are !Send and !Sync"<br>
<a href="https://docs.rs/monoio/latest/monoio/fn.spawn.html">https://docs.rs/monoio/latest/monoio/fn.spawn.html</a><br>
<a href="https://docs.rs/glommio/latest/glommio/fn.spawn_local.html">https://docs.rs/glommio/latest/glommio/fn.spawn_local.html</a><br>
<br>
So there are at least 3 different async runtimes (and I haven't even talked about async-std, smol, embassy, ...) which<br>
all have different use-cases, and methods of running "tasks" on threads. These runtimes exist, and are widely used,<br>
and applications make use of their thread-scheduling capabilities.<br>
<br>
So "async runtimes" do thread creation (and optionally CPU pinning) for the user.<br>
Other libraries like "Rayon" are thread-pool managers, those also have various CPU thread-create/pinning capabilities.<br>
If DPDK *also* wants to do thread creation/management and CPU-thread-to-core pinning for the user, that creates tension.<br>
<br>
> Bruce wrote: "so having Rust (not DPDK) do all thread management is the way to go (again IMHO)."<br>
<br>
I think I agree here, in order to make the Rust DPDK crate usable from the Rust ecosystem,<br>
it must align itself with the existing Rust networking ecosystem.<br>
<br>
That means, the DPDK Rust crate should not FORCE the usage of lcore pinnings and mappings.<br>
Allowing a Rust application to decide how to best handle threading (via Rayon, Tokio, Monoio, etc)<br>
will allow much more "native" or "ergonomic" integration of DPDK into Rust applications.<br>
<br>
> Regards,<br>
> Gregory<br>
<br>
Apologies for the long-form, "wall of text" email, but I hope it captures the nuance of threading and<br>
async runtimes, which I believe in the long term will be very nice to capture "async offload" use-cases<br>
for DPDK. To put it another way, lookaside processing can be hidden behind async functions & runtimes,<br>
if we design the APIs right: and that would be really cool for making async-offload code easy to write correctly!<br>
<br>
Regards, -Harry<br>
</div>
</span></font></div>
</body>
</html>