[dpdk-users] Send and Receive packets to/from specific core

[Kyle Larose]

Hi Mahdi,

On Fri, Mar 11, 2016 at 5:19 PM, Mahdi Moradmand Badie
<mahdi.mbadie at gmail.com> wrote:
> So you mean,
> each core works with its thread as a worker, and in the ring as a share
> memory, all cores(via their threads) could write and read in each part of
> the ring (but it is better each core access to its threads).
> My questions:
> 1) How I could access to cache/register/etc of each core if so?

As far as I'm aware, x86 CPUs do not give direct access to the cache.
This is handled by the CPU itself through its hierarchical caching
structures. Regarding register access, you'll need to write some
assembly to get access to those.

Now, maybe CAT exposes this level of information. I always thought it
was about controlling access to the cache by configuring some stuff,
but maybe it requires intervention from the kernel to manage it. If
so, you could possibly look at doing that (I.e. write a kmod, or use
some sort of existing user-space<->communication channel that lets you
do it).

> 2) How could I consider ring as share cache (L3), because here we use ring
> as a share memory but if I wanna use share cache how I could do it, is there
> any availability in DPDK?

The cache will be used automatically.

> 3) Is there any possibility to use ring as real memory, I mean ,Can I ask
> the core to write the message in the specific part of ring (from this
> address to this address)?

The ring *is* real memory. It's a ring buffer of fixed-size entries,
so typically you will "write" an entry to the ring in one process, and
"read" that entry in another. You could read/write an arbitrary offset
from the beginning of the ring, but it's not really meant for that.
You'd probably want to build your own data structure on top of shared
memory if you need that.

> 4) I wanna have a share memory (ring or whatever) with for example the size
> 4 (4 free place for read and write) which filled at first with 1, 7, 3, 4.
> Then each core (0, 1, 2, 3 and I have totally 4 cores in my machine) read
> the slot and add 1 to it and again write it in share memory so the result
> should be  (2, 8, 4, 5), this is possible do with ring? if so how? and each
> core just could access to its specific part (or better in order to prevent
> bad design)?
>
>

A ring isn't really appropriate here, since it's meant to produce and
consume messages in a fifo order. Here you want random access. An
array in shared memory would be better. Alternatively, you could
create a ring to each process. That only gets you the read and modify
though. You still need to write back the result. Where do you expect
to access it? \

If you want to use arrays, consider using the rte_memzone_reserve
function to reserve some memory. Take a look at the memzone api and
how it is used. Perhaps it will give you some ideas:
http://dpdk.org/doc/api/rte__memzone_8h.html.


> Sorry because I am new in DPDK and with not enough knowledge of C maybe my
> questions seems crazy :)
> Thanks,
>
> On 11 March 2016 at 16:50, Kyle Larose <eomereadig at gmail.com> wrote:
>>
>> On Fri, Mar 11, 2016 at 4:01 PM, Mahdi Moradmand Badie
>> <mahdi.mbadie at gmail.com> wrote:
>> > Yes, I found it before :), its a little ambiguous for me, I don't know
>> > how I
>> > could write the specific data(message or packet) into specific core,
>> > then
>> > change it and then read with the other core.
>>
>> Well, when one process writes to the other, it moves between threads.
>> Each thread is pinned to a specific core (i.e. processor affinity).
>> You can see that this is happening by looking at the output from the
>> example. It mentions on the page I linked something like "Starting
>> core 9" or "core 9: Received...". This means that the application has
>> started its worker thread and associated it with core 9. It will only
>> ever run on that core. Thus, in order for it to actually process the
>> message, the contents of the message must be copied into the
>> cache/registers/etc. of that core.
>>
>> To control which cores are used, you use the "-c" argument, and give
>> the "core mask" in hex. For example, -c 3 would use cores 0 and 1.
>>
>> The key things to take away from this are that threads are constrained
>> by the scheduler to only run on specific cores using processor
>> affinity, and that message queues in the form of rings can be used to
>> pass messages between these cores. A message processed on a thread
>> assigned to core X will be modified/accessed locally on that core.
>> This doesn't mean other cores couldn't access it. But, that should be
>> prevented by good design.
>>
>>
>> >
>> > On 11 March 2016 at 15:54, Kyle Larose <eomereadig at gmail.com> wrote:
>> >>
>> >> On Fri, Mar 11, 2016 at 3:47 PM, Mahdi Moradmand Badie
>> >> <mahdi.mbadie at gmail.com> wrote:
>> >> > My big challenge is having a tested code (run able with right
>> >> > functionality), then run and change it in order to have my task.
>> >> > I wanna use the share memory btw cores in order to write/read to/from
>> >> > and
>> >> > knowing how do that :), all available code which I found are based on
>> >> > NIC :(
>> >> > or maybe I am wrong!!!
>> >> > Thanks in advance,
>> >>
>> >> Take a look at this example:
>> >>
>> >>
>> >>
>> >> http://dpdk.org/doc/guides/sample_app_ug/multi_process.html#basic-multi-process-example
>> >>
>> >> IIRC, it uses rings to send messages back and forth between the cores.
>> >> The code is available with the dpdk source, so it should serve as a
>> >> good starting point to see how to set up the shared memory and use it.
>> >>
>> >> Does that help?
>> >>
>> >> >
>> >> > On 11 March 2016 at 15:43, Kyle Larose <eomereadig at gmail.com> wrote:
>> >> >>
>> >> >> On Fri, Mar 11, 2016 at 3:22 PM, Mahdi Moradmand Badie
>> >> >> <mahdi.mbadie at gmail.com> wrote:
>> >> >> > Dear All,
>> >> >> >
>> >> >> > I wanna Send(write) a Packet(s) with specific data (for example x
>> >> >> > =
>> >> >> > 10)
>> >> >> > from core 0 to share memory and Receive (Read) it with another
>> >> >> > Core
>> >> >> > (for
>> >> >> > example Core 1), change it in Core 1 (for example X = X + 1) and
>> >> >> > write
>> >> >> > again in share memory.
>> >> >> > I really don't know How I could do it, I wanna do this without
>> >> >> > using
>> >> >> > NIC
>> >> >> > ot
>> >> >> > rte_eth at all, so simple and easy but I confused.
>> >> >> > Please help me.
>> >> >> >
>> >> >> > PS. I know there are many example do more complex than this small
>> >> >> > exercise
>> >> >> > but all did it via NIC.
>> >> >>
>> >> >> What is your biggest challenge? Is it moving information between
>> >> >> cores, or getting information into your application?
>> >> >>
>> >> >> Consider that most of the multiprocess examples in DPDK do two
>> >> >> things:
>> >> >> 1) Send/Receive packets to/from a NIC
>> >> >> 2) Send packets between cores
>> >> >>
>> >> >> You obviously want to do #2, and looking at those examples should
>> >> >> make
>> >> >> how to do it fairly obvious. Is your challenge replacing #1 with
>> >> >> something other than a NIC? If so, you *could* consider using a pcap
>> >> >> PMD to just read packets from a file. In the past, I have used ring
>> >> >> PMDs in conjunction with a secondary process which generates packets
>> >> >> to inject arbitrarily formatted packets into my program.
>> >> >>
>> >> >> >
>> >> >> > --
>> >> >> > M at hdi Mor at dm@nd B at die
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > M at hdi Mor at dm@nd B at die
>> >
>> >
>> >
>> >
>> > --
>> > M at hdi Mor at dm@nd B at die
>
>
>
>
> --
> M at hdi Mor at dm@nd B at die