[dpdk-dev] Api in dpdk to get total free physical memory

[Burakov, Anatoly]

On 09-Mar-18 9:59 AM, Venumadhav Josyula wrote:
> Hi Anatoly,
> 
> Like we have api, rte_eal_get_physmem_size, which returns the total 
> memory physical Ram memory. This is eal_common_memory.c, we need 
> following following which would return free memory. For that that I was 
> referring we need to api get the free physical ram memory 
> ‘rte_eal_get_physmem_free’ for the we wanted to submit patch.
> 
> Thanks,
> 
> Regards
> 
> Venumadhav

Hi Venumadhav,

Yes, i understand what you're referring to. I'm just saying that 1) 
getting malloc heap stats will provide you with similar (and arguably 
more practically useful) information, and 2) there are changes 
(hopefully) coming that will lessen usefulness of such an API because it 
would be possible to allocate hugepages at runtime, instead of 
preallocating them at startup.

You're welcome to submit your patches, of course, but i would question 
the usefulness of such an API in light of coming changes to memory 
subsystem, because the "free" memory will not be the actual amount of 
free memory you have. Imagine the following:

// you have roughly 1MB of free memory
free_mem = rte_eal_get_free_mem();
printf("%luMB\n", free_mem >> 20); // prints "1MB"

rte_malloc(1 << 20); // allocate 1MB of memory
                      // causes allocation of 1G page

free_mem = rte_eal_get_free_mem();
printf("%luMB\n", free_mem >> 20); // prints "1024MB"

A second ago you had 1MB of "free space", next second you have 1G free 
space, despite the fact that you didn't free any memory - you just 
allocated more pages. "Amount of free space" will have little meaning 
when you can allocate more pages on the fly. I hope i've explained it 
clearly enough.

> 
> *From:*dev [mailto:dev-bounces at dpdk.org] *On Behalf Of *Burakov, Anatoly
> *Sent:* Friday, March 09, 2018 2:36 PM
> *To:* Venumadhav Josyula <vjosyula at gmail.com>; dev at dpdk.org
> *Subject:* Re: [dpdk-dev] Api in dpdk to get total free physical memory
> 
> On 08-Mar-18 9:36 PM, Venumadhav Josyula wrote:
>  > Hi All,
>  >
>  >
>  >
>  > Like ‘rte_eal_get_physmem_size’ api to the total size of the physical
>  > memory. Is there an API to get to get total free memory physical memory
>  > available ?
>  >
>  >
>  >
>  > We want such API we are planning to implement such API for the same
>  >
>  >
>  >
>  > /* get the total size of memory */
>  >
>  > uint64_t
>  >
>  > rte_eal_get_physmem_free(int socket_id)
>  >
>  > {
>  >
>  > const struct rte_mem_config *mcfg;
>  >
>  > unsigned i = 0;
>  >
>  > uint64_t total_len = 0;
>  >
>  >
>  >
>  > /* get pointer to global configuration */
>  >
>  > mcfg = rte_eal_get_configuration()->mem_config;
>  >
>  >
>  >
>  > for (i=0; i<RTE_MAX_MEMSEG; i++) {
>  >
>  > if (mcfg->free_memseg[i].addr == NULL)
>  >
>  > break;
>  >
>  >
>  >
>  > if (mcfg->free_memseg[i].len == 0)
>  >
>  > continue;
>  >
>  >
>  >
>  > /* bad socket ID */
>  >
>  > if (socket_id != SOCKET_ID_ANY &&
>  >
>  > mcfg->free_memseg[i].socket_id != SOCKET_ID_ANY &&
>  >
>  > socket_id != mcfg->free_memseg[i].socket_id)
>  >
>  > continue;
>  >
>  >
>  >
>  > total_len += mcfg->free_memseg[i].len;
>  >
>  > }
>  >
>  >
>  >
>  > return total_len;
>  >
>  > }
>  >
>  >
>  >
>  > Thanks,
>  >
>  > Regards
>  >
>  > Venu
>  >
> 
> All memory is registered on the heap, so you might want to look at heap
> stats to get the same information :) It would also arguably be more
> useful because just the size of memory will not tell you how much you
> can allocate, because memory may be heavily fragmented, and heap stats
> will also tell you biggest free memory block size.
> 
> Bear in mind, however, that there is work in progress [1] to enable
> mapping/unmapping hugepages at runtime, which would make such an API
> more or less useless - just because you don't have much free space *now*
> doesn't mean you can't allocate more :)
> 
> [1] http://dpdk.org/ml/archives/dev/2018-March/092070.html
> 
> -- 
> Thanks,
> Anatoly
> 


-- 
Thanks,
Anatoly