[dpdk-dev] struct malloc_elem overrun/corruption

[Burakov, Anatoly]

On 27-Nov-18 10:38 AM, Burakov, Anatoly wrote:
> On 26-Nov-18 11:44 PM, He Huang wrote:
>> Hi,
>>
>> I’ve been troubleshooting a possible memory allocator corruption:
>> Program received signal SIGSEGV, Segmentation fault.
>> [Switching to Thread 0x7fffefdf0700 (LWP 1079)]
>> 0x00000000004794ee in malloc_elem_free_list_insert 
>> (elem=0x7ff82d265000) at dpdk/lib/librte_eal/common/malloc_elem.c:292
>> 292             LIST_INSERT_HEAD(&elem->heap->free_head[idx], elem, 
>> free_list);
>> (gdb) bt
>> #0  0x00000000004794ee in malloc_elem_free_list_insert 
>> (elem=0x7ff82d265000) at dpdk/lib/librte_eal/common/malloc_elem.c:292
>> #1  0x0000000000479971 in malloc_elem_free (elem=0x7ff82d265000) at 
>> dpdk/lib/librte_eal/common/malloc_elem.c:448
>> #2  0x000000000047b054 in malloc_heap_free (elem=0x7ff82d265fc0) at 
>> dpdk/lib/librte_eal/common/malloc_heap.c:628
>> #3  0x00000000004787f5 in rte_free (addr=0x7ff82d266000) at 
>> dpdk/lib/librte_eal/common/rte_malloc.c:32
>>
>> Looked like the 1st field of struct malloc_elem (i.e. the heap 
>> pointer: struct malloc_heap *heap) was corrupted. Everything else 
>> looked good:
>> (gdb) p *elem
>> $2 = {
>>    heap = 0x9e0,
>>    prev = 0x7ff82d254fc0,
>>    next = 0x7ff84ce9a000,
>>    free_list = {
>>      le_next = 0x7ff873c89000,
>>      le_prev = 0x7ff82bcbf018
>>    },
>>    msl = 0x7ffff7f3d07c,
>>    state = ELEM_FREE,
>>    pad = 0,
>>    size = 532893696
>> }
>> (gdb) p *elem->prev
>> $3 = {
>>    heap = 0x7ffff7f3f67c,
>>    prev = 0x7ff82ce14000,
>>    next = 0x7ff82d265000,
>>    free_list = {
>>     le_next = 0x0,
>>      le_prev = 0x0
>>    },
>>    msl = 0x7ffff7f3d07c,
>>    state = ELEM_BUSY,
>>    pad = 0,
>>    size = 65600
>> }
>>
>> I haven’t completely ruled out my own code had a buffer overrun and 
>> corrupted the first field of malloc_elem object yet, but I’m beginning 
>> to look at it as a possible DPDK internal corruption. The DPDK code 
>> isn’t the latest but it had malloc fixes up to commit 
>> 9554dbb50a8a22942128a0e5bcb52243a4f723ab.
>>
>> Ideas/suggestions greatly appreciated! BTW it’s DMA memory so I 
>> couldn’t just use malloc/free and debug with standard memory debuggers.
>>
>> Thanks,
>> Isaac
>>
> 
> Hi Isaac,
> 
> You might want to look into enabling malloc debug options.
> 

Also, it might be useful to apply this commit and check if it still happens:

71aae4b421da9b741d1fb73a190a9facfec555b9

http://patches.dpdk.org/patch/48080/

-- 
Thanks,
Anatoly