GNU Arm Embedded Toolchain

NewLib malloc on top of heap breaks more than needed

Asked by Petter Österlund

I have noticed a peculiarity with malloc(). I am not sure if it is a bug or feature but it is for sure unexpected.

  p1 = malloc(30000); // heap will peak at 30000.
  free(p1); // Will be put back in top bin
  p1 = malloc(40000); // heap will now peak at 70000

I would have expected that this scenario would not require more than 40000 bytes of RAM (and e.g. fit into 64 KB).

I think I found why this happens. Line 2561 and 2575 of mallocr.c handles the case where allocation is made from top bin and top bin is not big enough:

  2561: if (chunksize(top) < nb || remainder_size < (long)MINSIZE)
  ...
  2575: malloc_extend_top(RCALL nb);

So, since top only has 30000 another 40000 is sbrk() making top 70000, it would have sufficed to extend with only 10000 and not hitting the maximum.

I have also studied but not tested the nane-mallocr.c and it too seems to have the same semantics that if the while-loop in malloc() cannot find a large enough chunk it will end and sbrk() the requested size without considering what already is available.

Question information

Language:
English Edit question
Status:
Expired
For:
GNU Arm Embedded Toolchain Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
Petter Österlund (petter-osterlund) said : 		#1

Me again, adding info and a hope that it will be recognized as something worth fixing.

The dlmalloc implementation v2.6.5 by Doug Lea used in Newlib is a bit old and I found that in dlmalloc 2.7.0. Doug has fixed this issue.

Hence I can conclude that this is not a feature and can be fixed which would be beneficial for small embedded systems. Similar fix would be even more valuable for those using nano-newlib I guess.

I do not know if Newlib always wants to stick with dlmalloc 2.6.5 but for small embedded ARM solutions it would be nice to at least back-port that fix to 2.6.5.

Revision history for this message
Launchpad Janitor (janitor) said : 		#2

This question was expired because it remained in the 'Open' state without activity for the last 15 days.

Revision history for this message
rh (rh-maillist) said : 		#3

I'm also interested in a solution to this for dlmalloc and newlib-nano's malloc. Are there plans to improve the interaction with malloc and sbrk so it takes current 'top' size in to account when asking for more heap?

Revision history for this message
Petter Österlund (petter-osterlund) said : 		#4

I have made a patch for malloc in Newlib that can be used to compile a replacement for malloc() overriding the version in libc.a. It seems to solve the issue described here nicely.

1. Download Newlib mallocr.c for your binary newlib version, e.g. from newlib-2.3.0.20160104.tar.gz unpack newlib-2.3.0.20160104/newlib/libc/stdlib/mallocr.c

    note: latest version should be fine for most since as mallocr.c "never" seems to be updated in newlib.

2. Apply this patch, which only adds a few lines at the right place.

*** newlib-2.3.0.20160104/newlib/libc/stdlib/mallocr.c-orig 2016-02-10 16:39:35.475634500 +0100
--- newlib-2.3.0.20160104/newlib/libc/stdlib/mallocr.c 2016-02-11 18:43:40.563547700 +0100
***************
*** 2571,2576 ****
--- 2571,2589 ----
      }
  #endif

+ # if 1 /* patch to extend including whats available on top */
+ if (bytes > chunksize(top)) {
+ INTERNAL_SIZE_T nb_diff = request2size(long_sub_size_t(bytes, chunksize(top)));
+ malloc_extend_top(RCALL nb_diff);
+ remainder_size = long_sub_size_t(chunksize(top), nb);
+ if (chunksize(top) < nb || remainder_size < (long)MINSIZE)
+ {
+ /* top not large enough; failure or intervening sbrk() */
+ malloc_trim(RCALL 0);
+ malloc_extend_top(RCALL nb);
+ }
+ } else
+ # endif
      /* Try to extend */
      malloc_extend_top(RCALL nb);
      remainder_size = long_sub_size_t(chunksize(top), nb);

3. Compile and include mallocr.c/mallocr.o with your project similar to this, replace target architecture options as appropriate.

   arm-none-eabi-gcc -Wall -c -O2 -mcpu=cortex-m3 -mthumb -DINTERNAL_NEWLIB -DDEFINE_MALLOC -DSMALL_MEMORY newlib-2.3.0.20160104/newlib/libc/stdlib/mallocr.c

   Use of -DSMAL_MEMORY is optional and changes pagesize from 4096 to 128.

4. Make test program; allocate 30000, free, allocate 40000, free.
     Build as normal.
     Peak memory will be >=70 KB as reported by malloc_stats()
     Now rebuild but add the patched mallocr.c / mallocr.o to your build/link from step 3.
     Peak memory should now be close to 40K.

Note: The compilation in 3 only compiles malloc() and a few friends, other functions such as free() and malloc_stats() will still be taken from libc.a.