Bad BLX instruction on Cortex M4 with libm

Hi Branden,

Can you post the command line used to compile and link and add a '-v' to it and post the output here?

Cheers,
Andre

Hi Andre,

The command line used is:

```
arm-none-eabi-gcc -mcpu=cortex-m4 -mthumb -T/home/alevy/hack/helena/tock/src/chips/sam4l/loader.ld build/crt1.o build/arch.o build/main.o build/apps/c_sync.bin.o -o build/main.elf -ffreestanding -nostdlib -lc -lm -lgcc
```

The output after adding `-v` is:

```Using built-in specs.
COLLECT_GCC=arm-none-eabi-gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/arm-none-eabi/5.3.0/lto-wrapper
Target: arm-none-eabi
Configured with: /build/arm-none-eabi-gcc/src/gcc-5.3.0/configure --target=arm-none-eabi --prefix=/usr --with-sysroot=/usr/arm-none-eabi --with-native-system-header-dir=/include --libexecdir=/usr/lib --enable-languages=c,c++ --enable-plugins --disable-decimal-float --disable-libffi --disable-libgomp --disable-libmudflap --disable-libquadmath --disable-libssp --disable-libstdcxx-pch --disable-nls --disable-shared --disable-threads --disable-tls --with-gnu-as --with-gnu-ld --with-system-zlib --with-newlib --with-headers=/usr/arm-none-eabi/include --with-python-dir=share/gcc-arm-none-eabi --with-gmp --with-mpfr --with-mpc --with-isl --with-libelf --enable-gnu-indirect-function --with-host-libstdcxx='-static-libgcc -Wl,-Bstatic,-lstdc++,-Bdynamic -lm' --with-pkgversion='Arch Repository' --with-bugurl=https://bugs.archlinux.org/ --with-multilib-list=armv6-m,armv7-m,armv7e-m,armv7-r
Thread model: single
gcc version 5.3.0 (Arch Repository)
COMPILER_PATH=/usr/lib/gcc/arm-none-eabi/5.3.0/:/usr/lib/gcc/arm-none-eabi/5.3.0/:/usr/lib/gcc/arm-none-eabi/:/usr/lib/gcc/arm-none-eabi/5.3.0/:/usr/lib/gcc/arm-none-eabi/:/usr/lib/gcc/arm-none-eabi/5.3.0/../../../../arm-none-eabi/bin/
LIBRARY_PATH=/usr/lib/gcc/arm-none-eabi/5.3.0/armv7e-m/:/usr/lib/gcc/arm-none-eabi/5.3.0/../../../../arm-none-eabi/lib/armv7e-m/:/usr/arm-none-eabi/lib/armv7e-m/:/usr/lib/gcc/arm-none-eabi/5.3.0/:/usr/lib/gcc/arm-none-eabi/5.3.0/../../../../arm-none-eabi/lib/:/usr/arm-none-eabi/lib/
COLLECT_GCC_OPTIONS='-v' '-mcpu=cortex-m4' '-mthumb' '-T' '/home/alevy/hack/helena/tock/src/chips/sam4l/loader.ld' '-o' 'build/main.elf' '-ffreestanding' '-nostdlib'
 /usr/lib/gcc/arm-none-eabi/5.3.0/collect2 -plugin /usr/lib/gcc/arm-none-eabi/5.3.0/liblto_plugin.so -plugin-opt=/usr/lib/gcc/arm-none-eabi/5.3.0/lto-wrapper -plugin-opt=-fresolution=/tmp/ccpDVAKK.res --sysroot=/usr/arm-none-eabi -X -o build/main.elf -L/usr/lib/gcc/arm-none-eabi/5.3.0/armv7e-m -L/usr/lib/gcc/arm-none-eabi/5.3.0/../../../../arm-none-eabi/lib/armv7e-m -L/usr/arm-none-eabi/lib/armv7e-m -L/usr/lib/gcc/arm-none-eabi/5.3.0 -L/usr/lib/gcc/arm-none-eabi/5.3.0/../../../../arm-none-eabi/lib -L/usr/arm-none-eabi/lib build/crt1.o build/arch.o build/main.o build/apps/c_sync.bin.o -lc -lm -lgcc -T /home/alevy/hack/helena/tock/src/chips/sam4l/loader.ld
```

Cheers,
Amit (Branden and I are collaborating on the same codebase)

Ok so I am not familiar with Arch Linux' distribution of the arm-none-eabi toolchain and I cant reproduce this with our 2015Q1 release. Could you try downloading our Linux tarball and use that instead?

Also out of interested, could you perhaps do the following, this might help in finding out where the distribution you are using is going wrong:

Locate the powf being used by adding '-Wl,-Map=map' to your command line. If look in 'map' for '(powf), the line above it will show where the libm.a that was used was taken from and which object file in it contained powf.
Im suspecting it will have been: '/usr/lib/gcc/arm-none-eabi/5.3.0/armv7e-m/libm.a' and 'lib_a-wf_pow.o'.

If thats the case then do the following:
$cd /tmp/something
$arm-none-eabi-ar x /usr/lib/gcc/arm-none-eabi/5.3.0/armv7e-m/libm.a
$arm-none-eabi-objdump -D lib_a-wf_pow.o &> dump.D

Then inspect dump.D to make sure powf calls __errno using bl and not blx. If it uses blx then Im guessing the multilib was not correctly configured/built in this distribution or a different version might have been used.

There is also the possibility that the linker will have messed up the relocation, could you also post the output of:
$arm-none-eabi-ld --version

Thanks for the quick response.

Yes, sorry, I posted the output from my box (where I just used the Arch Linux packages). Other collaborators (including Branden, I think?) are using the binaries from here. Arch builds arm-none-eabi more or less from source unmodified, but I'll try with the binaries you provide ASAP for consistency.

Meanwhile, I followed your instructions:

* The map file shows that it's getting `powf` from /usr/arm-none-eabi/lib/armv7e-m/libm.a(lib_a-wf_pow.o).

* I dumped the contents of lib_a-wf_pow.o as you suggested and all calls to __errno are done with a BL, _not_ a BLX instruction.

* My arm-none-eabi --version: GNU ld (GNU Binutils) 2.25.1

Amit

Alright, I'm here on Ubuntu running arm-none-eabi-gcc version 4.9.3 20150529 and arm-none-eabi-ld 2.24.0.20150921. The command line flags are the same as Amit posted above.

```
arm-none-eabi-gcc -mcpu=cortex-m4 -mthumb -T/home/brghena/Dropbox/repos/tock/src/chips/sam4l/loader.ld build/crt1.o build/arch.o build/main.o build/apps/c_sync.bin.o -o build/main.elf -ffreestanding -nostdlib -lc -lm -lgcc -v

Using built-in specs.
COLLECT_GCC=arm-none-eabi-gcc
COLLECT_LTO_WRAPPER=/home/brghena/toolchains/gcc-arm-none-eabi-4_9-2015q3/bin/../lib/gcc/arm-none-eabi/4.9.3/lto-wrapper
Target: arm-none-eabi
Configured with: /home/build/work/GCC-4-9-build/src/gcc/configure --target=arm-none-eabi --prefix=/home/build/work/GCC-4-9-build/install-native --libexecdir=/home/build/work/GCC-4-9-build/install-native/lib --infodir=/home/build/work/GCC-4-9-build/install-native/share/doc/gcc-arm-none-eabi/info --mandir=/home/build/work/GCC-4-9-build/install-native/share/doc/gcc-arm-none-eabi/man --htmldir=/home/build/work/GCC-4-9-build/install-native/share/doc/gcc-arm-none-eabi/html --pdfdir=/home/build/work/GCC-4-9-build/install-native/share/doc/gcc-arm-none-eabi/pdf --enable-languages=c,c++ --enable-plugins --disable-decimal-float --disable-libffi --disable-libgomp --disable-libmudflap --disable-libquadmath --disable-libssp --disable-libstdcxx-pch --disable-nls --disable-shared --disable-threads --disable-tls --with-gnu-as --with-gnu-ld --with-newlib --with-headers=yes --with-python-dir=share/gcc-arm-none-eabi --with-sysroot=/home/build/work/GCC-4-9-build/install-native/arm-none-eabi --build=i686-linux-gnu --host=i686-linux-gnu --with-gmp=/home/build/work/GCC-4-9-build/build-native/host-libs/usr --with-mpfr=/home/build/work/GCC-4-9-build/build-native/host-libs/usr --with-mpc=/home/build/work/GCC-4-9-build/build-native/host-libs/usr --with-isl=/home/build/work/GCC-4-9-build/build-native/host-libs/usr --with-cloog=/home/build/work/GCC-4-9-build/build-native/host-libs/usr --with-libelf=/home/build/work/GCC-4-9-build/build-native/host-libs/usr --with-host-libstdcxx='-static-libgcc -Wl,-Bstatic,-lstdc++,-Bdynamic -lm' --with-pkgversion='GNU Tools for ARM Embedded Processors' --with-multilib-list=armv6-m,armv7-m,armv7e-m,cortex-m7,armv7-r
Thread model: single
gcc version 4.9.3 20150529 (release) [ARM/embedded-4_9-branch revision 227977] (GNU Tools for ARM Embedded Processors)
COMPILER_PATH=/home/brghena/toolchains/gcc-arm-none-eabi-4_9-2015q3/bin/../lib/gcc/arm-none-eabi/4.9.3/:/home/brghena/toolchains/gcc-arm-none-eabi-4_9-2015q3/bin/../lib/gcc/:/home/brghena/toolchains/gcc-arm-none-eabi-4_9-2015q3/bin/../lib/gcc/arm-none-eabi/4.9.3/../../../../arm-none-eabi/bin/
LIBRARY_PATH=/home/brghena/toolchains/gcc-arm-none-eabi-4_9-2015q3/bin/../lib/gcc/arm-none-eabi/4.9.3/armv7e-m/:/home/brghena/toolchains/gcc-arm-none-eabi-4_9-2015q3/bin/../lib/gcc/arm-none-eabi/4.9.3/../../../../arm-none-eabi/lib/armv7e-m/:/home/brghena/toolchains/gcc-arm-none-eabi-4_9-2015q3/bin/../arm-none-eabi/lib/armv7e-m/:/home/brghena/toolchains/gcc-arm-none-eabi-4_9-2015q3/bin/../lib/gcc/arm-none-eabi/4.9.3/:/home/brghena/toolchains/gcc-arm-none-eabi-4_9-2015q3/bin/../lib/gcc/:/home/brghena/toolchains/gcc-arm-none-eabi-4_9-2015q3/bin/../lib/gcc/arm-none-eabi/4.9.3/../../../../arm-none-eabi/lib/:/home/brghena/toolchains/gcc-arm-none-eabi-4_9-2015q3/bin/../arm-none-eabi/lib/
COLLECT_GCC_OPTIONS='-mcpu=cortex-m4' '-mthumb' '-T' '/home/brghena/Dropbox/repos/tock/src/chips/sam4l/loader.ld' '-o' 'build/main.elf' '-ffreestanding' '-nostdlib' '-v'
 /home/brghena/toolchains/gcc-arm-none-eabi-4_9-2015q3/bin/../lib/gcc/arm-none-eabi/4.9.3/collect2 -plugin /home/brghena/toolchains/gcc-arm-none-eabi-4_9-2015q3/bin/../lib/gcc/arm-none-eabi/4.9.3/liblto_plugin.so -plugin-opt=/home/brghena/toolchains/gcc-arm-none-eabi-4_9-2015q3/bin/../lib/gcc/arm-none-eabi/4.9.3/lto-wrapper -plugin-opt=-fresolution=/tmp/cc7VeT3i.res --sysroot=/home/brghena/toolchains/gcc-arm-none-eabi-4_9-2015q3/bin/../arm-none-eabi -X -o build/main.elf -L/home/brghena/toolchains/gcc-arm-none-eabi-4_9-2015q3/bin/../lib/gcc/arm-none-eabi/4.9.3/armv7e-m -L/home/brghena/toolchains/gcc-arm-none-eabi-4_9-2015q3/bin/../lib/gcc/arm-none-eabi/4.9.3/../../../../arm-none-eabi/lib/armv7e-m -L/home/brghena/toolchains/gcc-arm-none-eabi-4_9-2015q3/bin/../arm-none-eabi/lib/armv7e-m -L/home/brghena/toolchains/gcc-arm-none-eabi-4_9-2015q3/bin/../lib/gcc/arm-none-eabi/4.9.3 -L/home/brghena/toolchains/gcc-arm-none-eabi-4_9-2015q3/bin/../lib/gcc -L/home/brghena/toolchains/gcc-arm-none-eabi-4_9-2015q3/bin/../lib/gcc/arm-none-eabi/4.9.3/../../../../arm-none-eabi/lib -L/home/brghena/toolchains/gcc-arm-none-eabi-4_9-2015q3/bin/../arm-none-eabi/lib build/crt1.o build/arch.o build/main.o build/apps/c_sync.bin.o -lc -lm -lgcc -T /home/brghena/Dropbox/repos/tock/src/chips/sam4l/loader.ld
```

Creating the Map file and searching through it I find powf was included from:
`/home/brghena/toolchains/gcc-arm-none-eabi-4_9-2015q3/bin/../lib/gcc/arm-none-eabi/4.9.3/../../../../arm-none-eabi/lib/armv7e-m/libm.a(lib_a-wf_pow.o)`

I also confirm that a dump of lib_a-wf_pow.o shows `bl 0 <__errno>` rather than the blx to errno_veneer as above.

So yeh it seems the multilib was configured correctly, the GNU ld version I have here however is slightly higher: 2.25.90.

Let me know what the results were with the tarball.

Cheers,
Andre

Alright, I updated to the tarball from the homepage, arm-non-eabi-gcc 5.2.1 20151202 and arm-none-eabi-ld 2.25.90.20151217. The problem is still the same.

```
arm-none-eabi-gcc -mcpu=cortex-m4 -mthumb -T/home/brghena/Dropbox/repos/tock/src/chips/sam4l/loader.ld build/crt1.o build/arch.o build/main.o build/apps/c_sync.bin.o -o build/main.elf -ffreestanding -nostdlib -lc -lm -lgcc -Wl,-Map=build/main.map -v

Using built-in specs.
COLLECT_GCC=arm-none-eabi-gcc
COLLECT_LTO_WRAPPER=/home/brghena/toolchains/gcc-arm-none-eabi-5_2-2015q4/bin/../lib/gcc/arm-none-eabi/5.2.1/lto-wrapper
Target: arm-none-eabi
Configured with: /home/build/work/GCC-5-0-build/src/gcc/configure --target=arm-none-eabi --prefix=/home/build/work/GCC-5-0-build/install-native --libexecdir=/home/build/work/GCC-5-0-build/install-native/lib --infodir=/home/build/work/GCC-5-0-build/install-native/share/doc/gcc-arm-none-eabi/info --mandir=/home/build/work/GCC-5-0-build/install-native/share/doc/gcc-arm-none-eabi/man --htmldir=/home/build/work/GCC-5-0-build/install-native/share/doc/gcc-arm-none-eabi/html --pdfdir=/home/build/work/GCC-5-0-build/install-native/share/doc/gcc-arm-none-eabi/pdf --enable-languages=c,c++ --enable-plugins --disable-decimal-float --disable-libffi --disable-libgomp --disable-libmudflap --disable-libquadmath --disable-libssp --disable-libstdcxx-pch --disable-nls --disable-shared --disable-threads --disable-tls --with-gnu-as --with-gnu-ld --with-newlib --with-headers=yes --with-python-dir=share/gcc-arm-none-eabi --with-sysroot=/home/build/work/GCC-5-0-build/install-native/arm-none-eabi --build=i686-linux-gnu --host=i686-linux-gnu --with-gmp=/home/build/work/GCC-5-0-build/build-native/host-libs/usr --with-mpfr=/home/build/work/GCC-5-0-build/build-native/host-libs/usr --with-mpc=/home/build/work/GCC-5-0-build/build-native/host-libs/usr --with-isl=/home/build/work/GCC-5-0-build/build-native/host-libs/usr --with-cloog=/home/build/work/GCC-5-0-build/build-native/host-libs/usr --with-libelf=/home/build/work/GCC-5-0-build/build-native/host-libs/usr --with-host-libstdcxx='-static-libgcc -Wl,-Bstatic,-lstdc++,-Bdynamic -lm' --with-pkgversion='GNU Tools for ARM Embedded Processors' --with-multilib-list=armv6-m,armv7-m,armv7e-m,armv7-r,armv8-m.base,armv8-m.main
Thread model: single
gcc version 5.2.1 20151202 (release) [ARM/embedded-5-branch revision 231848] (GNU Tools for ARM Embedded Processors)
COMPILER_PATH=/home/brghena/toolchains/gcc-arm-none-eabi-5_2-2015q4/bin/../lib/gcc/arm-none-eabi/5.2.1/:/home/brghena/toolchains/gcc-arm-none-eabi-5_2-2015q4/bin/../lib/gcc/:/home/brghena/toolchains/gcc-arm-none-eabi-5_2-2015q4/bin/../lib/gcc/arm-none-eabi/5.2.1/../../../../arm-none-eabi/bin/
LIBRARY_PATH=/home/brghena/toolchains/gcc-arm-none-eabi-5_2-2015q4/bin/../lib/gcc/arm-none-eabi/5.2.1/armv7e-m/:/home/brghena/toolchains/gcc-arm-none-eabi-5_2-2015q4/bin/../lib/gcc/arm-none-eabi/5.2.1/../../../../arm-none-eabi/lib/armv7e-m/:/home/brghena/toolchains/gcc-arm-none-eabi-5_2-2015q4/bin/../arm-none-eabi/lib/armv7e-m/:/home/brghena/toolchains/gcc-arm-none-eabi-5_2-2015q4/bin/../lib/gcc/arm-none-eabi/5.2.1/:/home/brghena/toolchains/gcc-arm-none-eabi-5_2-2015q4/bin/../lib/gcc/:/home/brghena/toolchains/gcc-arm-none-eabi-5_2-2015q4/bin/../lib/gcc/arm-none-eabi/5.2.1/../../../../arm-none-eabi/lib/:/home/brghena/toolchains/gcc-arm-none-eabi-5_2-2015q4/bin/../arm-none-eabi/lib/
COLLECT_GCC_OPTIONS='-mcpu=cortex-m4' '-mthumb' '-T' '/home/brghena/Dropbox/repos/tock/src/chips/sam4l/loader.ld' '-o' 'build/main.elf' '-ffreestanding' '-nostdlib' '-v'
 /home/brghena/toolchains/gcc-arm-none-eabi-5_2-2015q4/bin/../lib/gcc/arm-none-eabi/5.2.1/collect2 -plugin /home/brghena/toolchains/gcc-arm-none-eabi-5_2-2015q4/bin/../lib/gcc/arm-none-eabi/5.2.1/liblto_plugin.so -plugin-opt=/home/brghena/toolchains/gcc-arm-none-eabi-5_2-2015q4/bin/../lib/gcc/arm-none-eabi/5.2.1/lto-wrapper -plugin-opt=-fresolution=/tmp/ccGKJSmW.res --sysroot=/home/brghena/toolchains/gcc-arm-none-eabi-5_2-2015q4/bin/../arm-none-eabi -X -o build/main.elf -L/home/brghena/toolchains/gcc-arm-none-eabi-5_2-2015q4/bin/../lib/gcc/arm-none-eabi/5.2.1/armv7e-m -L/home/brghena/toolchains/gcc-arm-none-eabi-5_2-2015q4/bin/../lib/gcc/arm-none-eabi/5.2.1/../../../../arm-none-eabi/lib/armv7e-m -L/home/brghena/toolchains/gcc-arm-none-eabi-5_2-2015q4/bin/../arm-none-eabi/lib/armv7e-m -L/home/brghena/toolchains/gcc-arm-none-eabi-5_2-2015q4/bin/../lib/gcc/arm-none-eabi/5.2.1 -L/home/brghena/toolchains/gcc-arm-none-eabi-5_2-2015q4/bin/../lib/gcc -L/home/brghena/toolchains/gcc-arm-none-eabi-5_2-2015q4/bin/../lib/gcc/arm-none-eabi/5.2.1/../../../../arm-none-eabi/lib -L/home/brghena/toolchains/gcc-arm-none-eabi-5_2-2015q4/bin/../arm-none-eabi/lib build/crt1.o build/arch.o build/main.o build/apps/c_sync.bin.o -lc -lm -lgcc -Map=build/main.map -T /home/brghena/Dropbox/repos/tock/src/chips/sam4l/loader.ld
```

Searching the map file shows I am now taking powf from:
`/home/brghena/toolchains/gcc-arm-none-eabi-5_2-2015q4/bin/../lib/gcc/arm-none-eabi/5.2.1/../../../../arm-none-eabi/lib/armv7e-m/libm.a(lib_a-wf_pow.o)`

Quick update...

After playing around some with a bare-bones reproduction of the bug, it seems to be related to the fact that we declare `__errno` as `__attribute__ ((weak)) int __errno`.

1. When I get rid of the weak attribute, the compiler correctly outputs BL instructions.

2. That definition for __errno is incorrect anyway, since it's meant to be a function.

3. When re-doing it as a function, the weak attribute has no negative affect.

I'm still not sure why the result of defining errno as an integer (instead of a function) was that the compiler spit out incorrect instructions.

Amit

Andre,

So our code seems more or less fixed right now (more or less because we're not 100% sure what the body of the __errno function should be, but that's not directly related to this question).

It seems like the actual behavior is still a bug of some sort, at least in the sense that output is an odd "error message" for _our_ bug :)

I don't know if it's helpful, but this is my guess about what might be going on:

We defined __errno as an int, so `&__errno` gets compiled to a pointer with an even memory location. So, when ____errno_veneer is linked, the toolchain (linker? compiler?) thinks the __errno "function" is an ARM (not thumb) function (because thumb functions are referenced by odd pointers). That assumption gets propogated up and we end up with a BLX instruction.

FWIW, the actual body of __errno_veneer also contained a non-thumb instruction (some sort of 32-bit LDR instruction).

Amit

Hi Amit,

This sounds like a bug we had seen before in a previous version of Binutils. I believed this was fixed in our 5-2015-q4-major , could you possibly try to reproduce it using this latest release and see if it still shows that behavior with the sources that were triggering the blx instruction?

Cheers,
Andre

Andre

I tried 5-2015-q4-major and posted the results in response #7 above. The error still exists. Was there some other information you want me to get with that version?

Thanks
Branden

Hi Amit,

Looking at ld's code the scenario you describe could indeed happen. When the linker process a relocation for a call from Thumb code to ARM code (based on the least significant bit value), it changes the bl in blx. Thumb only targets are not considered when doing this because a call to ARM mode would fail anyway.

Regarding the wrong instruction in __errno_veneer, please open a new question if this comes from a library provided by this toolchain and we will look into it.

Best regards.