Bug 2995

Summary:	-fstack-protector-all causes ssh to SIGSEGV
Product:	buildroot	Reporter:	Mario Klebsch <mario>
Component:	Other	Assignee:	unassigned
Status:	RESOLVED WORKSFORME
Severity:	normal	CC:	abeard, buildroot
Priority:	P5
Version:	unspecified
Target Milestone:	2010.08
Hardware:	PC
OS:	Linux
Host:		Target:
Build:

Description Mario Klebsch 2010-12-20 15:43:55 UTC

Hi!

I bought an AMD Geode alix 2D13 board and tried to get linux running on it. I compiled buildroot and from the first attempt on, every ssh program crashes (SIGSEGV). During booting, ssh-keygen crashes, but the other ssh programs started on the command line did not run, either.
:-(

To get some light on this, I built gdb for my target system and tried to run ssh-keygen under debugger control. It turned out, that the SIGSEGV was caused by the third assembly instruction in main. I added a small 'Hello World' to the openssh source tree and this program also crashed due to the same cause.

Here is the disassembly of the small hello word (which was compiled to xxx.o):


ds9 src # objdump --disassemble openssh-5.1p1/xxx.o 

openssh-5.1p1/xxx.o:     file format elf32-i386


Disassembly of section .text:

00000000 <main>:
   0:	8d 4c 24 04                     lea    0x4(%esp),%ecx
   4:	83 e4 f0                           and    $0xfffffff0,%esp
   7:	ff 71 fc                             pushl  -0x4(%ecx)
   a:	55                   	                  push   %ebp
   b:	89 e5                                mov    %esp,%ebp
   d:	51                                      push   %ecx
   e:	83 ec 20                           sub    $0x20,%esp
  11:65 a1 14 00 00 00          mov    %gs:0x14,%eax      <--- Here it crashed
  17:89 45 f8             	           mov    %eax,-0x8(%ebp)
  1a:	31 c0                                 xor    %eax,%eax
  1c:	68 00 00 00 00               push   $0x0
  21:e8 fc ff ff ff                      call   22 <main+0x22>
  26:83 c4 10                           add    $0x10,%esp
  29:8b 55 f8                            mov    -0x8(%ebp),%edx
  2c:	65 33 15 14 00 00 00    xor    %gs:0x14,%edx
  33:75 08                                jne    3d <main+0x3d>
  35:8b 4d fc                            mov    -0x4(%ebp),%ecx
  38:c9                                       leave  
  39:8d 61 fc                            lea    -0x4(%ecx),%esp
  3c:	c3                                       ret    
  3d:e8 fc ff ff ff                       call   3e <main+0x3e>

BTW, the code runs fine on my development system

gdb showed, that the gs register on my target device has a value of 0, while it is 33 on my development system.

It turned out, that the gcc option -fstack-protector-all, which is used for openssh, causes the failing instruction to be inserted into the object code.

I have no idea, what the gs register is good for, I was not able to google information about gs register usage on linux systems. Perhaps, my small linux kernel is missing some option to initialize this register, perhaps buildroot has an option to initialize the gs register,...

Comment 1 Peter Korsgaard 2010-12-20 15:56:01 UTC

Do you have "Enable stack protection support" (BR2_TOOLCHAIN_BUILDROOT_USE_SSP) enabled? If yes, do things work if you disable that and do a fresh rebuild (make clean; make)

Comment 2 Mario Klebsch 2010-12-20 16:52:10 UTC

(In reply to comment #1)
> Do you have "Enable stack protection support" (BR2_TOOLCHAIN_BUILDROOT_USE_SSP)
> enabled? If yes, do things work if you disable that and do a fresh rebuild
> (make clean; make)

Where can I find this option? I think, I searched all menuconfig branches, but was unable to find this option. I anso grep'ed all Config.in's for "Enable stack protection support" and for BR2_TOOLCHAIN_BUILDROOT_USE_SSP, but I cannot find this entry. :-(

73, Mario

Comment 3 Mario Klebsch 2010-12-20 16:56:25 UTC

(In reply to comment #2)
> Where can I find this option? I think, I searched all menuconfig branches, but
> was unable to find this option. I anso grep'ed all Config.in's for "Enable
> stack protection support" and for BR2_TOOLCHAIN_BUILDROOT_USE_SSP, but I cannot
> find this entry. :-(

I finally found it in toolchain/toolchain-buildroot/Config.in.2

The compiler is still running...

73, Mario

Comment 4 Mario Klebsch 2010-12-20 21:01:47 UTC

(In reply to comment #1)
> Do you have "Enable stack protection support" (BR2_TOOLCHAIN_BUILDROOT_USE_SSP)
> enabled? If yes, do things work if you disable that and do a fresh rebuild
> (make clean; make)

This option was enabled, and now, after I have disabled this option and rebuild everything, it seems to work fine.

- Is there an incomaptibility bewteen stack prot4ectiona nd ucLibC or my alix target system?

- Could it be, that some choosen value in my kernels .config might be the cause of my trouble?

- Will Buildroot 2010.11 allow me to use stack protection on my target system?

- How does the %gs register fit into this problem? I have learned, that this register is related to thread local storage, but it is hard to find information about x86 register usage on linux systems on the web.

73, Mario

Comment 5 Andrew Beard 2011-03-02 13:51:25 UTC

I seem to be having the same problem on a proprietary Geode LX board I'm working with.  I just updated to the 2011.02 and with uclibc 0.9.31 if I enable stack protection all ssh related executables immediately cause a segfault.  Disabling stack protection seems to alleviate the problem.

One thing I have found is that this problem does not seem to occur with uclibc 09.32-rc2, though.  With the new release candidate I can enable or disable stack protection without causing a problem.  At this point we're still trying to decide if we'd rather continue with a pre-release c library or without stack protection.

Comment 6 Thomas Petazzoni 2011-03-02 14:35:07 UTC

Maybe this should be reported on the uclibc list.

Also the official 0.9.32 should be released fairly soon now, so probably you'll be able to have *both* an official release and stack protection in your product :-)

Comment 7 Thomas Petazzoni 2011-09-18 10:18:19 UTC

Mario, Andrew, any news on this bug ?

Comment 8 Gustavo Zacarias 2013-11-28 17:15:19 UTC

I think it's safe to close this bug, i've done testing with stack protection and openssh and couldn't hit this issue.
Besides it seems to be solved by uclibc 0.9.32+ according to comment #5 and we don't do 0.9.31 for anything other than avr32 now.