On hardened distros, gcc will be configured to automatically compile code as position independent code as part of their hardening toolset.
code/qcommon/vm_x86:1792
contains the entry:
__asm__ volatile(
"calll *%3\n"
: "+S" (programStack), "+D" (opStack), "+b" (opStackOfs)
: "g" (entryPoint)
: "cc", "memory", "%eax", "%ecx", "%edx"
);
This doesn't work on hosts compiling with -fPIC with 32bit builds since ebx is used by pic.
-fno-pic would probably be the easiest option here, imo.
Hey Mark.
Indeed, I didn't think about PIC when I was updating the vm_x86.c JIT.
In fact, I still don't know very much about it.
Could you try compiling it PIC with gcc5?
https://software.intel.com/en-us/blogs/2014/12/26/new-optimizations-for-x86-in-upcoming-gcc-50-32bit-pic-mode
Otherwise, one could try to push / pop the EBX register in the inlined asm around the calll.
Furthermore, Syscalls would also need to restore EBX to the Global Offset Table address before jumping back into engine code. The address of the GOT should be known to the JIT Compiler and could be emitted by the EmitCallDoSyscall() to do just that.
I've been told and confirmed that in PIC code, all function epilogues set EBX to the respective GOT, so that last step won't be necessary.
So one really only needs to restore ebx to the GOT after VM code has returned. I did this in my repo on github: https://github.com/thiloschulz/ioq3
Pull my latest changes and try again. Maybe you'll have that PIE after all. :)
Is GCC 5.0 a requirement with the latest commit (491fffe21bef6dc5284af37fe9f0cc45452e0296) in your fork?
Reason I ask is cause the compiler currently in use here is 4.9.3 and gcc is throwing an impossible constraint error:
code/qcommon/vm_x86.c: In function 'VM_CallCompiled':
code/qcommon/vm_x86.c:1791:2: error: 'asm' operand has impossible constraints
__asm__ volatile(
^
I'll try with gcc 5 or even gcc 6 on a separate host in a few minutes.
With GCC 6.1.0:
code/qcommon/vm_x86.c:1794: Error: bad register name `%opStackOfs'
Makefile:2601: recipe for target 'build/release-linux-x86/ded/vm_x86.o' failed
clang-3.9:
code/qcommon/vm_x86.c:1795:4: error: invalid register name
"mov %%ebx, %%opStackOfs\n"
^
<inline asm>:4:11: note: instantiated into assembly here
mov %ebx, %opStackOfs
Admittedly, I'm not entirely sure how clang handles inline assembly but I'd figure I'd post it here for completeness.
Sadly, no go with the most recent push. Same error in impossible constraint violation with gcc 4.9.3.
That said, shortly before you pushed your latest commit I had modified the assembly to pretty much what you had:
"mov %%ebx, %2\n"
and compiled the resulting code with gcc 6.1.0 which resulted in the binary being compiled properly. In this case once the binary was compiled with all the object files using -fPIC (including vm_x86.c) with no errors, I enabled -pie on the linker to finalize the process.
Once I confirmed that was working, additional hardening compiler flags were set with gcc 6.1.0 and the binary was copied over to the hardened host which had no issues with the binary.
Interesting that the compiler constraint error occurs with gcc 4.9.3 but is fine with 6.1.0. a 2016-03 gcc post (https://gcc.gnu.org/ml/gcc-help/2016-03/msg00156.html) suggests that EBX was considered reserved on i686 when generating PIC code but was changed to no longer be reserved as of gcc 5. I guess it's possible gcc 4 considers all usage of ebx in inline assembly, regardless of manually saving and restoring ebx, as violating the reservation when -fPIC is enabled?
(In reply to Mark from comment #7)
> Sadly, no go with the most recent push. Same error in impossible constraint
> violation with gcc 4.9.3.
well, it does work for me. This is the compiler invocation:
cc -DDEDICATED -Wall -fno-strict-aliasing -Wimplicit -Wstrict-prototypes -pipe -DUSE_ICON -DARCH_STRING=\"x86_64\" -DNO_GZIP -Icode/zlib -DUSE_INTERNAL_JPEG -Icode/jpeg-8c -DUSE_LOCAL_HEADERS -DPRODUCT_VERSION=\"1.36_GIT_3d55e71-2016-06-15\" -Wformat=2 -Wno-format-zero-length -Wformat-security -Wno-format-nonliteral -Wstrict-aliasing=2 -Wmissing-format-attribute -Wdisabled-optimization -Werror-implicit-function-declaration -MMD -DUSE_VOIP -ggdb -O0 -fPIC -fPIE -m32 -o build/debug-linux-x86_64/ded/vm_x86.o -c code/qcommon/vm_x86.c
version:
gcc version 4.9.2 (Debian 4.9.2-10)
> Interesting that the compiler constraint error occurs with gcc 4.9.3 but is
> fine with 6.1.0. a 2016-03 gcc post
> (https://gcc.gnu.org/ml/gcc-help/2016-03/msg00156.html) suggests that EBX
> was considered reserved on i686 when generating PIC code but was changed to
> no longer be reserved as of gcc 5. I guess it's possible gcc 4 considers all
> usage of ebx in inline assembly, regardless of manually saving and restoring
> ebx, as violating the reservation when -fPIC is enabled?
Like I said in my first response, gcc5 seems to have done away with that requirement. Does it compile with vanilla ioquake3 (without any of my current modifications)?
I think I see where the problem lies when comparing your compiler string to the one being generated by my gcc version.
It seems as though it compiles cleanly when optimizations are disabled however when the release build occurs, -O3 is passed which immediately causes the error.
My compilation:
cc -DDEDICATED -Wall -fno-strict-aliasing -Wimplicit -Wstrict-prototypes -pipe -DUSE_ICON -DARCH_STRING=\"x86\" -m32 -DNO_GZIP -Icode/zlib -DUSE_INTERNAL_JPEG -Icode/jpeg-8c -DUSE_LOCAL_HEADERS -DPRODUCT_VERSION=\"1.36_GIT_3d55e71-2016-06-15\" -Wformat=2 -Wno-format-zero-length -Wformat-security -Wno-format-nonliteral -Wstrict-aliasing=2 -Wmissing-format-attribute -Wdisabled-optimization -Werror-implicit-function-declaration -MMD -DNDEBUG -O3 -march=i586 -ffast-math -o build/release-linux-x86/ded/vm_x86.o -c code/qcommon/vm_x86.c
code/qcommon/vm_x86.c: In function 'VM_CallCompiled':
code/qcommon/vm_x86.c:1791:2: error: 'asm' operand has impossible constraints
__asm__ volatile(
^
with no optimizations:
$ cc -DDEDICATED -Wall -fno-strict-aliasing -Wimplicit -Wstrict-prototypes -pipe -DUSE_ICON -DARCH_STRING=\"x86\" -m32 -DNO_GZIP -Icode/zlib -DUSE_INTERNAL_JPEG -Icode/jpeg-8c -DUSE_LOCAL_HEADERS -DPRODUCT_VERSION=\"1.36_GIT_3d55e71-2016-06-15\" -Wformat=2 -Wno-format-zero-length -Wformat-security -Wno-format-nonliteral -Wstrict-aliasing=2 -Wmissing-format-attribute -Wdisabled-optimization -Werror-implicit-function-declaration -MMD -DNDEBUG -O0 -march=i586 -ffast-math -o build/release-linux-x86/ded/vm_x86.o -c code/qcommon/vm_x86.c
$
Can you try compiling as a Release build and see if you can reproduce?
No I cannot. It compiles just as well with -fPIC.
cc -DDEDICATED -fPIC -Wall -fno-strict-aliasing -Wimplicit -Wstrict-prototypes -pipe -DUSE_ICON -DARCH_STRING=\"x86\" -m32 -DNO_GZIP -Icode/zlib -DUSE_INTERNAL_JPEG -Icode/jpeg-8c -DUSE_LOCAL_HEADERS -DPRODUCT_VERSION=\"1.36_GIT_3d55e71-2016-06-15\" -Wformat=2 -Wno-format-zero-length -Wformat-security -Wno-format-nonliteral -Wstrict-aliasing=2 -Wmissing-format-attribute -Wdisabled-optimization -Werror-implicit-function-declaration -MMD -DUSE_VOIP -DNDEBUG -O3 -march=i586 -ffast-math -o build/release-linux-x86/ded/vm_x86.o -c code/qcommon/vm_x86.c
cc -DDEDICATED -fPIE -Wall -fno-strict-aliasing -Wimplicit -Wstrict-prototypes -pipe -DUSE_ICON -DARCH_STRING=\"x86\" -m32 -DNO_GZIP -Icode/zlib -DUSE_INTERNAL_JPEG -Icode/jpeg-8c -DUSE_LOCAL_HEADERS -DPRODUCT_VERSION=\"1.36_GIT_3d55e71-2016-06-15\" -Wformat=2 -Wno-format-zero-length -Wformat-security -Wno-format-nonliteral -Wstrict-aliasing=2 -Wmissing-format-attribute -Wdisabled-optimization -Werror-implicit-function-declaration -MMD -DUSE_VOIP -DNDEBUG -O3 -march=i586 -ffast-math -o build/release-linux-x86/ded/vm_x86.o -c code/qcommon/vm_x86.c
all works for me, so do -fpic and -fpie.
Curiously, in that last comment of yours you don't seem to have any -fPIC or -fpic or -fPIE flag. Is this enabled per default in your hardened version of GCC?
Also, does gcc >= 5.0 -fPIC compile standard ioquake3 (that ioquake3 without my newest changes) without errors?
(In reply to Mark from comment #0)
> On hardened distros, gcc will be configured to automatically compile code as
> position independent code as part of their hardening toolset.
Which specific "hardened distro" are you using, and how have they configured or compiled their gcc?
On Debian, many packages (including ioquake3) are compiled as PIE (position-independent executables) on an opt-in basis, but our gcc does not (yet) default to producing PIEs.
I believe recent Ubuntu has taken a different approach, where all libraries are PIC and all executables are PIE unless compiler options are used to opt-out. Debian toolchain maintainers are also considering this, but haven't done it yet.
(In reply to Thilo Schulz from comment #10)
> Also, does gcc >= 5.0 -fPIC compile standard ioquake3 (that ioquake3 without
> my newest changes) without errors?
A slightly older ioquake3 snapshot (from January, I think it's 558da252) builds fine with Debian's approach to producing PIE: -fPIE when compiling objects for the executable and -fPIE -pie when linking them; -fPIC when compiling objects for the DLLs and -fPIC -pic when linking them. To achieve this I had to patch the Makefile to introduce $(NOTSHLIBLDFLAGS) mirroring $(SHLIBLDFLAGS) - I can send that upstream if there's interest.
This was with gcc 5.3.1, which was current in Debian unstable at the time. I'll try a newer ioquake3 soon, with our current gcc 5.4.
Complete build logs: https://buildd.debian.org/status/fetch.php?pkg=ioquake3&arch=i386&ver=1.36%2Bu20160122%2Bdfsg1-2&stamp=1458553671
Gentoo hardened. 5.3.0 is masked and not considered stable yet so I'm hesistant about installing it on that box.
I've tried gcc 4.9, 5.3 and 6.1 on a Ubuntu box and couldn't reproduce the compiler error, appears to only occur on the Gentoo box which is leading me to think that a Gentoo compiler patch might be to blame here. I'll be working on testing that theory this week but it may take me a couple days.
Thilo,
Were you able to successfully test your push/pop ebx changes with a QVM? On stock Ubuntu I'm seeing sigsegvs. As a workaround I've built the binary with 5.3.0 with necessary flags and merely scp'd it over which apparently worked, heh.
(In reply to Mark from comment #12)
> Thilo,
> Were you able to successfully test your push/pop ebx changes with a QVM? On
> stock Ubuntu I'm seeing sigsegvs. As a workaround I've built the binary with
> 5.3.0 with necessary flags and merely scp'd it over which apparently worked,
> heh.
No, I haven't, as I don't have a baseq3 installed on my development system at present.
You still haven't replied to my original question. Does gcc >= 5 produce working PIE binaries on x86 that don't crash when using vanilla ioquake3, i.e. commit 9d6a95d?
On hardened distros, gcc will be configured to automatically compile code as position independent code as part of their hardening toolset. code/qcommon/vm_x86:1792 contains the entry: __asm__ volatile( "calll *%3\n" : "+S" (programStack), "+D" (opStack), "+b" (opStackOfs) : "g" (entryPoint) : "cc", "memory", "%eax", "%ecx", "%edx" ); This doesn't work on hosts compiling with -fPIC with 32bit builds since ebx is used by pic. -fno-pic would probably be the easiest option here, imo.With GCC 6.1.0: code/qcommon/vm_x86.c:1794: Error: bad register name `%opStackOfs' Makefile:2601: recipe for target 'build/release-linux-x86/ded/vm_x86.o' failed clang-3.9: code/qcommon/vm_x86.c:1795:4: error: invalid register name "mov %%ebx, %%opStackOfs\n" ^ <inline asm>:4:11: note: instantiated into assembly here mov %ebx, %opStackOfs Admittedly, I'm not entirely sure how clang handles inline assembly but I'd figure I'd post it here for completeness.