I have this error when trying to execute ioquake3 in an amd64 linux pc with gentoo using a hardened profile (smashing stack protection, position independent code, ...)
*** stack smashing detected ***: quake3 - terminated
quake3: stack smashing attack in function R_AddMD3Surfaces - terminated
so it seems ioquake3 tries to execute code in the stack... bad thing
Do you have any additional game data except the default one? Try going from a base install and check out which pak file is causing this. As we are very well aware, ioquake3 has no protection against malicious game data whatsoever. As a matter of fact, the MD3 implementation can be used to trigger buffer overflows with specially crafted MD3 files, so your bug report does not come as a great surprise to me. Probably the md3 file is just faulty with no malicious intent, but it is worth checking this out.
I have no additional data, it's a fresh install.
I've attached the log , I hope it give some more information. The game crashes after introduction screen and just before the game menu appears.
Ok then. Could you please compile from svn with debugging symbols enabled ("make debug")?
Start quake3 in a debugger like gdb, preferrably from a console outside of X or via an ssh connection. The mouse cursor will freeze when the game crashes in gdb. as soon as the game has crashed enter the commands
bt
info locals
and ideally a few info about the structures and variables that seem to be involved in the crash. (The relevant source code line is shown as well.)
And give us the results here.
Forgot a few things still:
(In reply to comment #5)
> via an ssh connection. The mouse cursor will freeze when the game crashes in
> gdb. as soon as the game has crashed enter the commands
Your whole X Screen probably won't respond to any input whatsoever. You can fix this by killing gdb once you are done debugging.
> bt
> info locals
> and ideally a few info about the structures and variables that seem to be
> involved in the crash. (The relevant source code line is shown as well.)
> And give us the results here.
These commands have to be entered into gdb of course... hey, don't get me wrong I just don't know how much of gdb you know already, so I better explain a bit more than not enough.
Anyways, getting the content of structures or variables can be done via
print <variablename>
or
print *structurename
with the star.
Ok, that will take me a while, until now gentoo was compiling for me, I should do the steps manually now.
I have a few expecience with gdb, no problem, but I think it is usable with restrictions when used with gentoo hardened profile because of use of position independent code (mainly it means instructions addresses are not available)
I think I could recompile with smashing protection but without position independent code (these are the two default protection features of gcc under gentoo hardened) and the game should still crash the same way I hope.
There is no problem of running quake, in fact when the mouse freezes for the first time I start launching it from an xterm on a diferent X session.
I would post the results ASAP.
Note: Appart of the above error (or not, perhaps with shared libraries the game doesn't crash...) I am using these line when launching quake:
quake3 +set vm_cgame 1 +set vm_game 1 +set vm_ui 1
because if I set it to 0 the game doesn't find the shared libraries, it search in my home directory and in /usr/share/games/quake3/baseq3/ but gentoo install them in /usr/games/lib64/quake3/baseq3 should I do something with it? Is there a way to tell to search in this location or should I submmit these bug to gentoo?
so you're using the interpreter, not vm_x86_64.
you can prevent ioq3 from grabbing the X server by running
ioquake3 +set in_mouse 0 +set r_fullscreen 0
A backtrace of the regular gentoo (non-debug) version might already be sufficient to find the problematic piece of code, just try it.
Well I simply make soft links for libraries and now can use also the shared libraries with the same result anyway.
And I'm not able to debug ioquake3, this is the result:
(gdb) run +set in_mouse 0 +set r_fullscreen 0
--- (some output) ---
Started tty console (use +set ttycon 0 to disable)
*** stack smashing detected ***: ioquake3 - terminated
ioquake3: stack smashing attack in function R_AddMD3Surfaces - terminated
Report to http://bugs.gentoo.org/
Program terminated with signal SIGKILL, Killed.
The program no longer exists.
(gdb) bt
No stack.
(gdb) info locals
No frame selected.
(gdb)
I think when a smashing stack is detected the program is completely killed/unload from memory
As I said before with gentoo hardened profile, the gcc modified code generated is PIE (position independent executable) and it seems symbols and addresses are not available:
$ gdb ioquake3
GNU gdb 6.6
Copyright (C) 2006 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu"...
(no debugging symbols found)
Using host libthread_db library "/lib/libthread_db.so.1".
(gdb) break R_AddMD3Surfaces
Function "R_AddMD3Surfaces" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
-- the rest is exactly the same output as in the quake.log file attached --
Edit Makefile and change this:
$(B)/ioquake3.$(ARCH)$(BINEXT): $(Q3OBJ) $(Q3POBJ) $(LIBSDLMAIN)
$(echo_cmd) "LD $@"
$(Q)$(CC) -o $@ $(Q3OBJ) $(Q3POBJ) $(CLIENT_LDFLAGS) \
$(LDFLAGS) $(LIBSDLMAIN)
to this (just the 4th line is different):
$(B)/ioquake3.$(ARCH)$(BINEXT): $(Q3OBJ) $(Q3POBJ) $(LIBSDLMAIN)
$(echo_cmd) "LD $@"
$(Q)$(CC) -o $@ $(Q3OBJ) $(Q3POBJ) $(CLIENT_LDFLAGS) \
$(LDFLAGS) -nopie $(LIBSDLMAIN)
then run `make debug` and use the new client executable in gdb. That should make gdb work.
OK, what I've done insted of modify that file is change my gcc profile from
x86_64-pc-linux-gnu-3.4.6 to x86_64-pc-linux-gnu-3.4.6-hardenednopie
to compile quake3, so no pie code is generated, I've also modified the gentoo ebuild to solve the library problems and adding a debug flag to allow to compile with make debug method and copying the right executables.
So I have now a quake3 compiled without pie but smash stack protecction and now I can play a demo for example but when trying to play a single or multiplayer game the game crash with an error of this style
********************
ERROR: VM_CompileX86: mprotect failed
********************
----- Server Shutdown (Server crashed: VM_CompileX86: mprotect failed) -----
that I think means the same is a memory protection failure.
I've tried to debug putting a break and now symbols are available but this is the result:
(gdb) break R_AddMD3Surfaces
Breakpoint 1 at 0x50b0e5: file code/renderer/tr_mesh.c, line 288.
(gdb) run +set in_mouse 0 +set r_fullscreen 0
Starting program: /usr/games/bin/ioquake3 +set in_mouse 0 +set r_fullscreen 0
Warning:
Cannot insert breakpoint 1.
Error accessing memory address 0x50b0e5: Input/output error.
Cannot insert breakpoint -3.
Temporarily disabling shared library breakpoints:
breakpoint #-3
Another run without putting the break:
...
Sys_Error: recursive error after: VM_CompileX86: mprotect failed
ioquake3: code/sys/sys_main.c:172: Sys_Exit: Assertion `ex == 0' failed.
Program received signal SIGABRT, Aborted.
0x00003590cd75afb9 in ?? ()
(gdb) bt
#0 0x00003590cd75afb9 in ?? ()
#1 0x00003590cd75c48e in ?? ()
#2 0x0000000000000020 in ?? ()
#3 0x0000000000000000 in ?? ()
(gdb) info locals
No symbol table info available.
I've read something about Gentoo hardened profile and found this:
1) -O3 optimization breaks stack smashing protection
2) SELinux prevents mprotect() from setting execute permissions on memory block
ioquake3 compiled with -O2 shouldn't trigger stack smashing protection anymore. However, you'll have to disable SELinux mprotect() changes or compile with `make HAVE_VM_COMPILED=0` which will make everything _REALLY_ slow.
Quake3 uses QVM files as dynamically linked libraries containing game code. QVM uses platform indepent pseudo-RISC code. It must be compiled into native instructions during map load. The engine must then set execute permissions to be able to use the compiled bytecode.
Compiled with -O2 and disabling MPROTECT with paxctl the game runs fine and even with pie code and smashing stack protection enabled.
Are necesary qvm files now that quake3 code is public?
Will not be better to compile the code for each platform and have all the code precompiled so it is not necesary to allow the generation of code in memory at run time which is a bit more risky for security?
Created attachment 1488 [details] quake.log