Bug 5954 - DDOS with getchallenge
Status: ASSIGNED
Alias: None
Product: ioquake3
Classification: Unclassified
Component: Misc
Version: unspecified
Hardware: All All
: P3 blocker
Assignee: Zachary J. Slater
QA Contact: ioquake3 bugzilla mailing list
URL:
Depends on:
Blocks:
 
Reported: 2013-06-10 04:04 EDT by jawfin
Modified: 2015-08-10 18:16:43 EDT
2 users (show)

See Also:



Description jawfin 2013-06-10 04:04:09 EDT
I hope I'm posting in the correct forum, apologies if not.
I was directed here from: -
https://github.com/Razish/OpenJK/issues/281

This related to Jedi Knight:Jedi Academy, specifically the work done by the guys in the OpenJK project.

Although there is flood protection on getstatus and getinfo, its not aggressive enough to stop our server from lagging out with what's allowed through.

But my principle issue is protection for getchallenge.  A DDOS attack sending getchallenge packets completely lags the server and prevents legitimate connections from happening.  After a period of time the server crashes.

I can get more information if required, my server is hosted by EscapedTurkey and I can ask them for whatever is needed to solve this.

Thanks for your help
Cheers
Jonathan
Comment 1 Tim Angus 2013-06-10 15:37:11 EDT
I've added rate limiting to getchallenge in 7b15415. As far as the DDoSing goes, the numbers used are deliberately not that aggressive. The reason for this is that it is impossible to tell the difference between legitimate inbound requests from genuine clients and those from a DDoS cluster. If you prevent the DDoS, you also prevent legitimate use of your server. By all means try playing with the numbers if you think you have a set which is acceptably more aggressive without disrupting normal access. The existing limits were all picked with a finger in the air.
Comment 2 escapedturkey@escapedturkey.com 2013-06-10 15:42:42 EDT
Can you add a feature whereby the server administrator can adjust the numbers via a variable in the command line or server configuration file (the default being the current settings)? This way depending on the severity of the attack the server administrator can adjust numbers until he/she is content with the outcome.
Comment 3 escapedturkey@escapedturkey.com 2013-06-10 15:52:33 EDT
Can you add a feature whereby the server administrator can adjust the numbers via a variable in the command line or server configuration file (the default being the current settings)? This way depending on the severity of the attack the server administrator can adjust numbers until he/she is content with the outcome. 

Additionally, if the settings can be changed happen while the server is running, this would be optimal. This way the administrator can adjust the numbers via rcon as the attack is occurring. 

Thank you. =)
Comment 4 jawfin 2013-06-10 17:21:47 EDT
Whilst the server suffers from getinfo and getstatus, it does cope with it.  The getchallenge is the only critical concern.  If there is indeed no actual way to allow legitimate connections then we have to wait out the attack.  According to the attacker this should only take a decade.
Comment 5 jawfin 2013-06-14 21:48:24 EDT
I notice a person is assigned to this, does this mean it's being addressed?
Either way, I've noticed a flaw in the flood protect of getchallenge.

The server was successfully lagged by one person with one IP with a repeated packet which shows in the logs (developer 1) thusly: -
SV packet 12.4.2.5:-7392 : getchallenge
SV packet 12.4.2.5:-7392 : getchallenge
SV packet 12.4.2.5:-7392 : getchallenge
SV packet 12.4.2.5:-7392 : getchallenge
SV packet 12.4.2.5:-7392 : getchallenge
SV packet 12.4.2.5:-7392 : getchallenge
SV packet 12.4.2.5:-7392 : getchallenge
SV packet 12.4.2.5:-7392 : getchallenge
SV packet 12.4.2.5:-7392 : getchallenge
SV packet 12.4.2.5:-7392 : getchallenge
SV packet 12.4.2.5:-7392 : getchallenge
SV packet 12.4.2.5:-7392 : getchallenge

There was 134 on that port, then 156 on port -29022, then 192 on port -20149 etc
All told there was over 300,000 getchallenge's in less than 1 minute from IP 12.4.2.5

It could be a spoofed packet via a DDOS, but unlikely that so many bots would all be on ISP without Ingress filtering.  It may just be a spoofed packet from one person but not that IP.  Either way, it completely bypassed the flood filter.  I can't give anymore information here as the server is hosted and I don't have remove virtual control, and its unlikely EscapedTurkey was running wireshark for the 5 minutes this attack lasted.
Comment 6 Tim Angus 2013-06-15 10:37:13 EDT
This is a community project so whether or not something is being addressed is really down to time, inclination and charity on the part of the people who occasionally contribute.

Anyway, are you definitely running a version with the changes? You should be seeing messages along the lines of "SV_GetChallenge: rate limit from ... exceeded, dropping request", if you are. Presumably you have recompiled and updated the copy you're running since I made the change I mentioned in comment 1? If not an auto build for that change is available here: http://ioquake3.org/files/jenkins/7b1541504216ce09ef2c7987eb5a7117a2125a5a/gcc/no_options/
Comment 7 Zachary J. Slater 2015-08-10 18:16:43 EDT
A little over 2 years ago was when we last asked for information and received no response. Please consider this the "going twice" before the bug is marked resolved and let us know if there is still an issue that needs to be fixed.