I don't understand. Why do we want this?
The current protocol seems to be just fine to me.

I agree.

I'm far from up-to-date with the state of the master protocol(s) support in IOQ3, but if standalone games are supposed to use the dpmaster protocol to get a separate "namespace" on the master server, then at least this patch fixes HEARTBEAT_FOR_MASTER for them. You are supposed to send the protocol name with your heartbeat message (i.e. "QuakeArena-1" for the Q3A protocol, or "DarkPlaces" for dpmaster protocol), not a random string.

I would hope that we can have a better backwards-compatibility story than "half the $GAME players and servers query/are listed as DP, the other half as Q3 despite being otherwise network-compatible, and you have to configure com_useDpmasterProtocol to be in the same half of the world as your friends" (where values of $GAME include "OpenArena 0.8.x" and "Quake III Arena 1.32 or later").

(One additional backwards-compat problem is that current versions of OpenArena wrongly use the same sv_heartbeat as Quake III Arena - but with a different protocol number - and that can't usefully be fixed until OA next breaks network compatibility.)

(In reply to comment #4)
> current versions of OpenArena
> wrongly use the same sv_heartbeat as Quake III Arena

... or rather, they wrongly do this if Thilo was correct in r1815 and r1842.

If Mathieu is correct in Comment #3, then OA *correctly* uses QuakeArena-1...

I'm not sure that I understand the distinction between GAMENAME_FOR_MASTER and HEARTBEAT_FOR_MASTER fully. As far as I can see from the code:

* Quake 3 servers have always sent "heartbeat QuakeArena-1\n" to signal
  both heartbeat and flatlining

* OpenArena servers have always sent the same heartbeats as Q3

* Quake 3 clients query the master server with "getservers 68 ..."
  (IPv4 master), "getserversExt Quake3Arena 68 ..." (IPv6 master,
  dual-stack client) or "getserversExt Quake3Arena 71 ipv6 ..." (IPv6
  master, IPv6-only client)

* OpenArena clients query the master server with "getservers 71 ..."
  (IPv4 master), "getserversExt openarena 71 ..." (IPv6 master, dual-stack
  client) or "getserversExt openarena 71 ipv6 ..." (IPv6 master,
  IPv6-only client)

HEARTBEAT_FOR_MASTER is a bit misnamed I guess. Maybe MASTER_PROTOCOL or something like that would be better.

As for OpenArena, do you know what infostring it returns with its infoResponse? More importantly, is there a "gamename" value into it, or not?

Never mind, I found some recent dpmaster logs.

No gamename into the infostring, so OpenArena servers are considered as Q3A servers, with a higher network protocol number. And since the getserversExt queries ask for openarena servers, you won't get any server in the response.

(In reply to comment #7)
> Never mind, I found some recent dpmaster logs.
> 
> No gamename into the infostring, so OpenArena servers are considered as Q3A
> servers, with a higher network protocol number. And since the getserversExt
> queries ask for openarena servers, you won't get any server in the response.

Well, I actually _adapted_ quake3 to work with dpmaster.
The sv_heartbeat message is said QuakeArena-1 string.
However, for the getServersExt query, QuakeArena-1 is not the string to send when you want Quake servers. Look at dpmaster's own documentation:

"\xFF\xFF\xFF\xFFgetserversExt Quake3Arena 68 ipv4 empty ipv6"

Obviously, the dpmaster server admin configures the gamenames to the heartbeats, and the heartbeat string can differ from the gamename string sent in the getserversExt query.
That's why there is the separate cl_gamename.

I thought I got those bases pretty much covered, that's why you have me baffled with this new "DarkPlaces" protocol that we're supposed to support, too?

What I mean is this:

- on the server side: either you stick to the Q3A protocol (sending QuakeArena-1 heartbeats, and no gamename in the infoResponse), which IOQ3 does, and you'll be registered as a Q3A server (codename: Quake3Arena). Or, if you want a separate namespace/game, you use the DP protocol (sending DarkPlaces heartbeats, with the infoResponse messages containing a "gamename" in the infostring).

- on the client side, i.e. the "getservers" and "getserversExt" queries: if you don't specify a gamename before the protocol number in a "getservers" (you have to do it for "getserversExt"), the gamename will be guessed based on the protocol number you ask for.

(In reply to comment #9)
> - on the server side: either you stick to the Q3A protocol (sending
> QuakeArena-1 heartbeats, and no gamename in the infoResponse), which IOQ3 does,
> and you'll be registered as a Q3A server (codename: Quake3Arena).
> want a separate namespace/game, you use the DP protocol (sending DarkPlaces
> heartbeats, with the infoResponse messages containing a "gamename" in the
> infostring).

Well, the master server already differentiates games based on the heartbeat, and that will most likely not change for the time being, right? This will also give us this separate "namespace", right?

> - on the client side, i.e. the "getservers" and "getserversExt" queries: if you
> don't specify a gamename before the protocol number in a "getservers" (you have
> to do it for "getserversExt"), the gamename will be guessed based on the
> protocol number you ask for.

The getServersExt query is being used for ipv6 connections and, if the id guys upgraded to a newer version of dpmaster, we could completely remove the old getservers query. So there we have namespace separation as well.
That's why I'd reason: we don't need this "darkplaces" protocol

(In reply to comment #10)
> Well, the master server already differentiates games based on the heartbeat,
> and that will most likely not change for the time being, right? This will also
> give us this separate "namespace", right?

I shouldn't have used the term "namespace". I realize now it's confusing, sorry.

Just to be clear, all servers are stored in one big list. The Q3A servers are not different, they are just using "Quake3Arena" as their "gamename". In other words, there's no difference between declaring a server using the Q3A master protocol, and declaring it using the DP master protocol with "Quake3Arena" as its "gamename".

> The getServersExt query is being used for ipv6 connections and, if the id guys
> upgraded to a newer version of dpmaster, we could completely remove the old
> getservers query. So there we have namespace separation as well.
> That's why I'd reason: we don't need this "darkplaces" protocol

I wouldn't really see the point of removing the vanilla Q3A master protocol. It wouldn't simplify the code significantly, and IOQ3 would lose interoperability with other tools and master servers.

Anyway, the point is: indeed, IOQ3 itself doesn't need the DP protocol. But if a standalone game based on IOQ3 doesn't want to receive IP addresses of Q3 servers from the masters (and doesn't want to change the network protocol number just for that purpose), it will need to send a custom gamename. And the only way to do that is to use the DP master protocol.

If it's not already in the IOQ3 code base, standalone games will have to patch the code themselves. So the idea here is to put the code as an option into the IOQ3 trunk to avoid redundant work.

(In reply to comment #11)
> Just to be clear, all servers are stored in one big list. The Q3A servers are
> not different, they are just using "Quake3Arena" as their "gamename". In other
> words, there's no difference between declaring a server using the Q3A master
> protocol, and declaring it using the DP master protocol with "Quake3Arena" as
> its "gamename".

Yes.

> I wouldn't really see the point of removing the vanilla Q3A master protocol. It
> wouldn't simplify the code significantly, and IOQ3 would lose interoperability
> with other tools and master servers.

It would only lose interoperability with old master servers, not any server browser tools. And if id updates to a more recent version of dpmaster (or maybe id's server even already supports the getserversExt query?) even this point would be moot.

> Anyway, the point is: indeed, IOQ3 itself doesn't need the DP protocol. But if
> a standalone game based on IOQ3 doesn't want to receive IP addresses of Q3
> servers from the masters (and doesn't want to change the network protocol
> number just for that purpose), it will need to send a custom gamename. And the
> only way to do that is to use the DP master protocol.

As said above, the only master server still making problems would be the one above from id, and standalone games really shouldn't use that server. Are there even alternatives to dpmaster? Wouldn't standalone games use your server anyways?
I simply don't see the need for a "dpmasterprotocol" cvar in this case. Why not always use getserversExt query in the standalone case, even when going over ipv4? As has been established it is more than sufficient in my view to provide the game separation you strive for, without adding yet more code.

> If it's not already in the IOQ3 code base, standalone games will have to patch
> the code themselves. So the idea here is to put the code as an option into the
> IOQ3 trunk to avoid redundant work.

Your server is a nice piece of work. I was just looking at the many changes the patch introduces and I don't see the necessity in it because you did this nice work. I stumbled over the gamename vs. heartbeat problems when adding AAAA records to my master server for wop and realized the getserversExt query can be used perfectly to separate games without needing to resort to stuff such as parsing infostrings.

(In reply to comment #12)
> I simply don't see the need for a "dpmasterprotocol" cvar in this case. Why not
> always use getserversExt query in the standalone case, even when going over
> ipv4? As has been established it is more than sufficient in my view to provide
> the game separation you strive for, without adding yet more code.

I assume the answer is the same as various cvars I asked for: so Quake III Arena and standalone games can share an engine build, and choose whether to be standalone at runtime. I no longer have any objection to this patch, thanks for explaining the situation.

When switching from discriminating by sv_heartbeat to discriminating by com_gamename, perhaps we should bump com_protocol from 70 to something obviously different (1071?) to avoid collision with the 68+ range already used by standalone mods like OA? (I ask this particularly because it's about to collide with protocol number 71, which OA has used since 2008 - potential for confusion there.)

(In reply to comment #7)
> No gamename into the infostring, so OpenArena servers are considered as Q3A
> servers, with a higher network protocol number. And since the getserversExt
> queries ask for openarena servers, you won't get any server in the response.

Would it be possible to make dpmaster consider protocol number 71 in the QuakeArena-1 heartbeat protocol to imply a gamename of "openarena" instead of "Quake3Arena"? :-) (This would require ioQ3 avoiding protocol number 71, but I think the "non-legacy" protocol number should probably increase to something visually distinct anyway.)

Historical behaviour which can't really be changed now:

* Q3 1.32c heartbeats as "QuakeArena-1 68", and queries for protocol 68

* ioQ3 1.36 heartbeats as "QuakeArena-1 68", and queries for protocol 68
  (IPv4) or "Quake3Arena 68" (IPv6)

* OA 0.8.1 and 0.8.5 heartbeat as "QuakeArena-1 71",
  and query for protocol 71 (IPv4) or "openarena 71" (IPv6)

Things we can still change:

* OA engines from the future can heartbeat as "DarkPlaces Quake3Arena 71"
  or "DarkPlaces openarena 71", whichever would give better compatibility
  (updated OA engine snapshots based on ioQ3 are released occasionally,
  whereas updated OA game content hasn't happened for a long time; I think
  most people still use the engine that comes with the 0.8.5 game content)

* "the OA engine" in Debian/Fedora/Ubuntu (which is just ioQ3 with some
  command-line options) can do the same

* if ID make their master server DP-compatible, ioQ3 could start
  using heartbeat "DarkPlaces Quake3Arena 68" if that'd help

Possibly-mad idea: instead of having a cvar, would it perhaps make sense for ioQ3 to recognise and special-case the hostname of id's Q3 master server, use the legacy protocol for that hostname only, and use the DarkPlaces heartbeat protocol for all other masters?

(In reply to comment #13)
> I assume the answer is the same as various cvars I asked for: so Quake III
> Arena and standalone games can share an engine build, and choose whether to be
> standalone at runtime. I no longer have any objection to this patch, thanks for
> explaining the situation.

We have that already, right now. Maybe a little modification to only send getserversExt queries when in standalone mode. But there's no need for extensive changes.

> Would it be possible to make dpmaster consider protocol number 71 in the
> QuakeArena-1 heartbeat protocol to imply a gamename of "openarena" instead of
> "Quake3Arena"? :-) (This would require ioQ3 avoiding protocol number 71, but I
> think the "non-legacy" protocol number should probably increase to something
> visually distinct anyway.)

It is possible now already. You have to configure dpmaster via special command line parameters, see the dpmaster documentation. The sv_heartbeat cvar lets you configure any heartbeat message you want.
The cl_gamename lets you query clients for any game that you want.
I see now that the drawback is that the dpmaster operator has to configure the same gamename as you set in your cl_gamename variable.

> * if ID make their master server DP-compatible, ioQ3 could start
>   using heartbeat "DarkPlaces Quake3Arena 68" if that'd help

Alright. In this case however I'd completely drop support for the old master server protocol.

> Possibly-mad idea: instead of having a cvar, would it perhaps make sense for
> ioQ3 to recognise and special-case the hostname of id's Q3 master server, use
> the legacy protocol for that hostname only, and use the DarkPlaces heartbeat
> protocol for all other masters?

Possible, but not desirable to have this.

Thilo: I checked master.quake3arena.com, it doesn't reply to getserversExt. Though it doesn't seem to matter much, as gamename can be sent with getservers anyway (and id master only returns servers with that gamename). Using dpmaster protocol (with current patch) works on id master and all other masters I know of except Tremulous'.

ioquake3 uses dpmaster. master.quake3arena.com:27950
OpenArena uses dpmaster. dpmaster.deathmask.net:27950
World of Padman uses dpmaster. wopmaster.kickchat.com:27955
Smokin' Guns uses dpmaster. master.quake3arena.com:27950
Urban Terror uses dpmaster. master.urbanterror.net:27950
Tremulous uses modified dpmaster that doesn't support dpmaster protocol. (Or a rewritten master that doesn't support dpmaster protocol.)

Created attachment 2870 [details]
Use dpmaster protocol.

Dropped legacy master protocol support.

* Reject clients using new GAMENAME_FOR_SERVER define instead of sv_heartbeat. (ioq3 uses "QuakeArena-1" for compatibility with ioq3 r2075+, if you don't mind breaking compatibility could you change it to "Quake3Arena" for nicer rejection message.)
** Should probably add a cvar for this, not sure if it should use be com_gamename or a new cvar.
* Use dpmaster protocol and filter LAN servers using com_gamename.

(In reply to comment #14)
> > Would it be possible to make dpmaster consider protocol number 71 in the
> > QuakeArena-1 heartbeat protocol to imply a gamename of "openarena" instead of
> > "Quake3Arena"? :-) (This would require ioQ3 avoiding protocol number 71, but I
> > think the "non-legacy" protocol number should probably increase to something
> > visually distinct anyway.)
> 
> It is possible now already. You have to configure dpmaster via special command
> line parameters, see the dpmaster documentation.

Right, what I mean is: who runs dpmaster.deathmask.net (if it's not Mathieu), and can we ask them to do this? Or, Mathieu, could you add an on-by-default mapping for OA in the dpmaster source code, as you've done for Q3A, RTCW, WolfET? This would make older OA binaries interoperate nicely with what future OA versions will hopefully be made to do.

(In reply to comment #16)
> (ioq3 uses "QuakeArena-1" for compatibility with ioq3 r2075+, if you don't mind
> breaking compatibility could you change it to "Quake3Arena" for nicer rejection
> message.)

I think it should be "Quake3Arena", tbh - otherwise people will just mix up the two meanings of "game name", and indeed the two meanings of "QuakeArena-1" (more than has happened already). If the value of com_protocol increases at the same time, preferably to a value that's visually distinct (1071? the svn commit number that increases it?), then r2075 and future versions can still interoperate by falling back to the legacy protocol, I think?

> ** Should probably add a cvar for this, not sure if it should use be
> com_gamename or a new cvar.

Debian and Fedora will need a cvar for this, because we use the same ioquake3 binary for Q3A and OA, by setting cvars on the command-line. It'd be simpler for us if it can be com_gamename, since we need to set that anyway.

(In reply to comment #14)
> The sv_heartbeat cvar lets you
> configure any heartbeat message you want.

Now that you mention it, that's one of my problems with the current code: an heartbeat cannot be "any message you want". Any heartbeat other than "QuakeArena-1" (Q3A), "DarkPlaces" (DP), or the Wolfenstein heartbeats is considered malformed. The documentation has always been crystal-clear about that. The previous versions of dpmaster used to tolerate them, but the current dpmaster simply ignore them.

(In reply to comment #17)
> Right, what I mean is: who runs dpmaster.deathmask.net (if it's not Mathieu),
> and can we ask them to do this? Or, Mathieu, could you add an on-by-default
> mapping for OA in the dpmaster source code, as you've done for Q3A, RTCW,
> WolfET? This would make older OA binaries interoperate nicely with what future
> OA versions will hopefully be made to do.

Willis is the guy behind DeathMask.net <http://www.deathmask.net/contact.php>. But I have access to his server for maintaining dpmaster (not the webpages, just the service itself).

Regarding your problem, I don't think you will need me to do anything. If you upgrade the OA engine to a more recent version of IOQ3, you'll get a new protocol number, so the new OA servers won't be able to see or join the old servers anyway. Provided the code has been changed by then, to allow OA to send DP heartbeats and "OpenArena" as a gamename in its infoResponse and getservers messages, you will be right on track.  :)

(In reply to comment #18)
> Regarding your problem, I don't think you will need me to do anything. If you
> upgrade the OA engine to a more recent version of IOQ3, you'll get a new
> protocol number, so the new OA servers won't be able to see or join the old
> servers anyway.

I very much hope not - if that's true, then I'll have to go back to "one engine for Q3, and a remarkably similar but incompatible engine for OA", and probably drop ioquake3 from Debian main as a result. :-(

The recent "legacy protocol" code in ioquake3 results in it advertising protocol 68 (historical Q3 1.32 protocol) to master servers, but trying to use Thilo's improved protocol (currently 70) for the actual clients, with automatic fallback to protocol 68 if the client is old.

Similarly, an updated-but-compatible OA engine should advertise "protocol 71" (because that's what 0.8.1 and 0.8.5 used), but try to negotiate Thilo's protocol for the actual clients, and fall back to (what it calls) protocol 71 for old clients. OA 0.8.1 and 0.8.5 correspond to Q3 1.32 in this diagram :-)

The com_legacyprotocol cvar lets us do that, maintaining network compatibility with historical versions while also using a better protocol most of the time.

(In reply to comment #18)
> Now that you mention it, that's one of my problems with the current code: an
> heartbeat cannot be "any message you want". Any heartbeat other than
> "QuakeArena-1" (Q3A), "DarkPlaces" (DP), or the Wolfenstein heartbeats is
> considered malformed. The documentation has always been crystal-clear about
> that. The previous versions of dpmaster used to tolerate them, but the current
> dpmaster simply ignore them.

That is not quite true. I run the WorldofPadman master server which runs with these command line arguments for configuration:

-g Quake3Arena protocols=68,69 -g WorldofPadman protocols=70 heartbeat=WorldofPadman

This means that a master server admin can configure the heartbeats that are accepted. It also means he *must* configure it for each game he wants to support. I understand that this is probably a drawback.

As we know, id runs an instance of dpmaster, though probably an older one. Can we send "QuakeArena-1" heartbeats (instead of "DarkPlaces") with the gamename included in the infostring and get that game separation we want, without breaking compatibility to id's server?

Maybe we can talk the id guys into upgrading to newer versions so that it supports the getserversExt query? Maybe they could even add AAAA-records for their master server? If this happens, like I said, I will remove the "getservers" query completely from the ioq3 client in favor of "getserversExt".

(In reply to comment #19)
> I very much hope not - if that's true, then I'll have to go back to "one engine
> for Q3, and a remarkably similar but incompatible engine for OA", and probably
> drop ioquake3 from Debian main as a result. :-(

Honestly, I'm very reluctant to the idea of modifying the configuration of dpmaster.deathmask.net to fix a bug into a game (even a nice game). And even if I wanted to, I'm not sure I could do it, because the OA servers are actually sending perfectly valid Q3A advertisements. It means I would have to declare all Q3A servers as OA servers, basically, in order to make this work. Of course, I won't do such a thing.

I understand your concern regarding client and server fragmentation across 2 versions, but from my experience with various open source FPS, it's mostly theoretical. The vast majority of active players and servers updates their game in the few days following a new release, if you publish the announcement to the right websites. Since the OS games are free and easily available, why wouldn't they?

(In reply to comment #20)
> That is not quite true. I run the WorldofPadman master server which runs with
> these command line arguments for configuration:
> 
> -g Quake3Arena protocols=68,69 -g WorldofPadman protocols=70
> heartbeat=WorldofPadman
> 
> This means that a master server admin can configure the heartbeats that are
> accepted. It also means he *must* configure it for each game he wants to
> support. I understand that this is probably a drawback.

Well, yes you could do that, but then you defeat the whole purpose of dpmaster and its DP master protocol, which is to not have to configure anything on the master side. If World of Padman was using the DP master protocol, you wouldn't have to add those parameters, and - more importantly - you would gain the ability to use other dpmaster servers as "backup masters" for your game. For example, Nexuiz and the other DP games have 4 different master servers listed in the engine (including at least 1 running IPv6), and last time I checked Warsow had 4 too. Replication is good  :)

> As we know, id runs an instance of dpmaster, though probably an older one. Can
> we send "QuakeArena-1" heartbeats (instead of "DarkPlaces") with the gamename
> included in the infostring and get that game separation we want, without
> breaking compatibility to id's server?

It would only work if the gamename is "Quake3Arena". You cannot advertise a game in your heartbeat ("QuakeArena-1" means Q3A), and then a different one in your infoResponse. The packet would be rejected.

(In reply to comment #21)
> It would only work if the gamename is "Quake3Arena". You cannot advertise a
> game in your heartbeat ("QuakeArena-1" means Q3A), and then a different one in
> your infoResponse. The packet would be rejected.

Okay, but from what I have gathered it works if we use the "DarkPlaces" string, even for id's master server?

(In reply to comment #22)
> Okay, but from what I have gathered it works if we use the "DarkPlaces" string,
> even for id's master server?

Yes, it does.

BTW, my best guess is that the id master is a custom dpmaster 1.7 (modified for flood protection against abusive clients). Which means no getserversExt support and no IPv6 support.

(In reply to comment #23)
> (In reply to comment #22)
> > Okay, but from what I have gathered it works if we use the "DarkPlaces" string,
> > even for id's master server?
> 
> Yes, it does.
> 
> BTW, my best guess is that the id master is a custom dpmaster 1.7 (modified for
> flood protection against abusive clients). Which means no getserversExt support
> and no IPv6 support.

Alright, you have convinced me. We send "DarkPlaces" heartbeat messages instead of QuakeArena-1. At this point, I believe there really aren't any old master servers out there that would make a problem with us changing this.

Implemented in r2105.
I tested this with id's master server and game separation works fine! There's also no problem with getting a server list from legacy servers.

This will probably break compatibility (even protocol fallback support) between ioq3 clients and servers between r2075 and r2105, because we don't have an ugly separation between gamename and heartbeat message anymore.
I don't think this is going to cause alot of problems because there hasn't been any major release of ioq3 with dual protocol support out, yet.

Simon: Take note that sv_heartbeat and cl_gamename have been removed and have been consolidated into "com_gamename". For Openarena, you should set com_gamename to "OpenArena". As Mathieu said, openarena heartbeats from legacy servers are automatically sorted into the "OpenArena" game list because it recognizes the protocol version to be OA (yes, that's pretty ugly).
When you explicitly request the gamename that the dpmaster server stores OpenArena under, which I think is "OpenArena", you will even get those legacy servers. Mathieu, can you confirm please?

(In reply to comment #25)
> Simon: Take note that sv_heartbeat and cl_gamename have been removed and have
> been consolidated into "com_gamename". For Openarena, you should set
> com_gamename to "OpenArena".

OA has traditionally used "openarena" as its cl_gamename. Is dpmaster case-sensitive?

> As Mathieu said, openarena heartbeats from legacy
> servers are automatically sorted into the "OpenArena" game list because it
> recognizes the protocol version to be OA (yes, that's pretty ugly).

Oh? That's what I was trying to ask for in Comment #17, which I thought Mathieu refused in Comment #21. If this already worked, awesome :-)

> When you explicitly request the gamename that the dpmaster server stores
> OpenArena under, which I think is "OpenArena", you will even get those legacy
> servers. Mathieu, can you confirm please?

That's what I was hoping for (but preferably with "openarena", if dpmaster is case-sensitive).

(I probably won't be updating ioquake3 in Debian, or recommending to Sago007 that he updates the OA engine snapshots, until Thilo says he's finished changing the network protocol for a while - let me know when, or if you want me to try something out in experimental!)

(In reply to comment #26)
> OA has traditionally used "openarena" as its cl_gamename. Is dpmaster
> case-sensitive?

I think so. I just looked at the code:
diff = strcmp(game_names[middle], game_name);

As to "traditionally":
cl_gamename was only recently introduced, and really only had any impact at all if you would be using the getserversExt query, i.e. when you were querying over ipv6.
Most likely, the "openarena" gamename never worked in the first place for ipv6 queries, because dpmaster stores openarena as "OpenArena" internally, if Mathieu can confirm.
You can definitely change this var without breaking anything. In fact, you MUST change it, because with this revision the getservers query will now include the gamename, which it did not before, which is why it still worked for you, even with incorrect cl_gamename.

> (I probably won't be updating ioquake3 in Debian, or recommending to Sago007
> that he updates the OA engine snapshots, until Thilo says he's finished
> changing the network protocol for a while - let me know when, or if you want me
> to try something out in experimental!)

As you've seen we have been steadily reducing the numbers of bugs on this tracker and a are on track for a new ioquake3 release. I think that would be a pretty good give-away as to when the protocol will be stable :)

(In reply to comment #25)
> Simon: Take note that sv_heartbeat and cl_gamename have been removed and have
> been consolidated into "com_gamename". For Openarena, you should set
> com_gamename to "OpenArena". As Mathieu said, openarena heartbeats from legacy
> servers are automatically sorted into the "OpenArena" game list because it
> recognizes the protocol version to be OA (yes, that's pretty ugly).
> When you explicitly request the gamename that the dpmaster server stores
> OpenArena under, which I think is "OpenArena", you will even get those legacy
> servers. Mathieu, can you confirm please?

No no, I didn't say that.  :)  Dpmaster just stores the name the game sends (except for Q3A because it doesn't send any name, so I arbitrarily choose "Quake3Arena"). Since OA uses the Q3A master protocol, dpmaster consider OA servers as Q3A servers, storing "Quake3Arena" as its gamename. Which is the reason why I cannot tweak the dpmaster.deathmask.net config to handle them properly now, because dpmaster has no reason whatsoever to think the OA servers are anything but Q3A servers. It doesn't even bother to look into its internal game DB for a gamename. Again, the only way I can think of would be to link the Q3A heartbeat and protocols to "openarena" instead of Q3A.

(In reply to comment #27)
> Most likely, the "openarena" gamename never worked in the first place for ipv6
> queries, because dpmaster stores openarena as "OpenArena" internally, if
> Mathieu can confirm.
> You can definitely change this var without breaking anything. In fact, you MUST
> change it, because with this revision the getservers query will now include the
> gamename, which it did not before, which is why it still worked for you, even
> with incorrect cl_gamename.

I confirm "openarena" never worked in the first place. If I understand correctly, the only message mentioning "openarena" was a client query (getserversExt). From the master's point of view, it's just a client asking for servers of a game it never heard about.

Well, then I guess there's two ways you can go about this:

Either Mathieu uses on the deathmask master server:
-g OpenArena protocols=71 heartbeat=QuakeArena-1
if this works

Or Simon sets com_gamename to "Quake3Arena" until OpenArena puts outs out an official update with ioq3's newest changes.

(In reply to comment #28)
> ... It doesn't even bother to look into its internal
> game DB for a gamename. ...

I don't know why I wrote that, it's false. Dpmaster looks at its internal game DB every time it receives a non-DarkPlaces heartbeat, to do the heartbeat -> game name translation.

(In reply to comment #29)
> Either Mathieu uses on the deathmask master server:
> -g OpenArena protocols=71 heartbeat=QuakeArena-1
> if this works

It would work, but I would have to unlink the "QuakeArena-1" heartbeat from Q3A. The heartbeat properties are used when receiving a server request, and the protocol property when receiving a client request. I have no way with the current dpmaster code to say "heartbeat X *and* protocol Y", it's more "heartbeat X *or* protocol Y".

(In reply to comment #30)
> (In reply to comment #29)
> > Either Mathieu uses on the deathmask master server:
> > -g OpenArena protocols=71 heartbeat=QuakeArena-1
> > if this works
> 
> It would work, but I would have to unlink the "QuakeArena-1" heartbeat from
> Q3A. The heartbeat properties are used when receiving a server request, and the
> protocol property when receiving a client request. I have no way with the
> current dpmaster code to say "heartbeat X *and* protocol Y", it's more
> "heartbeat X *or* protocol Y".

I'll try to look into this soon, if you don't get there first; I was wondering about packaging dpmaster for Debian anyway :-)

(In reply to comment #30)
> The heartbeat properties are used when receiving a server request, and the
> protocol property when receiving a client request. I have no way with the
> current dpmaster code to say "heartbeat X *and* protocol Y", it's more
> "heartbeat X *or* protocol Y".

Here's a proof-of-concept patch series for dpmaster, with a regression test (I was delighted to see that dpmaster has regression tests, they're definitely the way forward!). When it receives a heartbeat that implies a particular game name, it considers it to be a first guess; when it receives an infoResponse, it re-evaluates the game based on the protocol.

It also adds OpenArena to the table of known games, calling it "openarena", so that the getserversExt issued by OA 0.8.5 will retroactively start working :-)

Adding backwards-compat for a mod that uses the Q3 heartbeat and protocol 666, and wants to use com_gamename=HypotheticalMod when upgrading to ioquake3 r2105, would work like this:

    ./src/dpmaster -g HypotheticalMod lookslike=Quake3Arena protocols=666

hg repository (I hope I got this right, I haven't used hg before):

    https://bitbucket.org/smcv/dpmaster/

(In reply to comment #32)
> Here's a proof-of-concept patch series for dpmaster, with a regression test (I
> was delighted to see that dpmaster has regression tests, they're definitely the
> way forward!). When it receives a heartbeat that implies a particular game
> name, it considers it to be a first guess; when it receives an infoResponse, it
> re-evaluates the game based on the protocol.
> 
> It also adds OpenArena to the table of known games, calling it "openarena", so
> that the getserversExt issued by OA 0.8.5 will retroactively start working :-)
> 
> Adding backwards-compat for a mod that uses the Q3 heartbeat and protocol 666,
> and wants to use com_gamename=HypotheticalMod when upgrading to ioquake3 r2105,
> would work like this:
> 
>     ./src/dpmaster -g HypotheticalMod lookslike=Quake3Arena protocols=666
> 
> hg repository (I hope I got this right, I haven't used hg before):
> 
>     https://bitbucket.org/smcv/dpmaster/

What? Wait, I never said I intended to add anything like that to dpmaster. If tomorrow someone modifies the original Q3A source and pick "71" as its new network protocol, I don't see any reason why the servers should be registered as "openarena". On a more general note, the game properties have been added to allow an easier integration of closed-source games, not to fix bugged open-source games.

The bottom line is: I'm not modifying dpmaster in any way to fix on the master side something that is broken on the game side. I've already refused to do that for other games that weren't using the protocol correctly. OA is no exception, sorry.

I can definitely integrate your modifications to the test suite, but not your "lookslike" property or OA hardcoded properties.

(In reply to comment #33)
> The bottom line is: I'm not modifying dpmaster in any way to fix on the master
> side something that is broken on the game side. I've already refused to do that
> for other games that weren't using the protocol correctly. OA is no exception,
> sorry.

I can understand that stance.

Simon: I'd suggest you quietly change com_gamename to "Quake3Arena" for OpenArena for the time being. It doesn't really matter whether getserversExt works with ipv6 in these early stages. You didn't even know it doesn't until I told you. At least for ipv4 play the master server will work.
Once OpenArena updates to our revision this problem will go away by itself hopefully. You should try and get these changes integrated upstream at their place. But with the current state of affairs, OpenArena probably would have never worked together with ipv6 anyways.

(In reply to comment #34)
> Simon: I'd suggest you quietly change com_gamename to "Quake3Arena" for
> OpenArena for the time being.

Yeah, looks like I'll have to go that route for the Debian packages, and Sago007 will probably want to do the same for his engine updates (which most OA users don't use or know about, I suspect - they seem to be intended as a way to get wider testing of engine changes, and only distributed from the forum).

> Once OpenArena updates to our revision this problem will go away by itself
> hopefully. You should try and get these changes integrated upstream at their
> place.

I'll recommend that they do this in the next official release, but don't hold your breath... it's been "coming soon" for a year and a half :-)

Mathieu, please feel free to pull in/apply any of my dpmaster changes that you do agree with; I don't really know my way around hg yet, but hopefully it has an equivalent of git cherry-pick?

(In reply to comment #35)
> Mathieu, please feel free to pull in/apply any of my dpmaster changes that you
> do agree with; I don't really know my way around hg yet, but hopefully it has
> an equivalent of git cherry-pick?

The equivalent would be the transplant extension I guess <http://mercurial.selenic.com/wiki/TransplantExtension>. But actually I exported and imported your changes instead because I wanted to edit out the diffstats from the commit messages, and to do a couple of minor changes. Although I didn't sync my Mercurial tree into the DarkPlaces' SVN repository yet, your patches have been integrated <http://hg.icculus.org/molivier/dpmaster>. Thanks!