[PSA] Don't Test With Bots

tyler

aa
Sep 11, 2013
5,102
4,621
DO NOT TEST WITH BOTS

I see a lot of people who are new to mapping say they tested with bots. It made me wonder how many people are testing their map and making changes based on bot behavior.

This is a really, really bad idea.

Bots are stupid. Even on their highest difficulty, they are stupid. They come nowhere near a human player in terms of AI. They choose classes randomly, without thinking about team composition or previous class--so an Engineer with a full base set up might randomly change to Heavy. They build and snipe in arbitrary spots--unless you bother to edit the nav mesh that is generated for your map.

They don't even jump.

That includes rocket and sticky jumping. That means no double jumping Scouts, either. And you know what else? They won't walk off anything higher than about a hundred units. Bots are stupid. (Their code is based on the AI for zombies from Left 4 Dead. Does that explain their stupidity yet?) This also means they always take the stairs--they will never just hop up a short ledge to grab health or ammo.

All this means that if your change your map based on what the bots do, you're bound to be heading in the wrong direction. You don't want your map in rotation on servers where only bots play, right? Then test with humans.

They aren't your only option, either.

TF2Maps (this website! the one you're on!) runs tests of custom maps all the time. It is free to join them and get your map played by real, human players. No bots, ever. Just watch the Events forum for game day announcements or other events and submit your map. You don't even have to be there! STV demos are automaticaly recorded and you can watch them later.

You can also join the Steam chat room and hope to be included in an impromptu test.

Bots are good for one thing, though.

It's fine to use them if you want to check your map's logic to ensure all doors and timers are functioning properly. But you should be able to do this yourself, alone. Just remember, whatever you do, don't pack the nav mesh into your bsp!
 
Last edited:

henke37

aa
Sep 23, 2011
2,076
513
At least there are a few upsides to bots:
  • They don't forget to play the game. Usually.
  • They manage to find routes that humans don't instinctively find.

But this hardly compensates for all the bad points you already brought up.
 

Spacek531

L4: Comfortable Member
Jan 28, 2012
165
94
I am curious why we are advised to not pack the .nav file in the .bz2. Is it for a space reason, or is it because with a .nav, server ops may spawn bots and mess up everyone's experience?
 

Sergis

L666: ])oo]v[
aa
Jul 22, 2009
1,874
1,257
I am curious why we are advised to not pack the .nav file in the .bz2. Is it for a space reason, or is it because with a .nav, server ops may spawn bots and mess up everyone's experience?

if you pack a .nav into the map, you mess up everyones' experience. including the server op.
 

fubarFX

The "raw" in "nodraw"
aa
Jun 1, 2009
1,721
1,975
I am curious why we are advised to not pack the .nav file in the .bz2. Is it for a space reason, or is it because with a .nav, server ops may spawn bots and mess up everyone's experience?

you can't override a paked nav file. if a server operator or w/e would make a better nav for your map, they won't be able to use it on their server. also, if the AI is ever updated and need new navs, you will need to release a new version of your map. keeping your nav separated is just more convenient.
 

henke37

aa
Sep 23, 2011
2,076
513
If bots are better at finding their way around your map than players, then there is a problem.

Yes, because players are dumber than bots when it comes to finding routes. At least while accounting for that bots are limited at movement.
 

Fruity Snacks

Creator of blackholes & memes. Destroyer of forums
aa
Sep 5, 2010
6,394
5,571
Yes, because players are dumber than bots when it comes to finding routes. At least while accounting for that bots are limited at movement.

I respectfully disagree, in one sense. Bots can "see" all the routes, and choose which one they think is right. But this is not what you want.

Humans choose what they think is right, but can't see all the routes. Humans are the ones you want to testing, because those are the ones playing. There is no way that a TF2 Bot is better than a human player, with the except of following commands.
 

YM

LVL100 YM
aa
Dec 5, 2007
7,135
6,056
Ehhh Bots have uses. This is one of those things where if you're the 'target audience' for this message, it applies 100% and if you're not it applies about 0%.
 

henke37

aa
Sep 23, 2011
2,076
513
I respectfully disagree, in one sense. Bots can "see" all the routes, and choose which one they think is right. But this is not what you want.

Humans choose what they think is right, but can't see all the routes. Humans are the ones you want to testing, because those are the ones playing. There is no way that a TF2 Bot is better than a human player, with the except of following commands.

You are exactly right, it's the players that will play. And if they don't see the route then you have a problem.
 

Fruity Snacks

Creator of blackholes & memes. Destroyer of forums
aa
Sep 5, 2010
6,394
5,571
Not exactly what I meant, and sorry, I was unclear.

Players can see routes, but they can't see/know about it Instantly, like a bot can. Learning is the big thing you need in getting good feedback. You can't get a good supply of feedback, if the tester knows EVERYTHING. Yes, you'll get feedback, but it won't be as good as a real player.


A Real player is ALWAYS better than a bot.
 

henke37

aa
Sep 23, 2011
2,076
513
I think the point here is that a map should be easy enough to learn. In my experiences it is closely related to being easily navigable, but not the same. You could make a map navigable by throwing signs everywhere.

Players are humans. Humans aren't rational. Humans aren't perfect. They will miss obvious stuff. They will notice trivial stuff. They fail the easy and makes the difficult look easy.

Players need a different kind of gudience than bots. Bots can instantly learn all the routes. But only a player can learn how the routes work. But at the same time players need to be able to play the map while learning. Some would say that a player never stops learning. Players learn what other players do and change their behavior in response. It's a never ending battle of wits (and some guns too). Bots do nothing even remotely close to that.
 
Mar 23, 2010
1,870
1,695
the main thing is sentries. bots cant take down sentries at all so any engy that isnt going against you will screw shit up.
 

henke37

aa
Sep 23, 2011
2,076
513
The issue wasn't that they couldn't play as engineers (which they only suck at), but that they are bad at attacking engineers.
 

xzzy

aa
Jan 30, 2010
815
531
They won't walk off anything higher than about a hundred units. Bots are stupid.

This may not be 100% true anymore. A map I'm working on has a one way route with two drops, each about 200 units. I've seen bots willingly walk off these ledges on several occasions recently. The only caveat is that in my case the drop is the only option.. it's either jump or head back to spawn, and their code probably prefers moving forward.

Not that this makes bots suddenly awesome map testers, they aren't, but I think someone at Valve still works on the code and occasionally implements tweaks.
 

tyler

aa
Sep 11, 2013
5,102
4,621
I think it mostly has to do with how their meshes are generated. In that case it'd tell them to just keep going, but in most cases it'd tell them to prefer the stairs or the ramp or whatever.