Monday, February 22, 2010

Beware of Second-Order Effects

Games are basically glorified math problems with metaphors partially covering the numbers and fancy symbols. The best games are usually interesting and engaging metaphors plastered over difficult math problems.

Unfortunately, game designers are a bit less apt at math than programmers. Being both a programmer and a wannabe designer, I approach game design from equal parts design and math. I’m no mathematician, but I have a toolbox of mathematical concepts that I use to better understand game design.

Many games fail to accomplish a semblance of balance because designers add mechanics and reward systems without a view of the broader math those mechanics will force their will upon. One such concept often missed: Second-order effects. Whereas a player’s superior skill has an additive impact on their effectiveness (a first-order effect), complex and over-rambunctious reward systems can multiply that bonus of skill into an insurmountable obstacle for less-skilled players.

The “Slippery Slope” of Success in Strategy Games

Imagine two players of measurably uneven skill at a certain strategy game. If they were to play against one another in this game, let’s say it’s a competitive turn-based strategy game whose result is based entirely on skill, the more skillful player would increase his advantage every turn. Every single turn the less skillful player would be losing by a wider margin than the turn before. The less skillful player can never win such a game and will lose the game each time by a predictably wide margin that has compounded each turn into a massive deficit. This is the nature of skill: this process of compounding advantage/disadvantage cannot be altered but by adding elements of chance and asymmetry to the game.

In reality, a player’s pure skill doesn’t directly map to a higher advantage turn-over-turn. The dynamics of a strategy game usually mitigate skill differences in the early-game (the first four moves in chess, for example) because there are fewer possible moves and those moves are usually somewhat obvious to even an advanced beginner. In most western strategy games, the number of possible moves explodes as the game progresses, peaking at some point in the mid-game, then tapering off due to fewer pieces being on the board, or the board position becoming more and more determinate as more pieces are places and static positions grow. A noticeable exception to this pattern is Go, which moves from a completely empty to a mostly full board throughout the game—leading to a continual restriction of possible moves and possible good moves.

As the game progresses and the strategy space widens, the player who can better distinguish successful strategies from unsuccessful ones will win. This discrimination between strategies is the essence of skill—players who are more skillful are better at discriminating between strategies that might, to a lesser player, seem to have the same utility.

At each point of decision, the more skillful player will choose a better strategy. The more viable strategies to discriminate between, the more of an effect skill will have. At each decision point, the better player compounds his advantage, while the less player sees the game slowly slipping away. The worse player can never make up ground.

(Humans do not behave uniformly, though, and skill is not static. We need to play against others in order to determine there skill, and players behave with different skill at different times. Playing a strategy game doesn’t degenerate as quickly or regularly as a describe here, because I’m describing ideal conditions—human flaws add much appeal to strategy games where otherwise such games would be repeated drubbings of less-skilled players.)

Skill => Success => Reward => Less Skill Required?

Designers love to reward displays of skill with game mechanical perks. Kill enough goblins and you’ll level up! You played well, so now the game is going to become easier. This kind of design is surprisingly dangerous if you want to design a game of skill and not a timesink or casual game. But it’s so intuitive—it rewards you for doing something well, which will make you want to continue to play. Using positive feedback to reinforce the actions in games that you want players to keep doing—this seems like a completely reasonable thought process.

Here we see a second-order effect. Certain mechanics multiply the player’s effectiveness beyond the normal turn-over-turn addition caused by outplaying an opponent.

Rewards can cause second-order effects that ruin the balance in games of skill; A second-order effect that quickly make matches unwinnable for the less skilled side by multiplying the effectiveness of the more skilled player, increasing his advantage by leaps and bounds as the match progresses. In lopsided and poorly-designed reward systems, actions at the beginning of the game are multiplied and lead to surprisingly enormous benefits later in the game, instead of actions at the beginning of the game leading intuitively to equally important actions later in the game.

I approach games of skill that involve vertical character advancement and gear collection with intense suspicion because I’m aware of how easily second-order effects can skew the players’ apparent skill levels and sap the fun out of what could otherwise be a strategically interesting game. Most of the time, game designers appear oblivious to what they are doing when they add rewards. They tack on external reward structures to make their games more addictive, but in the process they’re punishing new players who are necessarily less-skilled by giving a multiplicative bonus to the effectiveness of veteran, higher-skill players.

When designing a competitive skill-based games, reward structures need to be looked at with the utmost suspicion. They have a tendency to compound in broken ways—ways that quickly transform a game that would otherwise have numerous viable strategies into a one-dimensional, strategically uninteresting race for the first imbalanced reward.


Anonymous said...

"The less skillful player can never win such a game and will lose the game each time by a predictably wide margin that has compounded each turn into a massive deficit."

Do you play many board games? I only ask because with some of the really well designed ones, although the more skilfull player will win, the game will be designed to stay fun even for the losing players right up to the end.

I think there's a lot that could be learned from that sort of design BUT it requires a game that has an opening, a mid game, and an end game built in.

Zerai said...

I tried to find a system to stop this once, but most i could find was grouping bonuses (or flanking, since bigger groups have more chances to flank) or defense bonuses

Also, this kind of balancing makes games last longer, but there has to be a chance to win, or is the "search that last guy in the corner of the map" phase

Kenny said...

Excellent post, my thoughts about skill-based games too. If you're more skillful then it gets easier for you to accomplish what you can already do? Doesn't make sense. The only conclusion I could end up with was to lower difficulty but not below a given minimum, meaning that you always have to perform. The minimum. Of course you might be able to outdo it, but lowering the diff can still mean to accomplish superhuman feats that would be impossible with no skill training. This way those who are more adept can do more - not necesserally easier but more things to do.

On a sidenote, if you don't yet know Robo Rally then it is definitely a game for oyu to check out and playplayplay, it puts an interesting twist on skill, luck and gameplay. Garfield, damn genious he is.

Dblade said...

Yeah, we call them force multipliers. Bad reward systems can make it impossible to win, but its a subtler problem: any game of skill will create compounded advantage over time.

Its not just about distinguishing strategy because each successful strategic "transaction" confers advantage. If I bring my queen out too early in chess, a skillful player can capture it, or restrict it's movement and offensive power. This compounds future transactions against the non-skilled player.

Or if I play online football, and someone gets three touchdowns on me in the first quarter, their skillful play was rewarded. But that just makes it harder for the non-skilled to come back, and its much easier to retain a lead than not.

EVE players use the term "failcascade" to show the effect. Even with no force multipliers failures compound easily in sequential games. Only real soultion is to remove the sequential part.

VRBones said...

As an international referee and tournament director, it is interesting that the underlying problem you are describing is a desirable attribute for competitive play.

The ability for minor enhancements of strategy or application of strategy to compound allows for a quicker discernment of a more skilled player. Any game that allows a catchup effect is frowned upon.

From a spectator's perspective though, this can lead to a game that is over far before the final whistle. The trick is to design games that allow comebacks if a greater amount of skill is displayed toward the end. Games like Baseball allow a team even at the bottom of the 9th to display enough skill to overcome any score. Parallels can be drawn to fighting game health bars where even on 1 health you can still compete and eliminate your opponent, or MR15 matches in counterstrike where it is possible (although less unlikel) to come back from a defecit.

These games have 2 parts; an ability to reset the game back to an even standing (usually through discrete, repeatable chunks of gameplay), and the win condition not limited to time but on point accumulation of those chunks. Due to the length of one 'play', most strategy games do not conform well to this model.

Rewards for success can certainly amplify skill discrepancy, but is only relevant if you are continually playing against the same people. In a competitive environment, the player should now be exposed to other players of similar rank who would have also received similar rewards. The rewards can now be used to discern finer and finer skill discrepancies to accurately rank a player.

Most of the issues you raised are fine for competitive environments, but most games are attempting to also be social and enjoyable. Can't find the link right now, but the Teaching philosophy of praising 90% of the time and punishing 10% of the time was borne out of analysis of how we percieve an enjoyable, learning environment to be. The conflict arises from this difference between 90% winning for enjoyment and an average 50% win rate in competitive environments. Why challenge yourself when the game allows you to pwn noobs to your heart's content?

Kenny said...

@VRBones: "These games have 2 parts; an ability to reset the game back to an even standing (usually through discrete, repeatable chunks of gameplay), and the win condition not limited to time but on point accumulation of those chunks."

Well, this is one problem because if a game allows this then most often than not the best strategy is to mindlessly repeat said small actions ad infinitum (until you win) instead of emphasizing strategy or tactics as a broader method.

VRBones said...

Kenny: "Well, this is one problem because if a game allows this then most often than not the best strategy is to mindlessly repeat said small actions ad infinitum (until you win) instead of emphasizing strategy or tactics as a broader method."

Take for example a point in tennis. Discrete chunks of tennis play that aggregate to form games, sets and matches to find a winner. Each point allows the players to demonstrate their skill, but each new point 'resets' back to an equal start. Players analyze their previous point results to formulate a new strategy to win the next point. This is the metagame and is a highly desired strategic element.

A computer game example would be Street Fighter IV competitions where each individual round allows players to demonsrate skill, but the result of a round is accumulated into best-of-3 games, then best-of-7 games per match to find a winner. After each round and game there is space for reflection, analysis and formulation of new strategies.

If you continually repeat the same actions over and over for each round, a skillful player should be able to counter your moves and win convincingly.

Most strategy games have scope for the metagame built into them (like scouting early to see if your opponent is massing tanks or going air), however they too can benefit from reflection and formulation of new strategies between games.

For the metagame to flourish, all strategies must be counterable. Without a counter to a dominant strategy, games become one-dimensional (like an arm wrestle) and skill only helps in efficiency of execution.

Kenny said...

Ahh ok, I kind of missed/misintrepreted that "reset back" condition. With that it of course makes perfect sense.

evizaer said...

Second-order effects are unavoidable in competitive games. This post was to merely bring them to light and suggest that they be carefully tracked in an effort to better balance games. I'm not suggesting that all such effects be removed or that they are some kind of aberration--instead, I say we must be keenly aware of the runaway feedback present in games instead of blindly repeating the mistakes we've seen in so many strategy games of years past.

RaydenUni said...

Sirlin has written a good article on the topic. There are 3 options, positive reinforcement, neutral, or negative reinforcement. It is just as easy to design a game where the more you win, the more your opponent has an advantage.

I don't think one is inherently better than the other two, depends no the type of game.