The “overlearning the game” problem

When I was in 5th grade, the kids in my neighborhood and I invented a great game. The game mechanics are a bit lengthy to describe here, but they involved two teams, each taking turns to question the other team on an imaginary object. The team with the most consistent answers won. We played this game for months, and enjoyed it a great deal. As time went on, we got better and better at it, making it very challenging to trip the other team up with difficult questions, and thus making the game very exciting and intellectually stimulating.

And then, we got too good at it. We overlearned the game. Within the bounds of the rules of the game, we realized that there is a strategy to crafting answers to questions that guaranteed a win. The opposing teams would try to defeat this approach with more and more difficult questions, but in the end, we gave up, and stopped playing the game.

I think this problem, of overlearning the game to a point where you exploit the rules to achieve goals that are far removed, or even opposed, to the original intent of the game, is systemic in human society and permeates almost all aspects of our lives.

Some examples:

Patents

The original purpose of patents was to protect inventors from others copying their paid-for-by-sweat-and-tears inventions. If there was no such protection, and barrier to entry was not too large, people and companies would not waste time and capital developing new inventions that ultimately benefit all of us.

However, people have now overlearned the game of patent litigation, leading to the situation where patents, especially in the software and high-tech industry, are now mostly used as attack weapons against potential competitors, and not even by the people or companies who invented what is covered by the patents. Patents are amassed, through purchases or acquisitions, by big corporations in order to beef up their patent arsenal for planned attacks against other companies or defense from attacks by other companies.

Long gone is the original purpose of patents.

Stocks

For this example, I’ll quote an excellent blog post by Mark Cuban

“… Wall Street is no longer what it was designed to be. Wall Street was designed to be a market to which companies provide securities (stocks/bonds), from which they received capital that would help them start/grow/sell businesses. Investors made their money by recognizing value where others did not, or by simply committing to a company and growing with it as a shareholder, receiving dividends or appreciation in their holdings.

Its primary business is no longer creating capital for business. Creating capital for business has to be less than 1pct of the volume on Wall Street in any given period.

… To traders, whether day traders or high frequency or somewhere in between, … Wall Street is a platform. It’s a platform to be exploited by every technological and intellectual means possible.”

Politics: Elected officials

The original purpose of having elected officials is to find the best person for the job of running a country, or state, or city.

People have overlearned this game too, and the result is that people who win elections are not necessarily those who are best suited for the job, but those who have gotten very good at getting elected. That is, people who can raise a lot of money, hire the best campaign strategists and consultants, pose for the best photo ops, and in general are able to tick off the most check marks on the list of what makes a great campaign.

Law

The main purpose behind having an impartial set of laws is to ensure the smooth functioning of society, and for a more abstract ‘make sure justice is served’ goal. Even this abstract goal has a concrete effect since, if people in a society can reasonably expect that justice is served most or all of the time, they are more likely to engage in many activities with their fellow citizens, such as trade, and the situation is more likely overall to lead to a proper-functioning and safe society with a high standard of living. To see why this is the case, one only needs to look at how well societies do when corruption is widespread or the rule of law collapses.

Unfortunately, there is a large group of people, namely a large majority of the legal profession, who have overlearned the game, and are now using the law as a weapon against people and companies, to further their clients’ interests at all costs, subverting the original intent of the law. An example of this is people and companies using the fear of the cost of litigation as, effectively, a blackmail mechanism to prevent competitors from engaging in activities that would affect that person’s or company’s bottom line (because even if the defendant wins, and knows they can win, that would still cost more in legal fees than just settling out of court, or simply avoiding the behavior that can trigger the lawsuit).

Subverting the original intent of the law to the exclusive benefit of their clients, while staying within the letter of the law, is a highly-prized and highly-paid skill that the top attorneys in the country have, and which of course does nothing for society at large or for the overall ‘make sure justice is served’ goal. And the fact that people are more and more aware of this is damaging to the foundations of a well-functioning society.

Test taking

Even on smaller-scale issues, overlearning the game is problematic. For example, the original purpose behind test taking was to find out which students have learned the most, or which students are most likely to do well at a particular university, or which candidates are best suited for a job. The original approach to doing better on tests was to simply study more, which also had the benefit of you becoming better in that subject.

The new approach, based on overlearning the rules of the game, is to over-emphasize test taking, leading to a large number of students who are great at getting good test scores, but don’t know how to attack problems that are outside the limited set of problems that show up on tests. And this is detrimental to their performance in their jobs, because in real life problems don’t fall under neat categories that can be solved with approaches from the focus-on-test-taking toolbox.

It should be noted that in all the above examples, people are still playing within the rules. With fixed rules, people are eventually able to find ways to exploit the rules to their benefit, and usually to the detriment of society at large. When societies notice this, and modify the rules to exclude some of that undesirable behavior, it doesn’t take long for people to find ways to exploit the new rules. (A prime example of this is campaign financing, and all the loopholes people have found over the years of circumventing campaign finance laws to get the money to the candidates they want to fund, no matter how many new campaign finance laws are passed.)

As the rules keep being modified to prevent abuse, and as people continue to learn to exploit the new rules, there comes a point where there is not much room any more to change the rules. There is only so much you can change about patent laws, or election laws, or stock market rules, etc, before they no longer serve their original purpose.

In the game I mentioned at the beginning of this post, there was only so much we could change about the rules to stop the undesirable strategy that made the game unplayable. In the end, we couldn’t end that behavior, and we stopped playing the game. But, in real life, we need to keep “playing the game”: we need to have elections, and protections for inventors, and laws that govern society, and a market where companies can raise money.

The question is, what do we do when we have to keep playing the game, given the endemic problem of the emergence of strategies that subvert the intent of the game.

Originally published at https://andrewoneverything.com. Original responses:

• frankydp (Twitter) responded: Farnk_normal Simply play a game in which no player is the winner. Change the game to benefit everyone not someone.
• kevinsimper (Twitter) liked this post.
• etorreborre (Twitter) liked this post.
• Fuzzy responded: Fuzzy I arrived at this post because I thought it was going to be about competitive computer games, an area where this problem occurs a lot. In general when major balance issues are found either: 1) people stop playing the game 2) the developer patches the game, nerfing whatever was over powered 3) the players eventually find a game mechanic (often a bug) which can actually defeat the over powered tactic Or finally, most interestingly to me: 4) a set of gentlemans’ rules are formed by the players themselves An excellent example of this is the street fighter scene in Japan, where some characters are deemed too strong and simple not used by any of the top players. Of course, much like the examples of real world problems, this relies on all parties having a common idea of the ethics or morals. Most importantly these ideas are ALWAYS led by the top players in the game. As a side note, I’ve just finished reading Sirlin’s playing to win articles and thinking about real world similarities is quite a good mental exercise. http://www.sirlin.net/ptw
• AhmadAlhour (Twitter) liked this post.
• naps92 (Twitter) liked this post.
• Hussain M Elius liked this post.
• overtnibble liked this post.
• EnigmaticDev (Twitter) liked this post.
• sp4zzpp2 (Twitter) liked this post.
• sp4zzpp2 (Twitter) responded: Missing-user-35 Finding a set of rules that in the end cause the intended behaviour is a NP-Problem that only can be solved by running a lot of concurrent tests, I guess.
• mikellewellyn (Twitter) responded: Mikemttam Great article, and I think a powerful analogy. I have been thinking for some time now that some of these major games need to be almost completely ditched, or changed dramatically. In effect, your solution of no longer playing that game because it was too flawed, but I expect you went on to play other games. Society needs to do the same. The big problem we have is all the vested interests in the status quo, who have the most to lose, are also typically the people with the most power and who are themselves in a position to do something about it. Patents — the system is clearly totally failing now. A possible solution might be — convene a group of respected and varied individuals to design a new system. Pull perhaps 12 people, one from each major area, and have them debate and analyse options and propose a new solution that needs say 9 of them to agree to it. Give them maybe 3–5 years. To choose the 12, maybe have some kind of voting system as used on a tv reality show ;) Politics — I am finding it increasingly difficult now to reconcile our advances in technology, psychology, sociology, and philosophy, with the 19th century-politics that seems to be prolific in Western “democracies”. What I mean is, the politics of our countries seems generally to reflect the attitude and behaviour of the stupid people of society, rather than the smart people. And the smart people are LETTING THAT HAPPEN. If we are going to solve the major problems facing the long term prosperity of the entire human race, we need to evolve/find/design a more effective system. Money should not be the determining factor for the organisation of society.
• jaygalvezo (Twitter) liked this post.
• Vytautas Jakutis liked this post.
• mikellewellyn (Twitter) liked this post.
• joelbot3000 (Twitter) responded: Qnzzzvsbma5dyvd4v32z_normal It seems like there needs to be collective responsibility from all game-players, that refocuses on the overall “spirit” of the system when the intent of the game is violated. The problem is that in our system there is a vested interest (often in direct opposition to the average citizen) in exploiting the system (e.g. patent trolls, political parties, high-freq trading firms).
• Spiggi liked this post.
• pattheflip (Twitter) responded: Haggar_normal fuzzy: the “gentlemen’s rules” in Japan are really only applicable to Super Turbo, where Akuma was “soft-banned”-you could still pick him in tournaments, but it was considered bad form. this stands in stark contrast to the remaining 99% of competitive street fighter, where all players everywhere are more than happy to break the game because that’s what we do to win.
• bshock responded: bshock I too find this a very powerful analogy, and I admire the thinking behind it. I’m interested in other examples where “overlearning the game” may have occurred. For instance, what about free markets in a capitalist system? The basic idea is that companies compete in the market with products/services, and the companies with the lowest prices and better products/services gain a greater market share. However, we seemed to have reached a point where some companies have learned to game the system in various ways. One obvious example is regulatory capture, where a company or companies convince the government to create regulatory laws/agencies that negatively impact smaller competitors that might disrupt the existing market. Perhaps a less obvious example is to control or distort the flow of market information. In the simple market scenario, lower prices and better quality help a company succeed because customers compare and disseminate information about their experiences; you buy what consensus suggests is best. Flow of information can be distorted through improved advertising/marketing, which exploit flaws in human thought processes to push and maintain inaccurate information. Flow of information can also be distorted through subverting media through ownership or other methods of influence, pushing an agenda of counter-factual information or simply creating confusion by muddying informational waters.
• BrianTRice (Twitter) liked this post.
• createdevelop (Twitter) liked this post.
• seththe responded: seththe To me, on a visceral level, the characters gaming the system in all of the examples seem immature. I know that there are many rewards to gaming the system and ‘winning.’ but this shortsighted winning isn’t ultimately satisfying. I’m not totally sure about this but it is my instinct. A tract which is relevant to what I’m trying to get at is James Carse’s Finite and Infinite Games: http://www.glg.net/pdf/Finite_Infinite_Games.pdf. I guess I ultimately think that the problem Andrew is exploring is a cultural problem. How can we continue playing the games when some shortsighted players are willing to shortcut the process and beat the games for their own ‘profit?’ I guess the Japanese street fighters are a good example of there being enough cultural leadership that collectively people make the decision to continue playing the game rather than winning decisively even though that opportunity is built into the game.
• Pinke2020 (Twitter) liked this post.
• martindevans (Twitter) liked this post.
• Twisted_Majic (Twitter) liked this post.
• tito (Twitter) liked this post.
• Yaakov responded: Yaakov I wonder if what is going on here is not overlearning but mere learning. If the goal of any “game” is to win (and each of your examples is about gaining advantage by winning in one way or another) then finding ways to achieve the goal of individual gain in the most efficient way *is* learning. That is to say, you are assuming an ethical component to these “games”, that one should “play” them to get the result the rule maker intended. However, while the goal of, say, patents was one thing: to foster innovation, the goal of the innovator was to make profit. So, the rules were set up with the *intention* that an innovator should benefit from them. As it turns out, people wishing to “win”, in this case, make a profit, have no reason to consider the ethics of abiding by the intention of the rule maker. Put another way: it’s only “cheating” if you actually break the rules. In your game example, a player might, say, accumulate points or finish in the shortest time. That being the goal, even if they have to step outside the notional framework of the game (e.g.: using mechanisms in the game in a way the designer clearly did not intend), they haven’t overlearned it they’ve learned it. They’ve found ways to treat the game so they can maximize the points, etc. Of course, when the game stops being a challenge, they will become bored and stop. But, then, that’s true of every game, even if the rules are followed precisely and without mischief.
• josex responded: josex You played a game: come up with good rules. You played another game: try to win under those rules. You now play another game: try to improve the rules. You play yet another game: try to win under the modified rules. And this keeps going and going. Innovation does not stay still. Security is not a static effort. No human endeavor is simply mission accomplished for all time. As long as you have people playing the game to build a certain ruleset to achieve “someone’s” idea of justice, you will have people playing to achieve their personal interpretation of “justice” or at least to overcome the goals in a certain way. You have dueling intentions because everyone is different and has a different idea of what should be the end result. As long as you care about that particular game (ie, as “rule maker” or as traditional “player”) and you think improvement is possible, you will keep playing it or thinking about improving.
• AdamRNeary (Twitter) responded: Headshot_normal Great post. The problem, as you identify, is that everyone is working within the “rules” and yet the gaming of the systems seems to be an accepted/expected behavior. Maybe it sounds naive, but I think the only solution is to try to build a culture that values politicians governing, bankers driving the economy rather than sucking money out of it. I think we need to build a culture where people should be ashamed to admit they gamed the system at a bbq rather than eager to brag about how they did it. It’s a start, anyway…
• Yaakov responded: Yaakov @AdamRNeary In my comment, above, I identified what you are talking about as the “ethical” component that is generally missing. This is not an inevitable state of affairs, in my opinion. What is lacking is an agreement on what is right and a race to the lowest common denominator of self-interest. Because people have agreed that “so long as you follow the rules you are ‘right’” we can’t advance, but that’s not the only possible common ground. Ethical public and corporate (groups of all kinds acting as agent) behavior, where ethical means something more humane, is, I believe, possible.
• Taiss liked this post.
• phil gauvin responded: phil gauvin great post and great question to ask.
• sam250 responded: sam250 Humans work systems to their advantage? What a surprise! The original intent of parents, stocks and even the law are a result of humans working lower level rule sets to their advantage. Even if all that were stripped away they would still be doing the same thing to the rules of physics, making better clubs and bows for killing each other. That it human nature and not something to be criticized — this is just a matter of strong, intelligent people getting somewhere other people cannot. All you need be concerned with is how YOU play the game to make what you want happen, happen.
• Spencer responded: @sam250: Well, not really. One of the primary reasons people develop societies and governing structures is that most people don’t really like that kind of Darwinism. They find that conflicts with human nature-that it prevents them from leading happy, fulfilled (whatever that may mean to each person) lives. Surely, there are people who think like you, but human history suggests they are the minority, and that they are spoilers for the rest of us.
• sam250 responded: sam250 “…most people don’t really like that kind of Darwinism.” lol Don’t you see that fighting “Dawinism”, promoting “liberalism” or whatever, is itself Darwinism? Laws and morals for people to protect themselves without having to be physically strong, ect? You can’t get away from it. It’s what we are.
• Spencer responded: “Promoting ‘liberalism’” could be understood to be a form of Darwinism. Though to be clear, that would mean that people build societies not just for protection from the strong, but because they believe the structures they erect will empower them over the vast majority of others. That is a view of history you could hold. It just doesn’t seem to me that most people build societies and communities, or live, with that in mind. It seems like people want to live in relative peace and security and be able to do what they feel fulfilled by and value. This is ultimately a probabilistic empirical question. Neither of us can know we’re right, but I think the approach of your view is somewhat problematic. For starters, it assumes that human nature is static and that it can’t be affected by things like philosophy; that is, it can’t develop in any way. That assumption doesn’t seem obvious. What’s more, your view is self-fulfilling, and self-serving. Believing that there’s no reason to behave altruistically because everyone else is being self-serving chooses the one of two (or more) possibilities that immediately serve you. That seems like a conflict of interest. Faced with the uncertainty of two choices of belief, I pick the one that describes a constructive multi-generational project, rather a philosophy of, basically, enlightened hedonism. Admittedly, I suppose the greater fulfillment the former provides is something of a conflict of interest of its own.
• LaMangaLaManga (Twitter) liked this post.