A quick look up on the internet, gives us the definition of Game Theory as:
The branch of mathematics concerned with the analysis of strategies for dealing with competitive situations where the outcome of a participant's choice of action depends critically on the actions of other participants.
Having a game theory optimal (GTO) strategy means that we find the best solution for our situation irrespective of what our opponent does. A standard example is the Prisoner's Dilemma. Here's a link to the Wikipedia entry and a YouTube video if you didn't know about this already.
What does GTO mean in poker?
Being a good poker player means making the best decisions based on the incomplete information you have. To bridge this very chasm between incomplete information and the best decision, we take the help of game theory. Just like in the Prisoner's Dilemma, we do not know what the opponent's strategy is. So we make the most optimal game theory decision in a vacuum. Then, as we gain more information about an opponent, we adjust your strategy based on this new information.
1. Bet Size
On the flop, your opponent bets 1/3rd pot. What do you do with middle pair? Most likely, you would call. What if he makes it 1/2 pot? Or 3/4 pot? Or 2x pot? 4x pot? At what point do you decide that you won't call this bet?
As your opponent's bet size increases, the odds that you get become worse. In order to not get exploited, we have to start letting go of some hands. So we start folding our weak pairs, then middle pairs good kickers, then middle pairs strong kickers, and sometimes even top pairs with weak-medium kickers.
2. Bluff catching
Similar logic goes into bluff catching. For example, say you have JTo and board runs out Jack high - you have top pair medium kicker. Your opponent, whom you don't know anything about, bets big all three streets. What do you do? If you always call down with just a top pair, you will be often up against QQ+ or better top pairs. If you always fold, you open yourself up to getting bluffed. To find a balance, we use a GTO approach.
As we move from flop to turn and to the river, we start folding the bottom of our range. This means that out of all the hands that we could have in this spot, we start folding the worst ones. If we find ourselves here with top pairs often, we start folding the ones with the worst kickers. If we call with every Jack on the flop, we fold some like J9, JT on the turn. On the river, since we arrive with QJ+, we have to fold some of these and call with some. So we can fold the worst Jacks - QJ, KJ and call down with AJ.
In the first example, we did not know what our opponent bet-sizes meant. In the second, we do not know when our opponent is bluffing or when he is value-betting. Hence we chose a strategy that allows us to make the best decision no matter what he does. Does it mean that we will win every pot? No. But it does mean that in the long run, after playing thousands of hands, and facing the same situations many times, the summation of all our decisions will come out to be a profitable number.
How does GTO compare with Exploitative strategy?
Exploitative strategy means that when you have absolute reads on your opponents, you deviate from the GTO approach to make the best decision in that particular instance. For example, if you know that the villain never bluffs when he bets big all three streets, but you are at the top of your range, it maybe a fine GTO call, but you can make an exploitative fold knowing that you are always beat.
No single strategy is better than the other. Usually a combination of both is the way to go - because even though GTO style is profitable, mixing it with exploitative might be even more profitable. This is especially true when you are playing against weaker opponents. They have certain tendencies and leaks that are better suited for an exploitative style of play. But against stronger opponents, who have fewer obvious leaks, a GTO style is much more favorable.
As the poker software and poker AI has become better, more and more players are employing a GTO approach, especially when the stakes are high and the competition is good. But since no player is playing a perfectly GTO strategy, adding an exploitative style to your game is a good idea.
Hopefully, this primer was useful and may it help you make more profitable decisions in the future. And next time try employing this strategy in any one aspect of your game - value-betting, check-raising, bluffing etc.