Sunday, June 26, 2022
No menu items!

The artificial intelligences have taken over the poker as well

It was not easy, given certain peculiarities of the game, and not everyone is happy that it happened

Must Read

Like many other games that are on a table and which are regulated by clear and shared rules, poker has also received the attention of those who deal with artificial intelligence systems. In particular, its Texas hold ’em variant, which was codified in Texas (eh already) in the 1960s and which has long been the most popular. Compared to games such as chess, or the very difficult go , Texas hold ’em has some peculiarities: first of all up to a maximum of ten players can play it simultaneously and, more importantly, every player and every artificial intelligence player is all obscure of the cards in the hands of each opponent. In other words, the table does not contain all the information necessary to develop a winning strategy.

Despite these peculiarities, even Texas hold ’em has struck a fate similar to that of chess and go  and, as the New York Times Magazine wrote , ” it was conquered by artificial intelligences”. However, more than in other games, there remain pockets of resistance to this conquest, and considerable space for a human approach.

Texas hold ’em – often also called just hold’ em – requires that at the beginning of a hand each player receives two cards from a deck of 52. They are the so-called hole cards and each player looks at his cards but does not know those of the opponents . At this point, in a series of successive phases, five cards common to all players are added to the table, who seeing first three ( flop ) and then, one at a time, the following two ( turn and river ) can decide whether to bet, calling, raising or folding, in this case leaving the previous bets on the table. The aim is to use one’s cards and community cards to obtain , according to the rules of poker, the best possible combination of five cards.

Then there are variants, specific terms and various complications, but the gist is that you have to get to the end, episode after episode, and hope for two outcomes: that the others, not convinced by their cards, abandon first; or, if someone is still in the game, that one or two of their cards guarantee, if combined with the common ones, the victory of the hand and therefore of the pot, that is the sum of all the bets made by the players during the hand.

Since poker existed before the Texas hold ’em rules were codified, already in the first half of the twentieth century, on the wave of studies and reasoning on game theory, there were the first attempts to develop effective models aimed at understanding what an optimal game strategy was for each player. How convenient poker could be to game theory is well explained in a sentence by mathematician John von Neumann, co-author – together with economist Oskar Morgenstern – of the book Theory of Games and Economic Behavior . Complaining that many games were unsuitable for his purposes, von Neumann said: “Real life is made up of deceptions, little lies, it’s full of moments when we ask ourselves what others think I want to do, and that’s what games are in my theory.”

But the deceptions, bluffs and incomplete information that each player has about everyone else are also what made poker a very human activity for decades, in which, as the New York Times Magazine wrote , “even computers had to simply interpret the plays of the opponents without knowing with certainty which cards they had in their hands “.

Neil Burch, a computer scientist who now works for the artificial intelligence firm DeepMind and who previously spent a couple of decades studying poker as a University of Alberta researcher, told The New York Times Magazine that for a long time the results were disappointing: “If we pitted a good poker player against a computer and played them, the computer fell apart.”

As the New York Times Magazine explained , game theorists use a tree diagram to represent the various paths a game can take. “In games like paper-scissor-rock, the tree is small: three branches for paper, scissor and rock, each with three branches for each of the opponent’s three possible moves.” For Texas hold ’em, even if you want to take a simple version, with only two players and with predetermined bets and winnings, the tree has 316,000,000,000,000,000 branches, that is 316 million billion branches. In the case of even more complex versions, Burch said, the branches become even greater than the number of atoms in the universe.

Despite the many zeros of these numbers, for some years artificial intelligences have managed to conquer even poker.

Computer scientists and researchers started with simplified versions, in which, for example, a pair of 9s was considered equivalent to a pair of 10s, so as to allow artificial intelligences to get carried away. Then they raised the bar, mainly thanks to an algorithm based on counterfactual regret minimization , the counterfactual minimization of regrets.

In other words, again those of the New York Times Magazine , “Computers were told to identify optimal strategies by playing against themselves billions of times and noting which decisions, which streets in the game tree, were least profitable. “. In short, they were told to remember which moves had generated regrets, so as to minimize them in subsequent moves, and it worked, as explained in an academic article published  in Science in 2015. All of this, however, continued to concern the one-on-one hold ’em games. one and with fixed bets.

That article – titled ” The one-on-one hold ’em limit has been solved ” – may not say much to many, but for those who played online poker (an activity already very popular in 2015) it was a sort of DeepBlue-beat-Kasparov moment. «I remember well» said former professional poker player Terrence Chan «that after reading it, I and others said to each other ‘it was good while it lasted, but now it’s over’.

Things weren’t that drastic, but they certainly gave online poker a big hit. Between those who wanted to develop programs to win online, and those who wanted to develop programs to understand who used other programs to win online, there was a certain commotion. Also in 2016, Polish programmer and former poker player Piotrek Lopusiewicz began selling PioSOLVER online for a couple of hundred dollars, a program that “approximated the solutions for the more complicated versions of the game.”

Since then, PioSOLVER has been used, along with other similar programs , by many professional players and these programs, as the New York Times Magazine wrote , “have radically transformed the way poker is played, particularly at its highest levels. “. The programs, also known as solvers (solvers) are now quite accessible and it is very difficult to find high-level players who do not use them before or after a game, to prepare or to critically analyze their hands.

One of them is Jason Koon, who has been playing at a high level since 2006 and has amassed tens of millions of dollars in prizes in his career. “In the pre-solver era,” he said, “I was a decent player, but from the moment the solvers  arrived, I dedicated a lot to it and started improving very, very, very quickly.”

Specifically, Koon uses PioSOLVER to train himself to understand, at any given moment, what is the optimal choice, that is, the one that – based on the limited information available to the software – offers the greatest chance of success.

Indeed, the solvers are able to suggest which bets are best to make and, in their own way, to evaluate even if and when it may be the case to bluff, that is, behaving in a way that makes others believe that they have cards that you do not have. During the games, Koon and all those like him who use solvers obviously cannot consult them, they can however – as chess players do after training with computers – try to do what, according to their memory and experience, a solver would do. .

But poker players also need to keep in mind that solvers  tend to act by assuming they are playing against other players intent on optimizing their play. Simplifying a little, it can therefore happen that inexperienced players or very skilled in doing non- solver  things can to some extent send the solvers or those who try to emulate them into crisis. «Against weaker players» wrote the New York Times Magazine «Koons sometimes deviates from theoretically perfect poker, bluffing more than necessary or betting heavy when instead the artificial intelligence would suggest to be cautious in order to better exploit the mistakes of others».

Simplifying even more, software has the advantage of not being emotional, not getting excited after many good hands, or darkening after it runs badly for a while; but also, at times, the disadvantage of being too analytical.

In some of their quick and simplified versions, the solvers  can also be used online: there are already cases of players suspected of using them and, if in doubt, there are those who now participate almost only in live tournaments, despite many sites making an effort to prevent that from happening. However, it seems to be one of those cases in which prohibitions and counter-moves will hardly be able to cope with the ever greater, better and faster ways to cheat.

The New York Times Magazine also wrote that the search for optimal solver -like strategies has changed the way hold ’em is played at the highest level, in many ways making talent less important and study more important. Doug Polk, who pretty much quit playing in 2017, said: “I think [the solvers ] took the soul out of poker: before there was a lot of room for creativity, now it’s just about memorizing things and putting them into practice.” .

For Lopusiewicz, the developer of PioSOLVER, solvers are nothing more than “a more powerful weapon” in a context, based on what is essentially a game of chance, in which there had been an arms race for some time. », For example with software that made it possible to analyze and study every previous move of every possible opponent. He has a similar idea Koon, who said: “ Solvers  don’t tell you why they do what they do, they just do it; it’s up to you to understand why “.


Please enter your comment!
Please enter your name here

Latest News

What to do after an accident?

What should I do at the accident site? Being in an accident can be very stressful. Find out now about the...

More Articles Like This