Random Numbers

A visitor recently contacted me with concerns about my Pick 'em Poker simulator. He stated that he was encountering significant losses when running 5,000-hand simulations, and wanted to know if there was a problem. He mentioned Bob Dancer's and Liam Daily's book, "A Winner's Guide to Pick 'em Poker," where the average loss in the worst 5% of 100,000 simulations was 1,860 coins (a return of 92.56%). The visitor pointed out that many sessions were showing large losses which conflicted with both his personal experiences playing the game, as well as the results of Bob Dancer's and Liam Daily's simulations.

I checked the code thoroughly, and did not find anything wrong. I replied to the visitor that everything appeared normal. I also discussed how Pick 'em Poker is a stud poker game, which makes it inherently more volatile than draw poker games, and the fact that 67% of all hands are losers. But just to make sure, I would investigate further by running 100,000 simulations of my own - using a new simulator that I would create from scratch. I created the new simulator, tested it thoroughly to make sure it was working properly, and began running simulations of 5,000-hand sessions. After 7,500 simulations, I stopped it and took a peek at the results.

Out of the 7,500 simulations, the worst session showed a loss of 3,510 coins (a return of 85.96%). I then began running 5,000-hand simulations using the simulator on the website, and the second one I ran had a loss which exceeded 3,510 coins. I ran a few more, and again exceeded that magnitude of a loss. I was now convinced that there was indeed something wrong.

I went back to examine the code again, and still could not find anything. The cards were being determined using the programming language's pseudo-random number generator, and the cards selected were correctly being tested to ensure that the same card was not used more than once in the same hand. I then began comparing the logic between the new simulator and the website simulator, and I finally realized what the problem was.

The .NET platform provides a Random object which generates pseudo-random numbers. A computer cannot generate truly random numbers without some sort of external (and unpredictable) input, therefore the numbers produced are considered pseudo-random. I was using the Random object in both simulators, but with a significant, fundamental difference: in the new simulator, I only created one Random object and used it throughout the entire program. In the website simulator, I was creating a new instance of the Random object every time a hand was simulated - 5,000 different instances.

When the Random object is created, the computer's time is used to determine the seed . This causes multiple successive instances which are created within a short time of each other to produce identical numbers, and that was the problem. The numbers being generated weren't so random after all.

To illustrate how significant the difference between these two methods is, I created a program which obtains "random" sequences of zeroes and ones, and plotted the results as pixels in an image. A black pixel indicates that a 0 was produced, and a white pixel indicates that a 1 was produced. I generated an image using both methods, and here are the results:

Old Method		New Method

The old method resembles a bar code, because blocks of the same number were repeatedly being produced. The new method represents how randomly selected numbers should look - like visual "noise", with no detectable pattern. The net effect of the old method was that the same initial cards and replacement cards were being repeatedly dealt for several hands in a row, until enough time passed that the time-based seed would produce different numbers. Then the new set of numbers would be used for several more hands, and so on, until all 5,000 hands had been simulated.

The real kicker is that I knew better. Pseudo-random number generators have traditionally always used the current time as the initial seed. And it wasn't just Pick 'em Poker that was affected: the simulators for every single game shared the same problem.

The corrected simulators were uploaded on May 13, 2008. I was recently poking around Random.org and came across images which represent random number sequences (similar to those above), which is what inspired me to create my own. I would like to again thank the visitor who discovered the problem in the first place, and took the time to ask about it. I might never have noticed that anything was wrong if he hadn't contacted me.