Solo Game

During the Kickstarter campaign (October 2019), we announce that we are including an app-powered solo mode for The Search for Planet X. We added it to the app in May 2020. Here is some history about its development and progress.

2019 February
Alex plays the game at DundraCon, and he develops proof-of-concept scripts that creates a game state and solves it. These scripts calculate that there are 1,138,272 arrangements of objects that follow all the logic rules.

2019 May
We meet at KublaCon to brainstorm and discuss approaches for developing the digital bot and integrating it into a solo mode for the app.

2019 August
We meet at Gen Con to finalize the approach and discuss the design. We officially hire Alex to create the bot.

2019 September
Alex submits the first complete working version of the bot, and we playtest against it.

2019 October
We launch the Kickstarter campaign for The Search for Planet X. Alex revises the bot based on our playtesting feedback, creating numerous configuration parameters we can use to tune the game play and the difficulty of the bot.

2020 February
We design the format that we will use to store the bot’s moves (what actions it will take and what theories it knows) in the maps. We modify Alex’s bot code to generate its output in this format.

2020 March
We add the solo buttons to the app interface. Two new buttons appear on the main action menu (“Bot: Take Turn” and “Bot: Submit Theory”) that perform the appropriate move based on where the bot’s player pawn should be on the time track. We track whether the bot or the player has located Planet X and adjust the buttons accordingly. If the player finds Planet X first, we determine what the bot will do with its final scoring opportunity. Version 1.5.0 with these changes was uploaded to the stores on March 10 for beta playtesting.

2020 April
We published a draft of the rules sheet for the solo online for open beta testing. Version 1.6.0 was uploaded to the stores on April 19, which includes a link to this draft rules sheet. We worked with Dan King to make sure that the instructions for the solo mode would be included in the How To Play video.

2020 May
We continued to test and tweak the algorithms for the bots actions and theory submissions. We added 40 more game codes to the Standard Mode of play. We fulfilled rewards to backers in the United States, and many began to playtest the solo mode and send us feedback.

2020 August
We added a session report form link to the app for solo players to submit detailed data and history for each solo game.

We have received numerous questions about how the bot selects its actions and makes its deductions. This document provides insight into how it all works.

BACKGROUND: GOAL FOR THE SOLO GAME

As board game developers, our goal for the solo game is to provide a fun way for people to experience the game even when they don’t have anyone else available to play with them. We want players to FEEL, as close possible, what they feel when playing against another player. As a game publisher, we want customers to get as much value from buying this game as they can.

We are not machine learning researchers. We are not trying to create a definitive competition between artificial and human intelligence. We are not creating a benchmark for players to measure their own intelligence. If computers can learn to beat humans at Chess and Go, I’m sure they could learn to beat humans at The Search for Planet X. But it is not our goal or our interest to pursure that; we created the bot to simply add more fun to the game.

APPROACH: HOW BOT ACTIONS ARE SELECTED

The bot developer created an algorithm that can perform actions and locate Planet X. It starts with a list of all possible arrangements, calculates what actions might provide the best information, and performs one of those actions. It then looks at the result of the action and compares that with every possible arrangement to eliminate any that do not produce the same result. That gives it a smaller number of possible arrangements to consider for the next action. The algorithm repeats this process until it locates Planet X.

If I run the algorithm against a particular game code multiple times, it can produce very different results. Even on the same game code, it can take 5 actions one time and 11 actions another time. Sometimes it can’t publish any theories until it can locate Planet X: it might not know anything for sure until it learns many things all at once. This happens because two actions may have the same potential value but, for any given game code, have different actual value depending on the results of the actions.

Some of these outcomes are not fun to play against. We don’t want it to locate Planet X too quickly so that the game ends before you get a chance to get going. We don’t want it to locate Planet X too slowly so that it provides no tension. We want the bot to publish theories so you can be racing against it to score points and trying to deduce things from its actions.

To achieve this, we don’t include the bot algorithm with the app. Instead, we run the bot algorithm against a particular game code multiple times and analyze the different results. Does the bot locate Planet X too quickly? too slowly? Does it learn enough to publish theories? We restrict the actions it can do so that it is more likely to be fun to play against, and we reject any outcomes that are too far of an outlier. Then we pick one outcome and store it with the game code in the app. So every time you play a particular game code solo in the app, the bot takes the same actions. And it doesn’t take minutes between turns to calculate its actions; those have already been calculated. The drawback to this approach is that the bot cannot incorporate your actions and your theories into its deduction.

QUESTION: Does the bot get starting information?
For all the Standard Mode game codes, the bot currently starts with no starting information. For all the Expert Mode game codes, the bot starts with 4 or 8 facts of starting information. We are working on storing that within the game code so that we can show that to players in a future version of the app.

QUESTION: Does the bot cheat?
The bot performs real actions and uses only the information from those results in the deductions. It does not get any information that a real player would not. The bot gets to play the same game code multiple times, which might seem unfair. But we aren’t taking the outcome that gives the bot the best chance of winning; we take the outcome that we believe will be the most fun to play against. That does mean the bot won’t usually be too unlucky in its actions; otherwise it wouldn’t learn enough to move the game forward and be fun to play against. In a multi-player game, if you get unlucky in your actions, there is information from other players that can help get you back on track. The bot doesn’t get that, so letting it play the game code multiple times eliminate the outliers is a way we can adjust for that.

BOT ALGORITHM DESIGN

By Alexander Mont

Here are some notes from Alex about the design of the bot:

The AI starts out by enumerating all possible sector maps that are consistent with the game’s Rules of Astronomy (there are 1,148,272 such maps for the “expert version” with 18 sectors, and 4,446 such maps for the “normal version” with 12 sectors.). It stores this as a “knowledge state” that represents everything it knows so far. Then, each time it receives information (from a conference clue, a scan/target, or prior knowledge) it removes from the knowledge state all sector maps that are not consistent with that information.

A knowledge state can be “scored” based on how valuable it is to the AI. We use a formula that rewards being able to safely place theories, finding the location of Planet X, and reducing the overall number of possible remaining sector maps. The AI will only place a theory if it is sure that it is correct (i.e. all possible maps in its knowledge state are consistent with the theory)

When the AI decides what scan to do, it does the following for every possible scan:

• Divides the current knowledge state into groups based on what the result of the scan would be if that were the true sector map. Thus, the groups represent the possible future knowledge states after the scan is completed.
• Score each of the potential possible future knowledge states.
• Figure out the average score of those knowledge states, weighted by the probability that the AI will end up in each state. Determine how much the score is expected to improve from the score before the scan.

Then it selects the scan with the highest ratio of expected score improvement to the time cost of the scan.