Developer Diary
An infrequently-updated set of posts
It’s beginning to become clear just how difficult a fine-grained undo will be to implement.
Let’s consider what a fine-grained undo might look like with a real-world case: a player plays Imported Nitrogen.
Don’t even worry about any complications like using Helion, a simple Imported Nitrogen. This player has 2 cards that take microbes and 2 cards that take animals. So in a normal operation, the player will be asked to choose to place microbes, and then to place animals.
The code looks like this:
public play(player: Player) {
player.plants += 4;
player.increaseTerraformRating();
player.game.defer(new AddResourcesToCard(player, CardResource.MICROBE, {count: 3}));
player.game.defer(new AddResourcesToCard(player, CardResource.ANIMAL, {count: 2}));
return undefined;
}
This means the deferredActionQueue will look like this:
[add microbes, add animal].
The engine goes through a series of steps which I’m sure during this first write-up will be incomplete and consequently unclear:
Player is presented with choosing which card gets the microbes Player is presented with choosing which card gets the animal.
At this point, the deferredActionQueue is empty: []
And waitingFor is SelectCard (for Animals)
The player wants to undo. What needs to happen?
The card with microbes has to be removed.
The deferredActionQueue has to be rolled back
The waitingFor needs to be SelectCard (for Microbes)
How do we do that? How do we keep track of these kinds of changes?
There are several approaches, but here’s one:
- Keep track of the state of the game every time a player input gets resolved, and the next playerInput is sent out. That state could be saved to the database, or possibly just stored, compressed, in memory. It’s not important. (I mean, it’s important, just not for discussing this problem.)
- When rolling back, the game takes the most recent game state and deserializes it. Now, normally this is done by creating a whole new
Gameinstance, which means whole newPlayerinstances. To update the existingGameandPlayerwithout touching existing instances would be new and different -- it’s not how deserializing currently works. (Later you’ll think about that second approach, so I’ll get back to it.) - Somehow we need to restore the state of both
deferredActionQueue,waitingForand alsowaitingForCb.
That’s where things get really, really messy.
Let’s say we decide that the right approach is to keep every deferred action and every waitingFor until the player’s turn is fully over, and that way, it would be possible to not only restore the game state, but to restore it such that the player can continue with the appropriate PlayerInput.
However, PlayerInputs refer to a Game and Player by their instance, and so if a rollback is done by deserializing, that rollback will have different instances of Game and Player, and those will not line up with the ones in the waitingFor and deferredActionQueue.
So I guess that means having to write a restore that does not disrupt instances. And those instances are not just Gameand Player, but any Card or ISpace. Consequently, restoring is tricky.
Some alternatives are to be able to serialize every deferredAction. This also gets messy pretty fast. Here’s a deferredAction:
export type Options = {
count?: number;
restrictedTag?: Tags;
title?: string;
filter?: (card: ICard) => boolean;
log?: () => void;
}
export class AddResourcesToCard extends DeferredAction {
constructor(
player: Player,
public resourceType: CardResource | undefined,
public options: Options = {},
) {
super(player, Priority.GAIN_RESOURCE_OR_PRODUCTION);
}
AddResourcesToCard has a filter. How does that get serialized? What about log?
By and large, each of these is manageable individually, but there are a couple hundred such operations. Making each serializable is a huge chunk of work, possibly too much?
It’s too bad that it would be quite that much work, because it would be fantastic to save them not just for small undo operations, but also when a turn is interrupted by restart.
I've really been almost exclusively focusing on database API, which means that you'll see very few features for a while. It does seem that I broke undo, and I really need to prioritize fixing it. I will. It's just something I know so little about, and I'd rather just kill it for now. :D
I've been changing and updating the API, and generating a couple of tests in the process. Here's what I've come to understand:
-
A game is completely eliminated from the server after 10 days. You can't even clone it. There are good reasons to host your own server. This is one of them.
-
Logs seem to take up about 30% of the
gamestable. I had expected it to be much more. For reasons I won't go into, about half the log entries are duplicates, so a logs optimization would still only remove about 15% storage. Not the 80% drop I was hoping for. So even if we focus on significantly droping logs, it won't make a meaningful difference.
I added a new feature which turned out to be dead simple if I didn't care about internationalization or good UI. It's the new help icon. It could open a deluge of questions and options, so one thing at a time.
There was recent issue with Aridor; it was totally broken. I found the source of that break finally. Kind of embarrassing poor code on my part. But, it's fixed. What a relief.
Happy Memorial Day weekend in America! I wanted to throw out a couple of updates.
Thanks to everyone who is providing translations! I wish it were easier. More big translations are coming for German - the translations are done I just need to get to it.
Looks like I broken undo. A couple of reports are saying so. We don't support undo very well, and I almost don't want to have to go fix it, but I'll take a look shortly.
I posted a possible roadmap earlier this month at https://docs.google.com/document/d/1r1F4CFJDy6_yXei-4dsCdBL4B6SDlvSJHw3g1dPmjsI/edit#heading=h.yj2prikbf94n and it almost generated no communication at all. That's too bad.
I was going to start trying to deal with an undo solution, but there's a saying in software when trying to fix one thing means fixing something else first, which means something else. It's called "Yak Shaving" because the story is that in trying to fix the prior thing that is necessary for you to get progress, you eventually find yourself in a field with shears and a yak.
Well, in order to implement undo I would need to raise database consumption. How much do we use?
terraforming-mars::DATABASE=> SELECT pg_size_pretty( pg_database_size('REDACTED') );
pg_size_pretty
----------------
6124 MB
(1 row)
terraforming-mars::DATABASE=> SELECT pg_size_pretty( pg_total_relation_size('games') );
pg_size_pretty
----------------
5703 MB
(1 row)
terraforming-mars::DATABASE=> SELECT pg_size_pretty( pg_total_relation_size('game_results') );
pg_size_pretty
----------------
413 MB
(1 row)
Our database holds 10GB, which means that if we double the amount of rows in the database, we'll run out of space.
So, OK, I guess my priority is to reduce database size. But before I can analyze that, I need to rewrite all our our database code so it is easier to test. What would make the database code easier to test? Using modern async await standards. Which is what I am doing now. But what's getting in the way? I broke the most recent undo. Yak shaving.