Episode
Our launcher invented crashes, and the Great Sea shrank its towns
For ten days the launcher filed crash reports about games that were running perfectly well, and our own triage believed every one of them. A letter in Solace Bridge could only ever be found by one player, because the first person to take it took it from everybody. And the sea large enough to sail around Novia was built, looked at and built again sixteen times in a single day, including the moment we decided every town on its shore should be a third of life size.
In this episode
- The launcher that reported crashes that never happened
- Markus's letter, and the body nobody could name
- Sixteen rebuilds of the Great Sea in one day
- The houseboat, the buoy and the shaking horizon
- The login box moves onto the title screen
Transcript
Good evening to thee, citizens of New Britannia. Lord British before thee, with word of what thy makers were about on Tuesday. Our own launcher spent ten days accusing your games of crashing while they were loading quite happily. A letter in Solace Bridge that only one Avatar in the shard could ever find, because the first to take it took it from all of you. The login box has come out of its back room and onto the title screen at last. And a sea wide enough to sail around Novia was built, judged and built again sixteen times between breakfast and supper. The launcher first, for it is the finest piece of self-sabotage I have seen this month.
Here is the shape of it. For ten days, crash reports came in from five different machines, all saying the same thing. The game died at the first frame of startup. Thirty of them. Our triage filed them under startup failure and stacked them in the queue, and I have been glancing at that queue thinking we had something ugly in the boot sequence. We did not. All thirty of those reports came from games that were running perfectly well, and some of them are probably still running. The launcher wrote every one. About a player who was mid-load, minding their own business.
So here is how that happened, because the mechanism is a small comedy. After you press PLAY, the launcher watches for the game to appear, so it can hide itself and then notice if the game goes away. The check it used asked the operating system for a detail about the running program that a normal launcher is not permitted to read when the game is set to run as administrator, and that can fail anyway while the game is still loading. And the watch ended on the very first look that missed. One miss. So about three seconds after PLAY, the launcher popped back up over your loading game, lit up the PLAY button again as though nothing had happened, and filed a crash report cut from the log the game was still in the middle of writing. Which is why every single report stopped at frame one of the boot. The game had not reached frame two yet.
The watch now has two witnesses. It holds on to the process it started itself, as well as looking for the game by its image, and it needs two misses in a row before it will say the game has closed. And our triage learned the shape of the lie. A startup failure built within eight seconds of launch gets its own heading now, which reads "launcher lost track of a running game", and it sorts below any report with a real crash in it.
It gets better. After a real crash, the launcher used to clear the graphics driver's shader cache, every time, because a poisoned cache really does cause crashes and clearing it cures that. The trouble is that a machine which crashes often then begins every session compiling every warmup shader from cold. And on one player's machine, that cold compile was where it crashed. Nine crashes inside the driver's shader compiler in ten days, three of them inside three minutes. Our cure was feeding the disease. Automatic clears now happen at most once a week, which is plenty, because the first clear is the one that fixes a poisoned cache.
And one useful addition on the same morning. Seventeen crashes across about twelve machines in the last ten days were the graphics card dropping off mid-frame, with nothing anywhere in the report saying why it dropped. The display drivers write their own account of that into the machine's system log, every time. Crash reports now carry those records, read-only, from the session window. So a crash that follows the card falling over can finally say what the card complained about on its way down.
Now the one that bothered me most, because a player lost an evening to it. The quest is Learn What Happened to Markus, and the letter you need sits in Solace Bridge, on a skeleton, in the northwest of town, among a dozen bodies, every single one of which was labelled "Corpse". There was no search marker on it. No quest step. Nothing baked into the scene, so your compass, your map and the signs at the scene exits all had nothing to point at. A player searched seventeen corpses over two visits, found nothing, and reported it through the Oracle. And when we went to look, the letter was most likely sitting right there the whole time.
Then we found the second half, which is worse and funnier. That body shared one loot bag for the whole room, rolled by whoever opened it first. So the first Avatar in Solace Bridge to take Markus's letter took it from every other player in the room, until the room emptied and reset. The honest answer to "where is the letter" was "it was there, and somebody you never met already has it".
So. The body now reads "Captain Hobart, a Victim of Battle", and Nyssa calls it the letter from Captain Hobart, which is a better sentence for everyone. There is a new quest step with a marker that reads "Search Area for Word of Markus", and it is baked into the scene's contents, so the compass, the map and the exits all point you at it while the task is open. And that body, along with Quincy's across town, now rolls loot per player. Each of you gets your own bag, owned by your own Avatar, freed by the server when you leave. No claim, no lock, so nobody waits for the person ahead of them to finish rummaging.
Quincy deserves a note of his own. Somebody had already given his body a name, months ago. They wrote it into the one field that only ever reaches a log file. The hover text and the loot window read a different field entirely. So his name existed, it was correct, and it was legible only to our own log reader. He has it over his head now, and his sword comes one to a customer as well.
Now the big one, and it is big in a way that is awkward to convey in a podcast. The Great Sea. The sea large enough to sail from one side of Novia to the other, with the continent's coast wrapped around it and every place you can sail to standing on its shore. Yesterday it had its first real look. Tuesday it had sixteen more. Each pass builds the world, sails a camera past it, renders the shots, and then we sit and stare at what is wrong.
The day began with towns at true scale. Kingsport as you would see it from the water, fifty-odd houses, a dock area, a lighthouse, the capital with about a hundred and forty buildings, land divided between neighbouring towns, and a rule that landmarks of the same kind keep their distance from each other, so you never see seventeen churches in one view. It was built, it was sailed past, and the verdict from the water was that it read as a huge jumble of overlapping models. So within the hour, every piece of every town became a model at a third of life size, laid on its own patch of shore with turned footprints tested so nothing ever overlaps, standing on its lowest corner, and rising on a gentle lift behind the waterfront so the rows at the back show over the rows at the front. Then we walled off fifty metres around every town piece on the sea chart, leaving each harbour mouth open, so no ship can ever get near enough for the trick to come apart. It is a stage set, and we keep the audience at the back of the theatre.
After that it was a day of looking and fixing. The miniature towns read as rubble, so the houses went into streets that follow the shore's own contours, a compact block climbing the rise, tavern on the front row, church a few rows up near the middle, a tower at the top. Those streets then stood on a hillside too steep to build on, so each street's band is levelled to its own height and stepped about two metres up from the one in front, with a bank between. Kingsport's houses now run unbroken from five to fifty metres back, where before it had a waterfront row, a crest row, and bare grass in between.
The sea itself was the interesting one. Below high reflection quality, the planar reflection was never being drawn, but the shader was still told to use it, so for most players the sea was a dark slab under a bright horizon. It now takes the sky off the wave normals instead, and the reflection cube is rendered from the ship's own camera. Before that, a stale origin was putting another coast's hills into the water as olive seams. It reads blurrier mips with distance, because a sharp land edge broke into confetti on every wave, and any ray that dips below the horizon turns back up into the sky. At night its gain doubles. Also, fifty-five thousand trees now grow in closed stands, and I should admit that for a few passes every broadleaf in Novia was wearing fir needles, because the tree materials had collided.
Two more from the Great Sea, and then I will stop. There was an inland bay with twelve ports in it, joined to the ocean by a creek between eight and sixteen metres wide. The generator now cuts a hundred and fifty metre strait out of any such bay, along the cheapest line, never within fifty metres of a town. And the piers had no boats for the simplest reason imaginable. The clearance a boat needed was smaller than the two margins we were measuring against, so not one boat had ever fitted alongside. A hundred and seven sloops and a hundred and forty six rowboats are tied up now. Cost came down too, by dropping every part of every model no ship will ever be close enough to see. Off Central Brittany that is about two thousand five hundred draw calls down to about eighteen hundred.
Which brings me to the boats you actually steer. First, a connection fix that matters more than it sounds. Riding a boat while the server's position updates arrived late, the hull was stepping. It would stop, then jump ahead, and now and then jump backwards. The camera follows you rigidly, you are parented to the hull, and the sea's own grid is laid out from the camera, so every one of those steps moved the entire view against the shore. The shoreline jerked. So did the nameplates of everyone standing on deck with you. The boat clock was following a late arrival by two percent per update, so playback drifted behind the newest position and sat at the limit of what it could extrapolate. It now takes the quickest arrival of the last second, measures how late arrivals are running, plays back a little behind by that much, and leans toward where it should be gently instead of snapping. In a simulation of four bad connections, two minutes each, the old playback froze the hull up to ninety four times, jumped it up to two hundred and forty eight times and moved it backwards up to a hundred and twenty nine. The new one did none of those three, in any of the four.
And then there is the houseboat that could not leave its own cove. The channel out of the test harbour had ten buoys, standing in pairs twenty six metres apart. The houseboat is seventeen metres wide. And the inner buoy on the north side sat ten centimetres off the exact line the boat takes when it leaves its berth. In the simulator it struck that buoy twenty four seconds after casting off and stuck there. Twenty four seconds. The pairs now stand forty metres apart from the harbour mouth to just ahead of the berth, and there is a new test that steers the houseboat from its berth to open sea, which it now does in a hundred and five seconds without touching a thing.
The ship homes gained two things in the same pass. Rope ladders over each side, cloned whole so they climb aboard the way a ladder works ashore, with ten to twenty ends on a galleon. And solid footing over the cabin roofs and deck panels that had no collision at all. That one came from a player who walked onto the stern roof of a galleon and fell straight through it into the cabin. Between two and six hundred square metres a ship, depending on the hull. And last, the sail. At the wheel, the camera sits behind the helmsman, and the sail filled the middle of the screen, so you were steering toward a coast you could not see. While you hold the helm, your own view now eases that sail down to about half drawn over half a second, and eases it back when you let go. Everyone else always sees it whole. Boats are still switched off on the chaos shard, before anyone goes looking.
Now, the front door of the game. The title screen has the hooded Avatar standing in it, and until Tuesday you had to click Play Online to open a login plate, which arrived in the middle of the screen and covered him up. Two clicks to do one thing, and the second click landed on the artwork. The login form now stands on the title screen itself, above Play Online and to the left of the figure, placed by its bottom edge so it grows upward into the sky as it fills. Play Online is the form's submit, or press Enter if your hands are already there. It builds no Log In button and no Cancel button, because there is nothing left for them to do. If your window is too narrow for the form to sit beside the figure, it centres above the Play buttons instead.
The login line also stopped carrying the server's news of the day and the old welcome seed. It reports failures and nothing else, which is what a login line is for. The chat message of the day is unchanged, so you will still be greeted once you are in. And the links underneath are tidier, with "I have an invite code" sitting beside "New to SotA: Create Account" rather than stranded on its own row.
Some smaller things, grouped by where you will meet them. On the volcanic island chain, the sky now follows you. Its sky dome stood at the centre of a four kilometre map and reached only fifteen hundred metres out, so from the outer islands you were standing outside your own sky, looking at its edge sitting on the horizon at night. The dome is now kept centred under the camera. Also, every house in the pirate village was carrying a bare copy of its own front door, parked at the exact position of the real one, with no hinge and no collider. So the working door swung open behind a frozen duplicate of itself and the whole village looked abandoned. Fourteen of those copies are gone, and the tavern, warehouse, lighthouse, drydock, cabana and houses all swing again. Five lane lanterns that stood in the middle of plank walkways moved to the edge with their arms reaching over the planks, the creek now springs from between a cluster of dark boulders instead of appearing out of bare dirt, and a treasure chest spilling gold beside the harbourmaster's house has been taken away.
Three about your house. The Grand Gothic Manor took its first round of QA reports and an owner walk-through on the same day, and it ate them. Looking up in the upper rooms showed you the inside of the roof, the beams and the backs of the dormers, so eight hundred and thirty square metres of plain plaster ceiling went in underneath. Strips of floor that ran out past the outer walls are pulled back. The rear door stood eight tenths of a metre back in a bare hole, and now sits in the outer wall with a stone sill down to its steps. Doors that opened into walls and columns open the other way. Two upstairs rooms sitting eleven centimetres low are level with the floor. Windows that had been walled up on the inside now light their rooms, the bathroom's narrow single door is a double, every flight of stairs has a ramp along the nose of the treads so you walk up it properly, and the house now arrives empty of light fixtures so you can light it yourself. Separately, taking ownership of another player's decorations checks every room of a lot, house, basement and dungeon, before it moves anything, so a room that cannot accept the items stops the whole action instead of leaving it half done.
Two about towns and maps. Player-owned towns build their decorations in one pass behind the loading screen again, the way the old client did, instead of stopping ten times a second to update the percentage. Decoration loading in player towns had quietly drifted two and a half seconds slower than the old client for the same players, and that is why. And on the website's world map, player towns now stand where the overworld actually puts them rather than on the middle of the hex you enter them from. Two hundred and forty one towns moved, most of them by a few metres. Journeys End Marketplace is back on the Longfall shore with Snug Harbor beside it to the west, which is the third attempt at those two, since they were drawn on one spot and then last week's fix put them the wrong way round. Town pages also draw all their lots again for towns whose governor owns houses in their own town.
Lightning round. Nothing says ancient menace quite like a phoenix that opens the fight by flying a hundred and fifteen metres away from you. The run a flyer takes before lifting off had grown to between a hundred and fifteen and a hundred and forty four metres, flown at one and a half times combat speed, and it could be the very first move the creature chose. It is about thirty metres now. A creature actually fleeing at low health still runs its full distance, because that one is supposed to hurt your feelings.
A handful of small insults removed. The keys F16 through F24, which macro pads and larger keyboards send, show their names in Keybindings and on hotbar slots instead of reading as a question mark with the key name inside, and they no longer log a client error every time a label is drawn. A party list too tall for a short screen stays on the left under your own frame and is drawn smaller until it fits, instead of walking off to the right. Gold ore rocks and barrels no longer leave you standing in the air beside them. An add-on asking the game which recipes make a given item no longer freezes everything for four and a half seconds on the first call, because it reads the baked item catalog rather than loading every item that every known recipe can produce. Guild leaders and officers, and player-owned town governors and stewards, can now put up to a dozen links onto every member's Desk, which a player asked us for through the Oracle. And the Call to Arms prize list was rebuilt so newer items like the Peacock companion pet can be given as rewards, with every release rebuilding it from now on.
New character names can no longer carry a pledge title. No Lord, no Sir, no Baron, no Royal Founder, and no SirRobin either, since running it together was the obvious next move. Real names that merely begin with one of those words, Sirius and Earlene among them, still pass. I will note that the list of names we were previously happy to allow included Lord British. There is one of those, and he is talking to you. Elsewhere, the Oracle learned answers to about fifty things players asked her and she could not answer, from where Ulfheim and the War Bears are to how to play on a second monitor and why player dungeon kills count toward no Call to Arms. QA players can always enter test scenes now, having been refused for lack of access. Seventeen books whose words changed got read aloud again overnight. And the release stopped sending every connected player a chat line promising a patch in about four hours, which was both true and unwelcome. Thirty one tracker issues closed, across a hundred and one commits.
That is Tuesday, and it was a loud one. What I am looking forward to is the first evening someone sails out of that cove for real, with the sail half drawn in front of them, the sky sitting in the water, and a third-scale Kingsport on the shore doing a perfectly convincing impression of a city from fifty metres away.
And spare a thought for that channel buoy. It held its post, it did its duty, it stopped a houseboat dead in twenty four seconds, and for its trouble we shoved it seven metres out into the dark. A clear case of mistaken buoy-dentity. Good night from New Britannia.