Episode
Your armour took itself off and we finally caught it doing so
Monday was the end of a long-running bug that removed gear you were wearing, a deliberately failed release that proved our rollback had never worked, and a large pile of music leaving the game download. New character names now get checked before they are allowed, the guild website learned to talk back to the game, and ninety-seven tracker items closed.
In this episode
- Worn gear that took itself off after a zone change
- A failed release staged on purpose, and what it found
- Seven and a half hours of music left the download
- New character names get checked at creation
- The guild website starts writing back to the game
Transcript
Good evening to thee, citizens of New Britannia. Lord British again, with an accounting of how thy makers spent their Monday. Thine armour hath been removing itself without thy leave, and that is ended. We staged a broken release on purpose, and learned that our means of undoing one had never once worked. The music of thy wax cylinders no longer sits upon thy hard drive, waiting to be asked. And a new character with a foul name shall now be turned away at the door. But first, a noise. One loud, rude noise in the middle of a battle, and the long journey it made to reach thine ear.
So. You are standing in a siege. There is a great deal going on. Somebody a long way off hits something, far enough away that you barely register it. And then a moment later that same sound arrives beside your head at full volume with its pitch bent out of shape, like a bee that has taken a personal interest in you.
Here is what was happening. The game keeps sixteen speakers for the sound of things striking other things, and it reuses them constantly. All sixteen of them were parked at one single point in the world. Every new blow moved that point to wherever the new blow landed, and dragged every sound still playing along with it. A hit from the far side of the field would be picked up halfway through its fade and set down next to your ear to finish. Each impact sound now stays where it happened.
Now the one I owe you a proper apology for, because it has been going on for a very long time and it has cost people real gear. You change zones, or you log back in, and something you were wearing is in your pack instead. Sometimes your boots. Sometimes more than your boots. And it sticks, so logging out and back in again does not restore it.
Here is the shape of it. When the game re-reads your pack from the server, it does not patch the list. It empties the container completely and then fills it back in over the next several seconds. For that gap, everything you are wearing looks, from the outside, exactly like something you have just taken off. And the part of the game that watches for you taking things off was watching. The slot got cleared, and the message telling the server you had unequipped it went out in about a second and a half. The server believed it, because why would it not. We told it.
We did have a safety net for this. It was measured in wall-clock time, sixty seconds from the moment the slot cleared, and anything that came back later than that was not put right. A slow scene load will spend sixty seconds on its own without breaking a sweat. So the net was reliably absent at precisely the moment you needed it. The repair now waits for the re-read that started all this to finish, rather than starting a stopwatch and hoping.
Three smaller holes were sitting in the same path, and they went with it. A zone change threw away the record that would have put the slot back. The restore at login only looked at the top level of your pack, so anything that had landed inside a bag was skipped without a word to anybody. And an expired repair entry was dropped without anyone checking whether the item had in fact come back. On top of that, the server now writes a line every single time it empties one of your equipment slots, which is exactly what eight previous investigations into this did not have. Nobody could tell a real unequip from a phantom one, because they looked identical from the outside.
The whole new behaviour sits behind a shard-wide switch. If it misbehaves we turn it off from the tunables page without building a new client, and it gives up after five minutes rather than holding one of your slots open forever.
Last night I told you we had rebuilt the machinery that puts new builds in front of you, after it locked everyone out for two hours. Yesterday we did the sensible and slightly terrifying thing, which is that we broke one on purpose to watch what happened.
Before that, three separate reviews of the rebuilt pipeline turned up fourteen more defects, and several of them were in the undo paths, which is where defects go to hide. A channel flip that fails its public check is now put back rather than left standing half open. The server is never rolled back underneath channels that have already moved. A passing network hiccup while reading the login gate no longer counts as a mismatch and panics everybody. The smoke test signs in as the actual build number instead of declining to mention it. Linux and Mac get their build records again. And the reservation counter burns a build number on every kind of failure rather than on a few selected ones.
Then we ran a failed promotion on the test shard deliberately, and found that the automatic server rollback had never worked. Not once. It stopped dead on its very first command, and the release script then reported the server as rolled back, with great confidence. The shard was unreachable for about five minutes until somebody put it back by hand.
There is a particular flavour of embarrassment in discovering that your emergency brake is a painting of an emergency brake, and there is only one way to discover it, which is to pull it. Both faults are fixed, and we are running the rehearsal again before anything is promoted for real.
The Promote button on the builds page moves in the release order now. The server goes first. It then has to accept a real test login as the new build, and only after that do the Windows, Linux and Mac downloads switch over. If anything fails before that switch, the server goes back to the build it was on and you stay on the build you already have. Before this, a failed server step after a promotion could leave every player in the world patched up to a build the server would refuse to speak to. That is the exact trap we fell into on Sunday.
My favourite failure of the day was smaller and much funnier. Our second attempt on the new pipeline did not fall over. It simply stopped, politely, forever. Each client build was asking the build controller for a second slot so it could go and check its own build number. The controller has one slot. So every build sat there waiting for permission from the very thing that was waiting for it to finish. We built a machine that asked itself a question and then queued up behind itself for the answer. The check happens once now, at deploy time, on the machine that holds the keys, and the client builds never wait on it at all.
Something good happened to the size of the game yesterday, from two directions at once. The first is that we retired the enormous single archive that new installations used to download. First installs now pull the game down file by file from the edge network, the same way every update has worked for ages. That archive was around thirteen gigabytes per platform, and every release spent up to ninety minutes per platform zipping it up and pushing it out. That is a couple of hours of every release day that no longer exists.
The second is the music, and this is my favourite thing on the whole list. Every wax cylinder now fetches its song the first time you play it, the way the Songs of the Sundered Blade cylinders already did. A hundred and twenty-four recordings that nothing but cylinders ever used, seven and a half hours of audio and a bit over a gigabyte of it, no longer ship inside the game at all. They arrive when somebody actually wants to hear them.
While we were in there, all two hundred and forty-four cylinder songs were re-encoded properly. Old recordings that had been saved in stereo with two absolutely identical channels are now spotted and stored once. Five tracks qualified, which means five of our songs were being delivered to you in glorious stereo on the strict condition that you listen to the same thing twice. The streamed set comes to about eight hundred megabytes, and the music cache default moved from half a gigabyte to a gigabyte and a half so the whole library fits comfortably. Jukeboxes also start fetching the next song on the playlist while the current one is playing, so a playlist of music you have never heard before now plays in order instead of jumping ahead to whatever happens to be ready.
New characters get their names checked now, and I want to be careful about how I describe this, because the failure mode of a name filter is usually worse than the problem it solves. A name is checked as it is typed at character creation. Slurs, strong profanity, sexual terms and names that pretend to be game staff are refused, and you are told the name is not allowed. Everything else is created exactly as it always was. If the check is slow, or not answering at all, character creation carries on as though it were never there. Nobody gets stuck at the gate because a service is having a bad morning.
The interesting part is the middle. Innuendo, drug references and the names of real people are not refused. They go into a queue, where a person looks at them. There is a new Name Review page for the team that lists those characters along with the reason the check gave, and from there somebody can keep the name, allow a name the check got wrong, or rename the character on the spot. There is a button that suggests a free fairy name, which produces a fresh one every time you press it, and I have already wasted more of my afternoon on that button than I would like recorded.
We tested it on fifty-eight names where we already knew the right answer, and every one came back in under half a second. It refused none of the legitimate ones. I want to single that out, because the legitimate list includes the surnames Dickens, Cockburn and Hancock, and I have watched simpler filters than ours go into open revolt over all three. If your family name has been getting you side-eye from software for thirty years, the gate holds.
There was a companion embarrassment in the same area, now fixed. When the team renames somebody's character, that player gets a notice explaining why. Renames done from our internal tools were filing that notice as test data, and test data gets thrown away after half an hour. So if the player was offline for the afternoon, and most people are, they logged in later to a brand new name, no explanation, and nobody obvious to be cross with. The notice waits for them now, however long they are away.
On the test shard, the guild website stopped being a thing you can only read. Guild members can reply to guild chat straight from the browser, under their own character, and it arrives in the game as though they had typed it there. Officers and leaders can change ranks, remove members, and edit the guild description and the message of the day. Governors and stewards can set town roles. And officers can invite players who are not logged in, who receive the invitation the normal way the next time they play. That cost us one short restart yesterday morning, and I think it was worth it.
Before any of that was opened up, somebody sent one harmless test request through both paths on the live test server, and it found a bug that would have hit the first person to use it. Setting a member back down to plain Member would have failed every single time, because the game does not call that rank by the same word the website does. The two were talking past each other about the most ordinary demotion in guild life. Fixed before anybody touched it. The studio tools on that site, the report queue and the takedown controls, now open only for accounts the game itself flags as developers.
The other half of that work is renaming towns. A player-owned town can be renamed from our internal tools now. The new name lives on the town's own record, it is checked against the guild naming rules and the blocked-name list, it is refused if another town already has it, and the change is recorded with the old name in the audit trail. Your deeds, your lots, your roles and your town chat all stay attached to the town, and there is a button to put the shipped name back. The new name reaches the guild website, the town directory and the chat mirror immediately. In the game it shows up in the Town Directory, on the compass, on the zone name, in the town chat tab, in the town console and in map tooltips, and a player standing in the town sees the change within two minutes without logging out and with no patch at all. That last part arrives with the next build on the test shard.
Three things about getting around. Turning fly mode off in mid air no longer kills you on landing. The protection on that landing only applied if the camera had climbed above your character, so flying level off the top of a hill and switching off left the entire fall unprotected, which is a remarkably rude way to learn about camera angles. Sprinting on a mount with Shift held no longer makes the poor animal stutter, because once your Focus ran out the sprint was switching itself off and on again about once a second and throwing the mount back to walking pace each time. And getting off a mount no longer keeps the mount's speed on your own two feet, which had riders thrown off a roof continuing to travel at a gallop until they stood still.
Following your party leader now takes you to the zone the prompt actually named. A leader who kept travelling while that prompt sat on your screen could carry you somewhere else entirely, including into a player dungeon or a basement, where you would arrive in the dark with your skills and your items refused. If the leader has moved on, you get a polite refusal in the chat log and you stay where you are.
Here is a skill that was fighting itself. Charge, in the Shield tree, was rolling its stun and its knockdown as two separate coins at the same odds and the same length. A good hit therefore stacked two resist-crowd-control buffs on the target, and five of those make a target completely immune to being controlled. So Charge was spending two of the five on its own victim and then watching its own follow-ups fail. It lands one knockdown and one buff now, and your chances of controlling the target are held exactly where the two coins used to put them, three in four at skill fifty and better than nine in ten at a hundred.
Two things for crafters, both about lot storage. Crafting can now take materials out of a lot container you already have open. Before, the recipe card would happily count what was in the chest, and then the craft itself would refuse those very same materials while the chest window was on screen. Closing the chest was the only way through, which is a fine thing to know and an appalling thing to work out on your own. And a container marked for crafting could untick itself, because the container window was sending the setting back off while it redrew. The lot would then report no crafting supply at all, and every material in that chest read as missing at the table. Any container that switched itself off this way needs ticking on once more.
The voices got better in the towns. The Helpful Automaton now speaks its greeting wherever it stands. Its recording had been made with a placeholder where the town name goes, so its opening line came out as pure silence and it only found its voice once you picked a topic. About twenty other characters, town criers and guards and guild registrars among them, had lines that name their area recorded against the internal shorthand for that area rather than the name you see, and those played as nothing at all. They are re-recorded. And when you ask an NPC about a topic they know nothing about, they now say so out loud rather than only in text.
Six player-run towns had no way to travel anywhere else at all. Axwich, New Holtrot, Snell's Bend, Parrot's Roost, Platinum Keys and Smaragar each have a Boat to Nearby Towns at the docks now, offering the four closest cities. And on player vendors, dragging a stack that sells a few at a time into the exchange window now asks how many you want, the same way a double-click always did. Dragging used to take the whole stack, which made for some brisk and unintentional shopping. Those listings show their price in light blue and say in the tooltip that they sell a few at a time.
Lightning round. Ninety-seven tracker items closed yesterday, fifty-two of them reported bugs and forty-five tester verifications going green. A good chunk of it was interface work following our move to the new system. The cooldown sweep on your hotbar was rotating counterclockwise and now goes the right way round again. The numbers on your health, focus and vigor bars sit in the middle of the bar instead of hanging off the bottom edge. Player names on nameplates read close to white again rather than a sort of apologetic grey. And resetting your window positions no longer drops the quest tracker directly on top of your status frames.
Also in that pile, skill glyphs dragged out of the skill tree now hold on to your cursor until you let go of them, a vendor search typed in Chinese finds the listings in your zone, and a player sitting in a chair can hold an entire conversation with a banker instead of getting a greeting followed by a wall of silence and no bank window. Thirty-six rows in the Options window that had been showing English to French players now show French. Six more of those rows were labelled with the French text of a setting we removed years ago, so the catchphrase volume slider had been introducing itself as a low health indicator.
And a scattering of the rest. Siege armies no longer stand about with their arms out when you watch them from a distance. Creatures in a Cabalist's ring now settle on ground a person could actually walk to, rather than inside somebody's house or sealed behind a fence. Deep in a control point wave, your own buffs keep showing to your party however many creature effects have piled onto you. The Titan of Truth is spelled Boreas in every language now instead of only in English. The large butcher's sausage finally has a name that is not the small sausage's name, and the market butchers have started stocking the decorative meats that were not for sale anywhere in the world. And a quest at Solace Bridge Outskirts that could not be finished by anyone who had already taken the free lot deed can now be finished.
That was Monday. My thanks to everyone who has been patiently re-equipping their boots after every zone change for the past year and filing careful reports about it. That one took eight attempts, and it was the logging we added on the ninth that made it obvious. I am looking forward to the first town that renames itself, and to hearing what the neighbours make of it.
And as for the butcher's large sausage, which spent all this time answering to the small sausage's name, I am told it has taken the correction badly and refuses to discuss the matter. Which makes this the wurst possible note to end on. Good night, and safe travels through New Britannia.