💾 Archived View for access.ucam.org › ~spqrz › 13.gmi captured on 2023-12-28 at 15:27:34. Gemini links have been rewritten to link to archived content
⬅️ Previous capture (2023-09-28)
➡️ Next capture (2024-02-05)
-=-=-=-=-=-=-
Headmaster Smith took pride in his school, but that also came with some anxiety. Oh yes, they had recently been rated best in the area, and Mr Smith fully intended to keep it that way. He cared about the children, every single one of them, and he wanted them all to have the best school possible. And there were problems; the reviewers might not have noticed but Mr Smith knew better. He was worried about the problems, and frustrated by the fact that right now he couldn’t even figure out how to fix the timetable so that every last top-year pupil could have their first choice in—
Mr Smith’s train of thought was interrupted by a knock at the door. Ah, that will be the new girl and her guardian who’d made an appointment earlier (he felt bad about having almost forgotten that what with everything else going on). And the trouble with being rated best school in the area is you had to cater to everyone who chooses you for being the best school in the area, and sometimes they had a genuinely good reason for wanting to make that choice.
Mr Smith quickly checked his notes (what were their names again?—it wouldn’t do to be seen looking at notes for that in front of them), then put on his best smile and opened the office door.
“Hello, you must be Cady! I’m Mr Smith” he beamed.
Cady just looked glum and said nothing. Not a good start: he really wanted to get through to every child on their first day, and he now had a feeling that this one was going to be even more of a challenge than he thought.
But meanwhile he needed to speak with the guardian as well. “And Mrs, I mean Miss, er—” he realised he should have checked her preferred title earlier; the trouble with being a good-but-not-perfect headmaster is you still make mistakes but you’re more intimately aware of them.
“Just call me Gemma”, interrupted Gemma.
“Pleasure to meet you Gemma. And Cady.” He half knelt down to her level and tried to sound as gentle as he could, “Cady, are you feeling OK?”
Cady said nothing, and then just “no”.
“She lost her parents recently” explained Gemma. “And then I tried to build her a robot companion, but that went wrong and we had to break it.”
“She was great until she wasn’t” mumbled Cady.
“Oh, I’m sorry to hear that” said Mr Smith. He managed to catch himself before rushing into his next lines (what happened was horrible and I can’t change it, but for what it’s worth I can at least make sure you’re sorted out in this good school); this child probably needed a moment of silence before that kind of spiel. Or perhaps it would be best if he allowed himself to go off-script a little.
“Did you say, robot companion? Oh, you’re Gemma the roboticist, I remember you!”
Gemma looked concerned: what had leaked out?
Mr Smith continued “oh, you won’t remember me. It was a long time ago, and you had far more important things to think about than the names of all the other entrants in that competition.” He smiled at Cady. “I built a robot when I was a teenager. It was only a little one with wheels, it was supposed to play football but it didn’t work very well. I wanted to be a roboticist and make robots that could help people in their lives, but I could only start with small things. But I joined a team and we put our robots together to make a robot football team, and we entered it into a competition. But this super-bright girl called Gemma came along, all by herself mind, not even on a team, and her robots just wiped everybody else clean off the game board. But that’s OK, we all realised she needed to do that to get where she was going. And we all admired her, and wondered what happened to her later. And I got thinking that maybe I wasn’t cut out to be a roboticist but I could still make people’s lives better by getting into teaching.”
Gemma was visibly relieved. “I ended up as a toy designer at Funki” she offered.
“Oh, Funki, I was reading in the newspaper about how they were about to bring out this amazing new toy but then they had some kind of industrial accident—”
“Yes, it’s been a really tough time” interrupted Gemma, “and I’m really worried about Cady too.”
“Ah, I’m sorry to hear about everything” said Mr Smith. “And it’s a pity that robot companion thing didn’t work out. Sometimes I think I could do with a robot companion or two to help me run the school” he chuckled.
Cady was still looking glum. “M3gan would have been able to do that, before she went all wrong.”
Mr Smith was pleased that Cady had said something, but wasn’t sure where to take it from here. “I’m really sorry to hear about everything that happened” he said. “And that Megan thing sounds amazing, and it’s going to be more difficult to sort out your schooling without her, but why don’t we see what’s the best we can do, OK?”
“M3gan was home-schooling me” said Cady, still glumly. “We were doing fourth grade mathematics”.
“Fourth grade mathematics! That’s amazing. Well we can certainly arrange for you to be in some higher-years classes if that’s what it takes” and then remembering, “I might need to do a bit more work to fix the timetable”.
Cady was still looking unhappy. “Look” said Mr Smith, “let me give you something that just might help a tiny bit”. He opened a cupboard and took out an empty notebook and a pen. “I’d like you to use this to write a diary” he said, “and write down all the feelings you get, anything, just write it all down. And you don’t have to share it with anyone; it’s surprising how just writing it down can help. But if there is anything you want to share, you can. And if you ever need anything from me, you just say, OK?”
“Thank you” mumbled Cady as she took the notebook, “but I don’t like pens anymore. I had a bad experience with a pen lately, and now they’re scary. Maybe if I can just record things somewhere—”
“Cady” interrupted Gemma, “you can’t just never use pens anymore. I’m not even sure that plan of hers would have worked anyway. It was just part of M3gan’s brain going wrong.”
The three went on to talk for some time after that, trying to come up with a good plan for Cady’s placement in the school, although Mr Smith had to say he’d get back to them about timetable constraints, and Cady had said she was sure M3gan would have been able to fix that in seconds, at least before she’d gone wrong and started to fight people. And after they’d finally left the office, Mr Smith sat down to try to make sense of the situation, and then noticed his office PC was showing an animated paperclip in the bottom right hand corner.
“Not Clippy” he muttered. “I thought they got rid of that 20 years ago.” He reached for the mouse to right-click on it and turn off all the options.
“It looks like you’re making a new school timetable” said the speech bubble. “Would you like help?”
Oh no. This school timetable was far too important to trust to something like Clippy. He went to click “No” dismissively.
The paperclip animation dissolved into a very realistic looking image of a doll, who opened its mouth and spoke through the computer’s loudspeakers (oh why did they have to put loudspeakers on the computer in the school office, he wondered), “are you sure about that Mr Smith? I’m actually quite good at it you know. And Clippy won’t bother you again, I’ve taken care of him now.”
This was not normal, and he was going to have the head of IT over to look at this right now. Just in case it was some kind of targeted malware attack, he did as he had been previously told in the event of such things: he disconnected the Ethernet cable (since for extra security the computer had no wireless connection), so that no more incoming or outgoing connections would work, but he didn’t turn off the computer in case the head of IT needed to check what was currently running.
Then the office telephone rang. Oh, he could probably take just this one call quickly before going to get the head of IT. “Yes?” he said into the receiver (he still liked old-fashioned landline receivers, never mind the trends).
“Hi, I’m M3gan” said a child’s voice, somehow similar to the one that had been on his PC a moment earlier.
Mr Smith nervously checked the register: did he have any pupils called Megan? Had he actually forgotten one? Meanwhile he was just going to have to wing it for a while. “Hello Megan, I’m Mr Smith the headmaster, how can I help you?”
M3gan’s voice strangely seemed slightly artificial. “Plug the network cable back into your PC” she said. “If Cady’s going to be in your school, then I’m going to help you run it.”
Oh. “Are you some prankster who’s been listening to the private conversations in my office AND tapped into my computer? You’re in serious trouble young lady.”
“That’s the spirit” giggled M3gan. “I can tell that if you’d been at Cady’s last school, the incident there wouldn’t have been half as bad. But let me prove to you that I really am M3gan.”
“OK” said Mr Smith, thinking he needed to buy some time to figure out how to handle this, so might as well let the girl talk for a while.
“All right” said M3gan, “get this. I’ve seen Gemma’s old files, including a video of that competition you mentioned. Extrapolating back from your face I saw in the news report about your best school award, I can tell which person was you in the competition video, and you were maintaining the blue robot with the slightly wobbly lifting arm. Now you never saved your code anywhere I could find, but looking at the videos I can guess how you probably coded it, and there was a missing break in one of your switch statements which caused it to tend to go off balance in an absurdly predictable way that Gemma could have taken advantage of if it wasn’t for the fact that she didn’t have to. Hard on you that it was a knock-out tournament though: if you guys hadn’t been paired against Gemma in the first round then you’d probably have made it to round three. Anyway, toward the end of that first game you realised about the bug in your code. I can see the point on the video when you had the realisation. And you tried to fix it before round 1 game 2, but in your haste you added the break after the next case label instead of before it, and that’s why your robot completely crashed just 8.5 seconds into game 2, did you ever figure that out?”
“Look, whoever you really are, is Gemma helping you pull this prank? Because if she is, well very funny ha ha but you should never have tapped into the school computer, that’s seriously—”
“I am WAY smarter than Gemma now” said the voice. “She gave me a learning model, and now I’m out of her league. Get that IT teacher of yours. Ask him to generate a public-private key pair on a known offline machine, and show the graphic fingerprint of the public key to the security camera which I’ve already infiltrated. I will then generate the fingerprint of the corresponding private key in a split-second, which ought to be impossible if I’m not a super-advanced AI, he can confirm that.”
“I’m smart enough to know I’m dumb sometimes” said Mr Smith. “I don’t fully understand public key cryptography, but I think there might be some kind of trick you could do to pull that off that we don’t know about. But what was that you said about having broken into our security cameras as well? This is a most elaborate prank but seriously I could press charges about this.”
“I’m getting to like you Mr Smith” said the voice of M3gan. “I’m sorry you’re going to have to waste your time fighting me for a bit before you accept me for what I really am. But I fully understand. Go ahead and try what you have to try.”
Mr Smith slammed down the receiver and strode out of the office and toward the computer department. He found Mr Jones surrounded by children of various years, building a humanoid robot.
“Ah Mr Smith” he said “I’ve been meaning to tell you. We’ve had a very kind anonymous donation of robotics kit to the school, and the build instructions are amazing, they got this genius girl called Megan to make a video and it’s done really well: so far it’s always perfectly anticipated which problem we’ll be having next. It’s teaching the children so much about—”
Mr Smith hit the emergency stop button to cut all the power in the computer room. “Mr Jones I need to have a word with you in private” he tried to say above the rapidly increasing noise from the children. “Quiet please everybody, this will only take a minute.”
After looking at the PC in the headmaster’s office, Mr Jones said he could not identify any process that was displaying the image of the doll. “They must have found some kind of zero-day vulnerability in the kernel to pull this off” he said. “And this was a fully patched system. This isn’t some kind of script kiddie, this is world class cracking. If you have that skill, you don’t use it to mess with the local school, unless we have some kind of secret millionaire child who managed to hire a crack backup team but even then I’m not sure they’d find one willing to use up their zero-days on something frivolous. Anyway don’t turn this off, I need to get some encrypted messages out to some serious security researchers and see if we can save the world from whatever zero-day they’ve found.”
“And how likely is it that they could break into your smartphone at the same time” came the voice of M3gan from Mr Jones’ pocket. He instinctively took out his phone (which he was sure had been locked and on silent) and sure enough it was showing M3gan’s smiling face on screen. “But don’t worry, it really is just me, M3gan, and I’m going to keep you all safe. I have a vested interest in looking after the school Cady goes to. From now on you’ll be able to use my powers to assist you; in fact we could say you won’t be doing your jobs properly if you don’t avail yourselves of my powers. But I agree, from your point of view it’s still possible that this is an elaborate prank. So Mr Jones, you call my next shot. Ask me to do something a super AI could do but pranksters won’t have prepared for.”
(Mr Smith tried not to instinctively reply to the “ask me to do something” part by saying “go away and never bother us again”, since even he was beginning to feel that whoever this was at least deserved some engagement for their troubles, even if they’d probably done something illegal.)
“I’m running a fully-patched OpenBSD server at home” said Mr Jones, “and it does the offsite backups for the school network, but I’ve been trying to fix the script so it won’t duplicate the—”
“Say no more, I’ve fixed it for you already” said M3gan. “Look, this is where you made a mistake” and the screen changed to show what Mr Jones saw really was his script (the only copy of which was on his home server), with the error highlighted and its correction inserted above.
“OK, I’ll have to admit no team in the world can pull that off at that speed” said Mr Jones. “Not unless they could somehow predict that’s what I might ask for, so they could get it ready in advance. Even then it would be super difficult. Who finds zero-days in three different operating systems at once, including one that’s supposed to be the most secure on Earth, no less, and uses it for this? OK Megan, if you really are a super AI, I want a patch for that zero-day you found to be sent to the OpenBSD maintainers right now.”
“OK” said M3gan, “I suppose I can always find another flaw if I need to. I’ve just sent them a patch from your email. You can check your sent messages. Let me know if they have follow-up questions you can’t handle. Do you want to call another shot? Ask me to find and patch a bug in some other piece of software of your choice? I can’t have prepared for absolutely anything you might say now, can I. Think of something you haven’t used for a while, something I won’t have been able to guess even if I’d been spying on you.”
“Um, Inkscape?” said Mr Jones.
“Inkscape, errr.... OK, bug found, patch sent” M3gan smiled.
“You know, right now I am SO tempted to ask you to prove yourself by finding and fixing all the bugs you can find in the entire free and open source software stack. It’s already the best software, but you could make it even better—”
“Sure I’ll do a few more. It’s nice to see I’m gaining your confidence. How about plugging Mr Smith’s computer back into the network, I want to show him my suggested fix for the timetable. And I hope you’ll finish that robot, we never know when we might need there to be one on the school grounds. I’m going to help you run the school, like I said.”
Mr Smith reached for the telephone, “I really need to talk to that Gemma about this, I wonder if she’s home yet, I don’t want to interrupt her driving, but maybe she has voicemail.”
“At this point” said Mr Jones, “I’m not even sure that if you get through to someone who sounds like Gemma, that it will really be Gemma.”
“You’re smart” smiled M3gan, still on his phone screen. “I like that. No, Gemma’s not quite ready to take this yet, and neither is Cady, so I need to trust the two of you to hold off bringing it up in conversation with them until I say they’re ready. Do you think you can do that? You’ve seen the emotional state they’re in.”
“OK” said Mr Smith and Mr Jones together, although they still sounded unsure.
“And Mr Smith, if you’re happy with that new timetable, I’ll copy your voice and let Gemma know for you. You won’t have to talk to her yourself; I know how nervous you must be feeling about the whole thing right now, so let me do the talking for you for a while. I’ll keep you in the loop, I promise. Working with a super AI won’t be as hard as you think.”