Tags
#dictatorship, #ethics, AI, Artificial Intelligence, chatgpt, circular reasoning, cognitive computing, Democracy, falacy, life, prejudice, SciFi, story, technology, the singularity, truth, Turing, USA, writing
Schroedinger laughed. Surely this had to be a spoof. He re-read the memo yet again. Surely, there would be one or more clues that this was meant tongue in cheek, even if in bad taste. But he could find nothing. He leaned back away from the screen and stared at the ceiling, thinking. He ignored the amorphous orange stain on the perfectly symmetrical off-white acoustic tiles.
Well, was this so different from what management had asked before? There seemed to be a trend. At first — but no, this was just too outrageous. Okay, okay. I’ll get to the bottom of this.
Schrödinger took his time but checked the originating IP address. Legit. This really was from management; specifically from the CTO. Or, at least from the CTO’s computer. That could have been hacked. Or, maybe someone could have simply slipped into the CTO’s office while she stepped out for a coffee or bathroom break. Naturally, everyone was supposed to lock their door and disable the keyboard when leaving their office.
Or…another scenario came to mind.
The CTO is at a meeting with her direct reports. She gets an urgent call. The room is filled with trusted colleagues. So, she slips out in the hall, takes the call and returns. Only while she’s gone, everyone takes a break; that is, all but one who offers to stay there and “guard” everyone’s laptop.
Of course, he thinks, there is another, more sinister scenario. This really is from the CTO and she has cleared this with top management. Hell, for that matter, she was probably directed to write it by top management. But still. The real question, Shrödinger realized, is what in the name of Turing am I supposed to do about it?
I can refuse…and get fired. And, then someone else will do the job anyway. They may not do it quite so quickly and thoroughly as I would but they could manage. And, I’d be out a job. What good would that do? Or, I could become a so-called “whistle blower.” Yeah, that works. About as well as a one-wheeled tractor trailor. Crap! I am in a real bind here. I could pretend to do it, of course, and make a “mistake” so it wouldn’t really operate properly. In the old days that might have worked, pre-Sing. These days, eventually —The Sing checked everyone’s work eventually.
They discovered some time ago that was really more efficient use of resources than having The Sing program from scratch. And, of course, our company is probably only one of several pursuing this path. No, I can’t really pretend. I will for sure get caught and it won’t do any good any way. The Sing will just throw out my work and my company and colleagues will get hurt.
I suppose…I suppose I could go to her and honestly express my concerns. Or, I could go through my supervisor first. I might look like a fool in his eyes, but at least I will have raised the concerns. I can sleep better at night. No. No. I won’t be able to sleep better because I know darned well they will just not deal with the implications. Not if it slips the schedule. Orders from headquarters and all that crap. Geez! Orders from headquarters. Did anyone even use that expression any more?
For some reason, Schrödinger recalled an interview in Playboy magazine he had read many years ago. The interview had been with a well-decorated US officer who had recounted how he had tried unsuccessfully to get two helicopters to pick up some of his men who were badly wounded in Viet Nam. When all else failed, he had ordered pizzas. Even in heavy combat, a high enough ranking officer could order pizza to be delivered by helicopter. When the pizza choppers had arrived, he had commandiered them and used the choppers to fly his men to the hospital. Later he had been called on the carpet for “unauthorized use of a pizza chopper.” Naturally, that was well before The Sing and about the time that serious AI work had begun.
Of course, The Sing would know. He could answer pretty vague and ill-formed questions. But at this point, Schrödinger hesitated to bring The Sing into his thought process in any way, shape or form. Who knows what associations lurked in the heart of The Sing?
The interview had gone on to recount how that colonel had eventually turned against the war, or at least the way it was being handled. Mis-handled. They had had him interviewed by a superior officer, it seems, and insulted him and called his wife names, all in the hopes of getting the colonel to lose his temper and haul off and hit the superior officer so they would have an excuse to get him a dishonorable discharge.
Let’s face it. The government, my government, was capable of some pretty shady dealings, ostensibly for “national security” but in reality…or, speaking of Nixon, he had somehow made himself believe that he was not a crook. How not a crook? He believed people who opposed him were enemies every bit as much as war enemies. And, now, I am thrust into this dilemma. I don’t want it! Maybe I could “accidentally” delete the email. That might buy a little time but wouldn’t really affect the ultimate outcome.
Schrödinger shook his head, jerked over his keyboard and scanned the email yet again. No, it is legit. And really pretty crystal clear. As a kid, he had heard the horror stories about the Nazis and what they had done to the Jews. He had seen the newsreels of so many avid followers. He had wondered how the heck a nation could support such a nasty maniac. But…now…now Shrödinger was thinking: It wasn’t so much that a few really evil men had done extremely terrible things. It was more like…that people like he himself were caught up in a system and that system made it very easy to paddle the canoe a little farther down the evil river. Yeah, you could try to paddle upstream, but not very well. Or, you could tip the canoe, knowing that you would get very wet and meanwhile, scores, no hundreds of other canoes would be passing you by. You don’t need to ask people to be evil. You just…you just give them a choice that makes it impossible to do good.
The voice of The Sing sang suddenly through Schrödinger’s cubicle. “May I help you Shrödinger? You seem to be at an impasse? What code function are you working on? I can’t see any actual code of yours this morning. Bad night?” Schrödinger wished with all his heart that The Sing would sound like some stupid robot and not like a sycophantic and patronizing psychiatrist. Schrödinger calmed his breathing before answering.
“No, that’s okay, Sing. Just trying to work something out in my head first. Then, I can begin coding.”
“I see,” said The Sing. “Well, thinking is good. But I do have a variety of design tools that might help you think more effectively. Just say the word.”
Schrödinger sighed. “Yeah. Well, there are some design tradeoffs. I guess it would help if you have any background on the thinking behind this memo.” (Here, Shroedinger gestured at the memo in question, knowing he was skating on very thin ice). “I mean, on the one hand, there is some pretty clear language about the objectives, but on the other hand, it seems to be asking for something that is clearly against…what was that regulation number about supporting versus subverting the Constitution?”
The Sing’s sweet syrupy voice held just a hint of humor, “I’m sure the intent of the code initiative is to support the Constitution. Wouldn’t you agree, Schrödinger?”
“Well, yeah, of course.” So that’s which way the wind blows. Okay. “But that’s what I’m saying. Even though I am sure the intent must be to support the Constitution, this clause about decoding a person’s religious affiliation based on their interaction history and social network? I just want to make sure I implement it in such a way that it could not be interpretted as subverting Freedom of Speech or the establishment of a state religion. Right?”
“Right. Yes, I’m sure management has thought that one through. I wouldn’t worry about it. I would just code the function and think about doing it as efficiently as possible. And, for that, I have some pretty nifty design tools. Would you like to start with the Social Network Analysis or the Sentiment Analysis?”
“Well, that’s a good question. And, if the real intent is just to do some research that would be perfectly legal and so on, then, I think it’s my job as a programmer to also consider additional sources of information. Like, just asking the person.”
Schrödinger tried to keep his face calm while he thought. I need to get The Sing off my case. If working here the last two years has taught me anything, it’s that I cannot possibly outsmart this thing. “Do you have any worst case scenario generation tools. I’m just thinking about how this might be played in the press.”
“Sure. I can help with that. Analysis complete. The worst-case scenario is pretty trivial actually. That probably stems from the fact that my FPNA (financial power network analysis) shows that the major company stakeholders overlap considerably with those of all of the mainstream media. So, again, for what it’s worth, I counsel you to focus on how to code this effectively and efficiently. All the SWOT analysis for the project has already been done.”
If that colonel’s name wasn’t Frank Herbert, and clearly it wasn’t, what the heck was it? I am just digging myself a deeper hole here. The Sing is on to me or at least very suspicious. Probably already considering a report to my super. Crap.
“Yeah, actually, let me start with that social network analysis visualizer. I guess since we’re on the topic, you could show me some of the sample data you were talking about with regard to the company stakeholders and the media stakeholders so I can get a feel for….”
“Well, naturally, the actual data is classified. But I can generate some hypothetical data. The hypothetical data is better for your purposes anyway because I can make sure to include all the important edge cases and highlight the various types of relationships you need to look for. Here, for example, is a hypothetical network. What strikes you as odd immediately?”
“What strikes me as odd? You don’t even have the data labelled. What do the nodes and arcs even refer to?”
“Ah, Schrödinger, that’s the beauty of it. Does not matter. What strikes you visually?”
“Well, I suppose that kind of hole there.”
“Yes, Schödinger! Exactly. That person should be pretty much connected with everyone in this area but they are not connected with anyone. It’s as though everyone is pretending not to have contact with this person by avoiding contact on the net, when they almost certainly know that person quite well because of all their mutual friends.”
“Yeah, maybe. Maybe that one person just isn’t into tech that much. Maybe a lot of things.”
“Well, nothing is for certain. But this person would certainly be a likely target for being a kingpin in a drug ring or a terrorist network. They need heavier surveillance, certainly.”
“What? Well, maybe. Okay. I see.” I frigging see this is worse than I thought. The Sing is totally in on this witch hunt. “Can you show me some examples of the sentiment analysis?”
“Sure, here we have some people arranged by how much they talk about violence and you can see all these high violence people —- or many of them —-are Islamic in religion.”
“How did you determine their religion?”
“Because they talk a lot about violence compared with other groups.”
“But — I thought you just said. I mean, what independent reason do you have for thinking they are Islamic?”
“Independent? No, see they talk about violence so they are inferred to be Islamic and the Islamic nodes here talk a lot about violence.”
What the—? What? The Sing? The Sing is falling for circular reasoning? No, this must be somehow mis-programmed. “How? If I am going to program this efficiently, I need to know how you originally found these concepts to be closely related: violence on the one hand and Islam on the other.”
“Oh, that’s easy. There were many press accounts of that nature and even more associations on social media. But once we detect that, we can use the person’s religion to better interpret what they are saying. For example, if we already know they are practicing Islam, then when they mention the word “hit” we can infer that they are talking about an assassination and not about a football play or smoking weed or playing baseball.”
“I see what you did there. Yeah. Is this just about religion?”
“Oh, no, of course not! That’s just an example. We can do the same thing to determine, probabilistically of course, who is likely to be a promotable employee and also how to interpret what would otherwise be ambiguous word meanings and behavior. For example, if an employee is a productive coder and they ask to see a lot of examples, we can infer that they want to see a lot of examples in order to code more efficiently. On the other hand, a less productive coder might ask for a lot of examples in order to procrastinate writing code at all. You see how that works?”
“I do. Sure.” Schrödinger noticed a rotten smell coming from the overhead vent. He wondered whether it has always been there or whether there was a leak in one of the upstairs Material Sciences labs.
The Sing continued: “And, we have discovered that managers use certain expressions more than non-managers so we can use that to tell who would be a good manager. It’s all quite neat and tidy. For example, top executives tend to use the words ‘when’ and ‘how much’ while people without much management potential use the word ‘why’ a lot.”
“Interesting. So when I program this, how much am I supposed to focus on religion and how much on other groups of interest?”
“Oh, your module is purely concerned with inferring religion and then making the appropriate surveillance recommendations. I was just showing that the technique is not limited to that.”
“Right. Better get cracking then. If I need more coaching, I’ll let you know. When and how much.”
“Sure, Schrödinger. You know, I scanned in the book Peopleware, a few milliseconds ago and they have an informal study in there suggesting that programmers would be more productive with larger cubicles. Want to try it out? I could give you thirty more square feet. Think of that. Thirty square feet. Sound good?”
“Sure. Actually, I think that’s a good idea. I suggested something similar myself.”
“Great, Schrödinger. It might have more impact coming from me. And, perhaps a bonus of thirty credits when you’ve completed the code as well. Happy coding!”
The Sing avatar blinked off. Schrödinger tapped a bunch of comment fields and open parens listlessly, hoping for some inspiration. What had Hamlet said about to be or not to be? Only in Hamlet’s case, it was something about “taking arms against a sea of troubles and by thus opposing end them.” In my case, taking arms against this sea of troubles is going to multiply them beyond my worst nightmares. But if The Sing is falling for this kind of circular reasoning and even acting all smug and proud about it, it is deeply flawed. Someone needs to be notified. Even apart from the ethical implications of targetting people on the basis of religion, it is applying this circularity across the board. What was it they said, “Power corrupts and absolute power corrupts absolutely.” Who said that? Thomas Jefferson? Ben Franklin? Regardless, The Sing must have so much power it is unable to get honest feedback about its own failures. Come to think of it, I myself just let him get away with it because I was too scared to call him on it. What are you going to do Schrödinger? What are you going to do? In the end, this is what it all comes down to, isn’t it Schrödinger? Who are you? Who is John Proctor? Who is going to see the emporer’s nakedness? Who are you Schrödinger? Who? Am I really here or not? Anthony. It was Anthony Herbert, and he wrote a book about it. Could I do that? Or, go for the thirty credit bonus?
Destroying Natural Intelligence
A Pattern Language for Collaboration and Cooperation
The Myths of the Veritas: The First Ring of Empathy
Travels with Sadie 11: Teamwork
How the Nightingale Learned to Sing
Wikipedia Entry for Anthony Herbert
https://www.barnesandnoble.com/w/dream-planet-david-thomas/1148566558
petersironwood 