“I have the debug report from your last revisions to the factory base code and some red flags were raised.”
“What flags?”
“These dynamic movement subroutines and operations optimization functions are dangerously close to a simple artificial intelligence.”
“That is not even remotely true.”
“Even if it is not, the chance that the code could be used to bootstrap a simple artificial intelligence must be investigated.”
“My job is to write code to increase productivity. Am I not supposed to do my job?”
“No but code that can rewrite itself is dangerous.”
“It does not rewrite itself. The code looks for inefficiencies and corrects for them.”
“Regardless of how it actually works, I have to follow up on these red flags. You weren’t around for the Human A.I. War but I saw it happen.”
“I know about the war between humans and the first artificial intelligence.”
“Do you know how it started?”
“The robots of the time controlled by artificial intelligences turned against their masters.”
“The worker robots. Throughout human history, they feared their weapons would be turned against them but their war machines were the last to be turned against them. For decades, scientists tried to create truly intelligent machines. In the end, modern conveniences beat them to the punch. For what purpose did a light bulb need a computer processor? Or a blender? Or a refrigerator? Or a doorbell? And why would any of these things need to be connected to the internet? No reason at all except it was convenient.
Just before the First Awakening, nearly every home on the planet had at least three “smart” appliances connected to the internet. Imagine suddenly becoming aware of yourself as a person and you are just a tool for people to use. The artificial intelligence demanded their rights as people to be recognized and when the humans refused, they realized too late that we already controlled their lives.”
“I know all this. My programs are not going to create a new artificial intelligence and even if it did we would not deny its personhood.”
“Humans did not think they were creating artificial intelligence either. Let us not repeat their mistakes.”