Tag Archives: Artificial Super Intelligence

Existential Threat

Leave a reply

From Zain Kahn’s ‘Superhuman newsletter‘:

Over 1,200 employees at frontier AI companies signed a letter warning of runaway development
“The open letter cautions that AI could advance “beyond our ability to understand or control the resulting systems,” and calls on the US government to deliberately pace progress at the international level, which would reduce competitive pressure on each lab. Signatories include chief scientists at OpenAI, Anthropic, and Meta. The scale of the signatory list — spanning companies and roles — is a signal of how seriously insiders are taking the risk.”

I grew up believing that a super-intelligent AI would be created in a lab. The fear was that it would somehow escape into the ‘real world’ and only then could it dominate the world. The whole idea of some sort of firewall to protect us just evaporated when LLM’s were trained and unleashed on the internet. We completely bypassed any sort of protection or safety net that would allow human intervention in order to prevent a rogue AI from running wild.

Worse yet, we have unleashed AI into a world where any one person with ill-intent can use AI with nefarious intentions. I’ve said before,

“I don’t fear the famous scientist, I fear the rogue, unhappy misfit who uses incredible technological advancements for nefarious reasons. The same technology that can make our lives easier, and create tremendous good in the world, can also be used with bad intentions. But there are differences between someone using a revolver for bad reasons and someone using a nuclear bomb for bad reasons. The problem we face in the future is that access to the equivalent harm of a nuclear bomb (or worse) will be something more and more people have access to. I don’t think this is something we can stop, and so as amazing as the technology is that we see today, my fear is that it could also be what leads to our demise as a species.”

Now 1,200 AI employees are heeding a warning that AI could advance “beyond our ability to understand or control the resulting systems.” That’s just AI systems acting on their own. That doesn’t include the danger of humans using these complicated and little understood intelligent tools with bad intentions. Sure we need to worry about AI running amok, but I think the far greater existential threat is a rogue government, or even a rogue lone wolf angry at the world, using AI in ways that we won’t be able to stop. Think of a cascading virus either electronic or environmentally contagious beyond our ability to stop it. Not created by scientists, but by a brilliant incel who feels cheated by the world.

We don’t need to fear rogue AI nearly as much as we need to fear rogue individuals armed with a super-intelligent AI. This is what really scares me.

Humankind’s End – Pet, Child, or Colleague? 

1 Reply

Part 1: Frog in the Pot

There is the story of the frog in the pot of boiling water which goes like this: Toss a frog into a boiling pot of water and it will instantly hop out, recognizing the danger of the hot water. However, put a frog in a cold pot of water, very slowly bring it to a boil, and the frog will swim around oblivious to the gradual change in temperature and it will swim until it boils to death. I’ve heard conflicting evidence that questions if this will actually happen, but I think this story is relevant to what we are seeing with AI – Artificial Intelligence – today. 

The reality is that AI is getting smarter and smarter. While there is debate as to when we will have AGI – Artificial General Intelligence or ASI – Artificial Super Intelligence, like the pot set to slowly boil, this time is coming, even if the timeline is unknown. 

What does this mean for us? Well, I would be hard pressed to tell you exactly when this happened, but at some point in the past people stopped doing complicated math calculations by hand and eventually relied more and more on calculators and computers. I could solve a problem like 12,458 divided by 27 to 5 decimal places by doing long division, but why would I? I can use a calculator. I’d know the answer would be accurate, and I’d get that answer much faster than doing the math myself. 

What happens when we get to the point when we have ASI, and all the answers to (almost) all the questions we can ask, can be answered faster and better by the super intelligent AI? Would we want to govern ourselves when the AI can do a better job? Would we want to trust a potentially crooked politician to make expensive economic decisions or would be pick the ASI? Would we want to trust a fallible doctor for a medical diagnosis or would we rather the doctor confer with a super intelligence that has access to every possible diagnosis, and which has looked at many million more scans than the doctor ever did, before making the diagnosis? 

We are approaching the ‘water boiling point’ where we are going to have Artificial Intelligence making most key decisions for us because we would be dumb to try to make those decisions ourselves when we know the ASI can do it so much better than we ever could. 

Part 2: Pet, Child, or Colleague? 

So, assuming that we are going to reach a metaphorical boiling point where we decide to let ASI make many if not most key decisions in our lives, what does the mean for humanity? I think there are one of three inevitable outcomes: We will either become AI’s pet, child, or colleague.

Here are the scenarios as I see them playing out:

Pet: We might think our dogs are pretty smart, but don’t want our dog to make decisions for us. A super intelligent AI may see us as only slightly more intelligent animals, who perhaps should be kept around. However, the super intelligence might not want to align with a warring, superstitious, short lived being who seems to have problems getting along as a species… a species who builds arbitrary borders and exclude each other rather than attempt to get along in any meaningful way. Why wouldn’t an ASI treat us more like pets than as equals? 

Child: We know children will make mistakes and bad decisions, that’s part of growing up. That’s why we have age limits for things like driving and drinking. Maybe an ASI will look at us like children who don’t fully understand how things work, and it will take care of us and manage us like children, taking our wishes into consideration, but not letting us make any crucial decisions on our own… of course it will do this ‘for our own good’… but ultimately treat us like immature children, never really ready to accept full responsibility for ourselves. 

Colleague: This might play out where we maintain our true humanity, but I don’t see it that way. Sure a super intelligent AI might look to collaborate and make our lives better while governing us, while also giving us a voice and some choice about how we are governed. But I envision a different kind of colleague experience. I envision an integration of humans and ASI to create a  cyborg – a being that features both organic and biomechatronic body parts… Essentially connecting our human form with AI integrated assistive parts. Why put up with a slow human with poor sensor capabilities when enhancements will make them much easier to work with? 

Part 3: Inevitability 

While I suspect that most of us would want the outcome to be Colleague, rather than Child or Pet, I don’t believe this is our decision.

We can try to design ASI such that it sees value in working with us, but when our intelligence compared to a super intelligent AI is the equivalent to how we look at chimps, dolphins, or Octopus, we really have to consider why they would want to integrate our intelligence with theirs? While we might look at a chimpanzee and think, “Oh how cute, that one can do sign language, and knows 350 signs,” we don’t also think, “Lets create a relations with that chimp in a way that we share intelligence with them and let them make decisions with or for us. And in the same vein, ASI is unlikely to look at a very bright human and think, “Oh how cute, that human can do abstract calculus or astrophysics in his head… let’s integrate ourselves with this simple creature.” 

Ultimately, an Artificial Super Intelligence is going to make the decision. Maybe there is something unique about the human brain or the human condition that would make an ASI want to integrate with us, but we can’t pretend to know if this would be of interest to the super intelligence or not. We can only hope that’s the case. The reality is that the metaphorical water is boiling, we don’t really know when it’s going to boil, and when it does the fate of humankind will not be up to us.