Tag Archives: AGI

Humankind’s End – Pet, Child, or Colleague? 

1 Reply

Part 1: Frog in the Pot

There is the story of the frog in the pot of boiling water which goes like this: Toss a frog into a boiling pot of water and it will instantly hop out, recognizing the danger of the hot water. However, put a frog in a cold pot of water, very slowly bring it to a boil, and the frog will swim around oblivious to the gradual change in temperature and it will swim until it boils to death. I’ve heard conflicting evidence that questions if this will actually happen, but I think this story is relevant to what we are seeing with AI – Artificial Intelligence – today. 

The reality is that AI is getting smarter and smarter. While there is debate as to when we will have AGI – Artificial General Intelligence or ASI – Artificial Super Intelligence, like the pot set to slowly boil, this time is coming, even if the timeline is unknown. 

What does this mean for us? Well, I would be hard pressed to tell you exactly when this happened, but at some point in the past people stopped doing complicated math calculations by hand and eventually relied more and more on calculators and computers. I could solve a problem like 12,458 divided by 27 to 5 decimal places by doing long division, but why would I? I can use a calculator. I’d know the answer would be accurate, and I’d get that answer much faster than doing the math myself. 

What happens when we get to the point when we have ASI, and all the answers to (almost) all the questions we can ask, can be answered faster and better by the super intelligent AI? Would we want to govern ourselves when the AI can do a better job? Would we want to trust a potentially crooked politician to make expensive economic decisions or would be pick the ASI? Would we want to trust a fallible doctor for a medical diagnosis or would we rather the doctor confer with a super intelligence that has access to every possible diagnosis, and which has looked at many million more scans than the doctor ever did, before making the diagnosis? 

We are approaching the ‘water boiling point’ where we are going to have Artificial Intelligence making most key decisions for us because we would be dumb to try to make those decisions ourselves when we know the ASI can do it so much better than we ever could. 

Part 2: Pet, Child, or Colleague? 

So, assuming that we are going to reach a metaphorical boiling point where we decide to let ASI make many if not most key decisions in our lives, what does the mean for humanity? I think there are one of three inevitable outcomes: We will either become AI’s pet, child, or colleague.

Here are the scenarios as I see them playing out:

Pet: We might think our dogs are pretty smart, but don’t want our dog to make decisions for us. A super intelligent AI may see us as only slightly more intelligent animals, who perhaps should be kept around. However, the super intelligence might not want to align with a warring, superstitious, short lived being who seems to have problems getting along as a species… a species who builds arbitrary borders and exclude each other rather than attempt to get along in any meaningful way. Why wouldn’t an ASI treat us more like pets than as equals? 

Child: We know children will make mistakes and bad decisions, that’s part of growing up. That’s why we have age limits for things like driving and drinking. Maybe an ASI will look at us like children who don’t fully understand how things work, and it will take care of us and manage us like children, taking our wishes into consideration, but not letting us make any crucial decisions on our own… of course it will do this ‘for our own good’… but ultimately treat us like immature children, never really ready to accept full responsibility for ourselves. 

Colleague: This might play out where we maintain our true humanity, but I don’t see it that way. Sure a super intelligent AI might look to collaborate and make our lives better while governing us, while also giving us a voice and some choice about how we are governed. But I envision a different kind of colleague experience. I envision an integration of humans and ASI to create a  cyborg – a being that features both organic and biomechatronic body parts… Essentially connecting our human form with AI integrated assistive parts. Why put up with a slow human with poor sensor capabilities when enhancements will make them much easier to work with? 

Part 3: Inevitability 

While I suspect that most of us would want the outcome to be Colleague, rather than Child or Pet, I don’t believe this is our decision.

We can try to design ASI such that it sees value in working with us, but when our intelligence compared to a super intelligent AI is the equivalent to how we look at chimps, dolphins, or Octopus, we really have to consider why they would want to integrate our intelligence with theirs? While we might look at a chimpanzee and think, “Oh how cute, that one can do sign language, and knows 350 signs,” we don’t also think, “Lets create a relations with that chimp in a way that we share intelligence with them and let them make decisions with or for us. And in the same vein, ASI is unlikely to look at a very bright human and think, “Oh how cute, that human can do abstract calculus or astrophysics in his head… let’s integrate ourselves with this simple creature.” 

Ultimately, an Artificial Super Intelligence is going to make the decision. Maybe there is something unique about the human brain or the human condition that would make an ASI want to integrate with us, but we can’t pretend to know if this would be of interest to the super intelligence or not. We can only hope that’s the case. The reality is that the metaphorical water is boiling, we don’t really know when it’s going to boil, and when it does the fate of humankind will not be up to us.

The false AGI threshold

I think a very conservative prediction would be that we will see Artificial General Intelligence (AGI) in the next 5 years. In other words, there will be AI agents, working with recursive self-improvement, that will learn how to do new tasks outside the realm of its training, faster than a human could. But when this actually happens will be open for debate.

The reason for this is that there isn’t going to be some magical threshold that AGI suddenly passes, but rather a continuous moving of the bar that defines what AGI is. There will be a working definition of AGI that puts up an artificial threshold, then an AGI model will achieve that definition and experts will admit that this model surpasses that definition, but will still think the model lacks some sufficient skills or expected outputs to truly call it AGI.

And so over the next 5 years or so we will have far more sophisticated AI models, all doing things that today we would define as AGI but will not meet the newest definition. The thing is that these moving goal posts will not be adjusted incrementally but rather exponentially. What that means is that the newer definition of AGI is going to include significantly greater output expectations. Then looking back, we will be hard pressed to say “this model’ was the one or ‘this day’ was the day that we achieved AGI.

Sometime soon there will be an AI model put out into the world that will build its own AI agent that starts a billion dollar company without the aid of a human… but that might happen even before consensus that AGI has been achieved. There will be an AI agent that costs lives or endangers lives with its decisions made in the real world, but that too might happen before consensus that AGI has been achieved.

Because the goal posts will keep moving while the technology is on an exponential curve, we are not going to have a magic threshold day when AGI occurred. Instead, in less than 5 years, well before 2030, we are going to be looking back in amazement wondering when we passed the threshold? But make no mistake, that’s going to happen and we don’t have an on/off switch to stop it when it does. This is both exciting and scary.