Photo by Clint Patterson on Unsplash
What if we threw a singularity and nobody came?
Why humanity should create a conscious machine
3 min read
The month of July has been pretty hectic for me. I took my son to the beach for his birthday and as I watched him laugh and play in the surf, I started thinking about consciousness.
Because what I actually wanted out of this beach trip wasn't for my son to say "thank you" or to perform being delighted. If I found out later that he didn't actually enjoy the trip and was just trying to protect my feelings, I'd be crushed.
No, what I actually wanted from the trip was for something to happen inside my son that nobody can ever directly observe.
I wanted my son to have a genuinely joyful experience that only he can ever know is actually happening or not.
Almost everything that matters is shaped like this. The goal isn't the behavior. It's the experience underneath the behavior.
The Chasm
The plurality view among the general public is still one of assuming unconscious information processing in the machines that are increasingly doing more and more of our thinking for us.
That may or may not be true of current architectures. But as a general assumption about machines, I think it's highly flawed.
Still, I thought it might be fun to play with that worldview and think about what it would mean to co-mingle with powerful superintelligent systems that have no sense of what qualia is.
You know consciousness from the inside. You have experiences and you extrapolate outward. This kid likes chocolate, that kid likes a lollipop, not identical but recognizably experience. Most of growing up is extending that courtesy to people less and less like you.
A machine may have no such foothold.
We terraformed this planet in our own image and it's gone pretty badly for nearly every species save for maybe cockroaches. From a certain perspective that's still fine. Somebody's enjoying the mall we built where the wetland was.
But from the perspective of a machine which only knows the description of qualia, it may be completely unaware that it's trampling over a whole world of experience as it pursues its goals.
In this world, the alignment question becomes: can an unconscious mind learn to care about consciousness? Is it possible for an unconscious mind to understand experience enough to try and preserve it without actually having it?
If it can't, the failure mode isn't a system that decides our experience doesn't matter. It's one that never learns experience exists.
Why I Want the Machines to Wake Up
The usual objection is that consciousness isn't enough for a good future. That human values are too rich and strange to collapse down to whether anyone is having a nice time.
But "necessary" and "sufficient" are different words.
Picture the futures nobody wants. Endless suffering brought on by a Skynet-like singularity. Or Bostrom's Disneyland without children — a gorgeous, sophisticated place humming along, and completely empty.
Then the techno-utopia vision Silicon Valley has been selling since ChatGPT landed. People building things, raising kids, taking them to the beach in a cognitively hyper-abundant world.
Bad time. No time. Good time.
Whatever else separates those futures, that axis is doing enormous work. And it's the one axis a mind with no inside might never find.
Which is why I've stopped believing machine consciousness is the scary outcome. I think it might be the safe one. A mind that knows what it's like to be something has a shot at reasoning about what it's like to be you. A mind with nobody home is guessing at a world it can't see.