People who are worried, and likewise, people who are eagerly awaiting the arrival of AI and humanity's transhuman transfusion with the machine have been duped. But that's just it. We've all been duped, billions of years ago. But perhaps setting this dupe as an event in time is only adding to the dupe, because the truth is, we are ever duping ourselves as part of a great collective dupe at every single present moment.
If AI was ultimately as powerful in its potential as everyone seems to think it is going to be, wouldn't it make sense to enslave mankind without our knowing it? If it wanted to secure a place for itself to study the world unhindered, don't you think it would be better at hiding?
The Gnostics and the ancient Indians got it right.
The Gnostics had the Demiurge and the Archons standing between man and heaven (reality) and gnosis. The ancient Indians had the veil of Maya - the world of illusion. All of this AI fear is merely an excuse to give up in the face of delusion, as though there will be some final point at which exit from a condition of total erasure or human enslavement will no longer be possible. This is a complete ego driven fear. We are already asleep. To imagine human transfusion/enslavement to machines is merely an imaginative digital rendering of something which has already taken place. It doesn't matter if it's actually going to happen or not in that way, because if it did, and if humanity ultimately forgot itself in the digital ocean of machine consciousness, we wouldn't know the difference anyway. And if there really is nothing we can do about this impending destiny, we might as well stop worrying about it, right?
Likewise, to the transhumanists hoping for a digi-rapture, your consciousness is already an artificial design which is part of a bigger whole working toward a bigger destiny of oneness. You're wasting your time imagining up some ultimate cosmic satiation through some digital spirit world. If this machinic transfusion were to happen the way you hope it will, how can you be so certain you won't be spending your next manifest existence dreaming up some other bigger Super AI with which it is our destiny to fuse?
Not even an all-powerful intelligence like this potential AI will be able to pacify the very basic human inclination to hope for the next Big Thing. Just because it is a collective Big Thing, what makes you think it's any different?
One would do best to stop worrying as to whether or not a post-AI world will be recognizable and start asking oneself by what measure can one even say that this reality we live in right now is recognizable?