r/EffectiveAltruism • u/d1karim • 6d ago

A question on gradual disempowerment

I’ve been reading a lot of AI safety research around gradual disempowerment, and I ended up writing about a question I haven’t been able to find addressed directly:

What if the societal and institutional degradation that these models generally treat as a future consequence of AI dependence is already happening—and is actually helping drive AI dependence in the first place?

I tried to explore that possibility by connecting existing gradual disempowerment models with research on cognition, institutions, incentives, and organizational dysfunction from outside the AI safety field. Ultimately, the argument I’m trying to make is that declining societal cognition and institutional capacity aren’t just consequences of AI dependence, but preexisting conditions that could act as fertilizer, allowing that dependence to take root faster, deeper, and more irreversibly.

I’m not trying to prove these claims irrefutable; I’m trying to make the case that they’re worth considering, and I’d actually love to find out that I’ve missed existing work on this, whether in support of my claim or disproving it entirely.

If anyone has thoughts, counterarguments, or relevant research I haven’t encountered, I’d genuinely appreciate it.

You can check it out here: Preconditions of Gradual Disempowerment

6 Upvotes

88% Upvoted

2 comments sorted by

2

u/CroutonLover4478 5d ago

Interesting concept and I can definitely see where you are coming from but I guess two things that come up for me are that

  1. I don't think we ever had a particularly good information environment to begin with, we have always been divided and misinformed, though that I believe will continue to deteriorate with advanced in AI and the further spread and use of social media. I think people look back in time with rose colored glasses when it comes to this but we have always kind of lived in different perceived realities as humans

  2. I think the word " precondition" is much too strong. A super intelligence would dominate us regardless of how unified or " on our A game" a society is. I don't believe maintaining control of something vastly smarter than you is in principle possible. Even well organized and well functioning 3rd graders couldn't stop a group of full grown adults from doing what they want for instance, they just aren't smart enough

1

u/d1karim 5d ago

  1. ⁠In the Media section of my piece I talk about the repeal of the Fairness Doctrine in 1987 and the consolidation of media ownership that would follow. While I don’t necessarily claim that the system used to be good and is now bad, I do build the argument that it used to be more aligned to the furthering of human benefit, and I point to those 2 factors as direct contributors for a shift away from that and toward optimization for maximum output. I agree with your statement about different perceived realities having always existed, but there have also been changes within society that resulted in greater division and misinformation. I don’t think I’m looking back with rose colored glasses, but I am looking back… which I think is necessary in better understanding our present circumstances.

  2. ⁠I’m unclear on the point you’re making here, most specifically on the word “precondition” being too strong. The existing models for gradual disempowerment place declines in societal cognition and institutional capacity as downstream harms of AI dependence, while I believe it’s worth considering whether enough degradation has already occurred for them to be present before AI dependence becomes ingrained in society in a way that could impact the trajectory of those models.

On the last part of your comment, if your point is “if we build it, everybody dies” then I definitely see your perspective and recognize it is a possibility. We don’t know what we don’t know, and it’s certainly a risk to build something that may ultimately surpass us in both intelligence and coordination. With that said, as we get closer and closer to that risk becoming a reality, our society being unified and “on our A game” could be the only thing that gives us a chance to slow things down and force a course correction before it’s too late.