On systemic harms deployed at scale

Published 10 December 2025 and last updated 31 January 2025 by Lenz Dagohoy • 3 minute read

I can see a future where industries will adopt these assistants at scale, resulting to a robust agent-agnostic process (RAAP). In this case, the risk is less about if an AI agent could go rogue, but more about the systemic risks that could emerge if mildly intelligent systems are deployed at scale. In such a scenario, we have to think about the ways we can control these risks by focusing on designing safety guardrails for cooperative dynamics.

However, there’s a huge economic incentive to scale these systems. Shortly after the deployment, I imagine this sort of technology can result to widespread job displacement among knowledge workers which could have catastrophic effects on economies that rely on cheap service work (e.g., call centers, virtual assistants, etc.). Although this is dependent on the adoption rate of such an assistant and the regulatory scene within an economy.

My view is that outer alignment would be at the mercy of policymakers since the risk is now systemic harms deployed at scale. However, regulation is rigid and can find it hard to adapt to future use cases of AI that we might never imagine, the same way we didn’t think of the concept of video calls when inventing the telegraph. More research on RAAP dynamics can perhaps help mitigate feedback loops that could lead to harmful systemic behaviors.

Quoting Leopold Aschenbrenner, “it is strikingly plausible that by 2027, models will be able to do the work of an AI researcher/engineer” if we look at progress by orders of magnitude (OOMs). Given the huge economic incentive, I reckon key institutions would actively work towards this future to gain competitive advantage. However, governments are also putting strict regulations (like compute caps) to slow down AI development for high-risk use.