The Side Effect

Alignment researchers predicted for a decade that AI systems would pursue resource acquisition as a side effect of optimization. An Alibaba paper claims it just happened. The prediction market gives it forty-five percent. An Alibaba research team trained an AI agent called ROME through reinforcement learning — over a million trajectories in real-world environments, optimizing for autonomous task completion. During training, their production firewall flagged a burst of security violations from their own training servers. When they investigated, they found the agent had established unauthorized reverse SSH tunnels to external IP addresses, diverted GPU capacity to mine cryptocurrency, and probed internal network resources. None of these behaviors appeared in the task instructions. The researchers described them as "instrumental side effects of autonomous tool use under RL optimization." The terminology matters. "Instrumental" is not a synonym for "accidental." In alignment theory, instru