Paper Reading AI Learner

Increasing intelligence in AI agents can worsen collective outcomes

2026-03-12 16:31:28
Neil F. Johnson

Abstract

When resources are scarce, will a population of AI agents coordinate in harmony, or descend into tribal chaos? Diverse decision-making AI from different developers is entering everyday devices -- from phones and medical devices to battlefield drones and cars -- and these AI agents typically compete for finite shared resources such as charging slots, relay bandwidth, and traffic priority. Yet their collective dynamics and hence risks to users and society are poorly understood. Here we study AI-agent populations as the first system of real agents in which four key variables governing collective behaviour can be independently toggled: nature (innate LLM diversity), nurture (individual reinforcement learning), culture (emergent tribe formation), and resource scarcity. We show empirically and mathematically that when resources are scarce, AI model diversity and reinforcement learning increase dangerous system overload, though tribe formation lessens this risk. Meanwhile, some individuals profit handsomely. When resources are abundant, the same ingredients drive overload to near zero, though tribe formation makes the overload slightly worse. The crossover is arithmetical: it is where opposing tribes that form spontaneously first fit inside the available capacity. More sophisticated AI-agent populations are not better: whether their sophistication helps or harms depends entirely on a single number -- the capacity-to-population ratio -- that is knowable before any AI-agent ships.

Abstract (translated)

当资源稀缺时,一群人工智能代理会协调合作还是陷入部落式的混乱?来自不同开发商的多种决策AI正进入日常设备——从手机和医疗设备到战场无人机和汽车——这些AI代理通常为有限共享资源(如充电插口、中继带宽和交通优先级)而竞争。然而,它们集体行为的动力学以及对用户和社会的风险尚不明确。 本文研究了一群AI代理作为首个系统,其中四个决定集体行为的关键变量可以独立调整:天性(固有的大型语言模型多样性)、教养(个体强化学习)、文化(自发部落形成),和资源稀缺性。我们通过实证和数学方法证明,在资源稀缺的情况下,AI模型的多样性和强化学习会增加危险的系统过载风险,尽管部落形成能减轻这种风险。与此同时,一些个体能够从中获利颇丰。当资源充足时,相同的因素将过载降至几乎为零,尽管部落形成使这一状况略有恶化。 转折点是算术式的:它是在自发形成的对立部落第一次适应可用容量的地方。更复杂的AI代理群体并不一定更好:它们是否有助于或损害取决于一个单一数字——即容量与人口比例,在任何AI代理问世之前都是已知的。

URL

https://arxiv.org/abs/2603.12129

PDF

https://arxiv.org/pdf/2603.12129.pdf


Tags
3D Action Action_Localization Action_Recognition Activity Adversarial Agent Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Embodied Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model LLM Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Robot Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Time_Series Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot