Article
Levels of Autonomy for AI Agents Working Paper - arXiv
arxiv.org
Quoted on this wiki
Every place a page here uses this source, in the order the words come in it.
Abstract “We argue that an agent’s level of autonomy can be treated as a deliberate design decision, separate from its capability and operational environment.” 1Introduction
Abstract “In this work, we define five levels of escalating agent autonomy,” 1Introduction
Abstract “characterized by the roles a user can take when interacting with an agent: operator, collaborator, consultant, approver, and observer.” 1Introduction
The earliest grand visions for AI promised intelligent agents that can learn our preferences, autonomously perform tasks on our behalf, and transform the way we consume information [24, 26]. Today, rapid growth in the capabilities of frontier generative AI models—in particular, large language models (LLMs)—renders these visions more achievable than ever. These models, which have demonstrated sophisticated understanding of natural language and our visual world, can equip AI agents with impressive autonomous capabilities, such as proactively resolving GitHub issues [18, 44], shopping online [45], and generally using computers in a human-like manner [1, 32]. Excitement over autonomous agents11 1 Henceforth, our use of “agents” will refer to AI agents rather than human agents, unless explicitly stated otherwise. is pervasive—autonomy is often touted as a new paradigm shift in AI by companies, developers, and the press [29]. This enthusiasm has been reinforced by product releases such as OpenAI’s Operator [32], Cognition’s Devin [41], and Google’s Deep Research [11], accelerating agents’ inclusion into society. “Despite these concerns, research and development of agents continue to focus on automating tasks, likely in part due to how the current evaluation paradigm of using benchmarks narrowly focuses on autonomous task completion accuracy [19].” In this essay, we argue that rather than treating autonomy as an inevitable consequence of increasing agent capability, autonomy can instead be a deliberate design decision made by agent developers. That is, given a fixed set of capabilities and a fixed operational environment, developers can make intentional choices about the level of autonomy within which an agent operates, and guide the design of their agent with those choices. But what do these levels entail, and how might they be applied in practice?
The earliest grand visions for AI promised intelligent agents that can learn our preferences, autonomously perform tasks on our behalf, and transform the way we consume information [24, 26]. Today, rapid growth in the capabilities of frontier generative AI models—in particular, large language models (LLMs)—renders these visions more achievable than ever. These models, which have demonstrated sophisticated understanding of natural language and our visual world, can equip AI agents with impressive autonomous capabilities, such as proactively resolving GitHub issues [18, 44], shopping online [45], and generally using computers in a human-like manner [1, 32]. Excitement over autonomous agents11 1 Henceforth, our use of “agents” will refer to AI agents rather than human agents, unless explicitly stated otherwise. is pervasive—autonomy is often touted as a new paradigm shift in AI by companies, developers, and the press [29]. This enthusiasm has been reinforced by product releases such as OpenAI’s Operator [32], Cognition’s Devin [41], and Google’s Deep Research [11], accelerating agents’ inclusion into society. “Such harms may manifest immediately (e.g., scams [12] and leaking private information [23]) or more gradually (e.g., human deskilling and loss of critical thinking [8], societal disempowerment [22]).” In this essay, we argue that rather than treating autonomy as an inevitable consequence of increasing agent capability, autonomy can instead be a deliberate design decision made by agent developers. That is, given a fixed set of capabilities and a fixed operational environment, developers can make intentional choices about the level of autonomy within which an agent operates, and guide the design of their agent with those choices. But what do these levels entail, and how might they be applied in practice?
Autonomy. In robotics and human-robot interaction, autonomy is broadly interpreted as “the ability to operate without a human operator for a protracted period of time” [3, 20]. We build off this definition and use autonomy to refer to the extent to which an AI agent is designed to operate without user involvement. Per our earlier definition, “user” can be a human or another AI agent. We use “designed to operate” because, as we later argue, autonomy is a design decision accompanied by a set of model- and user interface-based constraints that developers can use to shape agent behavior. “Involvement” is multifaceted and includes a spectrum of actions from direct control to light supervision. The multifaceted nature of involvement forms the foundation of our autonomy framework we will introduce later. “An agent may have low agency if the set of tools it has access to is limited (e.g., it can only call a calculator API but cannot browse the web), thereby restricting its capacity to carry out an intentional action.” 3Five Levels of Autonomy for AI Agents
→The user’s request is not straightforward in the sense that it needs to be decomposed into smaller subtasks to be completed effectively. The agent leaves this planning process to the user. As the user works, the agent begins paying attention to the user’s activities in the environment. This way, the user remains in charge of the workflow, while the agent provides the user with contextual assistance when requested, or even proactively suggests ways it can help. For example, when the user opens a web browser and navigates to a search engine, the agent suggests search queries related to the use of generative AI in the United States. As the user reads relevant reports on the topic, they can request agent summaries in a low-friction way (such as with the click of a button or a keyboard shortcut). The user can also highlight certain concepts they want to more deeply investigate, and the agent searches the web for related literature and presents them to the user. Later on, the user downloads some relevant datasets and opens a code editor to transform and visualize the data. The agent ‘‘follows’’ the user to the code editor and suggests code autocompletions. “if the agent proactively suggests actions, it does do not execute them until they are approved by the user.” Given this, some open questions agent developers may wrestle the when developing effective L1 agents may include:
Quotes are verbatim and belong to their publishers. Where a quote has been translated, the original wording is kept below it.