Documentation on agent control flow

[dyllamt]

I've been trying to understand the control flow for agent runs, which I think others are also trying to learn about. Here are some of the related issues for context:

#642
#142
#127

It would be very helpful for someone with deep knowledge of the library to create a documentation section for control flow within agents. These are some examples of what I want to understand:

1. inside agent.run() what is executed sequential v.s. concurrently (post to LLM provider, tool calls, result validation, etc.)?
2. are there points where coroutines within agent.run() can be cancelled?
3. at what points will control be given back to the caller of agent.run()?
4. which of these points are considered "success" cases and which ones "error" cases?

I'm trying to write a user application, where the agent might be sending data back to the user while it's running (see example below), so it's really helpful to be crystal clear on what's happening inside the agent.

import dataclasses

from fastapi import WebSocket
from pydantic_ai import Agent, RunContext


@dataclasses.dataclass
class AgentDeps:
    websocket: WebSocket


agent = Agent(
    "openai:gpt-4o",
)


@agent.tool
async def do_something(ctx: RunContext[AgentDeps], message: str) -> None:
    """Do the thing.

    Args:
        ctx: The call context.
        message: The info from my llm.
    """
    await ctx.deps.websocket.send(f"The llm sent me this: {message}")
    print("Agent did the thing.")


async def run_context(websocket: WebSocket, user_message: str):
    # Doing something before running the agent
    print("Doing something before...")

    # Blocks while agent is running
    await agent.run(user_prompt=user_message, deps=AgentDeps(websocket=websocket))

    # Doing something after running the agent
    print("Doing something after...")

I've also seen some examples thrown around where agent-runs are initiated in tools. The documentation would also help us reason about how that pattern would be executed under the hood.

[samuelcolvin]

Thanks for the issue, just curious, have you read through https://ai.pydantic.dev/agents/?

[dyllamt]

I've read that one - it gives a good overview on how to invoke and configure an agent, but not a lot of details on the internal control flow or state machine of the agent.

[dyllamt]

One potential way forward: I can put together a diagram with what I think is happening, and someone with more knowledge on the implementation can point out where I am going wrong?

[Kej11]

I use Logfire for this because it logs all the substeps of the flow automatically.

Tool calls can be either sequential or concurrent. In my app the LLM can do either and it dynamically picks the best strategy. For agents as tools, the control flow isn't changed, it just calls the other agent as a tool.