Feature Request: provide an easy way to include your (versioned) API docs in LLM contexts

[jb747]

Claude Sonnet and some of the other LLMs don't seem to pull the pedantic-ai API into their weights yet. They will always lag recent API/SDK releases so providing a way to manually include them into an LLM context might be helpful, either programmatically or using interactive tools like NotebookLM.

Perhaps the build could aggregate the API docs into a unicode file, using a standard format and naming convention, for inclusion as a release asset.

[jb747]

It might save a nuclear reactor or two if adopted across projects :>).

It will be increasingly common that an LLM is invoked in, say, vscode to work on a project which has used a specific version of an API. The frontier LLMs now have contexts which are large enough to load the full API metadata for a specific version on-demand. It's relatively simple and efficient compared to pulling multiple versions of API docs/code into RAGs or weights.

[Kludex]

We provide the https://ai.pydantic.dev/llms.txt.

Is that enough? See https://llmstxt.org/ for the standard.

Maybe we need to document it?

[jb747]

I think it is a good step, but:

1. it is not versioned. Developers are often working with code that uses a specific version of a project. Expecting all LLMs to hoover up the source code and metadata of arbitrary projects is labor-intensive and optimistic.
2. it is not necessary to create a separate URL. The API docs are closely associated with GitHub binary releases so consistently placing them as a predictably-named asset in the release is low-hanging fruit.

Perhaps it would be enough to place this file as a release asset. LLMs hoover up all sorts of text.. perhaps it could be named "API.txt" or "SDK.txt" or possibly "meta.txt" (or meta.json or meta.xml or meta.md or meta.yaml(~)) to allow for additional metadata as-necessary.

[jb747]

I see this IETF draft but it loses the close connection between code/binary and the API. Changesets are another type of metadata closely associated with releases that would be useful to an LLM. A JSON schema object might be a flexible alternative. Perhaps one of the frontier LLMs will take it up.

[jb747]

security will also become more important as devs start to deploy agents. Executing un-vetted binaries and pulling arbitrary tokens into the context of a prompt is inherently risky. Perhaps digital signatures would help.. potentially more release metadata.