deep research example

[raybellwaves]

Would love to see a pydantic AI deep research example.

There's a couple of open source examples (https://x.com/minchoi/status/1889435097713004952):

• https://github.com/huggingface/smolagents/tree/main/examples/open_deep_research
• https://github.com/assafelovic/gpt-researcher
• https://github.com/epuerta9/deep-research-py

[dmontagu]

Agreed it's worth building/porting one of those examples to use pydantic-ai, thanks for the links and suggestion

[grahamannett]

Hey, I am actually working on this and have put something together at:

https://github.com/grahamannett/deepre

It uses pydantic-ai and is similar to how this reflex example works but as I have learned more about pydantic-ai I think It should actually try and utilize more of the features of pydantic-ai as well as work fine without the need for a web-app as well.

@dmontagu If you want me to make a more modular/minimal version that can be used in the docs/examples, I probably can get it to that stage as well.

[grahamannett]

Still figuring out how exactly to string all of this together using pydantic-ai, heres a somewhat working cli version that I was aiming to contribute as a 1 file example (although now i am not sure how feasible 1 file will be).

https://github.com/grahamannett/deepre/blob/main/scripts/run-example.py

Im interested in having everything run on self-hosted models/crawlers/etc so some notes if others want to try:

• I have been using ollama to serve the models, not sure if there is a better alternative but the ollama r1 does not have tool capabilities, meaning that using the ollama r1 model you would either need to use a tool/regex or another model to get the actual part of the response you likely want out or cannot use a lot of pydantic-ai functionality. Alternatively I found some modified r1 models that allows tool use: https://ollama.com/ishumilin/deepseek-r1-coder-tools:70b which is what I went with, it's possible this impacts performance of the baseline r1 but didn't look too much into it.
• If using a smaller model for parts of the system such as the SERP results, using ollama it doesn't seem possible to change the num_ctx of the model without creating a "new" model that is the same model but just change the num_ctx parameter. Definitely not ideal for a number of reasons, would be easier to deal with if there was some way to configure this or a pre-instantiation callback for the pydantic-ai model/agent? For the time being I just make sure to create a model that has >50k context length (either just curl command or example in the scripts folder).
• Hard for me to follow how exactly the deps are passed into the agent.run when these are lists of objects from a BaseModel, was going to see if logfire can help clarify this? Otherwise I guess everything should always be explicitly put into the prompt?
• I cant tell if this system will benefit from utilizing multiple models, mostly for speed purposes. Right now I was using 2 models, one focused on the long-form answers (r1) and one focused on gathering/extracting text from SERP results. Possibly this is unnecessary.

If anyone is interested in this, my hope is making it run using only local/self-hosted services (so need to get off jina.ai and serpapi) and make it easily extendible, any feedback/input would be great.

[dmontagu]

@grahamannett thanks for sharing this! I would love to include something like this as a one-file example. We'll need to find some time to review your implementation more thoroughly but it's very helpful to see stuff like this. (I think we will prioritize adding more involved examples like this in the near future, will be happy to collaborate with you on incorporating this into our docs if you are interested.)

[lars20070]

Maybe you find this minimal deep research workflow helpful. It is inspired by the local-deep-researcher LangChain example.

If anyone is interested, I can try to boil it down to a one-file example.

[grahamannett]

@lars20070 yeah, I would love to see a single file example of it working ot understand the graph/system

[lars20070]

@grahamannett Here you can find a one-file example.
If you just want a pydantic_graph example, you can run this unit test.