Architecting a Deep Research System

A deep research system answers complex queries by running a comprehensive, multi-step analysis over a large document corpus. Instead of a single retrieval pass, it imitates how a person researches: plan the work, delegate pieces to specialized sub-agents, then synthesize the findings into one answer. This is the pattern behind the “deep research” features in tools like ChatGPT and Gemini.

When you actually need one

A deep research system is heavy machinery. Reach for the simplest method that works first:

• Direct LLM input — fine when everything fits in the context window (roughly under 1M tokens).
• Standard RAG — good for questions answered by retrieving specific facts or text chunks via vector similarity.
• Keyword search — good when you know the dataset and the exact terms to look for.

Escalate to a deep research system only when those fail: when the answer spans many sources, isn’t contained in any single chunk, or when you don’t yet know the corpus well enough to search it directly.

System architecture

The system is an orchestrator agent directing several sub-agents, each holding tools that touch the data.

1. Orchestrator agent — the brain. It takes the query, builds a research plan, dispatches sub-agents, evaluates what comes back, and synthesizes the final answer.
2. Sub-agents — workers that run narrow tasks the orchestrator hands them: a keyword search, a document summary, a figure extraction.
3. Tool library — the functions agents call to gather information.
4. Corpus and indices — the knowledge base, indexed two ways for efficient retrieval:
5. Keyword index for literal term matching (e.g. BM25).
6. Vector index for semantic similarity, built by chunking, embedding, and storing documents in a vector database.

Implementation

1. Aggregate and index

Gather the source material (Google Drive, Notion, Salesforce, and so on) into one place — say, PDFs in a single folder — then build both indices: a keyword index for specific terms, names, and titles, and a vector index by chunking documents, embedding each chunk, and storing the vectors for semantic search.

2. Build the tools

Write a small library of functions the agents can call. Keep them simple and reliable:

@tool
def keyword_search(query: str) -> str:
    """Searches the document corpus for specific keywords and returns matching text."""
    results = perform_keyword_search(query)
    # Format results for the LLM
    formatted_results = "\n".join([f"Source: {res['file_name']}\nContent: {res['content']}" for res in results])
    return formatted_results

@tool
def vector_search(query: str) -> str:
    """Performs a semantic search on the document corpus to find conceptually similar text."""
    results = perform_vector_search(query)
    # Format results for the LLM
    formatted_results = "\n".join([f"Source: {res['file_name']}\nContent: {res['content']}" for res in results])
    return formatted_results

Other useful tools include internet_search, filename_search, or custom accessors for specific databases.

3. Run the agentic loop

The orchestrator drives a loop:

1. Plan — take the query (asking a clarifying question if scope is unclear) and formulate a multi-step plan.
2. Delegate — dispatch sub-agents to run the first steps, handing each the tools and instructions it needs.
3. Execute & summarize — sub-agents gather information, summarize it, and report back.
4. Aggregate & iterate — the orchestrator collects summaries and decides whether they answer the query. If not, it refines the plan and delegates again, looping back to step 2.
5. Synthesize — once the evidence is sufficient, the orchestrator composes a single coherent, well-supported answer.

4. Choose models by role

Match model strength to the job to balance quality and cost:

• Orchestrator — needs strong reasoning; a high-capability tier such as Claude Opus 4.1 or the OpenAI o1-series.
• Sub-agents — narrower tasks, so faster and cheaper models fit: Claude Sonnet 4.5, Gemini 1.5 Flash, or GPT-4o mini.

For deeper model comparisons, see ChatGPT, Gemini, and Claude.

Trade-offs

The payoff is depth: by working a corpus far more thoroughly than single-pass RAG, the system delivers comprehensive answers and rarely misses critical information. The cost is latency and complexity — the iterative, multi-agent process is inherently slower. Use it where the quality and depth of the answer matter more than the speed of getting it.