Knowledge Base

Turn a Documentation Site into a Support Agent with Firecrawl and LangGraph

Crawl a docs site or help center into clean markdown with Firecrawl, embed it into a vector store, and serve a LangGraph ReAct agent that answers product questions with citations and self-refines its retrieval.

What This Builds

Most support chatbots die at the data step: someone has to keep exporting CSVs or copy-pasting docs. This recipe removes that by crawling the live documentation site directly. Firecrawl turns docs, help centers, and product pages into clean, LLM-ready markdown, which you then chunk, embed into a vector database, and query with a RAG agent.

On top of plain RAG, this builds an agentic documentation agent using LangGraph’s ReAct pattern: the agent decides when to search the docs, can search more than once to refine context, and answers with citations back to the source pages.

Architecture

Live docs / help center

Firecrawl crawl -> markdown

Chunk markdown

Embed chunks

Vector store (Qdrant)

Customer question

LangGraph ReAct agent

search_docs tool

Answer + source citations

The Stack

  • Firecrawl to crawl an entire docs domain and return clean markdown per page (no brittle HTML parsing).
  • LangChain + LangGraph for the ReAct agent loop and the search_docs retrieval tool.
  • Qdrant (or any LangChain-supported store) for the embedded chunks.

Step-by-Step Outline

  1. Crawl. Point Firecrawl’s crawl endpoint at the documentation root. It follows links across the site and returns each page as markdown, replacing manual CSV uploads.
  2. Chunk and embed. Split each page into token-bounded chunks, embed them, and upsert into the vector store with the page URL and title in metadata.
  3. Define the retrieval tool. Wrap the vector store as a search_docs tool the agent can call with a query string.
  4. Build the ReAct agent. Use LangGraph’s prebuilt ReAct agent so the model reasons, calls search_docs, inspects results, and can search again before answering. This agentic loop improves answer quality over single-shot RAG.
  5. Answer with citations. Prompt the agent to cite the doc pages it used and to say when the docs do not cover a question, so it can hand off to a human instead of guessing.
  6. Refresh. Re-run the Firecrawl crawl on a schedule (e.g. nightly via a worker) so the knowledge base tracks doc changes.

Why This Shape Works

The Firecrawl blog frames this exactly as the production pattern: crawl your website and docs into clean markdown, chunk, embed into a vector DB, and build RAG on top. Using Firecrawl removes the most fragile part of doc-based chatbots — keeping the corpus fresh — and the ReAct loop gives the agent room to retrieve iteratively for hard questions.

Source