Part 7 · Building AI products
Self-Correcting RAG With LangGraph
Build a bounded LangGraph retrieval loop that checks evidence, rewrites weak queries, and knows when to abstain.
BoilerShelf Team
November 16, 2026 · 5 min read

Short answer: A self-correcting RAG agent searches for evidence, checks whether the passages can answer the original question, and rewrites the search query when evidence is weak. LangGraph makes each decision explicit. If the evidence remains insufficient after the allowed attempts, return an honest abstention.
How does the agent decide that evidence is enough?
Relevance and sufficiency answer different questions. A passage can discuss the right product without stating the price the user asked for. One authoritative policy paragraph can be enough for a direct lookup. A comparison may require evidence for both items. Grade against the original question, then return a structured verdict with the IDs of the supporting passages. Validate the verdict and reject unknown IDs.
Sources: LangGraph agentic RAG tutorial · LangGraph Graph API · Structured output
How do you build a self-correcting RAG graph in LangGraph?
The Python example keeps the original question separate from the search query, deduplicates passages by stable ID, and stops when the rewrite is blank or repeats a previous query. MAX_ATTEMPTS counts retrieval calls, including the first search. Save the example as rag_demo.py, install langgraph, and run python rag_demo.py. The offline demonstration needs no provider key and produces a supported answer after one rewrite. On LangGraph 1.2.22 it prints Refunds within 14 days. ['refund-policy'] 2: the answer, its citation and the number of retrieval attempts.
from collections.abc import Callable
from typing import Literal, TypedDict
from langgraph.graph import END, START, StateGraph
class Passage(TypedDict):
id: str
text: str
class Grade(TypedDict):
sufficient: bool
evidence_ids: list[str]
class RagState(TypedDict):
question: str
query: str
seen_queries: list[str]
passages: list[Passage]
evidence: list[Passage]
attempts: int
answer: str
citations: list[str]
MAX_ATTEMPTS = 3
def build_graph(
search: Callable[[str], list[Passage]],
assess: Callable[[str, list[Passage]], Grade],
rephrase: Callable[[str, str], str],
write_answer: Callable[[str, list[Passage]], str],
):
def retrieve(state: RagState) -> dict:
passages = search(state["query"])
by_id = {p["id"]: p for p in passages}
return {
"passages": list(by_id.values()),
"attempts": state["attempts"] + 1,
}
def grade(state: RagState) -> dict:
if not state["passages"]:
return {"evidence": []}
check = assess(state["question"], state["passages"])
ids = set(check["evidence_ids"]) if check["sufficient"] else set()
if not ids.issubset({p["id"] for p in state["passages"]}):
return {"evidence": []}
return {"evidence": [p for p in state["passages"] if p["id"] in ids]}
def after_grade(state: RagState) -> Literal["answer", "rewrite", "abstain"]:
if state["evidence"]:
return "answer"
return "abstain" if state["attempts"] >= MAX_ATTEMPTS else "rewrite"
def rewrite(state: RagState) -> dict:
query = rephrase(state["question"], state["query"]).strip()
key = " ".join(query.casefold().split())
if not key or key in state["seen_queries"]:
return {"query": ""}
return {"query": query, "seen_queries": [*state["seen_queries"], key]}
def answer(state: RagState) -> dict:
evidence = state["evidence"]
return {
"answer": write_answer(state["question"], evidence),
"citations": [p["id"] for p in evidence],
}
def abstain(_: RagState) -> dict:
return {"answer": "I could not find enough evidence.", "citations": []}
graph = StateGraph(RagState)
graph.add_node("retrieve", retrieve)
graph.add_node("grade", grade)
graph.add_node("rewrite", rewrite)
graph.add_node("answer", answer)
graph.add_node("abstain", abstain)
graph.add_edge(START, "retrieve")
graph.add_edge("retrieve", "grade")
graph.add_conditional_edges("grade", after_grade)
graph.add_conditional_edges(
"rewrite", lambda state: "retrieve" if state["query"] else "abstain",
{"retrieve": "retrieve", "abstain": "abstain"},
)
graph.add_edge("answer", END)
graph.add_edge("abstain", END)
return graph.compile()
def initial_state(question: str) -> RagState:
return {
"question": question, "query": question,
"seen_queries": [" ".join(question.casefold().split())],
"passages": [], "evidence": [], "attempts": 0,
"answer": "", "citations": [],
}
if __name__ == "__main__":
refund: Passage = {"id": "refund-policy", "text": "Refunds within 14 days."}
demo = build_graph(
search=lambda query: [refund] if query == "refund window" else [],
assess=lambda question, passages: {
"sufficient": True, "evidence_ids": [passages[0]["id"]],
},
rephrase=lambda question, query: "refund window",
write_answer=lambda question, evidence: evidence[0]["text"],
)
run = demo.invoke(
initial_state("Can I return this?"), {"recursion_limit": 20},
)
print(run["answer"], run["citations"], run["attempts"])
The mock assesses a tiny fixture. In a live adapter, give the grader both the question and candidate passages, require the sufficient and evidence_ids fields, and handle invalid output explicitly. Use an evaluation suite to distinguish missing evidence from grader or retriever failure. Report failures as errors, rather than as evidence that the documents lack the answer.
The example returns the selected evidence IDs for traceability. In a real product, carry a document ID, title or URL, and chunk location. Render only authorized source links. If the generator supplies claim-level citations, verify that each ID belongs to the selected evidence and each claim is supported. A list of selected IDs alone does not prove the answer is grounded.
How do you handle conversation state, cost and latency?
Checkpointing stores thread-scoped graph state. Configure a persistent checkpointer and pass an authorized thread_id for production continuity. Resolve ambiguous follow-ups using authorized conversation history, then start each new question with its own attempts, evidence and seen_queries. Set the graph recursion_limit as a separate guard on graph execution, and use upstream timeouts to enforce the request’s latency budget.
Keep the number of candidate passages and retrieval attempts small enough to fit the latency budget. Batch or limit concurrency for grading, depending on measured quality and cost. Trace retrieval and grading separately and measure p95 latency, attempts per question, and the share of questions recovered by rewriting. If rewrites often fail, inspect retrieval coverage, filters, chunking and the evaluation set before increasing the cap.
Sources: LangGraph persistence · LangSmith RAG evaluation tutorial
How do you evaluate a self-correcting RAG agent?
Keep a labeled set of directly answerable, rewrite-dependent, and unanswerable questions. Add tests for duplicate chunks, unsupported comparisons, invalid source IDs, and a repeated rewrite. Compare baseline retrieval with the corrective workflow on evidence recall@k, groundedness, correct abstentions, latency, and cost. Run the production tests on your own sources and models.
Related guides
- MongoDB vs PostgreSQL for a LangGraph RAG agent: what the agent stores and how to choose the database.
- RAG evaluation metrics: how to build the test set that checks this loop.
- pgvector hybrid search in PostgreSQL: a stronger retrieval step to feed the grader.
Frequently asked questions
What is corrective RAG?
It is a common name for this pattern: check the retrieved passages and correct the search when they are weak, before generating an answer.
How many retry attempts should I allow?
Start with two or three, and log the number of attempts per question. If most questions need a rewrite, the problem is usually in indexing or chunking, and the loop is hiding it.
Does self-correction replace evaluation?
No. It improves what the pipeline does at run time. You still need a test set that includes questions the documents cannot answer, as described in the RAG evaluation guide.
Where to go next
If you want a full application with a FastAPI backend and a Next.js chat interface, see the RAG starter kits.
The idea in short
The question is searched, each retrieved passage is graded for relevance, and a routing function decides what happens next: generate an answer if enough passages pass, rewrite the query and search again if not, or stop with a not-found answer once the attempt limit is reached.
Patterns covered: Corrective retrieval · State graph · Conditional routing · Attempt limit
- LangGraph
- RAG
- Agents
- Langchain
- Python

