03. CRAG (Corrective RAG)

CRAG: Corrective RAG

This tutorial Corrective RAG (CRAG) Covers how to improve RAG-based systems using strategies.

CRAG is an approach that elaborates search-generated piplines, including self-reflection and self-evaluation steps for searched documents.

---

What is CRAG?

Corrective-RAG (CRAG) In the RAG (Retrieval Augmented Generation) strategy Methodology that evaluates the documents found in the search process and adds steps to refine knowledge is. This includes a series of processes to check the search results prior to creation, perform an auxiliary search if necessary, and finally generate a high quality answer.

CRAG's key ideas are:

Paper (Corrective Retrieval Augmented Generation) link

1. If one or more of the searched documents exceeds the predefined relevance threshold, proceed to the creation phase.

2. Perform knowledge purification steps before creation.

3. Segment documents into "knowledge stripes". (Here, the number of document search results, k means.)

4. Each knowledge strip is evaluated and the relevance is score. (Here we evaluate in document chunk.)

5. If all documents are below the relevance threshold or if the evaluation results have low reliability, they are reinforced with additional data sources (e.g. web browsing).

6. Optimize your search results with reinforcement poetry through web search, and query-Rewrite.

---

Main contents

In this tutorial, LangGraph is used to implement some ideas from the CRAG approach.

here Knowledge purification steps are omitted And, if necessary, design in a form that can be added as a node.

Also, If there is no relevant document We will use web search to complement your search.

Web search Tavily Search Use and introduce Question Re-writing to optimize your search.

---

Key steps overview

• Retrieval Grader : Evaluate the relevance of the searched document

• Generate : Generate answers via LLM

• Question Re-writer : Optimization of search quality through rewriting questions

• Web Search Tool : Using web search through Tavily Search

• Create Graph : Generating CRAG strategy graphs through LangGraph

• Use the graph : How to utilize the generated graph

---

Reference

• LangGraph CRAG tutorial (official documentation)

Preferences

# Configuration file for managing API keys as environment variables
from dotenv import load_dotenv

# Load API key information
load_dotenv()

 True 

# Set up LangSmith tracking. https://smith.langchain.com
# !pip install -qU langchain-teddynote
from langchain_teddynote import logging

# Enter a project name.
logging.langsmith("CH17-LangGraph-Use-Cases")

The result is returned as (yes/no) whether it is relevant for a single document.

Here we conduct an evaluation of one single document, not a set of documents.

retrieval_grader Evaluate documents using

from langchain_core.prompts import ChatPromptTemplate
from langchain_openai import ChatOpenAI
from langchain_teddynote.models import get_model_name, LLMs
from pydantic import BaseModel, Field

# Get model name
MODEL_NAME = get_model_name(LLMs.GPT4)


# A data model that evaluates the relevance of retrieved documents using a binary score.
class GradeDocuments(BaseModel):
    """A binary score to determine the relevance of the retrieved document."""

    # A field indicating 'yes' or 'no' whether the document is relevant to the question.
    binary_score: str = Field(
        description="Documents are relevant to the question, 'yes' or 'no'"
    )


# LLM Initialization
llm = ChatOpenAI(model=MODEL_NAME, temperature=0)

# LLM that generates structured output using the GradeDocuments data model
structured_llm_grader = llm.with_structured_output(GradeDocuments)

# Define system prompt
system = """You are a grader assessing relevance of a retrieved document to a user question. \n 
    If the document contains keyword(s) or semantic meaning related to the question, grade it as relevant. \n
    Give a binary score 'yes' or 'no' score to indicate whether the document is relevant to the question."""

# Create a chat prompt template
grade_prompt = ChatPromptTemplate.from_messages(
    [
        ("system", system),
        ("human", "Retrieved document: \n\n {document} \n\n User question: {question}"),
    ]
)

# Retrieval Evaluator Initialization
retrieval_grader = grade_prompt | structured_llm_grader

First, the evaluator to evaluate the retrieved document ( retrieval-grader ) Generates.

Evaluating the relevance of a searched document is a step in evaluating whether the searched document is related to a question.

Evaluation of the relevance of the searched documents (Question-Retrieval Evaluation)

from rag.pdf import PDFRetrievalChain

# Load PDF document
pdf = PDFRetrievalChain(["data/SPRI_AI_Brief_2023년12월호_F.pdf"]).create_chain()

# Create a retriever and chain
pdf_retriever = pdf.retriever
pdf_chain = pdf.chain

Files downloaded for practice data Please copy to folder

• Author: Jaeheung Lee (AI Policy Institute Office Liability Institute), Lee Ji-soo (AI Policy Lab Yi Phyang Institute)

• Link: https://spri.kr/posts/view/23669

• File name: SPRI_AI_Brief_2023년12월호_F.pdf

Software Policy Institute (SPRi)-December 2023

Documents utilized for practice

• As covered in the previous tutorial, we omit the detailed description.

Reference

However, LangGraph creates Retriever and Chain separately. Only then can you do detailed processing for each node.

Here, we create a Retrieval Chain based on PDF documents. Retrieval Chain with the simplest structure.

Basic PDF-based Retrieval Chain creation

Start tracking LangSmith. 
[Project name] 
CH17-LangGraph-Use-Cases 

 [{'title': "Samsung, created Ai'Gaus' released...Starting next year, it will be mounted on Galaxys24, etc.”,'url':'https://www.aitimes.com/news/articleView.html?idxno=155034','content': "Samsung, creation AI ‘Gauss’ released...Starting next year, the creation satellite (AI)'Gauss', developed by the Samsung Temple on GalaxyS24, etc., was first released. Samsung (CEO Han Jong-hee, Boundary Hyun) held the ‘Samsung AI Forum 2023’ second day event at the 8th Samsung SeoulR&D Campus, and unveiled its own development-generating AI Gauss and Ondice AI plans.  
(...sway...) 
△Features of large language models and direction of development in the future (openAI orthopedic source) △\'Multi-modal interactive AI\'(Professor Seo Hong-seok Choi) △Efficient code generation technology utilized by generated AI (Seoul University's Hwang Seung-won Lab) △High Density 2D-3D Indoor Forecast (Seoul Kim Gun-hee's Lab) △Language Model Assessment Method (KAIST Professor Seo-G\ The status of the generated AI, such as the creation technology (Yeon-Yeon's final faculty laboratory), and future research directions were also shared.'}] 

# Run the web search tool
results = web_search_tool.invoke({"query": question})
print(results)

# Reset web search tool
from langchain_teddynote.tools.tavily import TavilySearch

# Set maximum search results to 3
web_search_tool = TavilySearch(max_results=3)

• The need for web browsing : When all documents do not meet the relevance threshold or the evaluator is unsure, we will get additional data through web browsing.

• Using Tavily Search : Perform a web search using Tavily Search. This optimizes search queries and provides more relevant results.

• Rewrite question : Improve search queries by rewriting questions to optimize web browsing.

Web search tools Is used to reinforce the context.

Web search tools

 [Original question]: "Explain the creationAI developed by the Samsung." 
[Query Recomposition]: "Please elaborate on the features and features of the generated AI developed by the Samsung." 

# Run and check the results
print(f'[원본 질문]: "{question}"')
print("[쿼리 재작성]:", question_rewriter.invoke({"question": question}))

question_rewriter Use to rewrite the question.

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_openai import ChatOpenAI

# LLM settings
llm = ChatOpenAI(model=MODEL_NAME, temperature=0)

# Query Rewrite System Prompt
system = """You a question re-writer that converts an input question to a better version that is optimized 
for web search. Look at the input and try to reason about the underlying semantic intent / meaning."""

# Prompt definition
re_write_prompt = ChatPromptTemplate.from_messages(
    [
        ("system", system),
        (
            "human",
            "Here is the initial question: \n\n {question} \n Formulate an improved question.",
        ),
    ]
)

# Question Re-writer Chain Initialization
question_rewriter = re_write_prompt | llm | StrOutputParser()

Rewriting queries is a step in rewriting questions to optimize web browsing.

Question Re-writer

 The creation AI'Samsung Gauss' developed by the Samsung Electronics works on the Ondevice and consists of three models: language, code, and images. This model has no risk of user information leakage and plans to be phased in to various products. Samsung Gauss has been learned with secure data, and models of optimized size are available. 

**Source** 
-data/SPRI_AI_Brief_2023 December issue_F.pdf (page 13) 

from langchain import hub
from langchain_core.output_parsers import StrOutputParser
from langchain_openai import ChatOpenAI


# Get RAG prompt from LangChain Hub and use it
prompt = hub.pull("teddynote/rag-prompt")

# LLM Initialization
llm = ChatOpenAI(model_name=MODEL_NAME, temperature=0)


# Document Formatting
def format_docs(docs):
    return "\n\n".join(
        [
            f'<document><content>{doc.page_content}</content><source>{doc.metadata["source"]}</source><page>{doc.metadata["page"]+1}</page></document>'
            for doc in docs
        ]
    )


# create a chain
rag_chain = prompt | llm | StrOutputParser()


# Chain execution and output of results
generation = rag_chain.invoke({"context": format_docs(docs), "question": question})
print(generation)

It's a common Naive RAG chain we know.

The answer generation chain is a chain that generates answers based on the documents retrieved.

Reply generation chain

 binary_score='yes'

# question definition
question = "Explain the generative AI developed by Samsung Electronics."

# Document Search
docs = pdf_retriever.invoke(question)

# Extract the page contents of index document 1 among the searched documents
doc_txt = docs[1].page_content

# Run relevance assessment using retrieved documents and questions and output results
print(retrieval_grader.invoke({"question": question, "document": doc_txt}))

State

First, define the status above the CRAG graph.

web_search Indicates whether to utilize web search. Expressed as yes or no (yes: web search required, no: no required)

from typing import Annotated, List
from typing_extensions import TypedDict


# state definition
class GraphState(TypedDict):
    question: Annotated[str, "The question to answer"]
    generation: Annotated[str, "The generation from the LLM"]
    web_search: Annotated[str, "Whether to add search"]
    documents: Annotated[List[str], "The documents retrieved"]

node

Define the nodes to use for the CRAG graph.

Copyfrom langchain.schema import Document


# Document Search Node
def retrieve(state: GraphState):
    print("\n==== RETRIEVE ====\n")
    question = state["question"]

    # Perform a document search
    documents = pdf_retriever.invoke(question)
    return {"documents": documents}


# Generate Answer Node
def generate(state: GraphState):
    print("\n==== GENERATE ====\n")
    question = state["question"]
    documents = state["documents"]

    # Generating answers using RAG
    generation = rag_chain.invoke({"context": documents, "question": question})
    return {"generation": generation}


# Document Evaluation Node
def grade_documents(state: GraphState):
    print("\n==== [CHECK DOCUMENT RELEVANCE TO QUESTION] ====\n")
    question = state["question"]
    documents = state["documents"]

    # Filtered Documents
    filtered_docs = []
    relevant_doc_count = 0

    for d in documents:
        # Question-Document 의 관련성 평가
        score = retrieval_grader.invoke(
            {"question": question, "document": d.page_content}
        )
        grade = score.binary_score

        if grade == "yes":
            print("==== [GRADE: DOCUMENT RELEVANT] ====")
            # Add relevant documents to filtered_docs
            filtered_docs.append(d)
            relevant_doc_count += 1
        else:
            print("==== [GRADE: DOCUMENT NOT RELEVANT] ====")
            continue

    # If no relevant documentation is found, perform a web search.
    web_search = "Yes" if relevant_doc_count == 0 else "No"
    return {"documents": filtered_docs, "web_search": web_search}


# Query Rewrite Node
def query_rewrite(state: GraphState):
    print("\n==== [REWRITE QUERY] ====\n")
    question = state["question"]

    # Rewrite the question
    better_question = question_rewriter.invoke({"question": question})
    return {"question": better_question}


# Web search node
def web_search(state: GraphState):
    print("\n==== [WEB SEARCH] ====\n")
    question = state["question"]
    documents = state["documents"]

    # Perform a web search
    docs = web_search_tool.invoke({"query": question})
    # Convert search results to document format
    web_results = "\n".join([d["content"] for d in docs])
    web_results = Document(page_content=web_results)
    documents.append(web_results)

    return {"documents": documents}

Functions to utilize for conditional edges

decide_to_generate After completing the relevance assessment, the function serves to route to the next node depending on whether the web is being searched.

web_search end Yes If query_rewrite After rewriting the query in the node, we do a web search.

if, web_search end No In the case of generate Perform to generate the final answer.

def decide_to_generate(state: GraphState):
    # Decide on next steps based on evaluated documents
    print("==== [ASSESS GRADED DOCUMENTS] ====")
    # Need to search the web?
    web_search = state["web_search"]

    if web_search == "Yes":
        # If you need to supplement your information with a web search
        print(
            "==== [DECISION: ALL DOCUMENTS ARE NOT RELEVANT TO QUESTION, QUERY REWRITE] ===="
        )
        # Routing to query rewrite node
        return "query_rewrite"
    else:
        # Since there is a related document, proceed to the generate answer step.
        print("==== [DECISION: GENERATE] ====")
        return "generate"

Graph generation

Now define the node and connect the edges to complete the graph.

Copyfrom langgraph.graph import END, StateGraph, START

# Initialize graph state
workflow = StateGraph(GraphState)

# node definition
workflow.add_node("retrieve", retrieve)
workflow.add_node("grade_documents", grade_documents)
workflow.add_node("generate", generate)
workflow.add_node("query_rewrite", query_rewrite)
workflow.add_node("web_search_node", web_search)

# Edge connection
workflow.add_edge(START, "retrieve")
workflow.add_edge("retrieve", "grade_documents")

# Add conditional edges to the document evaluation node
workflow.add_conditional_edges(
    "grade_documents",
    decide_to_generate,
    {
        "query_rewrite": "query_rewrite",
        "generate": "generate",
    },
)

# Edge connection
workflow.add_edge("query_rewrite", "web_search_node")
workflow.add_edge("web_search_node", "generate")
workflow.add_edge("generate", END)

# Compile the graph
app = workflow.compile()

Visualize the graph.

Copyfrom langchain_teddynote.graphs import visualize_graph

visualize_graph(app)

Graph execution

Now run the graph and check the results.

from langchain_core.runnables import RunnableConfig
from langchain_teddynote.messages import stream_graph, invoke_graph, random_uuid

# config settings (max recursion count, thread_id)
config = RunnableConfig(recursion_limit=20, configurable={"thread_id": random_uuid()})

# Enter your question
inputs = {
    "question": "The name of the generative AI developed by Samsung Electronics is?",
}

# Running graphs in streaming format
stream_graph(
    app,
    inputs,
    config,
    ["retrieve", "grade_documents", "query_rewrite", "web_search_node", "generate"],
)

 ==== RETRIEVE ==== 


==== [CHECK DOCUMENT RELEVANCE TO QUESTION] ==== 


================================================== 
🔄 Node: grade_documents 🔄 
- - - - - - - - - - - - - - - - - - - - - - - - - - - -  
==== [GRADE: DOCUMENT RELEVANT] ==== 
==== [GRADE: DOCUMENT RELEVANT] ==== 
==== [GRADE: DOCUMENT RELEVANT] ==== 
==== [GRADE: DOCUMENT RELEVANT] ==== 
==== [GRADE: DOCUMENT NOT RELEVANT] ==== 
==== [GRADE: DOCUMENT NOT RELEVANT] ==== 
==== [GRADE: DOCUMENT NOT RELEVANT] ==== 
==== [GRADE: DOCUMENT RELEVANT] ==== 
==== [GRADE: DOCUMENT NOT RELEVANT] ==== 
==== [GRADE: DOCUMENT NOT RELEVANT] ==== 
==== [ASSESS GRADED DOCUMENTS] ==== 
==== [DECISION: GENERATE] ==== 

==== GENERATE ==== 


================================================== 
🔄 Node: generate 🔄 
- - - - - - - - - - - - - - - - - - - - - - - - - - - -  
The name of the generated AI developed by the Samsung Electronics is'Samsung Gauss'. 

**Source** 
- data/SPRI_AI_Brief_2023 December _F.pdf (page 12) 

# config settings (max recursion count, thread_id)
config = RunnableConfig(recursion_limit=20, configurable={"thread_id": random_uuid()})

# Enter your question
inputs = {
    "question": "The name of the 2024 Nobel Prize in Literature winner is?",
}

Run the graph.

# running the graph
invoke_graph(app, inputs, config)

 ==== RETRIEVE ==== 


================================================== 
🔄 Node: retrieve 🔄 
- - - - - - - - - - - - - - - - - - - - - - - - - - - -  
page_content='CES 2024 
More than 500 Korean companies will participate in this exhibition. 
Period place homepage 
2024.1.9~12 USA, Las Vegas https://www.ces.tech/ 
-International conference on machine learning and application (AIMLA 2024) 
On the theory, methodology and practical approach of artificial intelligence and machine learning 
Share knowledge and latest research results 
-Key areas of artificial intelligence and mechanical learning in theory and practice 
For discussion, for researchers and practitioners in academia and industry 
AIMLA 2024 
Share the cutting-edge development news 
Period Place Homepage' 
(...sway...) 
Evaluate the AI system of external research groups by using funds to fulfill commitments 
'2.0','data/SPRI_AI_Brief_2023 Year's December','file_path':'data/SPRI_AI_Brief_2023 Year','page': 9,'total_pages' 
================================================== 

==== [CHECK DOCUMENT RELEVANCE TO QUESTION] ==== 

==== [GRADE: DOCUMENT NOT RELEVANT] ==== 
==== [GRADE: DOCUMENT NOT RELEVANT] ==== 
==== [GRADE: DOCUMENT NOT RELEVANT] ==== 
==== [GRADE: DOCUMENT NOT RELEVANT] ==== 
==== [GRADE: DOCUMENT NOT RELEVANT] ==== 
==== [GRADE: DOCUMENT NOT RELEVANT] ==== 
==== [GRADE: DOCUMENT NOT RELEVANT] ==== 
==== [GRADE: DOCUMENT NOT RELEVANT] ==== 
==== [GRADE: DOCUMENT NOT RELEVANT] ==== 
==== [GRADE: DOCUMENT NOT RELEVANT] ==== 
==== [ASSESS GRADED DOCUMENTS] ==== 
==== [DECISION: ALL DOCUMENTS ARE NOT RELEVANT TO QUESTION, QUERY REWRITE] ==== 

================================================== 
🔄 Node: grade_documents 🔄 
- - - - - - - - - - - - - - - - - - - - - - - - - - - -  
web_search: 
Yes 
================================================== 

==== [REWRITE QUERY] ==== 


================================================== 
🔄 Node: query_rewrite 🔄 
- - - - - - - - - - - - - - - - - - - - - - - - - - - -  
question: 
Who is the 2024 Nobel Prize for Literature? 
================================================== 

==== [WEB SEARCH] ==== 


================================================== 
🔄 Node: web_search_node 🔄 
- - - - - - - - - - - - - - - - - - - - - - - - - - - -  
page_content=' Copy text Next news Play ad Read more After the second of the ad SKIP SKIP Next news play Next news Play layer Close Next news view Subtitle / Subtitle Overall, Korean writer, Han Kang was selected as the Nobel Prize for Literature this year. It is the first Nobel Prize for Literature in Korea and the first Asian woman in the history of the Nobel Prize for Literature in 123 years. "The 2024 Nobel Prize for Literature is a Korean writer, Han Kang."   A novelist who wrote'Vegetarian','Boy is coming', Han Kang was the first Nobel Prize in Korean literature.  
(...sway...) 
 『The boy is coming The work called 』 won the 2014 Manchaeological Prize, the 2017 Italian Malaparte Literary Award, and has been translated into over 20 countries around the world. 『published in 2023. 』 was the first Korean to win the French Medici Foreign Literature Award and a year later he received a Nobel Prize in the Nobel Prize.Media outlets breaking the Nobel Prize for Literature by writer Han Kang, along with the title of the first Asian woman, capitalized on the amount of the prize (11 million krona / about 144 million won't be taxed), along with the names of the world literature masters Hemingway, Faulkner, Marquez, and Tony Morrison. There was a writer who trembled alongside the name. 
================================================== 

==== GENERATE ==== 


================================================== 
🔄 Node: generate 🔄 
- - - - - - - - - - - - - - - - - - - - - - - - - - - -  
generation: 
The 2024 Nobel Prize for Literature is Korean writer Han Kang. 

**Source** 
-(Document, page content) 
================================================== 

Run graphs in streaming format.

# running the graph
stream_graph(
    app,
    inputs,
    config,
    ["retrieve", "grade_documents", "query_rewrite", "generate"],
)

 ==== RETRIEVE ==== 


==== [CHECK DOCUMENT RELEVANCE TO QUESTION] ==== 


================================================== 
🔄 Node: grade_documents 🔄 
- - - - - - - - - - - - - - - - - - - - - - - - - - - -  
==== [GRADE: DOCUMENT NOT RELEVANT] ==== 
==== [GRADE: DOCUMENT NOT RELEVANT] ==== 
==== [GRADE: DOCUMENT NOT RELEVANT] ==== 
==== [GRADE: DOCUMENT NOT RELEVANT] ==== 
==== [GRADE: DOCUMENT NOT RELEVANT] ==== 
==== [GRADE: DOCUMENT NOT RELEVANT] ==== 
==== [GRADE: DOCUMENT NOT RELEVANT] ==== 
==== [GRADE: DOCUMENT NOT RELEVANT] ==== 
==== [GRADE: DOCUMENT NOT RELEVANT] ==== 
==== [GRADE: DOCUMENT NOT RELEVANT] ==== 
==== [ASSESS GRADED DOCUMENTS] ==== 
==== [DECISION: ALL DOCUMENTS ARE NOT RELEVANT TO QUESTION, QUERY REWRITE] ==== 

==== [REWRITE QUERY] ==== 


================================================== 
🔄 Node: query_rewrite 🔄 
- - - - - - - - - - - - - - - - - - - - - - - - - - - -  
Who is the 2024 Nobel Prize for Literature? 
==== [WEB SEARCH] ==== 


==== GENERATE ==== 


================================================== 
🔄 Node: generate 🔄 
- - - - - - - - - - - - - - - - - - - - - - - - - - - -  
The 2024 Nobel Prize for Literature is Korean writer Han Kang. 

**Source** 
-(Document, page content)