LangGraph ile Düzeltici RAG (CRAG) Uygulaması

Düzeltici RAG (CRAG) Nedir?

Düzeltici bilgi getirme destekli üretim (CRAG), dil modellerini daha doğru hale getirmeyi amaçlayan RAG’in geliştirilmiş bir sürümüdür.

Geleneksel RAG, metin üretimine yardımcı olmak için yalnızca getirilen belgeleri kullanırken, CRAG bunların ilgili ve doğru olduğundan emin olmak için bu belgeleri aktif olarak kontrol edip iyileştirerek bir adım daha ileri gider. Bu, modelin yanlış veya yanıltıcı bilgi ürettiği halüsinasyonları ve hataları azaltmaya yardımcı olur.

Kaynak: Shi-Qi Yan ve diğ., 2024

CRAG çerçevesi, bir getirme değerlendiricisi ve belirli düzeltici eylemleri içeren birkaç temel adım üzerinden çalışır.

Belirli bir girdi sorgusu için, standart bir getirici önce bilgi tabanından bir dizi belge çeker. Bu belgeler daha sonra her belgenin sorguyla ilişkisini belirlemek için bir getirme değerlendiricisi tarafından incelenir.

CRAG’de getirme değerlendiricisi, ince ayar yapılmış bir T5-large modelidir. Değerlendirici, her belgeye bir güven puanı atar ve bunları üç güven düzeyine ayırır:

1. Doğru: En az bir belge üst eşik değerinin üzerinde puan alırsa doğru kabul edilir. Sistem daha sonra, belgeler içindeki ilgisiz veya gürültülü verileri filtrelerken en önemli ve ilgili bilgi şeritlerini çıkarmak için ayrıştır-sonra-yeniden birleştir algoritmasını kullanan bir bilgi iyileştirme süreci uygular. Bu, üretim süreci için yalnızca en doğru ve ilgili bilginin tutulmasını sağlar.
2. Yanlış: Tüm belgeler alt eşik değerinin altında kalırsa yanlış olarak işaretlenir. Bu durumda CRAG getirilen tüm belgeleri atar ve bunun yerine daha doğru olabilecek yeni harici bilgi toplamak için bir web araması gerçekleştirir. Bu adım, web’deki geniş ve dinamik bilgiden yararlanarak getirme sürecini statik veya sınırlı bilgi tabanının ötesine taşır ve ilgili ve doğru verileri getirme olasılığını artırır.
3. Belirsiz: Getirilen belgeler karışık sonuçlar içerdiğinde belirsiz kabul edilir. Bu durumda CRAG her iki stratejiyi birleştirir: Başta getirilen belgelerden bilgiyi iyileştirir ve web aramalarından elde edilen ek bilgiyi dahil eder.

Bu eylemlerden biri uygulandıktan sonra, iyileştirilmiş bilgi nihai yanıtı üretmek için kullanılır.

CRAG ve geleneksel RAG

CRAG, geleneksel RAG’e göre birkaç önemli iyileştirme sunar. En büyük avantajlarından biri, getirdiği bilgideki hataları düzeltebilme yeteneğidir. CRAG’deki getirme değerlendiricisi, bilginin yanlış veya ilgisiz olduğunu tespit ederek bunun nihai çıktıyı etkilemeden önce düzeltilmesine yardımcı olur. Bu, CRAG’in daha doğru ve güvenilir bilgi sağlaması anlamına gelir; hataları ve yanlış bilgileri azaltır.

CRAG ayrıca bilginin hem ilgili hem de doğru olmasını sağlamada da öne çıkar. Geleneksel RAG yalnızca alaka puanlarını kontrol edebilecekken, CRAG belgeleri yalnızca ilgili değil aynı zamanda isabetli olacak şekilde iyileştirerek daha ileri gider. İlgisiz ayrıntıları ayıklar ve en önemli noktalara odaklanır; böylece üretilen metin doğru bilgiye dayanır.

LangGraph Kullanarak CRAG Uygulaması

Bu bölümde, LangGraph kullanarak CRAG’in nasıl uygulanacağına dair adım adım bir kılavuzdan geçeceğiz. Ortamınızı nasıl kuracağınızı, temel bir bilgi vektör deposu oluşturmayı ve CRAG için gereken kilit bileşenleri (getirme değerlendiricisi, soru yeniden yazıcı ve web aracı) yapılandırmayı öğreneceksiniz.

Ayrıca tüm bu parçaları bir araya getiren bir LangGraph iş akışının nasıl kurulacağını öğrenecek ve CRAG’in daha doğru ve güvenilir sonuçlar için çeşitli soru türlerini nasıl yönetebildiğini göstereceğiz.

Adım 1: Kurulum ve yükleme

Önce gerekli paketleri yükleyin. Bu adım, CRAG hattını çalıştırmak için ortamı hazırlar.

pip install langchain_community tiktoken langchain-openai langchainhub chromadb langchain langgraph tavily-python

Ardından Tavily ve OpenAI için API anahtarlarınızı yapılandırın:

import os
os.environ["TAVILY_API_KEY"] = "YOUR_TAVILY_API_KEY"
os.environ["OPENAI_API_KEY"] = "YOUR_OPENAI_API_KEY"

Adım 2: Vekil bir bilgi tabanı kurun

RAG gerçekleştirmek için önce belgelerle dolu bir bilgi tabanına ihtiyacımız var. Bu adımda bir Substack bülteninden örnek belgeler kazıyıp bir vektör deposu oluşturacağız; bu depo vekil bilgi tabanımız olarak işlev görecek. Vektör deposu, kullanıcı sorgularına göre ilgili belgeleri bulmamıza yardımcı olur.

Sağlanan URL’lerden belgeleri yükleyerek başlıyor ve bir metin ayırıcı ile daha küçük parçalara bölüyoruz. Bu bölümler daha sonra OpenAIEmbeddings kullanılarak gömülür ve belgelerin verimli şekilde getirilmesi için bir vektör veritabanında (Chroma) depolanır.

from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_community.document_loaders import WebBaseLoader
from langchain_community.vectorstores import Chroma
from langchain_openai import OpenAIEmbeddings
urls = [
    "<https://ryanocm.substack.com/p/mystery-gift-box-049-law-1-fill-your>",
    "<https://ryanocm.substack.com/p/105-the-bagel-method-in-relationships>",
    "<https://ryanocm.substack.com/p/098-i-have-read-100-productivity>",
]
docs = [WebBaseLoader(url).load() for url in urls]
docs_list = [item for sublist in docs for item in sublist]
text_splitter = RecursiveCharacterTextSplitter.from_tiktoken_encoder(
    chunk_size=250, chunk_overlap=0
)
doc_splits = text_splitter.split_documents(docs_list)
# Add to vectorDB
vectorstore = Chroma.from_documents(
    documents=doc_splits,
    collection_name="rag-chroma",
    embedding=OpenAIEmbeddings(),
)
retriever = vectorstore.as_retriever()

Adım 3: Bir RAG zinciri kurun

Bu adımda, bir kullanıcının sorusunu ve bir dizi belgeyi alıp bir yanıt üreten temel bir RAG zinciri kuruyoruz.

RAG zinciri, önceden tanımlı bir istem ve getirilen belgelere dayalı yanıtlar oluşturmak için bir dil modeli (GPT 4-o mini) kullanır. Bir çıktı ayrıştırıcısı da üretilen metni okunması kolay olacak şekilde biçimlendirir.

### Generate
from langchain import hub
from langchain_core.output_parsers import StrOutputParser
# Prompt
rag_prompt = hub.pull("rlm/rag-prompt")
# LLM
rag_llm = ChatOpenAI(model_name="gpt-4o-mini", temperature=0)
# Post-processing
def format_docs(docs):
    return "\\n\\n".join(doc.page_content for doc in docs)
# Chain
rag_chain = rag_prompt | rag_llm | StrOutputParser()
print(rag_prompt.messages[0].prompt.template)

You are an assistant for question-answering tasks. Use the following pieces of retrieved context to answer the question. If you don't know the answer, just say that you don't know. Use three sentences maximum and keep the answer concise.
Question: {question} 
Context: {context} 
Answer:

generation = rag_chain.invoke({"context": docs, "question": question})
print("Question: %s" % question)
print("----")
print("Documents:\\n")
print('\\n\\n'.join(['- %s' % x.page_content for x in docs]))
print("----")
print("Final answer: %s" % generation)

Question: what is the bagel method
----
Documents:
- the book was called The Bagel Method.The Bagel Method is designed to help partners be on the same team when dealing with differences and trying to find a compromise.The idea behind the method is that, to truly compromise, we need to figure out a way to include both partners’ dreams and core needs; things that are super important to us that giving up on them is too much.Let’s dive into the bagel 😜🚀 If you are new here…Hi, I’m Ryan ğŸ‘&lsaqua;ğŸ�¼ I am passionate about lifestyle gamification ğŸ�® and I am obsesssssssss with learning things that can help me live a happy and fulfilling life.And so, with The Limitless Playbook newsletter, I will share with you 1 actionable idea from the world's top thinkers every Sunday ğŸ�¯So visit us weekly for highly actionable insights :)…or even better, subscribe below and have all these information send straight to your inbox every Sunday 🥳Subscribe🥯
- #105 | The Bagel Method in Relationships 🥯
- The Limitless Playbook 🧬SubscribeSign inShare this post#105 | The Bagel Method in Relationships 🥯ryanocm.substack.comCopy linkFacebookEmailNoteOther#105 | The Bagel Method in Relationships 🥯A collection of the best hidden gems, mental models, and frameworks from the world’s top thinkers; to help you become 1% better and live a happier life ��Ryan Ong �Feb 25, 2024Share this post#105 | The Bagel Method in Relationships 🥯ryanocm.substack.comCopy linkFacebookEmailNoteOtherShareHello curious minds 🧠I recently finished the book Fight Right: How Successful Couples Turn Conflict into Connection and oh my days, I love every chapter of it!There were many repeating concepts but this time, it was applied in the context of conflicts in relationships. As usual with the Gottman’s books, I highlighted the hell out of the entire book 😄One of the cool exercises in
- The Bagel MethodThe Bagel Method involves mapping out your core needs and areas of flexibility so that you and your partner understand what's important and where there's room for flexibility.It’s called The Bagel Method because, just like a bagel, it has both the inner and outer circles representing your needs.Here are the steps:In the inner circle, list all the aspects of an issue that you can’t give in on. These are your non-negotiables that are usually very closely related to your core needs and dreams.In the outer circle, list all the aspects of an issue that you are able to compromise on IF you are able to have what’s in your inner circle.Now, talk to your partners about your inner and outer circle. Ask each other:Why are the things in your inner circle so important to you?How can I support your core needs here?Tell me more about your areas of flexibility. What does it look like to be flexible?Compare both your “bagel� of needsWhat do we agree on?What feelings do we have in common?What shared goals do we have?How might we accomplish these goals
----
Final answer: The Bagel Method is a relationship strategy that helps partners identify their core needs and areas where they can be flexible. It involves mapping out non-negotiables in the inner circle and compromise areas in the outer circle, facilitating open communication about each partner's priorities. This method aims to foster understanding and collaboration in resolving differences.

Adım 4: Bir getirme değerlendiricisi kurun

Üretilen içeriğin doğruluğunu artırmak için bir getirme değerlendiricisi kuruyoruz. Bu araç, yalnızca en faydalı bilginin kullanılmasını sağlamak için getirilen her belgenin ne kadar ilgili olduğunu kontrol eder.

Getirme değerlendiricisi bir istem ve dil modeliyle yapılandırılır. Belgelerin ilgili olup olmadığını belirler ve yanıt üretilmeden önce ilgisiz içeriği filtreler.

### Retrieval Evaluator
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.pydantic_v1 import BaseModel, Field
from langchain_openai import ChatOpenAI
# Data model
class RetrievalEvaluator(BaseModel):
    """Classify retrieved documents based on how relevant it is to the user's question."""
    binary_score: str = Field(
        description="Documents are relevant to the question, 'yes' or 'no'"
    )
# LLM with function call
retrieval_evaluator_llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)
structured_llm_evaluator = retrieval_evaluator_llm.with_structured_output(RetrievalEvaluator)
# Prompt
system = """You are a document retrieval evaluator that's responsible for checking the relevancy of a retrieved document to the user's question. \\n 
    If the document contains keyword(s) or semantic meaning related to the question, grade it as relevant. \\n
    Output a binary score 'yes' or 'no' to indicate whether the document is relevant to the question."""
retrieval_evaluator_prompt = ChatPromptTemplate.from_messages(
    [
        ("system", system),
        ("human", "Retrieved document: \\n\\n {document} \\n\\n User question: {question}"),
    ]
)
retrieval_grader = retrieval_evaluator_prompt | structured_llm_evaluator

Adım 5: Bir soru yeniden yazıcı kurun

Arama sürecini iyileştirmek için kullanıcı sorgularını daha açık ve spesifik hale getirecek bir soru yeniden yazıcı ekleyeceğiz.

Yeniden yazıcı, orijinal sorguyu daha odaklı hale getirerek daha iyi ve daha ilgili sonuçlara yol açar.

### Question Re-writer
# LLM
question_rewriter_llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)
# Prompt
system = """You are a question re-writer that converts an input question to a better version that is optimized \\n 
     for web search. Look at the input and try to reason about the underlying semantic intent / meaning."""
re_write_prompt = ChatPromptTemplate.from_messages(
    [
        ("system", system),
        (
            "human",
            "Here is the initial question: \\n\\n {question} \
 Formulate an improved question.",
        ),
    ]
)
question_rewriter = re_write_prompt | question_rewriter_llm | StrOutputParser()

Adım 6: Tavily web arama aracını başlatın

Bilgi tabanı yeterli bilgiye sahip değilse CRAG boşlukları doldurmak için web aramasına başvurur. Bu, olası bilgi kaynaklarının yelpazesini genişletir. Bu adımda, web’i aramak ve ek belgeler bulmak için Tavily API’sini kullanıyoruz.

### Search
from langchain_community.tools.tavily_search import TavilySearchResults
web_search_tool = TavilySearchResults(k=3)

Adım 7: LangGraph iş akışını kurun

LangGraph ile CRAG iş akışını oluşturmak için şu üç ana adımı izleyin:

1. Graf durumunu tanımlayın
2. İşlev düğümlerini tanımlayın
3. Tüm işlev düğümlerini bağlayın

Graf durumunu tanımlayın

İş akışı sırasında düğümler arasında hareket ederken verileri depolamak için paylaşılan bir durum oluşturun. Bu durum; kullanıcının sorusu, getirilen belgeler ve üretilen yanıtlar gibi tüm değişkenleri tutacaktır.

from typing import List
from typing_extensions import TypedDict
class GraphState(TypedDict):
    """
    Represents the state of our graph.
    Attributes:
        question: question
        generation: LLM generation
        web_search: whether to add search
        documents: list of documents
    """
    question: str
    generation: str
    web_search: str
    documents: List[str]

İşlev düğümlerini tanımlayın

LangGraph iş akışında her işlev düğümü, CRAG hattındaki belirli bir görevi üstlenir; örneğin belgeleri getirme, yanıt üretme, alaka değerlendirme, sorgu dönüştürme ve web araması. İşte her işlevin özeti:

retrieve işlevi, kullanıcının sorusuyla ilgili belgeleri bilgi tabanından bulur. Genellikle ön işlenmiş belgelerden oluşturulan bir vektör deposu olan bir getirici nesnesi kullanır. Bu işlev, kullanıcının sorusunu içeren mevcut durumu alır, getiriciyle ilgili belgeleri getirir ve bu belgeleri duruma ekler.

from langchain.schema import Document
def retrieve(state):
    """
    Retrieve documents
    Args:
        state (dict): The current graph state
    Returns:
        state (dict): New key added to state, documents, that contains retrieved documents
    """
    print("---RETRIEVE---")
    question = state["question"]
    # Retrieval
    documents = retriever.get_relevant_documents(question)
    return {"documents": documents, "question": question}

generate işlevi, getirilen belgeleri kullanarak kullanıcının sorusuna bir yanıt oluşturur. Bir istemi bir dil modeliyle birleştiren RAG zinciriyle çalışır. Bu işlev, getirilen belgeleri ve kullanıcının sorusunu alır, bunları RAG zincirinde işler ve ardından yanıtı duruma ekler.

def generate(state):
    """
    Generate answer
    Args:
        state (dict): The current graph state
    Returns:
        state (dict): New key added to state, generation, that contains LLM generation
    """
    print("---GENERATE---")
    question = state["question"]
    documents = state["documents"]
    # RAG generation
    generation = rag_chain.invoke({"context": documents, "question": question})
    return {"documents": documents, "question": question, "generation": generation}

evaluate_documents işlevi, getirilen her belgenin kullanıcının sorusuyla ne kadar ilgili olduğunu getirme değerlendiricisiyle kontrol eder. Bu, nihai yanıt için yalnızca faydalı bilginin kullanılmasını sağlar. İşlev, her belgenin alaka düzeyini derecelendirip işe yaramayanları eler. Ayrıca belgelerin çoğu ilgili değilse web araması gerekip gerekmediğini göstermek için durumu web_search bayrağıyla günceller.

def evaluate_documents(state):
    """
    Determines whether the retrieved documents are relevant to the question.
    Args:
        state (dict): The current graph state
    Returns:
        state (dict): Updates documents key with only filtered relevant documents
    """
    print("---CHECK DOCUMENT RELEVANCE TO QUESTION---")
    question = state["question"]
    documents = state["documents"]
    # Score each doc
    filtered_docs = []
    web_search = "No"
    for d in documents:
        score = retrieval_grader.invoke(
            {"question": question, "document": d.page_content}
        )
        grade = score.binary_score
        if grade == "yes":
            print("---GRADE: DOCUMENT RELEVANT---")
            filtered_docs.append(d)
        else:
            print("---GRADE: DOCUMENT NOT RELEVANT---")
            continue
    if len(filtered_docs) / len(documents) <= 0.7:
        web_search = "Yes"
    return {"documents": filtered_docs, "question": question, "web_search": web_search}

transform_query işlevi, özellikle orijinal sorgu ilgili belgeler bulamıyorsa daha iyi arama sonuçları elde etmek için kullanıcının sorusunu iyileştirir. Soruyu daha net ve spesifik hale getirmek için bir soru yeniden yazıcı kullanır. Daha iyi bir soru, hem bilgi tabanından hem de web aramalarından faydalı belgeler bulma şansını artırır.

def transform_query(state):
    """
    Transform the query to produce a better question.
    Args:
        state (dict): The current graph state
    Returns:
        state (dict): Updates question key with a re-phrased question
    """
    print("---TRANSFORM QUERY---")
    question = state["question"]
    documents = state["documents"]
    # Re-write question
    better_question = question_rewriter.invoke({"question": question})
    return {"documents": documents, "question": better_question}

web_search işlevi, iyileştirilmiş sorguyu kullanarak çevrimiçi ek bilgi arar. Bilgi tabanında yeterli bilgi olmadığında kullanılır ve daha fazla içerik toplamaya yardımcı olur. Bu işlev, web’de ek belgeler bulmak için Tavily web arama aracını kullanır; ardından bilgi tabanını iyileştirmek için mevcut belgelere bu sonuçlar eklenir.

def web_search(state):
    """
    Web search based on the re-phrased question.
    Args:
        state (dict): The current graph state
    Returns:
        state (dict): Updates documents key with appended web results
    """
    print("---WEB SEARCH---")
    question = state["question"]
    documents = state["documents"]
    # Web search
    docs = web_search_tool.invoke({"query": question})
    web_results = "\\n".join([d["content"] for d in docs])
    web_results = Document(page_content=web_results)
    documents.append(web_results)
    return {"documents": documents, "question": question}

decide_to_generate işlevi bir sonraki adımı belirler: mevcut belgelerle bir yanıt mı üretilecek, yoksa sorgu iyileştirilip tekrar mı aranacak? Bu seçimi, (önceden değerlendirilen) belgelerin alaka düzeyine göre yapar.

def decide_to_generate(state):
    """
    Determines whether to generate an answer, or re-generate a question.
    Args:
        state (dict): The current graph state
    Returns:
        str: Binary decision for next node to call
    """
    print("---ASSESS GRADED DOCUMENTS---")
    state["question"]
    web_search = state["web_search"]
    state["documents"]
    if web_search == "Yes":
        # All documents have been filtered check_relevance
        # We will re-generate a new query
        print(
            "---DECISION: ALL DOCUMENTS ARE NOT RELEVANT TO QUESTION, TRANSFORM QUERY---"
        )
        return "transform_query"
    else:
        # We have relevant documents, so generate answer
        print("---DECISION: GENERATE---")
        return "generate"

Tüm işlev düğümlerini bağlayın

Tüm işlev düğümleri tanımlandıktan sonra, CRAG hattını oluşturmak için LangGraph iş akışında bunları birbirine bağlayabiliriz. Bu, düğümleri kenarlarla bağlayarak bilgi ve karar akışını yönetmek; iş akışının her adımın sonucuna göre doğru şekilde çalışmasını sağlamak anlamına gelir.

from langgraph.graph import END, StateGraph, START
workflow = StateGraph(GraphState)
# Define the nodes
workflow.add_node("retrieve", retrieve)  # retrieve
workflow.add_node("grade_documents", evaluate_documents)  # evaluate documents
workflow.add_node("generate", generate)  # generate
workflow.add_node("transform_query", transform_query)  # transform_query
workflow.add_node("web_search_node", web_search)  # web search
# Build graph
workflow.add_edge(START, "retrieve")
workflow.add_edge("retrieve", "grade_documents")
workflow.add_conditional_edges(
    "grade_documents",
    decide_to_generate,
    {
        "transform_query": "transform_query",
        "generate": "generate",
    },
)
workflow.add_edge("transform_query", "web_search_node")
workflow.add_edge("web_search_node", "generate")
workflow.add_edge("generate", END)
# Compile
app = workflow.compile()

from IPython.display import Image, display
try:
    display(Image(app.get_graph(xray=True).draw_mermaid_png()))
except Exception:
    # This requires some extra dependencies and is optional
    pass

Adım 8: İş akışını test etme

Kurulumumuzu test etmek için, nasıl bilgi getirdiğini, belge alakasını nasıl değerlendirdiğini ve nasıl yanıtlar ürettiğini görmek üzere iş akışını örnek sorgularla çalıştırıyoruz.

İlk sorgu, CRAG’in bilgi tabanı içinde ne kadar iyi yanıt bulduğunu kontrol eder.

from pprint import pprint
# Run
inputs = {"question": "What's the bagel method?"}
for output in app.stream(inputs):
    for key, value in output.items():
        # Node
        pprint(f"Node '{key}':")
        # Optional: print full state at each node
        pprint(value, indent=2, width=80, depth=None)
    pprint("\\n---\\n")
# Final generation
pprint(value["generation"])

---RETRIEVE---
"Node 'retrieve':"
{ 'documents': [ Document(page_content="the book was called The Bagel Method.The Bagel Method is designed to help partners be on the same team when dealing with differences and trying to find a compromise.The idea behind the method is that, to truly compromise, we need to figure out a way to include both partners’ dreams and core needs; things that are super important to us that giving up on them is too much.Let’s dive into the bagel 😜🚀 If you are new here…Hi, I’m Ryan 👋� I am passionate about lifestyle gamification � and I am obsesssssssss with learning things that can help me live a happy and fulfilling life.And so, with The Limitless Playbook newsletter, I will share with you 1 actionable idea from the world's top thinkers every Sunday �So visit us weekly for highly actionable insights :)…or even better, subscribe below and have all these information send straight to your inbox every Sunday 🥳Subscribe🥯", metadata={'description': 'A collection of the best hidden gems, mental models, and frameworks from the world’s top thinkers; to help you become 1% better and live a happier life ��', 'language': 'en', 'source': '<https://ryanocm.substack.com/p/105-the-bagel-method-in-relationships>', 'title': '#105 | The Bagel Method in Relationships 🥯'}),
                 Document(page_content='#105 | The Bagel Method in Relationships 🥯', metadata={'description': 'A collection of the best hidden gems, mental models, and frameworks from the world’s top thinkers; to help you become 1% better and live a happier life ��', 'language': 'en', 'source': '<https://ryanocm.substack.com/p/105-the-bagel-method-in-relationships>', 'title': '#105 | The Bagel Method in Relationships 🥯'}),
                 Document(page_content='The Limitless Playbook 🧬SubscribeSign inShare this post#105 | The Bagel Method in Relationships 🥯ryanocm.substack.comCopy linkFacebookEmailNoteOther#105 | The Bagel Method in Relationships 🥯A collection of the best hidden gems, mental models, and frameworks from the world’s top thinkers; to help you become 1% better and live a happier life â�¤ï¸�Ryan Ong ğŸ�®Feb 25, 2024Share this post#105 | The Bagel Method in Relationships 🥯ryanocm.substack.comCopy linkFacebookEmailNoteOtherShareHello curious minds ğŸ§\\xa0I recently finished the book Fight Right: How Successful Couples Turn Conflict into Connection and oh my days, I love every chapter of it!There were many repeating concepts but this time, it was applied in the context of conflicts in relationships. As usual with the Gottman’s books, I highlighted the hell out of the entire book 😄One of the cool exercises in', metadata={'description': 'A collection of the best hidden gems, mental models, and frameworks from the world’s top thinkers; to help you become 1% better and live a happier life â�¤ï¸�', 'language': 'en', 'source': '<https://ryanocm.substack.com/p/105-the-bagel-method-in-relationships>', 'title': '#105 | The Bagel Method in Relationships 🥯'}),
                 Document(page_content="The Bagel MethodThe Bagel Method involves mapping out your core needs and areas of flexibility so that you and your partner understand what's important and where there's room for flexibility.It’s called The Bagel Method because, just like a bagel, it has both the inner and outer circles representing your needs.Here are the steps:In the inner circle, list all the aspects of an issue that you can’t give in on. These are your non-negotiables that are usually very closely related to your core needs and dreams.In the outer circle, list all the aspects of an issue that you are able to compromise on IF you are able to have what’s in your inner circle.Now, talk to your partners about your inner and outer circle. Ask each other:Why are the things in your inner circle so important to you?How can I support your core needs here?Tell me more about your areas of flexibility. What does it look like to be flexible?Compare both your “bagel� of needsWhat do we agree on?What feelings do we have in common?What shared goals do we have?How might we accomplish these goals", metadata={'description': 'A collection of the best hidden gems, mental models, and frameworks from the world’s top thinkers; to help you become 1% better and live a happier life ��', 'language': 'en', 'source': '<https://ryanocm.substack.com/p/105-the-bagel-method-in-relationships>', 'title': '#105 | The Bagel Method in Relationships 🥯'})],
  'question': "What's the bagel method?"}
'\\n---\\n'
---CHECK DOCUMENT RELEVANCE TO QUESTION---
---GRADE: DOCUMENT RELEVANT---
---GRADE: DOCUMENT RELEVANT---
---GRADE: DOCUMENT NOT RELEVANT---
---GRADE: DOCUMENT RELEVANT---
---ASSESS GRADED DOCUMENTS---
---DECISION: GENERATE---
"Node 'evaluate_documents':"
{ 'documents': [ Document(page_content="the book was called The Bagel Method.The Bagel Method is designed to help partners be on the same team when dealing with differences and trying to find a compromise.The idea behind the method is that, to truly compromise, we need to figure out a way to include both partners’ dreams and core needs; things that are super important to us that giving up on them is too much.Let’s dive into the bagel 😜🚀 If you are new here…Hi, I’m Ryan 👋� I am passionate about lifestyle gamification � and I am obsesssssssss with learning things that can help me live a happy and fulfilling life.And so, with The Limitless Playbook newsletter, I will share with you 1 actionable idea from the world's top thinkers every Sunday �So visit us weekly for highly actionable insights :)…or even better, subscribe below and have all these information send straight to your inbox every Sunday 🥳Subscribe🥯", metadata={'description': 'A collection of the best hidden gems, mental models, and frameworks from the world’s top thinkers; to help you become 1% better and live a happier life ��', 'language': 'en', 'source': '<https://ryanocm.substack.com/p/105-the-bagel-method-in-relationships>', 'title': '#105 | The Bagel Method in Relationships 🥯'}),
                 Document(page_content='#105 | The Bagel Method in Relationships 🥯', metadata={'description': 'A collection of the best hidden gems, mental models, and frameworks from the world’s top thinkers; to help you become 1% better and live a happier life ��', 'language': 'en', 'source': '<https://ryanocm.substack.com/p/105-the-bagel-method-in-relationships>', 'title': '#105 | The Bagel Method in Relationships 🥯'}),
                 Document(page_content="The Bagel MethodThe Bagel Method involves mapping out your core needs and areas of flexibility so that you and your partner understand what's important and where there's room for flexibility.It’s called The Bagel Method because, just like a bagel, it has both the inner and outer circles representing your needs.Here are the steps:In the inner circle, list all the aspects of an issue that you can’t give in on. These are your non-negotiables that are usually very closely related to your core needs and dreams.In the outer circle, list all the aspects of an issue that you are able to compromise on IF you are able to have what’s in your inner circle.Now, talk to your partners about your inner and outer circle. Ask each other:Why are the things in your inner circle so important to you?How can I support your core needs here?Tell me more about your areas of flexibility. What does it look like to be flexible?Compare both your “bagel� of needsWhat do we agree on?What feelings do we have in common?What shared goals do we have?How might we accomplish these goals", metadata={'description': 'A collection of the best hidden gems, mental models, and frameworks from the world’s top thinkers; to help you become 1% better and live a happier life ��', 'language': 'en', 'source': '<https://ryanocm.substack.com/p/105-the-bagel-method-in-relationships>', 'title': '#105 | The Bagel Method in Relationships 🥯'})],
  'question': "What's the bagel method?",
  'web_search': 'No'}
'\\n---\\n'
---GENERATE---
"Node 'generate':"
{ 'documents': [ Document(page_content="the book was called The Bagel Method.The Bagel Method is designed to help partners be on the same team when dealing with differences and trying to find a compromise.The idea behind the method is that, to truly compromise, we need to figure out a way to include both partners’ dreams and core needs; things that are super important to us that giving up on them is too much.Let’s dive into the bagel 😜🚀 If you are new here…Hi, I’m Ryan 👋� I am passionate about lifestyle gamification � and I am obsesssssssss with learning things that can help me live a happy and fulfilling life.And so, with The Limitless Playbook newsletter, I will share with you 1 actionable idea from the world's top thinkers every Sunday �So visit us weekly for highly actionable insights :)…or even better, subscribe below and have all these information send straight to your inbox every Sunday 🥳Subscribe🥯", metadata={'description': 'A collection of the best hidden gems, mental models, and frameworks from the world’s top thinkers; to help you become 1% better and live a happier life ��', 'language': 'en', 'source': '<https://ryanocm.substack.com/p/105-the-bagel-method-in-relationships>', 'title': '#105 | The Bagel Method in Relationships 🥯'}),
                 Document(page_content='#105 | The Bagel Method in Relationships 🥯', metadata={'description': 'A collection of the best hidden gems, mental models, and frameworks from the world’s top thinkers; to help you become 1% better and live a happier life ��', 'language': 'en', 'source': '<https://ryanocm.substack.com/p/105-the-bagel-method-in-relationships>', 'title': '#105 | The Bagel Method in Relationships 🥯'}),
                 Document(page_content="The Bagel MethodThe Bagel Method involves mapping out your core needs and areas of flexibility so that you and your partner understand what's important and where there's room for flexibility.It’s called The Bagel Method because, just like a bagel, it has both the inner and outer circles representing your needs.Here are the steps:In the inner circle, list all the aspects of an issue that you can’t give in on. These are your non-negotiables that are usually very closely related to your core needs and dreams.In the outer circle, list all the aspects of an issue that you are able to compromise on IF you are able to have what’s in your inner circle.Now, talk to your partners about your inner and outer circle. Ask each other:Why are the things in your inner circle so important to you?How can I support your core needs here?Tell me more about your areas of flexibility. What does it look like to be flexible?Compare both your “bagel� of needsWhat do we agree on?What feelings do we have in common?What shared goals do we have?How might we accomplish these goals", metadata={'description': 'A collection of the best hidden gems, mental models, and frameworks from the world’s top thinkers; to help you become 1% better and live a happier life ��', 'language': 'en', 'source': '<https://ryanocm.substack.com/p/105-the-bagel-method-in-relationships>', 'title': '#105 | The Bagel Method in Relationships 🥯'})],
  'generation': 'The Bagel Method is a framework designed to help partners '
                'navigate differences and find compromises by mapping out '
                'their core needs and areas of flexibility. It involves '
                'creating two circles: the inner circle for non-negotiable '
                'needs and the outer circle for aspects where compromise is '
                'possible. This method encourages open communication about '
                "each partner's priorities and shared goals.",
  'question': "What's the bagel method?"}
'\\n---\\n'
('The Bagel Method is a framework designed to help partners navigate '
 'differences and find compromises by mapping out their core needs and areas '
 'of flexibility. It involves creating two circles: the inner circle for '
 'non-negotiable needs and the outer circle for aspects where compromise is '
 "possible. This method encourages open communication about each partner's "
 'priorities and shared goals.')

İkinci sorgu ise, bilgi tabanı ilgili belgelere sahip olmadığında CRAG’in ek bilgi bulmak için web’i arama yeteneğini test eder.

from pprint import pprint
# Run
inputs = {"question": "What is prompt engineering?"}
for output in app.stream(inputs):
    for key, value in output.items():
        # Node
        pprint(f"Node '{key}':")
        # Optional: print full state at each node
        pprint(value, indent=2, width=80, depth=None)
    pprint("\\n---\\n")
# Final generation
pprint(value["generation"])

---RETRIEVE---
"Node 'retrieve':"
{ 'documents': [ Document(page_content='Mystery Gift Box #049 | Law 1: Fill your Five Buckets in the Right Order (The Diary of a CEO)', metadata={'description': "The best hidden gems I've found; interesting ideas and concepts, thought-provoking questions, mind-blowing books/podcasts, cool animes/films, and other mysteries ��", 'language': 'en', 'source': '<https://ryanocm.substack.com/p/mystery-gift-box-049-law-1-fill-your>', 'title': 'Mystery Gift Box #049 | Law 1: Fill your Five Buckets in the Right Order (The Diary of a CEO)'}),
                 Document(page_content='ğŸ�¦Â\\xa0Twitter, 👨ğŸ�»â€�💻Â\\xa0LinkedIn, ğŸŒ�Â\\xa0Personal Website, and 📸Â\\xa0InstagramShare this postMystery Gift Box #049 | Law 1: Fill your Five Buckets in the Right Order (The Diary of a CEO)ryanocm.substack.comCopy linkFacebookEmailNoteOtherSharePreviousNextCommentsTopLatestDiscussionsNo postsReady for more?Subscribe© 2024 Ryan Ong ğŸ�®Privacy ∙ Terms ∙ Collection notice Start WritingGet the appSubstack is the home for great cultureShareCopy linkFacebookEmailNoteOther', metadata={'description': "The best hidden gems I've found; interesting ideas and concepts, thought-provoking questions, mind-blowing books/podcasts, cool animes/films, and other mysteries â�¤ï¸�", 'language': 'en', 'source': '<https://ryanocm.substack.com/p/mystery-gift-box-049-law-1-fill-your>', 'title': 'Mystery Gift Box #049 | Law 1: Fill your Five Buckets in the Right Order (The Diary of a CEO)'}),
                 Document(page_content='skills are the foundation of which you build your life and career and it’s truly yours to own; you can lose your network, resources, and reputation but you will never lose your knowledge and skills.Never try to skip the first two buckets. If you try to jump straight to network, resources, and / or reputation bucket, you might “succeed� in the short-run but in the long run, your lack of knowledge and skill will catch on to you.There is no skipping the first two buckets of knowledge and skills if you’re playing long-term sustainable results. Any attempt to do so is equivalent to building your house on sand.💥 Key takeawayFocus on using your knowledge and skills to create lots of values in the world and the world will reward you with growing network (people will come to you), resources (people will pay for your services), and reputation (people will know what you are capable of).⛰ 4-4-4 Exploration ProjectEach month, I would explore one new thing; a skill, a subject, or an experience.January 2023: Writing and Storytelling (Subject)', metadata={'description': "The best hidden gems I've found; interesting ideas and concepts, thought-provoking questions, mind-blowing books/podcasts, cool animes/films, and other mysteries ��", 'language': 'en', 'source': '<https://ryanocm.substack.com/p/mystery-gift-box-049-law-1-fill-your>', 'title': 'Mystery Gift Box #049 | Law 1: Fill your Five Buckets in the Right Order (The Diary of a CEO)'}),
                 Document(page_content='This site requires JavaScript to run correctly. Please turn on JavaScript or unblock scripts', metadata={'description': 'A collection of the best hidden gems, mental models, and frameworks from the world’s top thinkers; to help you become 1% better and live a happier life ��', 'language': 'en', 'source': '<https://ryanocm.substack.com/p/105-the-bagel-method-in-relationships>', 'title': '#105 | The Bagel Method in Relationships 🥯'})],
  'question': 'What is prompt engineering?'}
'\\n---\\n'
---CHECK DOCUMENT RELEVANCE TO QUESTION---
---GRADE: DOCUMENT NOT RELEVANT---
---GRADE: DOCUMENT NOT RELEVANT---
---GRADE: DOCUMENT NOT RELEVANT---
---GRADE: DOCUMENT NOT RELEVANT---
---ASSESS GRADED DOCUMENTS---
---DECISION: ALL DOCUMENTS ARE NOT RELEVANT TO QUESTION, TRANSFORM QUERY---
"Node 'evaluate_documents':"
{ 'documents': [],
  'question': 'What is prompt engineering?',
  'web_search': 'Yes'}
'\\n---\\n'
---TRANSFORM QUERY---
"Node 'transform_query':"
{ 'documents': [],
  'question': 'What is the concept of prompt engineering and how is it applied '
              'in artificial intelligence?'}
'\\n---\\n'
---WEB SEARCH---
"Node 'web_search_node':"
{ 'documents': [ Document(page_content="Prompt engineering is the method to ask generative AI to produce what the individual needs. There are two main principles of building a successful prompt for any AI, specificity and iteration. The box below includes one example of a framework that can be applied when prompting any generative AI.\\nPrompt Engineering is the process of designing and refining text inputs (prompts) to achieve specific application objectives with AI models. Think of it as a two-step journey: Designing the Initial Prompt: Creating the initial input for the model to achieve the desired result. Refining the Prompt: Continuously adjusting the prompt to enhance ...\\nPrompt engineering is the process of designing and refining the inputs given to language models, like those in AI, to achieve desired outputs more effectively. It involves creatively crafting prompts that guide the model in generating responses that are accurate, relevant, and aligned with the user's intentions.. For students and researchers in higher education, mastering prompt engineering is ...\\nJune 25, 2024. Prompt engineering means writing precise instructions for AI models. These instructions are different from coding because they use natural language. And today, everybody does it—from software developers to artists and content creators. Prompt engineering can help you improve productivity and save time by automating repetitive ...\\nMaster Prompt Engineering - The (AI) Prompt\\nAI Takes Wall Street by Storm: C3.ai's Strong Forecast Sparks a Surge in AI Stocks\\nAsk Me Anything (AMA) Prompting\\nHow Self-Critique Improves Logic and Reasoning in LLMs Like ChatGPT\\nOptimizing Large Language Models to Maximize Performance\\nThe Black Box Problem: Opaque Inner Workings of Large Language Models\\nHow to Evaluate Large Language Models for Business Tasks\\nIntroduction to the AI Prompt Development Process\\nSubscribe to new posts\\nThe Official Source For Everything Prompt Engineering & Generative AI Defining Prompt Engineering\\nGiven that the prompt is the singular input channel to large language models, prompt engineering can be defined as:\\nPrompt Engineering can be thought of as any process that contributes to the development of a well-crafted prompt to generate quality, useful outputs from an AI system.\\n A Simplified Approach to Defining Prompt Engineering\\nThe Prompt is the Sole Input\\nWhen interacting with Generative AI Models such as large language models (LLMs), the prompt is the only thing that gets input into the AI system. Its applications cut across diverse sectors, from healthcare and education to business, securing its place as a cornerstone of our interactions with AI.\\nExploration of Essential Prompt Engineering Techniques and Concepts\\nIn the rapidly evolving landscape of Artificial Intelligence (AI), mastering key techniques of Prompt Engineering has become increasingly vital. The key concepts of Prompt Engineering include prompts and prompting the AI, training the AI, developing and maintaining a prompt library, and testing, evaluation, and categorization.\\n")],
  'question': 'What is the concept of prompt engineering and how is it applied '
              'in artificial intelligence?'}
'\\n---\\n'
---GENERATE---
"Node 'generate':"
{ 'documents': [ Document(page_content="Prompt engineering is the method to ask generative AI to produce what the individual needs. There are two main principles of building a successful prompt for any AI, specificity and iteration. The box below includes one example of a framework that can be applied when prompting any generative AI.\\nPrompt Engineering is the process of designing and refining text inputs (prompts) to achieve specific application objectives with AI models. Think of it as a two-step journey: Designing the Initial Prompt: Creating the initial input for the model to achieve the desired result. Refining the Prompt: Continuously adjusting the prompt to enhance ...\\nPrompt engineering is the process of designing and refining the inputs given to language models, like those in AI, to achieve desired outputs more effectively. It involves creatively crafting prompts that guide the model in generating responses that are accurate, relevant, and aligned with the user's intentions.. For students and researchers in higher education, mastering prompt engineering is ...\\nJune 25, 2024. Prompt engineering means writing precise instructions for AI models. These instructions are different from coding because they use natural language. And today, everybody does it—from software developers to artists and content creators. Prompt engineering can help you improve productivity and save time by automating repetitive ...\\nMaster Prompt Engineering - The (AI) Prompt\\nAI Takes Wall Street by Storm: C3.ai's Strong Forecast Sparks a Surge in AI Stocks\\nAsk Me Anything (AMA) Prompting\\nHow Self-Critique Improves Logic and Reasoning in LLMs Like ChatGPT\\nOptimizing Large Language Models to Maximize Performance\\nThe Black Box Problem: Opaque Inner Workings of Large Language Models\\nHow to Evaluate Large Language Models for Business Tasks\\nIntroduction to the AI Prompt Development Process\\nSubscribe to new posts\\nThe Official Source For Everything Prompt Engineering & Generative AI Defining Prompt Engineering\\nGiven that the prompt is the singular input channel to large language models, prompt engineering can be defined as:\\nPrompt Engineering can be thought of as any process that contributes to the development of a well-crafted prompt to generate quality, useful outputs from an AI system.\\n A Simplified Approach to Defining Prompt Engineering\\nThe Prompt is the Sole Input\\nWhen interacting with Generative AI Models such as large language models (LLMs), the prompt is the only thing that gets input into the AI system. Its applications cut across diverse sectors, from healthcare and education to business, securing its place as a cornerstone of our interactions with AI.\\nExploration of Essential Prompt Engineering Techniques and Concepts\\nIn the rapidly evolving landscape of Artificial Intelligence (AI), mastering key techniques of Prompt Engineering has become increasingly vital. The key concepts of Prompt Engineering include prompts and prompting the AI, training the AI, developing and maintaining a prompt library, and testing, evaluation, and categorization.\\n")],
  'generation': 'Prompt engineering is the process of designing and refining '
                'text inputs to guide AI models in generating desired outputs. '
                'It focuses on specificity and iteration to create effective '
                'prompts that align with user intentions. This technique is '
                'widely applicable across various sectors, enhancing '
                'productivity and automating tasks.',
  'question': 'What is the concept of prompt engineering and how is it applied '
              'in artificial intelligence?'}
'\\n---\\n'
('Prompt engineering is the process of designing and refining text inputs to '
 'guide AI models in generating desired outputs. It focuses on specificity and '
 'iteration to create effective prompts that align with user intentions. This '
 'technique is widely applicable across various sectors, enhancing '
 'productivity and automating tasks.')

CRAG’in Sınırlamaları

CRAG, geleneksel RAG’i geliştirse de dikkat gerektiren bazı sınırlamalara sahiptir.

Önemli sorunlardan biri, getirme değerlendiricisinin kalitesine bağımlılığıdır. Bu değerlendirici, getirilen belgelerin ilgili ve doğru olup olmadığını yargılamak için esastır. Ancak değerlendiricinin eğitilmesi ve ince ayar yapılması talepkârdır; çok miktarda yüksek kaliteli veri ve hesaplama gücü gerektirir. Değerlendiriciyi yeni sorgu türleri ve veri kaynaklarıyla güncel tutmak karmaşıklığı ve maliyeti artırır.

Başka bir sınırlama, CRAG’in yanlış veya belirsiz belgeleri bulmak ya da değiştirmek için web aramalarını kullanmasıdır. Bu yaklaşım daha güncel ve çeşitli bilgi sağlayabilse de önyargılı veya güvenilmez verileri dahil etme riski taşır. Web içeriğinin kalitesi büyük ölçüde değişkendir ve en doğru bilgiyi bulmak için bunu ayıklamak zor olabilir. Bu da, iyi eğitilmiş bir değerlendiricinin bile düşük kaliteli veya önyargılı bilgilerin dahil edilmesini tamamen engelleyemeyebileceği anlamına gelir.

Bu zorluklar, sürekli araştırma ve geliştirme ihtiyacını vurgular.

Sonuç

Genel olarak CRAG, getirilen bilgileri kontrol edip iyileştiren özellikler ekleyerek geleneksel RAG sistemlerini geliştirir; böylece dil modellerini daha doğru ve güvenilir hale getirir. Bu da CRAG’i pek çok farklı uygulama için faydalı bir araç yapar.

CRAG hakkında daha fazla bilgi edinmek için orijinal makaleye buradan göz atın.

RAG hakkında daha fazla öğrenme kaynağı arıyorsanız şu blogları öneririm:

• Llama 3.1 8B, Ollama ve Langchain ile RAG
• RAG için Yeniden Sıralama Aracısı olarak RankGPT
• Bir Bilgi Grafiği Kullanarak RAG Uygulaması Gerçekleştirme