Sairis

Appearance

Context vs. RAG: How They Work Together ​

Prompt Engineering 106

Introduction ​

Understanding how context and retrieval-augmented generation (RAG) interact is essential for designing effective AI interactions in Sairis. While both concepts influence AI responses, they serve distinct roles in guiding and shaping outputs.

  • Context refers to what the AI remembers within a conversation, including structured instructions (metaprompts) and conversation history.
  • RAG allows the AI to retrieve knowledge from external sources, such as knowledge bases and specific documents, to expand its understanding beyond conversation memory.

Knowing when to leverage context vs. when to enable RAG search is key to ensuring that AI provides accurate, relevant, and structured responses aligned with business needs.

Learning Objectives ​

By the end of this course, you will be able to:

  1. Differentiate between AI context and RAG-powered knowledge retrieval
    Understand how conversation history, instructions, and RAG interact in Sairis.

  2. Recognize when to use context vs. RAG
    Learn when to embed context within a prompt versus when to retrieve knowledge from external sources.

  3. Apply best practices for managing context and knowledge retrieval in AI interactions
    Ensure AI responses are structured, relevant, and efficient.

Understanding Context in AI ​

Context plays a critical role in shaping AI behavior and ensuring that responses remain coherent, relevant, and goal-oriented. In Sairis, context can be categorized into two types:

1. Context in Metaprompts (Instructions) ​

This refers to predefined guidance given to the AI that persists across interactions within a session. It includes:

  • Role definitions - Assigning the AI a specific function (e.g., "You are a legal assistant providing contract summaries.").
  • Behavioral guidelines - Ensuring AI follows specific rules (e.g., "Always provide responses in a professional tone.").
  • Response structure formatting - Standardizing output consistency (e.g., "Summarize findings in three key bullet points.").

Example:

"You are an HR specialist. Always provide policy guidance based on company regulations.  
When answering questions, structure your responses using:  
- Policy Reference 
- Explanation 
- Recommended Action."

2. Context as Conversation History ​

The AI remembers previous exchanges within a session, allowing it to build on past responses and maintain continuity. However, once the conversation ends, this memory resets unless information is explicitly stored elsewhere (e.g., Blueprints).

Example:

  • User (Message 1): "Summarize the key findings from the annual sales report."
  • AI Response: "The report highlights a 12% revenue increase, improved conversion rates, and market expansion into two new regions."
  • User (Message 2): "Great. Can you now compare this to last year's performance?"
  • AI understands the request because the prior message is still within the conversation's context.

Pro Tip: If your AI forgets key details mid-conversation, restate necessary context within your prompt to improve accuracy.

Understanding RAG (Retrieval-Augmented Generation) ​

RAG enhances AI's ability to pull relevant information from external sources instead of relying solely on conversation memory or pre-trained knowledge.

How RAG Works in Sairis ​

  1. Users select knowledge bases (information domains) to be AI-searchable.
  2. The AI retrieves relevant content from those sources when responding to a prompt.
  3. The AI generates a response based on retrieved knowledge, integrating it into the conversation.

Benefits of RAG ​

Accesses up-to-date company knowledge (e.g., policy updates, proprietary documents).
Reduces hallucination risks by ensuring AI only pulls from verified knowledge sources.
Enhances AI expertise beyond pre-trained general knowledge.

Example:

  • User Input (with RAG enabled): "What is our company's current PTO policy?"
  • AI searches the HR knowledge base and retrieves the latest policy document.
  • AI Response: "According to the latest HR guidelines, employees receive 20 PTO days per year, with an option to carry over up to 5 days."

When to Use Context vs. RAG ​

Use Context When: ​

  • The AI needs to maintain conversation continuity (e.g., remembering user preferences in-session).
  • Instructions must define AI behavior and formatting (e.g., structured output templates).
  • The required information is static or rule-based (e.g., "Always use a polite, professional tone.").

Use RAG When: ​

  • AI must retrieve specific knowledge beyond what is contained in session memory.
  • The response requires up-to-date or document-based information.
  • Users need dynamic knowledge lookups (e.g., "Find our latest customer support policy.").

Best Practices for Managing Context & RAG ​

1. Structuring Prompts for Context Clarity ​

  • Use clear and structured instructions to set AI behavior.
  • Restate key details in prompts when needed to reinforce context.
  • Be explicit about format, scope, and expectations.

2. Optimizing RAG Retrieval ​

  • Choose the right knowledge bases to ensure AI pulls from accurate sources.
  • Limit retrieval scope to prevent AI from pulling unnecessary details.
  • Test RAG responses to verify accuracy and avoid conflicting information.

Pro Tip: If AI provides irrelevant RAG results, refine the knowledge base selection or adjust retrieval parameters.

Examples in Practice ​

Example 1: Context-Driven AI Behavior ​

John needs AI to follow a structured format in responses. He sets an instruction in the metaprompt:

"You are an executive assistant.   
Always format meeting summaries as follows:   
- Key Discussion Points   
- Action Items   
- Follow-Up Steps   
- Assigned Responsibilities."

Every AI response follows this format, ensuring structured and useful summaries.

Sarah asks AI for a policy update. Instead of relying on memory, RAG retrieves the most recent document:

User: "What is our latest cybersecurity policy?"   
AI: "Let me check the company's security knowledge base...   
According to the latest cybersecurity policy (updated March 2024):   
- All employees must enable multi-factor authentication.   
- Passwords must be updated every 90 days.   
- External device access is restricted."

Since Sarah enabled RAG for security policies, the AI provides an accurate and up-to-date response.

Key Takeaways ​

  • Context and RAG serve different roles in AI interactions.
    • Context = Maintains conversation history and predefined AI instructions.
    • RAG = Retrieves external knowledge for dynamic, document-based responses.
  • Use context when AI needs to remember session details and follow structured behavior.
  • Use RAG when AI must retrieve factual, up-to-date knowledge from external sources.
  • Optimizing context structuring and RAG retrieval ensures AI accuracy, consistency, and efficiency.

By mastering when and how to use context vs. RAG, you can fine-tune AI performance in Sairis, ensuring that responses align with both business processes and knowledge governance.