Agent Core

From chatbot to acting agent — before MCP, before LangGraph

Prerequisites: LLM + RAG.

Why LLM alone is not an Agent

• No action — only emits text unless given tools
• No persistent memory — need RAG + store (LangGraph memory)
• No multi-step plan — one generation pass vs loops

ReAct framework

Reason + Act — paper: Yao et al., 2022

Loop until the model emits a final answer (no more tool_calls). Typical max iterations: 5–15.

ReAct trace: book a flight

Minimal loop (pure Python)

MAX_STEPS = 10
messages = [{"role": "user", "content": user_query}]

for step in range(MAX_STEPS):
    response = client.chat.completions.create(
        model="gpt-4o-mini", messages=messages, tools=TOOLS)
    msg = response.choices[0].message
    messages.append(msg)

    if not msg.tool_calls:
        return msg.content  # final answer

    for call in msg.tool_calls:
        result = run_tool(call.function.name, json.loads(call.function.arguments))
        messages.append({"role": "tool", "tool_call_id": call.id, "content": result})

Tool calling in practice

1. Define JSON Schema (OpenAI format)

tools = [{
  "type": "function",
  "function": {
    "name": "search_docs",
    "description": "Search internal knowledge base",
    "parameters": {
      "type": "object",
      "properties": {
        "query": {"type": "string"},
        "top_k": {"type": "integer", "default": 5}
      },
      "required": ["query"]
    }
  }
}]

2. Tool design checklist

3. Connect RAG as a tool (not inline)

def search_docs(query: str, top_k: int = 5) -> str:
    chunks = retriever.invoke(query)[:top_k]
    return json.dumps([{"id": c.metadata["id"], "text": c.page_content} for c in chunks])

Agent decides when to retrieve — useful for multi-step tasks ("compare refund vs exchange policy" → two searches).

Intent recognition & routing

First gate: send query to the right subsystem

Router implementation (LLM classifier)

ROUTER_PROMPT = '''Classify intent. Reply JSON only: {"intent": "...", "confidence": 0.0-1.0}
Intents: knowledge_qa | transaction | chitchat | escalation'''

def route(user_msg: str) -> str:
    r = llm.invoke(ROUTER_PROMPT + "\nUser: " + user_msg)
    intent = json.loads(r.content)["intent"]
    if intent == "knowledge_qa":
        return rag_chain.invoke(user_msg)
    if intent == "transaction":
        return agent_loop.invoke(user_msg)
    return llm.invoke(user_msg)  # chitchat

Alternative routers

• Embedding classifier — cheap, fast; train on labeled intents
• Rules + regex — order IDs, "track shipment"
• LangGraph conditional edge — see LangChain & LangGraph

Multi-turn context management

Token budget assembly (priority order)

1. System rules & safety instructions (never drop)
2. Current task state JSON (slots, pending tool results)
3. Retrieved docs / RAG chunks (top-k by relevance)
4. Recent conversation turns (newest first until budget full)
5. Summary of older turns (if summarize strategy)

Sliding window + summarize pattern

TOKEN_BUDGET = 12000

def build_context(history, retrieved, system):
    msgs = [system]
    msgs += retrieved  # RAG chunks as system or tool messages
    recent, old = split_at_token_limit(history, TOKEN_BUDGET * 0.6)
    if old:
        summary = llm.invoke(f"Summarize this conversation:\n{old}")
        msgs.append({"role": "system", "content": f"Earlier summary: {summary}"})
    msgs += recent
    return trim_to_budget(msgs, TOKEN_BUDGET)

Hands-on lab: build a ReAct agent

~60 min — no LangGraph required

pip install openai python-dotenv

1. Two tools: search_faq (hard-coded dict or RAG) + create_ticket (append to JSON file).
2. Implement the loop from #react-example; cap at 8 steps; log each Thought/Action/Observation.
3. Add router: chitchat bypasses tools; FAQ questions must call search_faq first.
4. Test cases: (a) "Hi" → no tool; (b) "Refund policy?" → search; (c) "File ticket for broken item" → create_ticket after confirm.
5. Context test: 10-turn conversation; verify turn 11 still knows order ID from turn 3 (sliding window or summary).
6. Failure injection: return tool error — agent should recover and explain to user.

Success criteria

Next: tune retrieval in Retrieval Engineering; frameworks in LangChain & LangGraph.

Agent evaluation

Offline eval (before deploy)

# Simple tool accuracy eval
def eval_tool_choice(test_cases, agent_fn):
    correct = 0
    for case in test_cases:
        trace = agent_fn(case["query"], return_trace=True)
        if trace["tools_called"] == case["expected_tools"]:
            correct += 1
    return correct / len(test_cases)

Online eval (production)

• Human thumbs up/down on completed tasks
• Escalation rate — % handed to human support
• Tool error rate — API failures / bad args
• Latency P95 — total loop time including tools

Benchmarks: SWE-bench (code), WebArena (web), AgentBench. Cross-course matrix: Evaluation Guide.