Building Fault-Tolerant AI Agents with Temporal and LangGraph

Building a simple LLM wrapper is easy. Building a production-grade agentic system that can handle multi-step reasoning, external tool execution, and long-running human-in-the-loop approvals is significantly harder.

When we move from simple 'chaining' to 'agentic' workflows, we introduce a massive reliability gap. In a standard Python script, if your server restarts or an API call times out during a 10-minute agentic reasoning loop, you lose everything. The state is gone, the progress is lost, and the cost of the tokens spent is wasted.

To solve this, we need to merge two distinct concepts: Stateful Logic Orchestration (LangGraph) and Durable Execution (Temporal). This article explores how to integrate these technologies to build agents that are not just smart, but practically indestructible.

The Problem: The Fragility of Agentic State

Agentic workflows are inherently non-linear and long-running. Unlike a standard REST API that responds in milliseconds, an agent might:

1. Search the web for information (30 seconds).
2. Process large documents (2 minutes).
3. Ask a human for clarification (could take hours or days).
4. Execute code in a sandbox (10 seconds).

If any part of the infrastructure fails during these steps—a pod restart in Kubernetes, a network partition, or a rate limit on the LLM—the entire execution context is usually lost. While LangGraph provides excellent tools for managing the logic of these cycles, it doesn't natively solve the infrastructure problem of durable execution across distributed systems.

Enter LangGraph: Logic and Cycles

LangGraph, developed by the LangChain team, excels at defining agents as state machines. Unlike linear chains, LangGraph allows for cycles, which are essential for agents that need to iterate on a problem until a condition is met.

Key features of LangGraph include:

• StateGraph: A way to define nodes (functions) and edges (transitions) where state is passed and updated.
• Persistence: Built-in 'checkpointers' that save the state of the graph after every node execution.
• Human-in-the-loop: The ability to 'interrupt' the graph, wait for input, and resume.

However, LangGraph’s persistence layer is primarily a storage mechanism (like a database). It doesn't handle the orchestration of retries, the scheduling of tasks across workers, or the management of timeouts in a distributed environment. That is where Temporal comes in.

Enter Temporal: The Durable Execution Engine

Temporal is a distributed, stateful engine for executing workflows. It ensures that code is executed exactly once, eventually, regardless of hardware or software failures.

In Temporal:

• Workflows are stateful functions that are persisted via event sourcing.
• Activities are the units of work (e.g., calling an LLM, hitting a database) that have automatic retries and timeouts.
• Workers poll for tasks and execute them.

If a Temporal worker dies mid-workflow, another worker picks up the execution exactly where it left off, reconstructing the state by replaying the event history.

The Architecture: LangGraph as the Brain, Temporal as the Nervous System

To build a fault-tolerant agent, we treat the LangGraph StateGraph as the logic layer and wrap it within a Temporal Workflow. There are two primary patterns for this integration.

Pattern A: The Atomic Activity Pattern

In this pattern, a single LangGraph execution is treated as a Temporal Activity. This is suitable for shorter agentic tasks (under a few minutes) where you want Temporal to handle the high-level retries of the entire agent loop.

# Temporal Activity
@activity.defn
async def run_agent_task(input_data: dict) -> dict:
    graph = compile_my_langgraph()
    final_state = await graph.ainvoke({"messages": [HumanMessage(content=input_data['query'])]})
    return final_state

While simple, this doesn't leverage Temporal's ability to survive crashes during the agent's reasoning process. If the activity fails, it starts from the very beginning of the LangGraph loop.

Pattern B: The Durable Orchestrator Pattern

This is the preferred approach for high-stakes, long-running agents. Here, the Temporal Workflow manages the state transitions, and each LangGraph node (or a group of nodes) is executed as a Temporal Activity.

In this model, we use LangGraph to define the flow, but we use Temporal to execute the steps. This gives us granular recovery: if the agent fails at step 4 of 10, Temporal resumes exactly at step 4.

Implementing Durable State Recovery

The core challenge is syncing LangGraph's checkpoint state with Temporal's history. To do this effectively, we can implement a custom Checkpointer in LangGraph that reads from and writes to a Temporal-managed state or a shared persistent store that Temporal acknowledges.

Step 1: Defining the LangGraph State

class AgentState(TypedDict):
    messages: Annotated[Sequence[BaseMessage], operator.add]
    research_done: bool
    approval_required: bool

Step 2: The Temporal Workflow

The Temporal Workflow acts as the long-lived container. It handles the 'Human-in-the-loop' signals and manages the overall lifecycle.

@workflow.defn
class AgenticWorkflow:
    @workflow.run
    async def run(self, initial_query: str):
        state = {"messages": [HumanMessage(content=initial_query)], "research_done": False}
        
        # Loop through LangGraph logic manually managed by Temporal
        while not state.get("finished"):
            # Execute a 'Reasoning' activity
            state = await workflow.execute_activity(
                run_reasoning_node,
                state,
                start_to_close_timeout=timedelta(minutes=5)
            )
            
            if state.get("approval_required"):
                # Temporal waits here indefinitely without consuming resources
                # It survives restarts until a signal is received
                approval = await workflow.wait_condition(lambda: self.user_approved)
                state["messages"].append(ChatMessage(content=f"User approved: {approval}"))
        
        return state

Handling Human-in-the-Loop with Signals

One of the most powerful features of combining these two is handling human intervention. In LangGraph, you might use a breakpoint. In a distributed system, that breakpoint needs to survive for potentially days.

Temporal's Signals are perfect for this. A user can send a signal to a running workflow (via a UI or Slack bot). The workflow, which has been 'sleeping' and consuming zero CPU/RAM, wakes up, processes the signal, and continues the LangGraph execution.

This eliminates the need for complex polling logic or manual state management in your database. The state is the code.

Practical Considerations for Senior Engineers

1. Determinism and Replay

Temporal relies on determinism for workflow replay. When embedding LangGraph, ensure that any non-deterministic actions (like calling an LLM or generating a UUID) happen inside Activities, not the Workflow logic itself. The Workflow should only coordinate the activities.

2. State Size

LLM histories can become large. Temporal's history has a size limit (usually 50MB). If your agentic conversation is massive, do not store the entire message history in the Temporal workflow state. Instead, store the history in a database (like Postgres or MongoDB) and pass only the thread_id or a summary between the Workflow and Activities.

3. Versioning

Agent logic evolves quickly. You might change your LangGraph prompt or add a new node. Temporal provides robust workflow versioning, allowing you to run 'v1' of your agent for existing executions while routing new requests to 'v2'. This is critical for long-running agents that shouldn't break mid-flight when you deploy new code.

Real-World Example: The Refund Agent

Imagine a 'Refund Processing Agent' for an e-commerce platform:

1. Node 1 (LangGraph): Analyze customer sentiment and policy. If the refund is > $100, set approval_required = True.
2. Temporal Check: The workflow sees approval_required. It pauses and sends an email to a manager.
3. External Event: Three days later, the manager clicks 'Approve'.
4. Temporal Signal: The signal hits the workflow.
5. Node 2 (LangGraph): The agent resumes, calls the Stripe API to process the refund, and drafts a confirmation email.

If the Stripe API is down, Temporal automatically retries with exponential backoff. If the server running the agent crashes during the email drafting, Temporal moves the task to another server. The customer never experiences a 'lost' request.

Conclusion: Why This Matters

Moving AI agents from 'cool demos' to 'mission-critical infrastructure' requires a shift in how we think about state. LangGraph provides the sophisticated cyclic reasoning needed for intelligence, but Temporal provides the durability needed for reliability.

Actionable Next Steps:

1. Audit your current agents: Identify where they fail when a network timeout occurs or a process restarts.
2. Decouple Logic from Execution: Use LangGraph to define your state transitions and tool-calling logic.
3. Wrap with Temporal: Use Temporal Workflows to manage the high-level lifecycle, especially for any steps involving human approval or multiple external API calls.
4. Implement Activities for LLM Calls: Always wrap LLM calls in Temporal Activities to benefit from automatic retries and observability.

By layering these two technologies, you build a system where the AI can be 'creative' and iterative, while the underlying infrastructure remains boringly predictable and resilient.