---
title: "Context Graphs and Decision Traces: Building the Data Plane for Agentic AI"
date: "2025-12-27"
author: "Pixeltable Team"
tags:
  - AI Agents
  - Context Graphs
  - Decision Traces
  - Infrastructure
  - Enterprise AI
  - Data Lineage
description: "The next trillion-dollar platforms won't just store data. They'll capture decision traces. Learn how Pixeltable provides the infrastructure for building context graphs that make AI agent reasoning auditable, reproducible, and improvable."
url: "https://pixeltable.com/blog/context-graphs-decision-traces-agentic-ai"
---

# Context Graphs and Decision Traces: Building the Data Plane for Agentic AI

The last generation of enterprise software created a trillion-dollar ecosystem by becoming systems of record. Salesforce for customers. Workday for employees. SAP for operations. The next generation will be built on something different: **systems of record for decisions**, not just objects.
 

 

 Foundation Capital's recent analysis of [context graphs as AI's trillion-dollar opportunity](https://foundationcapital.com/context-graphs-ais-trillion-dollar-opportunity/) crystallizes what many AI infrastructure teams are discovering: agents don't just need data. They need *decision traces*. The reasoning that connects data to action. The context that explains not just *what* happened, but *why it was allowed to happen*.
 

 

 This is where Pixeltable fits in. Not as another agent framework or orchestration layer, but as the **data plane that makes context graphs possible**: the infrastructure that captures, versions, and makes queryable every decision trace your AI systems produce.
 

 
## The Missing Layer: Decision Traces as Data

 

 Here's the distinction that matters:
 

 

 - **Rules** tell an agent what should happen in general ("use official ARR for reporting")

 - **Decision traces** capture what happened in this specific case ("we used X definition, under policy v3.2, with a VP exception, based on precedent Z")

 

 

 Traditional systems store the outcome: "20% discount applied." They don't store the reasoning: the three SEV-1 incidents from PagerDuty, the open escalation in Zendesk, the prior approval precedent from last quarter, the policy version evaluated, and the approver who signed off. That context, the decision trace, dies in Slack threads and people's heads.
 

 

 Pixeltable treats decision traces as first-class data. Every computation, every LLM call, every transformation is automatically versioned and linked in a queryable [dependency graph](/blog/dependency-graph-magic-computed-columns). When your agent makes a decision, you don't just get the output. You get the complete lineage of how that output was derived.
 

 
## Why Architecture Matters: Read Path vs. Write Path

 

 The Foundation Capital analysis identifies a critical architectural distinction: warehouses like Snowflake and Databricks sit in the **read path**: they receive data via ETL *after* decisions are made. By the time data lands in the warehouse, the decision context is gone.
 

 
> 
 "A system that only sees reads, after the fact, can't be the system of record for decision lineage. It can tell you what happened, but it can't tell you why."
 

 

 Pixeltable sits in the **write path**. Computations happen as data arrives, not after. When you define a computed column that calls an LLM, Pixeltable doesn't just store the result. It captures the full context at decision time:
 

 
```python
import pixeltable as pxt
from pixeltable.functions.openai import chat_completions

# Create a table that captures decision traces
decisions = pxt.create_table('ops.escalation_decisions', {
 'ticket_id': pxt.String,
 'customer_tier': pxt.String,
 'open_incidents': pxt.Json, # From PagerDuty
 'escalation_history': pxt.Json, # From Zendesk
 'churn_signals': pxt.Json, # From Slack/CRM
 'policy_version': pxt.String,
})

# Build the decision context from multiple sources
decisions.add_computed_column(
 decision_context=pxt.functions.string.format(
 'Incidents: {0}, History: {1}, Signals: {2}',
 decisions.open_incidents.astype(pxt.String),
 decisions.escalation_history.astype(pxt.String),
 decisions.churn_signals.astype(pxt.String)
 )
)

# The agent's reasoning is captured as a computed column
@pxt.udf
def build_prompt(policy: str, context: str) -> list[dict]:
 return [
 {'role': 'system', 'content': f'Policy version: {policy}. Evaluate escalation.'},
 {'role': 'user', 'content': context}
 ]

decisions.add_computed_column(
 prompt=build_prompt(decisions.policy_version, decisions.decision_context)
)

decisions.add_computed_column(
 agent_response=chat_completions(
 model='gpt-4o',
 messages=decisions.prompt
 )
)

decisions.add_computed_column(
 agent_recommendation=decisions.agent_response.choices[0].message.content
)
```

 

 Every row in this table is a complete decision trace. The inputs, the policy version, the prompt sent to the LLM, the full response, all captured at decision time, all versioned, all queryable. Six months later, you can ask: "Show me all escalation decisions for healthcare customers where the agent recommended override, along with the exact context it saw."
 

 
## Dependency Graphs Are Context Graphs

 

 Foundation Capital describes context graphs as "a living record of decision traces stitched across entities and time so precedent becomes searchable." This is exactly what [Pixeltable's dependency graph](/blog/dependency-graph-magic-computed-columns) provides, but built into the infrastructure layer rather than bolted on after the fact.
 

 

 When you create computed columns in Pixeltable, you're not just defining transformations. You're building a context graph:
 

 
```python
import pixeltable as pxt
from pixeltable.functions.openai import transcriptions

# The dependency graph captures the full decision lineage
support_analysis = pxt.create_table('support.ticket_analysis', {
 'ticket': pxt.Json,
 'customer_data': pxt.Json,
 'conversation_audio': pxt.Audio,
})

# Each computed column adds a node to the context graph
support_analysis.add_computed_column(
 transcript=transcriptions(
 audio=support_analysis.conversation_audio,
 model='whisper-1'
 )
)

@pxt.udf
def analyze_sentiment(text: str) -> dict:
 # Your sentiment analysis logic
 return {'score': 0.8, 'label': 'positive'}

support_analysis.add_computed_column(
 sentiment=analyze_sentiment(support_analysis.transcript.text)
)

@pxt.udf
def calculate_churn_risk(sentiment: dict, customer: dict) -> float:
 # Combine signals into risk score
 base_risk = 1.0 - sentiment.get('score', 0.5)
 tier_modifier = 0.8 if customer.get('tier') == 'enterprise' else 1.0
 return base_risk * tier_modifier

support_analysis.add_computed_column(
 churn_risk=calculate_churn_risk(
 support_analysis.sentiment,
 support_analysis.customer_data
 )
)

@pxt.udf
def agent_decide(transcript: str, risk: float, customer: dict) -> str:
 # Agent decision logic
 if risk > 0.7:
 return 'escalate_immediately'
 elif risk > 0.4:
 return 'schedule_followup'
 return 'standard_response'

support_analysis.add_computed_column(
 recommended_action=agent_decide(
 support_analysis.transcript.text,
 support_analysis.churn_risk,
 support_analysis.customer_data
 )
)
```

 

 The dependency graph shows exactly how `recommended_action` was derived: from the transcript (which came from the audio), combined with churn_risk (which came from sentiment analysis and customer data). This isn't metadata bolted on after the fact. It's the native structure of how Pixeltable processes data.
 

 
## Time Travel for Decisions: Replaying Context

 

 The Foundation Capital article notes that Salesforce "knows what the opportunity looks like now, not what it looked like when the decision was made." You can't replay the state of the world at decision time.
 

 

 [Pixeltable's automatic versioning](/blog/pixeltable-versioning-time-travel) solves this. Every change to every row is tracked. You can query the exact state of your context graph at any point in time:
 

 
```python
# Access the current table
decisions = pxt.get_table('ops.escalation_decisions')

# Time travel to a specific version
historical_decisions = pxt.get_table('ops.escalation_decisions:15')

# Query historical data with full context
past_decision = historical_decisions.where(
 historical_decisions.ticket_id == 'TICKET-4521'
).select(
 historical_decisions.ticket_id,
 historical_decisions.decision_context,
 historical_decisions.policy_version,
 historical_decisions.agent_recommendation
).collect()

# Compare how decisions changed across versions
current_version = decisions.select(
 decisions.ticket_id,
 decisions.agent_recommendation
).collect()

# Revert if a policy change caused issues
decisions.revert(version=15)
```

 

 This is what makes decision traces auditable. Not "here's what the system says now," but "here's exactly what the agent saw, with this policy version, using this model, at this moment in time."
 

 
## Multimodal Context: Beyond Text

 

 Here's where Pixeltable's unique architecture becomes even more relevant. Enterprise decision context isn't just text. It's:
 

 

 - The support call recording where the customer threatened to churn

 - The screenshot of the error message they submitted

 - The video walkthrough from the sales demo

 - The PDF contract with the custom terms

 

 

 Traditional systems can't treat multimodal artifacts as first-class decision context. Pixeltable can. It's built for [multimodal AI applications](/blog/building-multimodal-apps) from the ground up:
 

 
```python
deal_decisions = pxt.create_table('sales.deal_decisions', {
 'deal_id': pxt.String,
 'proposal_doc': pxt.Document,
 'demo_recording': pxt.Video,
 'competitor_screenshot': pxt.Image,
 'call_recording': pxt.Audio,
})

# Extract context from every modality
@pxt.udf
def extract_key_terms(doc_text: str) -> dict:
 # Extract pricing, terms, SLA from document
 return {'pricing': '...', 'terms': '...'}

deal_decisions.add_computed_column(
 doc_text=pxt.functions.document.extract_text(deal_decisions.proposal_doc)
)

deal_decisions.add_computed_column(
 proposal_summary=extract_key_terms(deal_decisions.doc_text)
)

# Transcribe and analyze the call
deal_decisions.add_computed_column(
 call_transcript=transcriptions(
 audio=deal_decisions.call_recording,
 model='whisper-1'
 )
)

# Use vision model on competitor screenshot
from pixeltable.functions.openai import chat_completions

@pxt.udf
def build_vision_prompt(img_url: str) -> list[dict]:
 return [{
 'role': 'user',
 'content': [
 {'type': 'text', 'text': 'Analyze this competitor pricing screenshot'},
 {'type': 'image_url', 'image_url': {'url': img_url}}
 ]
 }]

deal_decisions.add_computed_column(
 competitor_analysis=chat_completions(
 model='gpt-4o',
 messages=build_vision_prompt(deal_decisions.competitor_screenshot.localpath)
 ).choices[0].message.content
)

# The agent decision has full multimodal context
@pxt.udf
def pricing_agent(proposal: dict, transcript: str, competitor: str) -> dict:
 # Combine all context for pricing decision
 return {
 'recommended_discount': 15,
 'reasoning': 'Based on competitor pricing and customer sentiment...',
 'confidence': 0.85
 }

deal_decisions.add_computed_column(
 pricing_recommendation=pricing_agent(
 deal_decisions.proposal_summary,
 deal_decisions.call_transcript.text,
 deal_decisions.competitor_analysis
 )
)
```

 

 The decision trace includes everything: the extracted contract terms, the call transcript, the competitor analysis. All versioned. All queryable. All linked in the dependency graph.
 

 
## From Decision Traces to Searchable Precedent

 

 The real power of context graphs emerges over time. As Foundation Capital notes: "Captured decision traces become searchable precedent. And every automated decision adds another trace to the graph."
 

 

 With Pixeltable's [incremental embedding indexes](/blog/pixeltable-incremental-embedding-indexes), you can build this precedent search directly into your agent workflows:
 

 
```python
from pixeltable.functions.huggingface import sentence_transformer

# Create an embedding index over past decisions
decisions.add_embedding_index(
 column_name='decision_context',
 embedding=sentence_transformer.using(
 model_id='sentence-transformers/all-MiniLM-L6-v2'
 )
)

# New decisions can search for relevant precedent
# Find similar past decisions using vector similarity
sim_score = decisions.decision_context.similarity(string='healthcare customer with incident history')

similar_cases = decisions.order_by(sim_score, asc=False).limit(5).select(
 decisions.ticket_id,
 decisions.customer_tier,
 decisions.agent_recommendation,
 decisions.policy_version,
 similarity=sim_score
).collect()

# Build precedent-aware agent decisions
@pxt.udf
def find_precedent_context(current_context: str, similar_decisions: list) -> str:
 # Format similar decisions as precedent context
 precedent_text = "\n".join([
 f"- {d['ticket_id']}: {d['agent_recommendation']} (policy {d['policy_version']})"
 for d in similar_decisions
 ])
 return f"Similar past decisions:\n{precedent_text}\n\nCurrent context: {current_context}"

# Use precedent in reasoning - the embedding index updates automatically
# when new decisions are inserted
```

 

 Now your agents don't just follow rules. They learn from precedent. "Last quarter, for a similar healthcare customer with similar incident history, the VP approved a 15% exception. Here's why." The decision trace becomes organizational memory.
 

 
## Incremental Context Graphs: Growing Without Rebuilding

 

 Context graphs need to grow continuously. Every new decision adds context. Every exception becomes potential precedent. Pixeltable's incremental computation model means the graph grows efficiently:
 

 
```python
# Insert new decision data
decisions.insert([{
 'ticket_id': 'TICKET-7823',
 'customer_tier': 'enterprise', 
 'open_incidents': {'sev1': 3, 'sev2': 5},
 'escalation_history': {'previous_escalations': 2},
 'churn_signals': {'risk_score': 0.7},
 'policy_version': 'v3.2'
}])

# All computed columns update automatically:
# - decision_context is computed
# - prompt is built
# - agent_response is generated
# - agent_recommendation is extracted
# - The embedding index updates incrementally
# - The context graph grows without rebuilding
```

 

 No ETL pipelines. No batch rebuilds. The context graph is always current because computation happens at write time, not read time. For more on this architecture, see our deep dive on [building AI data infrastructure](/blog/building-ai-data-infrastructure-pixeltable-architecture).
 

 
## The Infrastructure Layer for Agentic Systems

 

 Foundation Capital identifies three paths for startups in the agentic era: replace existing systems of record, replace modules within systems, or create entirely new systems of record for decisions.
 

 

 Pixeltable enables all three by providing the infrastructure layer beneath them. Whether you're building:
 

 

 - **An AI-native CRM** that captures every sales decision with full context

 - **A support automation module** that tracks escalation reasoning

 - **A decision audit system** that makes agent reasoning queryable

 

 

 You need the same underlying capabilities: multimodal data handling, automatic versioning, dependency tracking, incremental computation, and vector search over decision traces. That's what Pixeltable provides.
 

 

 For building agents specifically, see our [practical guide to AI agent architecture](/blog/practical-guide-building-agents) and [building memory-powered stateful agents](/blog/building-memory-powered-ai-stateful-agents-pixeltable).
 

 
## Getting Started: Your First Context Graph

 

 Building context graphs with Pixeltable starts with treating your agent's decision process as data:
 

 
```python
import pixeltable as pxt

# 1. Define the decision table with all context inputs
pxt.create_dir('agent_decisions', if_exists='ignore')
context_table = pxt.create_table('agent_decisions.support_routing', {
 'request_id': pxt.String,
 'request_text': pxt.String,
 'customer_context': pxt.Json,
 'system_state': pxt.Json,
 'timestamp': pxt.Timestamp,
})

# 2. Add computed columns for each reasoning step
@pxt.udf
def classify_intent(text: str) -> str:
 # Your classification logic
 return 'support_request'

context_table.add_computed_column(
 intent_classification=classify_intent(context_table.request_text)
)

@pxt.udf
def calculate_priority(intent: str, customer: dict) -> int:
 base = 5 if intent == 'urgent' else 3
 tier_boost = 2 if customer.get('tier') == 'enterprise' else 0
 return base + tier_boost

context_table.add_computed_column(
 priority_score=calculate_priority(
 context_table.intent_classification,
 context_table.customer_context
 )
)

@pxt.udf
def route_request(intent: str, priority: int, state: dict) -> str:
 if priority > 7:
 return 'tier_3'
 elif priority > 4:
 return 'tier_2'
 return 'tier_1'

context_table.add_computed_column(
 routing_decision=route_request(
 context_table.intent_classification,
 context_table.priority_score,
 context_table.system_state
 )
)

# 3. Create searchable precedent with embedding index
from pixeltable.functions.huggingface import sentence_transformer

context_table.add_embedding_index(
 column_name='request_text',
 embedding=sentence_transformer.using(
 model_id='sentence-transformers/all-MiniLM-L6-v2'
 )
)

# 4. Query the context graph
# What decisions did we make for similar requests?
sim = context_table.request_text.similarity(string='customer cannot access dashboard')
similar = context_table.order_by(sim, asc=False).limit(5).collect()

# How did routing change after policy update v2.1?
# Access historical versions
old_routing = pxt.get_table('agent_decisions.support_routing:10')

# Which edge cases required human override?
overrides = context_table.where(
 context_table.routing_decision == 'manual_review'
).collect()
```

 

 Every row is a complete decision trace. Every computed column is a node in your context graph. Every version is preserved for replay and audit.
 

 
## The Data Plane for the Agentic Era

 

 The question isn't whether systems of record survive the shift to agents. They will. The question is whether the infrastructure exists to capture what agents actually need: not just data, but **decision traces**. The reasoning. The context. The precedent.
 

 

 Traditional data infrastructure can't do this. Warehouses sit in the read path. Operational systems store current state. Vector databases handle embeddings but not lineage.
 

 

 Pixeltable provides the data plane that makes context graphs possible:
 

 

 - **In the write path**: Computations happen at decision time, not after ETL

 - **Automatic versioning**: [Replay the exact state](/blog/pixeltable-versioning-time-travel) at any decision point

 - **Dependency graphs**: [Full lineage](/blog/dependency-graph-magic-computed-columns) of how outputs derived from inputs

 - **Multimodal native**: Images, video, audio, documents as first-class context

 - **Incremental**: [Context graphs grow](/blog/pixeltable-incremental-embedding-indexes) without rebuilding

 - **Queryable**: SQL + vector search over decision traces

 

 

 The next trillion-dollar platforms will be built on systems of record for decisions. Pixeltable is the infrastructure layer that makes them possible.
 

 

 [Get started with Pixeltable](https://docs.pixeltable.com) and build your first context graph today.