Ambasdr turns each user's profile, documents, and social links into a temporal knowledge graph so the AI agent can answer questions about them with structured, traceable context instead of a wall of text. This page explains what a knowledge graph is, how the upstream Graphiti framework works, and how Ambasdr's sidecar service wires it all together.
A knowledge graph is a network of entities (things) connected by typed relationships (how those things relate). It's a structured alternative to keeping everything as prose.
If you give an LLM a 5-page resume as raw text and ask “What companies has Gerald worked at?”, the model has to read the whole document, find the relevant lines, and infer the answer. If the same resume is parsed into a graph — a Person node connected by BelongsTo edges to several Organization nodes — the answer is a one-hop graph query that always returns the same result.
The shift from “blob of text” to “typed graph” matters for three reasons that show up directly in the Ambasdr product:
“What skills does this ambassador have?” becomes MATCH (a:Ambassador)-[:HasSkill]->(s:Skill) instead of a fuzzy text search. The agent gets a clean list every time.
“Which of their skills relate to the topics they write about?” is a two-hop traversal — trivial in a graph, painful in flat prose. This is what powers the Knowledge tab's visualization.
Every fact in the graph points back to the document or profile field it came from. The agent can say “according to your resume” rather than hallucinate.
Graphiti is an open-source framework from Zep for building temporal context graphs for AI agents. Ambasdr uses it as a library, wrapped in a FastAPI service.
What sets Graphiti apart from a plain graph database or a vanilla RAG pipeline:
| Capability | What It Means |
|---|---|
| Temporal facts | Each edge (“fact”) carries validity windows: when it became true (valid_at), when it stopped being true (invalid_at), and when the system learned about it (created_at) or superseded it (expired_at). This is “bi-temporal” modeling — real-world time and system time tracked separately. |
| Incremental ingestion | You feed it “episodes” (chunks of source data) one at a time. The graph evolves with each one. No batch recomputation. |
| Custom ontology | You define entity and edge types as Pydantic models. The LLM is constrained to extract instances of those types — you get a typed graph, not free-form “noun phrases.” Ambasdr defines 11 entity types and 17 edge types (see section 04). |
| Hybrid retrieval | Search combines semantic (vector cosine on embeddings), keyword (BM25 full-text), and graph traversal (n-hop neighborhood). Each retrieval method recalls different things; the combination beats any one alone. |
| Provenance | Every entity and edge traces back to the episode (raw input) that produced it. You can always answer “where did this fact come from?”. |
| Pluggable graph backend | Graphiti can sit on Neo4j, FalkorDB, Kuzu, or Amazon Neptune. Ambasdr uses Neo4j locally and in production. |
If you take one section away, take this one. These six terms appear everywhere in the code, the API, and the Neo4j schema.
If you've used a relational database: think “rows” for entities, “foreign keys with extra columns” for edges, “raw input batches” for episodes, and “tenant id” for group_id. The temporal columns are like a soft-delete timestamp, except they record real-world truthfulness, not row lifecycle.
| Term | Definition |
|---|---|
| Episode | A unit of source data that gets ingested. Can be plain text, structured JSON, or a chat message. In Ambasdr, one episode = one consolidated text blob built from a user's profile + documents + links + agent config. Episodes are the “ground truth” that everything else in the graph points back to. |
| Entity (node) | A typed thing in the graph: a Skill, an Organization, a Topic. Carries a name, summary, labels, and a vector embedding of its summary. Stored as a labeled node in Neo4j. |
| Edge / Fact | A typed relationship between two entities, with a natural-language fact string attached. Example: an edge of type HasSkill from Ambassador “Gerald Parker” to Skill “Go” with fact “Gerald Parker has the Go programming skill.” Edges are first-class — they have UUIDs, embeddings, and temporal fields of their own. |
| Validity window | Two timestamps on every edge: valid_at (when the fact became true in the real world) and invalid_at (when it stopped). When new data contradicts an existing fact, Graphiti invalidates the old fact by setting invalid_at — it doesn't delete it. Old facts remain queryable for “what was true on date X?”. |
| Provenance | Every entity and edge is linked back to the episode(s) that produced it via internal MENTIONS/HAS_EPISODE edges. You can always walk back from a derived fact to the raw source. |
| Group / org_id | Graphiti's tenant key. Every node and edge carries a group_id; queries are scoped by it. Ambasdr sets org_id = page_id — each user's page has its own isolated subgraph in the same Neo4j database. |
| Ontology | The set of entity types and edge types your application defines, expressed as Pydantic models. Graphiti passes the ontology to the extraction LLM as a constraint: “only emit instances of these types.” This is what makes the resulting graph typed and predictable instead of free-form. |
Visually, the relationship between these concepts looks like this:
SOURCE GRAPHITI NEO4J
───────────── ───────────────── ────────────────
┌──────────────┐
user profile ─┐ │ (Episode) │
│ ┌────>│ episode_1 │
uploaded docs ─┤ consolidated LLM extraction pluggable │ └──────┬───────┘
├──> episode (text) ──> guided by ──> graph driver writes ────────┤ MENTIONS │
social links ─┤ + reference_time ontology to backend │ ┌─────▼──────┐ HasSkill
│ ├────>│ Ambassador │ ─────────> Skill ─SkillAppliesTo─> Topic
agent config ─┘ │ │ "Gerald" │ ─────────> Org ─OrgRelatesTo──> Topic
│ └────────────┘ BelongsTo (group_id = page_id)
│
(Each edge carries: valid_at, invalid_at, created_at, expired_at)
Defined in graphiti_service/ontology/{models.py,edge_types.py}. Eleven entity types, seventeen edge types. The Ambassador node is the center of gravity; everything else describes them.
Entity types and what they represent:
| Entity | Represents |
|---|---|
Ambassador | The user whose page this is. Center of the graph. |
Skill | A capability, technology, or expertise (e.g. “Go”, “UX research”). Has a proficiency level. |
Topic | A subject area the ambassador knows or writes about. Categorized (professional / technical / creative / academic / personal / industry). |
Organization | A company, school, or group the ambassador belongs to. |
Project | A discrete piece of work the ambassador did. Has a status (active / completed / planned). |
Document | A file the user uploaded (resume, portfolio, article, certification). Holds the storage path and category. |
ContentFact | An atomic statement extracted from a document (“Led migration from monolith to microservices”). Connects back to the source Document. |
Achievement | A credential, certification, award, or degree. |
SocialPresence | A profile on an external platform (Twitter, LinkedIn, GitHub, …). |
Persona | A trait or focus area chosen during onboarding (creator, developer, consultant, …). |
Interest | A non-professional thing the ambassador cares about. |
A typical graph for one page looks like this (only the most common edges shown):
┌────────────────┐
│ Achievement │
└────────▲───────┘
│ Achieved
│
HasPersonaTrait │ HasPresenceOn
┌─────────┐ ◄────────────┐ ┌─────┴────────┐ ──────────► ┌─────────────────┐
│ Persona │ │ │ │ │ SocialPresence │
└─────────┘ │ │ │ └─────────────────┘
└──┤ Ambassador ├──┐
┌──────────┐ ◄─ Worked ─ │ │ │ HasSkill ┌─────────────┐
│ Project │ On │ │ └──────────► │ Skill │
└────┬─────┘ ┌──┤ ├──┐ └──────┬──────┘
│ ProjectInvolves │ └──────────────┘ │ │ SkillAppliesTo
▼ │ │ │ BelongsTo ▼
┌──────────┐ │ │ AuthoredDocument ┌──────────┐
│ Topic │ ◄─────────────┘ ▼ ▼ │ Topic │
└────┬─────┘ DocumentCoversTopic ┌─────────────┐ └────┬─────┘
│ │Organization │ │
│ FactRelatesTo └─────────────┘ │
▼ │
┌──────────────┐ ExtractedFrom │
│ ContentFact │ ◄─────────────────────── ┌──────────┐ │
└──────────────┘ │ Document │ │
└──────────┘ │
│
TopicRelatesTo ◄──────────┘
(Topic ─── Topic, undirected semantic)
The extraction prompt explicitly forbids orphan entities. If the LLM still emits one — or if Graphiti's deduplication expires a node's only edge during re-ingestion — the post-ingest /v1/repair/isolated-nodes step tries to LLM-reattach it; only nodes the LLM can't connect get deleted by /v1/graph/cleanup-isolated. Full pipeline in §06b.
Graphiti spans three local stores. Neo4j holds the graph itself; Postgres holds operational bookkeeping; MinIO holds the source documents.
| Store | What It Holds |
|---|---|
Neo4jport 7689 (bolt) |
The graph itself. All entity nodes (with embeddings + summaries), all edges (with facts + temporal fields), and Graphiti's internal Episodic nodes representing the raw episodes. Both the vector index (cosine similarity) and the full-text index (BM25, Lucene-backed) live in Neo4j — created automatically at startup by client.build_indices_and_constraints(). |
Postgresdb ambasdr_knowledge_graph on port 5438 |
Operational metadata. Three tables: ingested_episodes (idempotency ledger keyed by (org_id, content_hash)), dead_letter_queue (failed ingestions with payload, traceback, retry status), and audit_log (append-only event trail). Schema is recreated at startup by ensure_schema() from ingestion/schema.sql — it's safe to drop and recreate the DB as long as graphiti is restarted afterward. |
MinIOport 9000, bucket ambasdr-documents |
S3-compatible blob store for the source documents users upload (resumes, portfolios, articles). Graphiti reads from MinIO via POST /v1/extract to pull raw text out of PDFs and other formats before they enter the graph as episode content. |
Visually:
┌──────────────────────────────────────────────────────────────────────────────────┐
│ Graphiti FastAPI service │
│ (port 8050, uvicorn) │
└──────┬─────────────────────┬───────────────────────────────────┬─────────────────┘
│ │ │
│ extract │ graph reads/writes │ ops bookkeeping
│ (boto3) │ (bolt) │ (asyncpg)
▼ ▼ ▼
┌──────────────┐ ┌─────────────────────┐ ┌─────────────────────────┐
│ MinIO │ │ Neo4j │ │ Postgres (KG schema) │
│ port 9000 │ │ port 7689 (bolt) │ │ port 5438 │
│ │ │ │ │ │
│ ambasdr- │ │ Entity nodes │ │ ingested_episodes │
│ documents/ │ │ Edges (facts) │ │ dead_letter_queue │
│ (uploads) │ │ Episodic nodes │ │ audit_log │
│ │ │ Vector index │ │ │
│ │ │ Fulltext (BM25) │ │ (recreated by │
│ │ │ index │ │ ensure_schema) │
└──────────────┘ └─────────────────────┘ └─────────────────────────┘
A common misread is that the “knowledge graph” lives in Postgres. It doesn't — Postgres only tracks that an ingestion happened and whether it failed. The nodes and edges are in Neo4j.
A “Regenerate Ambasdr” click on the dashboard triggers a single consolidated ingestion. The pipeline is idempotent — re-running it on unchanged content is a no-op.
The sequence of calls and state transitions:
Frontend Go Backend Graphiti Neo4j KG Postgres
──────── ────────── ──────── ───── ──────────
│ │ │ │ │
│ click Regenerate │ │ │ │
├───────────────────>│ │ │ │
│ POST /agent- │ │ │ │
│ config/regenerate │ │ │ │
│ │ build episode body: │ │ │
│ │ user + page + docs + │ │ │
│ │ links + agent config │ │ │
│ │ │ │ │
│ │ POST /v1/extract (per doc)│ │ │
│ ├───────────────────────────>│ read MinIO blob, │ │
│ │ │ LLM-extract text │ │
│ │<───────────────────────────┤ return text │ │
│ │ │ │ │
│ │ status: ingesting │ │ │
│ ├──────────────────────────────────────────────────────────────────────>│
│ │ │ │ agent_configs
│ │ │ │ .kg_ingestion_status
│ │ │ │ │
│ │ POST /v1/ingest │ │ │
│ ├───────────────────────────>│ SHA-256(body) │ │
│ │ ├──────────────────────────┼───────────────>│
│ │ │ dedup lookup │ ingested_episodes
│ │ │ │ │
│ │ │ ┌─ if duplicate: │ │
│ │ │ └─ return prev UUID, exit │
│ │ │ │ │
│ │ │ audit: episode_received │ │
│ │ ├───────────────────────────────────────────>│
│ │ │ │ audit_log
│ │ │ │ │
│ │ │ LLM (Claude Sonnet): │ │
│ │ │ extract entities + edges│ │
│ │ │ per Ambasdr ontology │ │
│ │ │ │ │
│ │ │ embed each node/edge │ │
│ │ │ (OpenAI embeddings) │ │
│ │ │ │ │
│ │ │ merge into Neo4j │ │
│ │ ├─────────────────────────>│ │
│ │ │ (per-tenant: group_id) │ │
│ │ │ │ │
│ │ │ audit: ingestion_ │ │
│ │ │ succeeded │ │
│ │ ├───────────────────────────────────────────>│
│ │ │ │ │
│ │ │ insert into │ │
│ │ │ ingested_episodes │ │
│ │ ├───────────────────────────────────────────>│
│ │<───────────────────────────┤ IngestResponse │ │
│ │ │ │ │
│ poll status │ │ │ │
│ (every 3s) │ POST /v1/repair/isolated-nodes (LLM-evaluate orphans) │ │
├───────────────────>│───────────────────────────>│ │ │
│ │ DELETE /v1/graph/cleanup-isolated │ │
│ │───────────────────────────>│ │ │
│ │ │ │ │
│ │ status: completed │ │ │
│ ├──────────────────────────────────────────────────────────────────────>│
│ │ │ │ agent_configs
│ frontend sees │ │ │ .kg_ingestion_status
│ "synced", │ │ │ │
│ stops polling │ │ │ │
↓ ↓ ↓ ↓ ↓
Each numbered step explained:
The dashboard's Regenerate Ambasdr button POSTs to /v1/pages/{page_id}/agent-config/regenerate on the Go backend. The handler validates ownership, sets the agent-config generation status to pending, and fires an async goroutine. The HTTP response returns immediately — ingestion happens in the background.
The Go knowledge service (backend/internal/service/knowledge.go) pulls the page, user, documents, social links, and agent config from Postgres. For each document, it calls POST /v1/extract on Graphiti to convert the MinIO blob (PDF, etc.) into text. The Go service then concatenates everything into one episode body — a single string — and assigns it a stable external_source_id like page-complete-<page_id>.
The agent_configs.kg_ingestion_status column in the main app DB transitions from idle / failed / completed to ingesting. The frontend's useKnowledgeStats hook is polling status every 3 seconds — when it sees ingesting, it keeps polling until the status flips again.
On POST /v1/ingest, the pipeline computes SHA-256(body.strip().lower()) and checks ingested_episodes in Postgres for an existing row with the same (org_id, content_hash) or (org_id, external_source_id). If found, it returns the previous episode UUID and exits — no LLM calls, no graph writes. This is why hitting Regenerate on unchanged data is free.
Claude Sonnet is prompted with the Ambasdr ontology (as Pydantic models converted to a schema) and the extraction instructions from config.py. The model emits typed entities (Skill, Organization, etc.) and edges (HasSkill, BelongsTo, etc.) with natural-language fact strings. Each node and edge then gets an OpenAI text-embedding-3-small embedding for vector search.
Nodes and edges are merged into the per-tenant subgraph — everything keyed by group_id = org_id = page_id. Graphiti's internal Episodic node records the raw episode and is connected to every extracted entity via MENTIONS edges, giving you provenance for free. Temporal fields (valid_at, invalid_at) are set from the episode's reference time and any LLM-inferred dates.
If extraction or Neo4j writes fail, the pipeline retries with exponential backoff (up to MAX_RETRIES = 3, base 2 s, jitter 25%, cap 30 s). After exhaustion, the payload — plus error type, message, and traceback — is parked in dead_letter_queue with status pending. It's queryable via GET /v1/dlq and replayable via POST /v1/dlq/{id}/replay.
On a successful ingest, the backend follows up with two Graphiti calls in a fire-and-forget goroutine: POST /v1/repair/isolated-nodes tries to LLM-reattach any node whose active-edge count is zero, then DELETE /v1/graph/cleanup-isolated removes whatever the LLM couldn't connect. The full pipeline is documented in §06b.
agent_configs.kg_ingestion_status is set back to completed with a fresh kg_last_ingested_at. The frontend's status poll picks this up, stops polling, and re-renders the Knowledge tab with the new node and edge counts.
A best-effort recovery layer for entities that lose all their active edges. Runs automatically after every successful KG ingest. Designed to preserve information over discarding it — the cleanup that deletes nodes only fires after repair has tried and failed.
A node is isolated when it has zero active RELATES_TO edges — i.e. no edge with expired_at IS NULL. MENTIONS edges back to the source Episodic node don't count; they're provenance links, not entity-to-entity relationships, and the graph API filters them out of rendering.
This active-only definition is critical. The earlier predicate counted edges regardless of expired_at, which meant nodes whose only edges had been expired by Graphiti's de-duplication looked isolated in the UI but didn't qualify for repair. The current predicate matches the API's notion of "visible" exactly.
As of migration 000016_add_kg_content_hash, IngestPageData short-circuits when the page's hashable content (user, page metadata, docs, social links) is unchanged since the last successful ingest. Agent instruction regeneration no longer triggers re-ingestion — the single biggest source of bookkeeping expirations is closed. The repair pipeline below remains as defense in depth for the residual cases.
The dashboard surfaces this gate to the user via the needs_regeneration boolean on the agent-config response. The Regenerate Ambasdr button only enables when the page's current content hash differs from the stored hash (or no hash exists yet). Operators who need to force a re-ingest regardless can pass ?force=true to POST /v1/pages/:page_id/agent-config/regenerate, which routes through ForceIngestPageData on the backend.
The most common remaining cause is re-ingestion churn when content actually does change:
BelongsTo vs WORKED_AT) or omit relationships entirely.RELATES_TO edges expired — invisible to the graph API, isolated in the UI.Other rarer causes: extraction prompt accidentally yields an orphan; manual data load that omits relationships.
Implemented in knowledge-graph/graphiti_service/repair/isolated_nodes.py. Triggered by POST /v1/repair/isolated-nodes; the backend invokes this automatically after every successful ingestion (see "When repair runs" below). The flow per isolated node:
Cypher: MATCH (n:Entity {group_id: $org_id}) WHERE NOT EXISTS { (n)-[r:RELATES_TO]-() WHERE r.expired_at IS NULL } RETURN n. The predicate is byte-identical to the cleanup helper (verified by unit test) so find and delete never disagree on what counts as isolated.
For each isolated node, run Graphiti's hybrid search (cosine similarity + BM25) using the node's name and summary as the query. Returns up to 10 candidate connection partners scoped to the same group_id. The isolated node itself is filtered out of the result set. If the search returns zero candidates, the node moves to cleanup unchanged.
Format all candidates into a single prompt. Ask the LLM (defaults to Claude Sonnet) to evaluate which candidates have a meaningful relationship to the isolated node and, when so, propose a relationship name (preferring the ontology's edge types, falling back to UPPER_SNAKE_CASE) and a concise factual statement. The LLM returns a JSON array — one object per candidate — with should_connect, relationship_name, and fact.
For every approved connection, first delete any existing expired RELATES_TO edges between this exact pair (Cypher: MATCH (s)-[r:RELATES_TO]-(t) WHERE r.expired_at IS NOT NULL DELETE r), then call Graphiti's add_triplet() to create the new edge. The pre-clean is non-negotiable — without it Graphiti's contradiction detection runs against the entire edge history and re-expires the new edge the millisecond it lands. The repair reports success at the API level (edges_created += 1) but the node stays visually isolated. This was hit in production and is what the structural unit tests now guard against.
When you call add_triplet(), Graphiti runs two contradiction checks: (1) same-pair history — every prior edge between the source and target nodes (active OR expired) — and (2) cross-edge contradiction — an LLM that compares the new edge's fact against semantically similar edges elsewhere in the graph (e.g. "Senior Software Engineer at Cisco" vs. "Senior Software Engineer at Siemens"). Either can expire the new edge synchronously on insert. The repair defends against both: step 4's pre-clean removes the same-pair history, and step 5 (below) un-expires the new edge if the cross-edge layer stamps it.
After add_triplet returns success, run a Cypher pass scoped to (source_uuid, target_uuid, relationship_name) created in the last 30 seconds. If expired_at is set, clear it. This is intentionally a tight scope — only the edge we just created, with the exact relationship name, with a recent timestamp. Other historical edges between the same pair (cleaned in step 4) and other edges between unrelated pairs are untouched. The repair LLM already evaluated all candidates with full context; we treat its judgment as authoritative over Graphiti's cross-edge contradiction LLM, which doesn't see why the repair is being run.
The repair LLM is told to only use relationship names from AMBASDR_EDGE_TYPES. The prompt frames the constraint explicitly: "you MUST pick one of these for every relationship you propose — do not invent new relationship names." Despite the instruction, LLMs occasionally propose novel UPPER_SNAKE_CASE names (e.g. WORKED_AT when the ontology has BelongsTo). The validation step in evaluate_and_create_edges checks the proposed name against AMBASDR_EDGE_TYPES and skips any off-ontology proposal with a warning. This prevents semantically-duplicate edges from being created and reduces the surface area for the cross-edge contradiction layer to trigger.
After repair runs, the backend calls DELETE /v1/graph/cleanup-isolated (implemented in search/graph_explorer.py). Same predicate as repair — anything still without active RELATES_TO edges is deleted via DETACH DELETE. The two-phase design preserves information: a node only gets removed when the LLM evaluated its candidates and judged none of them connected.
| Trigger | Implementation |
|---|---|
| Automatic — after every successful KG ingestion | Fire-and-forget goroutine in backend/internal/service/knowledge.go (around line 293). Calls graphitiClient.RepairIsolatedNodes(ctx, orgID), then CleanupIsolatedNodes(ctx, orgID). Uses context.Background() so it survives the originating request's cancellation. Errors are logged at warn; ingestion is reported as successful regardless of repair outcome. |
| Manual — operator-initiated | curl -X POST -H "Content-Type: application/json" -d '{"org_id":"<page_id>"}' <kg-service>/v1/repair/isolated-nodes. Returns {nodes_found, edges_created, details}. Used during incident triage when re-ingesting isn't worth the cost. |
| Cleanup alone, without repair | curl -X DELETE <kg-service>/v1/graph/cleanup-isolated?org_id=<page_id>. Skips the LLM step and deletes any visually-isolated node immediately. Use only when you know the orphans are noise (e.g. after a known-bad extraction run). |
Every meaningful transition writes an audit row:
repair_edge_created — one row per new edge with source/target uuids, relationship_name, and factrepair_completed — summary row with nodes_processed and edges_createdQueryable via GET /v1/audit?org_id=<page_id>. Per-container logs flow through Loki — {compose_service="knowledge-graph"} |~ "repair|isolated" shows the structured log lines.
| Symptom | Likely cause | Fix |
|---|---|---|
Repair returns edges_created > 0 but the same node is still isolated on next query |
Pre-clean step missing or running after add_triplet(). Graphiti's contradiction logic immediately re-expires the new edge. Unit test test_repair_clears_expired_edges_before_add_triplet guards against this. |
Verify repair/isolated_nodes.py still does the DELETE before add_triplet. Re-run repair after fixing. |
Repair reports nodes_found > 0 but edges_created == 0 |
Either semantic search returned no candidates (low embedding quality for short single-word names), or the LLM judged none of the candidates relevant. Both are legitimate — cleanup will delete the node next. | Inspect via GET /v1/audit. If the candidates are clearly relevant and the LLM still rejected them, tune the evaluation prompt in repair/isolated_nodes.py. |
| Same nodes show up isolated after every regeneration | Re-ingestion churn — the extraction prompt isn't yielding consistent relationships across runs. Each regeneration expires the old edge and the new pass doesn't reattach. | Two paths: (a) make agent instruction regeneration conditional on content change so it doesn't trigger re-ingest, or (b) stabilize the extraction prompt so the same source text produces the same edge name twice in a row. |
| Repair endpoint returns 200 immediately, no work logged | The find step returned zero rows because the active-edge predicate is wrong (counting expired edges). The earlier WHERE NOT (n)-[:RELATES_TO]-() form had this bug. |
Verify find_isolated_nodes uses WHERE NOT EXISTS { (n)-[r:RELATES_TO]-() WHERE r.expired_at IS NULL }. Unit test test_find_isolated_nodes_filters_to_active_edges enforces this. |
Three complementary retrieval methods, combined. Each catches things the others miss.
Every node and edge has an OpenAI embedding of its summary/fact. A query is embedded, then Neo4j's vector index returns the nearest neighbors by cosine similarity. Catches paraphrases and concept-level matches (“leadership” finds “managed team of 5”).
Neo4j's full-text indexes (Lucene-backed) run BM25 ranking over node names, summaries, and edge facts. Catches exact terms and proper nouns that embeddings sometimes blur (“Cisco”, “Kubernetes”).
Starting from matched nodes, walk edges to bring in neighbors. The frontend's node panel uses /v1/graph/nodes/{uuid}/connections for exactly this — pivot on a node, see what's adjacent.
The POST /v1/search endpoint accepts a search_mode parameter:
| Mode | What Runs |
|---|---|
hybrid (default) | Cosine similarity + BM25 in parallel, results merged. Best general recall. |
semantic | Cosine only. Use when you want concept matches and exact wording shouldn't matter. |
keyword | BM25 only. Use when you need exact-term matches and embeddings are too fuzzy. |
Filters can be applied on top: by entity type (["Skill", "Organization"]), by edge type (["HasSkill", "BelongsTo"]), or by temporal window (created_after / created_before). All search calls are scoped to group_id = page_id — cross-page leakage isn't possible at the API level.
Ingestion talks to an LLM and a database over the network; either can fail. Graphiti's wrapper turns those failures into manageable state, not silent data loss.
Before any LLM call, the pipeline hashes the episode body and checks ingested_episodes. Duplicates short-circuit. Same external_source_id on a re-ingest also dedups. You can hit Regenerate as many times as you want with no extra cost.
Transient failures retry up to 3 times with exponential backoff (2 s, 4 s, 8 s, … capped at 30 s) plus 25% jitter. Handles flaky LLM 429s and intermittent network issues without user intervention.
After retries are exhausted, the payload is parked in dead_letter_queue with the error type, message, full traceback, attempt count, and a status (pending / retrying / resolved / abandoned). Inspect via /v1/dlq, replay via /v1/dlq/{id}/replay.
Every interesting transition writes an immutable row to audit_log: episode_received, duplicate_skipped, ingestion_succeeded, ingestion_failed, dead_lettered, dlq_replayed, dlq_resolved, repair_edge_created, repair_completed, graph_purged. Queryable via GET /v1/audit.
The DLQ and audit tables are created at startup by ensure_schema(), which executes ingestion/schema.sql against the KG Postgres database. The CREATE statements are all IF NOT EXISTS so it's safe to run on every boot. If you ever recreate the KG database while graphiti is running, you must restart the service — make kg-reset does this for you.
Rolling up everything above into the practical wins.
The agent can ask “skills the ambassador has” or “organizations they belong to” as graph queries rather than text search. Cleaner, more predictable answers.
The graph captures how entities relate — this skill applies to that topic, this fact came from that document. Retrieval can traverse edges instead of fishing through prose.
Content-hash dedup means re-running ingestion on unchanged data is a no-op. No LLM cost, no graph churn.
Failed ingestions don't disappear — they're parked with full traceback and can be replayed. Every state change is logged.
Every node, edge, audit row, and DLQ entry is scoped to group_id = page_id. Search and stats only ever see the requesting page's slice of the graph.
The frontend's Knowledge tab renders the graph live (d3-force layout), with search, node panels, and connection lookups. It's a paid-tier feature gated by model.IsPremium(plan_tier). The mobile Knowledge tab now mirrors this gate: free-tier users see the tab behind a non-dismissible paywall (blurred teaser + "Manage on web" upgrade CTA), premium users see the full page (mock data today, pending JUE-393). Shipped in mobile PR #97.
All endpoints are unauthenticated — graphiti runs as a sidecar reachable only from the Go backend over localhost. Multi-tenancy is enforced via the org_id query parameter on every call.
| Endpoint | Purpose |
|---|---|
GET /health | Liveness check. Returns {"status":"ok","service":"ambasdr-knowledge-graph"}. |
POST /v1/ingest | Ingest a single consolidated episode. Body: episode text, source description, source type, org_id, reference time, optional external_source_id. Idempotent. |
POST /v1/extract | Extract text from a MinIO document blob. Body: storage_path + content_type. Returns extracted text used as input to /v1/ingest. |
POST /v1/search | Hybrid vector + BM25 + graph search across a tenant's subgraph. Returns ranked edges with source/target node metadata. |
GET /v1/graph/nodes | List all nodes for an org_id up to limit. Used by the frontend to render the full graph. |
GET /v1/graph/nodes/{node_uuid} | Fetch a single node by UUID, scoped to org_id. |
GET /v1/graph/nodes/{node_uuid}/connections | Adjacent edges + neighbor nodes for a given node. Optional edge_types comma-list filter. Powers the side panel when a node is clicked. |
GET /v1/stats | Node count, edge count, and entity-type distribution for an org_id. Drives the Connections counter on the Knowledge tab. |
POST /v1/repair/isolated-nodes | LLM-evaluates orphaned nodes and tries to add edges that connect them. Called by the backend right after a successful ingest. |
DELETE /v1/graph/cleanup-isolated | Delete any nodes still isolated after repair. Treats them as extraction failures. |
DELETE /v1/graph/purge | Delete all nodes and edges for an org_id. Used when a page is destroyed. |
GET /v1/dlq | List failed ingestions for an org_id, with payload, error type, traceback, attempt count, and status. |
GET /v1/dlq/stats | Aggregate DLQ counts grouped by status and top error types. |
POST /v1/dlq/{dlq_id}/replay | Re-run a parked ingestion. Returns the same IngestResponse shape as a fresh ingest. |
POST /v1/dlq/{dlq_id}/resolve | Mark a DLQ entry as resolved or abandoned. 204 No Content. |
GET /v1/audit | Read the audit log for an org_id: episode_received, duplicate_skipped, ingestion_succeeded, ingestion_failed, dead_lettered, dlq_replayed, dlq_resolved, repair_edge_created, repair_completed, graph_purged. |
All targets are defined in the monorepo root Makefile. Background processes are managed by scripts/svc.sh with PID files in /tmp.
| Command | What It Does |
|---|---|
make kg-up | Single command to bring the full KG stack up: starts the Docker dependencies (Postgres, Neo4j, MinIO, minio-init) then launches uvicorn detached on port 8050. Idempotent — if graphiti is already running, reports that and exits 0. |
make kg-down | Stop the graphiti service and then docker compose stop postgres neo4j minio. Cleans up PID file and orphaned port-bound processes. |
make kg-reset | Wipe Neo4j (MATCH (n) DETACH DELETE n), drop and recreate the ambasdr_knowledge_graph Postgres DB, and empty the ambasdr-documents MinIO bucket. If graphiti is running, restarts it so ensure_schema() recreates the audit/DLQ/episodes tables in the fresh DB. |
make kg-up-service | Start only the graphiti FastAPI server (assumes infra is already up). Used by kg-reset internally. |
make kg-down-service | Stop only the graphiti FastAPI server, leaving Docker containers running. |
make kg-infra | Bring up just the Docker containers graphiti depends on (Postgres, Neo4j, MinIO, minio-init). Does not start uvicorn. |
make kg-run | Foreground uvicorn with --reload. For interactive development when you want logs streaming to your terminal. |
make kg-install | Create the Python 3.13 venv in knowledge-graph/.venv and install requirements.txt. |
make kg-build | Build the production Docker image ambasdr-knowledge-graph from knowledge-graph/Dockerfile. |
make kg-test | Run the full pytest suite (unit + integration) inside the KG venv. |
make kg-test-unit | Unit tests only. |
make kg-test-integration | Integration tests (require Docker dependencies up). |
make kg-lint | Run ruff check over graphiti_service/ and tests/. |
make db-reset-full | Reset the main app DB and wipe the KG stores in one shot. Bounces graphiti so ensure_schema() recreates the tables. |
kg-reset bounces graphitiGraphiti runs ensure_schema() exactly once at startup, against whatever DB exists at that moment. If you drop and recreate ambasdr_knowledge_graph while the service is up, the next ingest will hit UndefinedTableError: relation "audit_log" does not exist. The Makefile auto-restarts graphiti so the schema is recreated in the fresh DB.
Graphiti is a good default for temporal knowledge graphs and a bad default for cost. Its expensive behaviours are opt-out, not opt-in. Every figure below was measured on the same source document against the same purged starting graph, read from Langfuse traces — not estimated.
| Step | Change | Cost / ingestion | LLM calls |
|---|---|---|---|
| baseline | graphiti defaults on claude-sonnet-5 | $2.69 | 305 |
| PR #156 | removed every attribute field from the ontology | $0.3977 | ~154 |
| PR #160 | bounded the previous-episode context to 1 | $0.2494 | 77 |
About 91% off, with no provider change and no model downgrade. The graph came back at 78–83 nodes against an 84-node baseline — equivalent, not degraded.
Graphiti makes one extra LLM call per entity and per edge whose type declares any attribute field. Cost therefore scales with graph size, not document size. The baseline broke down as: 28% EdgeDuplicate and 18% EdgeTimestamps (graphiti internals), 25% edge attributes, 24% entity attributes — so roughly half the bill was attribute extraction.
14 of our 17 edge types declared exactly one field, memory_type, whose value is a constant per class: HasSkill is always SEMANTIC, AuthoredDocument always EVENT_BASED. 77 calls per ingestion were asking an LLM to recover a value already hardcoded in the class definition. All fields were removed from all 11 entity types and all 17 edge types; memory_type became the static EDGE_MEMORY_TYPES map plus memory_type_for().
_build_entity_types_context builds the extraction prompt from the class name and docstring (type_model.__doc__), never from the fields. Entities are still typed Skill / Project / Organization exactly as before; only the follow-up attribute call disappears. With all fields gone, a type is its docstring — so a test pins that every entity and edge type keeps a usable one, because losing one would silently degrade typing while everything else still passed.
add_episode, left to itself, calls retrieve_episodes(last_n=RELEVANT_SCHEMA_LIMIT) — ten previous episodes — and copies each one in full into four separate prompts: extract_nodes, node dedup, extract_edges, and the attribute/summary pass. Each adds ~1,633 input tokens, which is how a 6,481-character document cost 15,867 input tokens on ExtractedEdges.
That default fits graphiti's usual shape, where episodes are successive conversation turns and history carries information the current turn lacks. Ours are complete snapshots of the same page, so ten prior copies of one resume add cost without adding information. Bounded via previous_episode_uuids (a supported seam, not a patch of internals), tunable through KG_EPISODE_CONTEXT, clamped 0–10.
None means “retrieve ten”. Only an empty list means none. And this change has no user-visible signal: the graph is unchanged by design, health checks stay green, and the only symptom of a silent revert is the bill a month later — so two tests drive the ingest path and capture the kwargs. Before they existed, deleting the single previous_episode_uuids= line left all 142 tests green.
Four open models were measured against the same resume and the same purged graph. Every one produced zero nodes.
| Model | Calls | Time | Nodes | Cost |
|---|---|---|---|---|
claude-sonnet-5 | 305 | 173s | 84 | $2.69 |
qwen3-30b-a3b-instruct | 3 | 800s | 0 | $0.01 — 67% truncated |
qwen3-32b | 52 | 900s | 0 | $0.08 — 27% truncated |
moonshotai/kimi-k2 | ~1 | 410s | 0 | $0.01 — provider errors |
Two failure modes, only one about the model. Over-generation: Anthropic averages ~103 output tokens per call; qwen3-32b averaged 670 and 30b-a3b 11,700, exceeding max_tokens and truncating mid-JSON. A robustness gap that is not model-specific: openai_generic_client.py does response.choices[0].message.content, which raises TypeError when the gateway returns an error body with no choices; the retry wrapper then backs off, so the symptom presents as the model being slow while a direct probe returns in 0.8s. Any model can hit it.
Both environments therefore default to anthropic. The abstraction and instrumentation stand on their own as the harness for evaluating this properly later. Also worth keeping: a partial run is not evidence — qwen3-32b was reported mid-flight as “looks like it's working” at 0% truncation and finished at 27%.
The case for switching providers was built on the $2.69 baseline. PRs #156 and #160 removed 91% of it with no provider change at all, so the remaining prize was a fraction of what the work had been scoped against. Fix your own waste before shopping for a cheaper rate on it.
| Lever | What to know |
|---|---|
| Model tier routing | 93% of ingestion spend is graphiti's small tier, so pointing it at Haiku (half Sonnet's rate) is the biggest config-only saving. But graphiti's AnthropicClient accepts model_size and ignores it — it tags a tracing span and sends self.model regardless. Its OpenAI/Gemini/GLiNER clients do honour it. We route it ourselves via a ContextVar (not an attribute — graphiti runs up to 20 calls concurrently and mutation would race). |
| Content-hash dedup | Skips ingestion when the assembled episode is unchanged. Only persist the hash on a successful ingest: graphiti returns HTTP 200 even when the episode was dead-lettered, and recording the hash on that response makes the dedup gate suppress every retry permanently. |
| Reranker choice | Search goes through client._search() with the default RRF reranker — pure ranking arithmetic, no extra LLM call. A cross-encoder reranker would cost one model call per search, which at our search volume would exceed ingestion. |
| Split timeouts | A real single-page ingestion runs 97–170 seconds — it is an LLM pipeline, not a lookup. One shared 30s HTTP timeout abandoned every ingestion mid-flight and marked the page Failed while the KG service was still working. Deadlines are split per call site: KG_TIMEOUT_SECONDS=30 for reads, KG_BATCH_TIMEOUT_SECONDS=600 for ingest and extract. |
| Pinned graphiti-core | graphiti-core[neo4j,anthropic]==0.29.0. The entire attribute saving rests on two private guards (len(model_fields) == 0 in node_operations.py / edge_operations.py). A test calls the private path directly and asserts no LLM call is made for a zero-field type, so a bump that removes the guard fails a test instead of silently doubling the bill. |
Embeddings and reranking go through OpenAI clients and are not instrumented, so ingestion cost read from Langfuse traces excludes them. Also note that the Go process's in-process Langfuse callback cannot see an LLM call made in the Python KG service — that is why InstrumentedAnthropicClient exists, and why cost alerting looked like coverage while being structurally blind to the largest spender.
The portable version of this — the audit checklist, each lever with its traps, the silent-failure catalogue, the measurement protocol, and complete tenant deletion across Neo4j and the sidecar Postgres — is packaged as the graphiti-kg-optimization skill. Run its audit checklist first: it is eight greps and two queries and tells you which levers are unpulled before you change anything.