Skip to main content

​
Prompt management — UnifiedPromptManager + Langfuse

Status: Active. Lane A of Glad-Labs/poindexter#450 (the OSS migration umbrella) completed 2026-05-09 — production no longer carries inline prompt constants. The YAML prompt files were subsequently migrated to agentskills.io SKILL.md packs (Glad-Labs/poindexter#528); no prompts/*.yaml files ship anymore.

​
Where prompts live

The single source of truth for every production prompt is a set of SKILL.md packs under src/cofounder_agent/skills/<pack>/<skill>/SKILL.md — authoritative at runtime as of poindexter#825 (2026-07). Resolution:
  1. SKILL.md pack default — what ships with the repo (the in-memory default loaded at boot) and what production serves. Edits land via PR + the prompt contract tests.
  2. Langfuse production label — consulted FIRST only when app_settings.langfuse_prompt_overrides_enabled='true' (default false). The legacy override layer, kept as an escape hatch for deliberate live prompt experiments. Langfuse itself is otherwise a read-only mirror maintained by SyncPromptCatalogToLangfuseJob — see Langfuse as a read-only mirror.
  3. KeyError — if the key isn’t registered. Per feedback_no_silent_defaults.md, an unknown key is a configuration bug, not a quiet fallback.
The legacy prompts/*.yaml loader (_initialize_prompts) still runs at boot for backward compatibility, but the repo ships zero YAML prompt files — every key now comes from a SKILL.md pack via _initialize_skills. Skills load after YAML so a migrated SKILL.md transparently wins over any leftover YAML entry for the same key.

​
The cascade

caller: prompt = pm.get_prompt("qa.topic_delivery", topic=t, opening=o)
                    ↓
        UnifiedPromptManager.get_prompt(key, **kwargs)
                    ↓
        ┌───────────────────────────────────┐
        │ 0. langfuse_prompt_overrides_     │
        │    enabled == 'true'?  (default   │
        │    false — skip straight to 2)    │
        └───────────────────────────────────┘
                    ↓ (only when enabled)
        ┌───────────────────────────────────┐
        │ 1. Langfuse client                │
        │    .get_prompt(name=key,          │
        │     label="production")           │
        │    → returns template if hit      │
        └───────────────────────────────────┘
                    ↓ (if None or error)
        ┌───────────────────────────────────┐
        │ 2. self.prompts[key]              │
        │    (in-memory default, loaded     │
        │     from SKILL.md packs at boot)  │
        │    → returns template             │
        └───────────────────────────────────┘
                    ↓ (if KeyError)
                  raises
All paths feed the same template.format(**kwargs) step at the end, so the call-site contract (kwargs in, formatted string out) is identical regardless of which tier wins.

​
Available prompt keys

Each SKILL.md pack declares the keys it provides in its frontmatter metadata.prompts list — that frontmatter is the authoritative key inventory (the loader registers exactly those keys). The packs that ship today:
Pack (skills/<pack>/<skill>/SKILL.md)metadata.categorySurface
content/writercontentbase content-writer persona + narrative-pass seed
content/two-pass-writercontentTWO_PASS draft/revise prompts
content/blog-generationcontentinitial draft / SEO+social / iterative refinement
content/content-qa, content/qacontent_qathe qa.* review, critique, topic-delivery, consistency, aggregate-rewrite prompts + the DeepEval g-eval criterion
content/researchresearchsearch-result analysis + topic-candidate ranking
content/image-generationimagefeatured-image, search-query, image-director, and vision alt-text caption prompts
content/seo-metadataseo_metadataseo.* title / description / keywords / excerpt / category / tags
content/social-mediasocialtrend research + post creation
content/podcast, content/video, content/video-directormediamedia-script and shot-list prompts
content/atomscontentatoms.* system prompts (narrate_bundle, pipeline_architect)
content/utilityutilitycontent summarization / JSON conversion helpers
ops/automation, ops/business, ops/triage, ops/hygieneopstask.* business/automation/ops prompts + the retention/memory-hygiene summarizers
voice/agentvoicevoice.* system prompts — Emma persona + Claude-bridge TTS override ({surface})
The narrate_bundle and pipeline_architect templates carry the operator persona as {site_name} / {site_url} placeholders, rendered from the run-bound site_config by the calling atom before the text reaches the model.

​
How operators tune prompts

​
The one edit path — SKILL.md via PR

  1. Open the pack that owns the key (e.g. skills/content/seo-metadata/SKILL.md for seo.*).
  2. Edit the body in that key’s ## <key> section (in practice: ask an agent to).
  3. Commit / merge. The container picks up the new body on next deploy / restart, and SyncPromptCatalogToLangfuseJob pushes it into the Langfuse mirror within ~6h.
  4. The prompt contract tests (tests/unit/services/test_prompt_*.py, test_prompt_manager_skills.py, and per-surface tests like test_multi_model_qa_prompts.py) pin rendered bodies — update the affected expectation in the same PR.

​
Reviewing prompts — Langfuse UI (read-only)

Open <langfuse_host>/project/poindexter/prompts to browse every production prompt with full version history — the mirror job keeps it current. Edits made there do not take effect (the runtime serves the SKILL.md default); the mirror will replace a hand-edited version with the current default on its next cycle and page Discord about it.

​
Escape hatch — live overrides (off by default)

Set langfuse_prompt_overrides_enabled='true' to restore the legacy Langfuse-first lookup for deliberate live experiments (UI edit takes effect within ~60s, no deploy). While it’s on, remember the trade you’re making: any Langfuse production version shadows its SKILL.md default — the masking-trap behavior — so turn it back off (or delete the experimental versions) when the experiment ends. The contract tests serve two purposes:
  • Drift detection. A prod-side Langfuse edit that strays from the SKILL.md default still lands on the dashboard but ALSO trips the test in CI, forcing a conscious revert-or-update.
  • Migration safety. When prompts moved (f-string → YAML → SKILL.md), the tests verified the rendered body stayed byte-for-byte identical, so format-string gotchas ({{/}} doubling, trailing newlines) couldn’t sneak through.
Inline-fallback resilience (test_prompt_fallback_drift.py). Several resolvers keep an inline _*_FALLBACK copy so the pipeline survives a prompt-registry outage. A parametrized guard drives each resolver with the registry up (SKILL.md path) and down (inline path) and asserts they agree — so a fired fallback can never silently serve stale text. When a fallback DOES fire, the resolver logs at error (per feedback_self_heal_not_suppress: self-heal by serving the byte-identical inline copy, but surface the registry outage loudly rather than swallow it).

​
How callers fetch prompts

from services.prompt_manager import get_prompt_manager

pm = get_prompt_manager()  # cached singleton

prompt = pm.get_prompt(
    "qa.topic_delivery",
    topic="Why local LLMs win",
    opening="In 2026, the case for local AI...",
)
Sync API — even though Langfuse is involved, the call path doesn’t await. The Langfuse client caches in-process and the call returns the cached version immediately. Format string semantics — kwargs are substituted via str.format(**kwargs). Literal braces in the prompt body must be doubled ({{ / }}); the SKILL.md bodies preserve the doubling.

​
SKILL.md prompt format

A prompt-bearing pack is a single SKILL.md: YAML frontmatter declares the keys, the body holds one ## <key> section per prompt with the template in a fenced block. 
---
name: seo-metadata
description: >
  SEO metadata generation. Produce titles, meta descriptions, excerpts, ...
license: Apache-2.0
metadata:
  category: seo_metadata
  prompts:
    - key: seo.generate_title
      output_format: json # 'json' | 'text' — for downstream parsing
      description: 'Default prompt — premium packs ship as an add-on'
---

# SEO metadata skill

## seo.generate_title

\`\`\`
Generate one SEO-friendly title for the topic below.

Return ONLY a JSON object — {{"title": "..."}} — no markdown, no reasoning.

TOPIC: {topic}
\`\`\`
The loader (prompt_manager._initialize_skills) lives inside the package (<pkg>/skills/) so a package-relative path resolves identically on the host (src/cofounder_agent/skills) and in the worker container (/app/skills). Operator action skills (the repo-root skills/poindexter/ pack that wraps the CLI/MCP) are a different layer — they carry no metadata.prompts and are not loaded here.

​
Parsing + import-time validation

One parser reads SKILL.md everywhere: services/skill_frontmatter.py (parse_frontmatter + extract_section), shared by both the runtime loader (_initialize_skills) and the importer (poindexter skills import). It anchors the closing --- to the start of a line, so a --- inside a quoted frontmatter value — or a thematic break in the body — is never mistaken for the delimiter. (The two paths previously used different parsers; a pack could import clean and then fail to load.) poindexter skills import fails loud (per feedback_no_silent_defaults.md) when a declared key has no resolvable ## <key> section — it runs the same extract_section the loader uses, so a pack that imports clean is guaranteed to resolve every key it advertises. The check runs before anything is written to disk or recorded in skill_catalog.

​
Langfuse as a read-only mirror

SyncPromptCatalogToLangfuseJob (services/jobs/sync_prompt_catalog_to_langfuse.py, every 6h, gated by langfuse_prompt_mirror_enabled) keeps Langfuse showing every production prompt so the operator reviews the whole catalog in one UI. Per cycle:
  • Missing key → created with the production label and a config.source='skill_sync' provenance marker.
  • Default changed → new version pushed (the label moves with it), so the mirror always matches what production serves.
  • Hand-edited version (production version without the sync marker) → replaced with the current default + a warn prompt_catalog_drift finding (“Langfuse is a mirror — edit the SKILL.md”). The edited body stays in Langfuse version history.
  • Orphaned name (key renamed/deleted in the repo) → warn prompt_catalog_drift finding; deletion stays a human action.
Skips quietly when Langfuse isn’t configured (the OSS default).

​
History — why the mirror is one-way

The pre-2026-07 design was Langfuse-first resolution populated by a manual bulk import (scripts/import_prompts_to_langfuse.py, now retired; its imported_by versions are treated as sync-owned and upgraded in place). That produced the masking trap: the import left a production copy of every key, so later SKILL.md edits shipped green through CI while production silently served stale snapshots — a 2026-07-03 audit found 12 of 39 imported prompts masking newer defaults, including a same-day critic fix, plus 8 orphans under renamed/deleted keys. The fix is structural: SKILL.md is authoritative, the mirror is write-only from the repo’s perspective, and the override layer is an explicit opt-in (poindexter#824, #825).

​
Why Langfuse + SKILL.md, not just one

  • SKILL.md alone: no single place to review the live catalog with version history — you’d read 20+ pack files. Prompt bodies also wouldn’t be linkable from traces.
  • Langfuse alone: prompts wouldn’t ship with the product (the consumer stack has no Langfuse at all), couldn’t be edited atomically with the code that formats them, and would bypass PR review + the contract tests.
  • Both, with SKILL.md authoritative: the pack default is what ships, what CI tests, and what production serves; Langfuse mirrors it for review and keeps the override escape hatch for live experiments.