​

Module v1 — making the plugin substrate explicit

​

What shipped 2026-05-13

​

Why

1. Fuzzy module boundaries. The content pipeline calls into ai_content_generator.py (1,331 LOC) which calls back into the stages tree; brain/ has its own embeddings outside the main migration system (Glad-Labs/poindexter#328); the “Content Module” isn’t a thing you can install or describe in one sentence.
2. OSS / business mix is informal. scripts/sync-to-github.sh filters by a hand-maintained path list (web/public-site, web/storefront, mcp-server-gladlabs, marketing, premium dashboards, writing_samples, .shared-context, CLAUDE.md). The line keeps moving; a new private-overlay file requires editing the sync filter, which is easy to forget.
3. Pipeline orchestration drag. A new module currently needs plumbing across 6+ surfaces — Prefect flow + Telegram command + MCP tool + Grafana panel + brain probe + CLI subcommand. Each surface has its own registration path. Adding a finance module feels like a five-day plumbing project.

​

The 70% already built

• Plugin registry exists via setuptools entry_points() — plugins/registry.py, no custom registry code, same mechanism pytest / click / flask use. 19 entry-point groups defined: taps, probes, jobs, stages, reviewers, adapters, providers, packs, llm_providers, topic_sources, image_providers, audio_gen_providers, video_providers, tts_providers, caption_providers, publish_adapters.
• Plugin Protocol interfaces are formalised with @runtime_checkable Protocols in plugins/{tap,stage,reviewer, adapter,probe,job,llm_provider,pack,image_provider, audio_gen_provider,video_provider,tts_provider,caption_provider, media_compositor,publish_adapter,topic_source}.py. Each has a typed Result / Document / Completion data class.
• Sample plugins at plugins/samples/ — database_probe.py, hello_tap.py, noop_job.py — work as reference implementations.
• Substrate services are wired and load-bearing: services/cost_guard.py, services/prompt_manager.py (UnifiedPromptManager + Langfuse), services/site_config.py DI seam, services/audit_log.py, services/memory_client.py, the brain daemon (brain/).
• Operator surface is in place: a single Telegram bot (poindexter-pipeline-bot container, 24/7), the MCP server (25 tools), the FastAPI worker, the poindexter CLI.
• Orchestration is canonical: Prefect, as of the Stage 3 cutover (2026-05-13).
• Declarative data plane — 5 tables (external_taps, retention_policies, webhook_endpoints, publishing_adapters, qa_gates) feed 14 handlers across 5 surfaces.

​

What changes — Module v1 in five components

​

Component 1 — Module manifest + new entry-point group

[project.entry-points."poindexter.modules"]
content = "poindexter_module_content:ContentModule"

@runtime_checkable
class Module(Protocol):
    """A self-contained business function (content, finance, HR, ...).

    A Module BUNDLES the lower-level plugin contributions (stages,
    reviewers, probes, jobs, taps, adapters, providers, packs) plus
    the things the existing registry doesn't track (DB migrations,
    Grafana panels, HTTP routes, CLI commands).
    """

    name: str                       # canonical slug, e.g. "content"
    version: str                    # semver
    visibility: Literal["public", "private"]
    requires: list[str]             # ["substrate>=1.0", "module:memory"]

    def manifest(self) -> ModuleManifest: ...

    async def migrate(self, pool: asyncpg.Pool) -> None:
        """Apply this module's DB migrations. Idempotent."""

    def register_routes(self, app: FastAPI) -> None:
        """Mount this module's HTTP surface on the host app."""

    def register_cli(self, parser: CLISubparser) -> None:
        """Register `poindexter <module> <subcommand>` entries."""

    def register_dashboards(self, grafana: GrafanaProvisioner) -> None:
        """Contribute Grafana panels under a per-module folder."""

    def register_probes(self, brain: BrainProbeRegistry) -> None:
        """Brain probes specific to this module."""

    async def healthcheck(self, pool: asyncpg.Pool) -> ProbeResult:
        """Module-wide health (sums sub-probe results)."""

​

Component 2 — Per-module migrations

poindexter_module_content/
├── __init__.py
├── module.py            # ContentModule class
├── migrations/
│   ├── 20260514_120000_init.py
│   ├── 20260520_080000_add_dev_diary_template.py
│   └── ...
├── prompts/
├── flows/
├── routes/
├── dashboards/
└── probes/

​

Component 3 — Per-module Grafana dashboards

​

Component 4 — HTTP route auto-discovery

# in poindexter_module_content.module
def register_routes(self, app: FastAPI) -> None:
    from .routes import tasks, approval, posts
    app.include_router(tasks.router, prefix="/api/content/tasks", tags=["content"])
    app.include_router(approval.router, prefix="/api/content/approval", tags=["content"])
    app.include_router(posts.router, prefix="/api/content/posts", tags=["content"])

​

Component 5 — visibility flag replaces the sync filter

1. Walk poindexter.modules entry-points.
2. Include modules with visibility="public" in the OSS sync.
3. Exclude modules with visibility="private".

​

The OSS / business split made concrete

• poindexter (public OSS) ships:
  • the substrate (brain, prefect, langfuse, cost_guard, prompt_manager, site_config, audit_log, memory_client, telegram-bot, MCP server, CLI)
  • the content module as the reference implementation: canonical_blog template, multi-model QA, programmatic validator, image-gen/Pexels image stage, publish to a self-hosted Next.js + DB.
  • the base module that provides shared utilities every module needs (admin settings UI, audit_log explorer, healthcheck aggregator). Could also be inlined into substrate; the spec treats it as a module for symmetry.
• glad-labs-stack (private overlay) ships:
  • the gladlabs-business module (renamed from the current mcp-server-gladlabs/ + web/storefront/ + marketing scripts): Lemon Squeezy customer lookup, premium-prompts seeder, Glad-Labs brand assets, R2 license-delivery API, the storefront Next.js app.
  • web/public-site/ stays here too (Vercel deploys from this repo).
  • the Glad-Labs Postgres + Grafana + Telegram bot instances — deployment artifacts that depend on these modules but aren’t distributed.

​

Migration path from today

​

Phase 1 — Module manifest + registry (~1 day)

• Define plugins/module.py with the Module Protocol and a ModuleManifest data class.
• Add the poindexter.modules entry-point group constant.
• Extend plugins/registry.py with get_modules() -> list[Module].
• Pin behaviour with 5 unit tests (module discovery, manifest validation, dependency cycles).

​

Phase 2 — Per-module migration runner (~1 day)

• New services/module_migrations.py runner. Same shape as services/migrations/__init__.py but keyed on (module_name, migration_name).
• New module_schema_migrations table.
• Boot wiring: after substrate migrations apply, iterate registered modules and call await module.migrate(pool).

​

Phase 3 — Convert content/ into a Module (~1-2 days)

• Create poindexter_module_content/ package (could be in-tree initially under src/poindexter_module_content/ then split out to its own repo when the substrate stabilises).
• Move existing content-pipeline code: services/content_router_service.py, the stages tree, multi_model_qa.py, content_validator.py, the content-specific YAML prompts (the qa.* / image.decision / topic.ranking / narrative.* keys that landed via Lane A Glad-Labs/poindexter#450), the canonical_blog LangGraph template, the content_generation Prefect flow + deployment. prompt_manager.py (UnifiedPromptManager — the loader) stays in substrate; only the prompt files move.
• Move content-specific migrations from services/migrations/ to poindexter_module_content/migrations/.
• Register the ContentModule via entry-point.
• All existing tests keep passing; the change is structural, not behavioural.

​

Phase 4 — Per-module routes + dashboards (~1 day)

• Refactor main.py to iterate get_modules() and call register_routes(app) for each.
• Refactor infrastructure/grafana/ into per-module folders.
• Add Module.register_dashboards(provisioner) to seed.
• Update Grafana provisioning workflow.

​

Phase 4 (closeout) — register_probes wire-up (#239, 2026-05-27)

• plugins/probe_registry.py defines BrainProbeRegistry — a worker-side collector that modules write into during register_probes(registry). Rejects duplicate FQIDs per feedback_no_silent_defaults.
• main.py lifespan constructs the shared registry, hands it to every discovered module, then stashes it on app.state for request handlers.
• GET /api/modules/probes returns the registered probe specs (module / name / description / interval). 503 when the registry is missing — never silently empty.
• Cross-process bridge to the brain daemon (poll the endpoint, run the probes on the brain’s side) is intentionally deferred: no module ships a concrete probe yet, so the discovery half is enough to unblock the next module that needs one.

​

Phase 5 — visibility + sync rewrite (~0.5 day)

• Add visibility field to manifest.
• Rewrite scripts/sync-to-github.sh (or replace with a Python script) to iterate modules and include only visibility="public".
• Document the new sync contract in docs/operations/oss-sync.md.

​

Non-goals

• Module sandboxing / process isolation. Modules run in the same Python process as today. If a future module needs isolation (third-party untrusted code), that’s a separate spec.
• Module hot-reload. Modules are discovered at boot. Adding a module still requires a restart. Hot-reload doesn’t compose with Python’s import semantics.
• A plugin marketplace. Distribution is pip install <package> for now. A marketplace can come later if there’s demand.
• Cross-module RPC. Modules continue to communicate through shared substrate (Postgres, audit_log, brain knowledge graph) — no new RPC layer.
• Versioned migrations across modules. A module that depends on another module’s tables can declare requires: ["module:content>=1.0"] but the runner does not yet enforce migration ordering across modules. If module B needs a table from module A, ensure A is installed first. This may need revisiting once we have 3+ modules.

​

Testing strategy

• Phase 1 — tests/unit/plugins/test_module_registry.py: 5 cases covering discovery, manifest validation, missing fields, duplicate module names, dependency cycle detection.
• Phase 2 — tests/integration_db/test_module_migrations.py: fresh poindexter_test_<hex> DB, register two test modules, verify both their migrations apply, verify module_schema_migrations rows land correctly, verify reruns are no-ops.
• Phase 3 — tests/integration_db/test_content_module_e2e.py: with the content module registered, run one pipeline_tasks row end-to-end through the Prefect flow. Should produce the same awaiting_approval outcome as today’s path. Pin the contract.
• Phase 4 — tests/unit/routes/test_module_route_discovery.py: spin up a TestClient with two test modules registered, verify both modules’ routes are mounted at their declared prefixes.
• Phase 5 — tests/unit/scripts/test_visibility_filter.py: feed the sync filter a fake module registry with mixed visibility, verify only visibility="public" modules survive the filter.

​

Open questions / future work

• Naming. Is Module the right word, or does it collide with Python’s existing module semantics in confusing ways? Maybe Workspace or Domain or Slice? Defer until Phase 1 review. Resolved 2026-05-13: keep Module. Matt: “modules in reference to Python files etc. is very broad — we’ll know the difference.”
• Sub-modules. Should gladlabs-business further decompose into gladlabs-storefront, gladlabs-licenses, gladlabs-premium? Not now — get one private overlay module shipping first.
• In-tree vs out-of-tree packages. Phase 3 puts the content module package in-tree (src/poindexter_module_content/) for convenience. Once the contract is stable, extracting to its own repo + pip install is straightforward. Defer the split until there are 2 in-tree modules.

​

Ground truth references

• src/cofounder_agent/plugins/registry.py — existing entry-point discovery, basis for Component 1.
• src/cofounder_agent/plugins/{tap,stage,probe,job,...}.py — existing plugin Protocols that modules will bundle.
• src/cofounder_agent/services/migrations/__init__.py — runner shape that Component 2 will mirror.
• scripts/sync-to-github.sh — sync filter Component 5 replaces.
• CLAUDE.md — operator-facing description; will need an update when Phase 3 lands.