​

Plugin Architecture

⚠️ This is a design specification, not a shipping status document. It describes how Poindexter is evolving architecturally. Some sections describe shipped work; others describe future phases. For current deployment state, see CLAUDE.md.

Capability plugins vs business modules. The Phases A–J on this page describe capability plugins — atoms like Tap, Stage, Provider, LLMProvider. Module v1 is the next layer up: a Module bundles several capability plugins into a manifested, install-shaped business function. They’re orthogonal axes, not competing decompositions. Most extensions slot into the capability layer; you only reach for a Module when you’re adding a new business surface that needs its own DB migrations, jobs, routes, dashboards.

v3 locked decisions (2026-04-19):

• Secrets encryption-at-rest is core scope (Phase A delivers via pgcrypto), not “known gap.”
• Tracing + observability instrumentation (OTel, Tempo, OpenLLMetry) is deferred to a post-refactor Phase I. We’ll know what to monitor after the refactor is done.
• Phase J (LLMProvider plugin family) confirmed as core scope. Swap Ollama → vllm/llama.cpp/TGI/LocalAI by one app_settings row.
• Phase A0 (integration harness, #21) shipped 2026-04-19. Phase A is unblocked.
• Other v2 gaps (config schema versioning, resource limits, plugin-to-plugin deps) → Phase I.

​

Problem

​

Core insight: adopt standards, don’t invent

​

The original six plugin Protocols

​

1. Tap — data ingestion

class Tap(Protocol):
    name: str
    interval_seconds: int  # 0 = on-demand only
    async def extract(self, pool, config) -> AsyncIterator[Document]: ...

• Native Python Taps — for internal sources (our own posts, our own memory files, audit log). Fast, no subprocess overhead.
• SingerTap wrapper — wraps any Singer binary (tap-github, tap-slack, tap-gmail, tap-google-analytics). Subprocess + stdout JSON-lines. Unlocks the public Singer catalog with zero per-source code.

​

2. Probe — state checking

​

3. Job — scheduled maintenance

class Job(Protocol):
    name: str
    schedule: str  # cron or interval, handed to apscheduler
    async def run(
        self, pool, config, *, site_config: SiteConfig,
    ) -> JobResult: ...

​

4. Stage — pipeline transformer

• Reviewer(Stage) — scores content (programmatic_validator, llm_critic, seo_checker, url_verifier)
• Adapter(Stage) — publishes to a platform (the legacy social_adapters/ impls were retired 2026-06-29; distribution is now publishing.<name> handlers + Postiz)
• Provider(Stage) — generates media (Pexels, image-gen, AI-generation, future Midjourney/Flux)

​

5. Pack — bundled prompts + styles + configs

• Free (community): poindexter-pack-community on public pypi, Apache-2.0
• Premium: glad-labs-pack on private pypi, license-gated

​

6. LLMProvider — inference backend

class LLMProvider(Protocol):
    name: str
    async def complete(messages, model, **kwargs) -> Completion: ...
    async def stream(messages, model, **kwargs) -> AsyncIterator[Token]: ...
    async def embed(text, model) -> list[float]: ...

1. OpenAICompatProvider — generic HTTP client with configurable base_url + model. Reaches Ollama (/v1 endpoint), llama.cpp server, vllm, SGLang, HuggingFace TGI, LM Studio, LocalAI by config.
2. OllamaNativeProvider — keeps Ollama-specific features (electricity cost tracking, /api/embed, model pull). Default for the out-of-box experience.
3. LiteLLMProvider (shipped 2026-05-04, poindexter#199 phase 1) — see services/litellm_provider.md. Wraps the LiteLLM SDK so one plugin covers 100+ providers (Ollama, OpenAI, Anthropic, Gemini, Bedrock, Vertex, OpenRouter, etc.) with authoritative cost tracking + retries-with-backoff via mature OSS. Activates by setting plugin.llm_provider.primary.standard='litellm'. Phase 2 (delete services/model_router.py) waits on production validation.

​

Plugin discovery: setuptools entry_points

[project.entry-points."poindexter.taps"]
github = "poindexter_tap_github:GithubTap"
memory = "poindexter_tap_memory:MemoryFilesTap"

[project.entry-points."poindexter.llm_providers"]
openai_compat = "poindexter_llm_openai_compat:OpenAICompatProvider"

• Install: pip install poindexter-tap-slack
• Uninstall: pip uninstall poindexter-tap-slack
• Update: pip install -U poindexter-tap-slack
• List: pip list | grep poindexter- or importlib.metadata.entry_points()

​

Config boundary: DB vs file

{
  "key": "plugin.tap.github",
  "value": "{\"enabled\": true, \"interval_seconds\": 3600, \"config\": {\"repo\": \"Glad-Labs/poindexter\"}}",
  "category": "plugins"
}

​

Architectural invariants

1. Everything configurable — every plugin reads from app_settings (runtime) or repo files (infra rules); no hardcoded defaults that can’t be overridden.
2. Everything an adapter of a core functionality — six Protocols; every feature is one of them.
3. Everything in the database — runtime config, plugin state, secrets, metrics history. Only DATABASE_URL lives outside.
4. Drop-in additions — pip install + container restart picks up the new plugin. No central switch statements, no hardcoded plugin lists.
5. Core is OSS-only, community can extend — default shipping policy ships free/OSS backends (Ollama, Singer community taps, Prometheus). Community plugins can wrap paid providers and ship on their own pypi packages.
6. Standards over inventions — when an OSS standard exists (Singer, OpenAI-compat, apscheduler, Prometheus, entry_points), we wrap it, not reinvent.

​

Migration phases

​

Observability strategy

• Logs: structlog (already in stack). Request-ID propagation added as part of Phase D.
• LLM cost: native cost_logs table stays — cheapest source of truth for electricity + token counts.

​

Dependency audit

​

Known gaps (tracked, not blocking)

• Secrets encryption-at-rest — now core (pgcrypto) per 2026-04-19 lock-in.

• Plugin config schema versioning + migrations
• Per-plugin resource limits (CPU/memory quotas)
• Plugin-to-plugin dependency resolution
• Full observability stack (OTel, Tempo, OpenLLMetry, Langfuse)

• Bare except Exception: sweep (125 swallowing blocks across services + routes)
• memory_system.py dead code (966 lines — pyproject.toml marks it superseded by pgvector)
• unified_orchestrator.py + task_executor.py vs content_router_service.py — resolved 2026-05-16: Prefect Stage 4 deleted task_executor.py and unified_orchestrator.py; content_router_service.py is now a thin TemplateRunner dispatcher.
• Routes decomposition (task_publishing_routes.py 1049 lines, cms_routes.py 959 lines)
• 50-migration consolidation pass
• Dependency license audit
• Swap Google Analytics for Plausible/Matomo (only remaining non-OSS surface)
• Dependency license audit
• Swap GA for Plausible/Matomo

​

Further reading

• GitHub #64 — umbrella (v2)
• GitHub #53 — original plugin ecosystem vision (Taps / Probes / Prompt Packs)
• GitHub #20 — content_router split
• GitHub #56 — brain daemon pluggable watchdog
• Brain memory under plugin refactor v2 tag — mcp__poindexter__search_memory