phpbotscout: delivery plan¶

Authors

Matt Cockayne, Claude Opus 5 (AI drafting assistant)

Date

2026-07-26

Status

APPROVED — 2026-07-27

Implements

Delivery sequencing for phpbotscout: draft requirements (APPROVED) and phpbotscout: interface design (APPROVED).

1. Purpose¶

The requirements define twelve phases. This document plans them: what depends on what, what can run in parallel, what can start before any code is written, where each phase lands, and what could stall the critical path.

It deliberately contains no dates. This is one maintainer working to no external deadline; invented dates would be fiction that later gets treated as commitment. Phases carry relative size (S/M/L) and a dependency order, which is what actually governs sequencing.

How to work this plan¶

1. No phase begins without an approved spec. The requirements are approved; each phase still needs its own implementation spec citing the R-* identifiers it satisfies, per AGENTS.md.
2. A phase is done when its exit criterion is met — not when the code is written. Several exit criteria are measurements, not merges.
3. Every phase that touches user-facing behaviour enters shadow mode before it acts (R-OPS-7).

2. The dependency graph¶

  ┌─────────────────────────────────────────────────────────────────┐
  │  TRACK 0 — no dependencies, start immediately                   │
  │  Discord app + privileged intent · corpora · policy · test guild│
  └───────────────┬─────────────────────────────────────────────────┘
                  │ (intent approval gates every Discord phase)
                  ▼
  ┌──────────────────────────────┐    ┌──────────────────────────────┐
  │ TRACK A — Discord            │    │ TRACK B — retrieval          │
  │                              │    │                              │
  │ 1a-0  library spike     [S]  │    │ 2   index + ask CLI     [L]  │
  │   ↓   (upstream repos)       │    │   ↓                          │
  │ 1a    chat-platform×2   [L]  │    │ 2b  provider benchmark  [M]  │
  │   ↓   (upstream, released)   │    │   ↓                          │
  │ 1b    Discord ingest    [M]  │    │ 2c  hybrid spike        [S]  │
  └───────────────┬──────────────┘    └───────────────┬──────────────┘
                  │                                   │
                  └─────────────┬─────────────────────┘
                                ▼
                     ┌────────────────────────┐
                     │ 3  answering       [M] │ ← the first user-visible phase
                     └───────────┬────────────┘
                                 │
        ┌────────────────────────┼────────────────────────┐
        ▼                        ▼                        ▼
  ┌──────────────┐      ┌────────────────┐       ┌────────────────┐
  │ 4a forge     │      │ 5  moderation  │       │ 4c gap         │
  │    issues[M] │      │           [L]  │       │    explorer[S] │
  │ (upstream)   │      │ (needs 1b +    │       │ (needs 4b)     │
  └──────┬───────┘      │  mod capability│       └────────────────┘
         ▼              │  in provider)  │
  ┌──────────────┐      └────────────────┘
  │ 4b escalation│
  │          [M] │
  └──────┬───────┘
         ▼
  ┌──────────────┐
  │ 6  feedback  │
  │    loop  [M] │
  └──────────────┘

The finding that should change the order¶

Track B has no dependency on Track A. The knowledge index and the ask CLI need no Discord code, no upstream modules, and no privileged-intent approval — R-CLI-1 exists precisely so the retrieval path can be exercised from a terminal.

That matters more than it first appears, because the two tracks have opposite risk profiles:

Track B carries the product's largest genuine unknown and none of its lead time. Track A carries the lead time and a well-contained unknown.

Recommendation: run both, but if only one can progress, progress Track B. Numbering the phases 1 → 6 implies Discord comes first; the dependency graph says it need not, and the risk profile says it should not. Discovering that BM25 over your corpus answers 27 of 30 real questions — or only 11 — is the single most valuable thing that can be learned early, and it can be learned without writing a line of Discord code.

3. Track 0 — start now, no code required¶

None of this depends on anything, and all of it unblocks later phases.

0.3 is the load-bearing one. Three phases are measured against that corpus, and a benchmark built from invented questions measures nothing. Draw them from real support history — things actually asked in Discord, in issues, or by you at 22:40 — and include questions the corpus genuinely cannot answer, because R-KB-8 needs the failure path tested as hard as the success path.

4. Phase detail¶

Each phase lists where it lands, what spec it needs, and what "done" means. Sizes are relative: S ≈ a focused sitting, M ≈ a substantial piece of work, L ≈ the biggest units here.

Phase 1a-0 — Discord library spike · S · upstream¶

Lands in: a scratch worktree; artefacts discarded (R-ARCH-14). Spec: none — the rubric in §3.3 is the spec. Does: the same vertical slice — connect, receive on an allowlisted channel, post a threaded reply — against disgo, discordgo and arikawa, behind a draft contract. Exit: rubric scored; decision record written naming the rejected options and why; the draft contract revised where the spike exposed a bad boundary.

Remember the second objective: this validates the contract, not just the library. Contract changes forced by the spike are the point, not scope creep.

Phase 1a — chat-platform + chat-platform-discord · L · upstream¶

Lands in: two new repositories under gitlab.com/phpboyscout/go/. Spec: required, in the module repo — contract surface, registry, conformance harness. Does: the contract with its read/act split (R-ARCH-10), the optional moderation capability (R-ARCH-11), the depfootprint guard, and the Discord provider on the chosen library. Exit: both modules released; conformance suite green; depfootprint proves no vendor SDK in the contract module.

The largest single unit in the plan, and the one furthest from this repository.

Phase 1b — Discord ingest · M · phpbotscout¶

Needs: 1a released, 0.1 approved, 0.2. Does: consume the modules, register the gateway as a controls-managed service, read-only ingest with channel allowlisting. Posts nothing. Exit: the bot connects, logs messages from allowlisted channels, survives a forced disconnect with session resume, and shuts down gracefully.

Phase 2 — knowledge index + ask CLI · L · phpbotscout¶

Needs: 0.3, 0.7. Nothing from Track A. Does: source discovery against the predicate, the SQLite FTS5 index, incremental refresh, the Retriever contract, and the index and ask commands. Exit: ask answers benchmark questions from the terminal with correct citations, and index sources explains why each source qualified or was excluded.

The most valuable phase in the plan. It answers the product's central question and delivers something usable before any bot exists.

Phase 2b — provider benchmark · M · phpbotscout¶

Needs: 2, 0.3, 0.4. Does: run candidates through the identical retrieval path, scored on the seven-point rubric. Exit: rubric scored, decision record written, per-task provider selections made, and the benchmark committed so it can be re-run when a model is deprecated or a price changes.

Phase 2c — hybrid-retrieval spike · S · phpbotscout¶

Needs: 2, 0.3. Does: classify keyword-only failures by whether semantic search would plausibly have fixed them; estimate effort only if benefit is demonstrated. Exit: decision record with an explicit build-now / build-later / never verdict and the benefit figure behind it.

Phase 3 — Discord answering · M · phpbotscout¶

Needs: 1b, 2, 2b, 0.4, 0.6. The first phase users see. Does: triggers, threaded answers with citations, hedged confidence, feedback reactions, /ask and /docs with their visibility split, rate limiting, and the injection defences. Exit: 30 shadow answers reviewed for quality and citation accuracy (Q-D2), then live. Spend measured, so the R-OPS-3a ceiling can finally be set from evidence.

Publish the policy (0.6) before this goes live, not after.

Phase 4a — forge issue capability · M · upstream¶

Needs: nothing. Can run in parallel with everything. Lands in: go/forge and go/forge-gitlab. Does: the optional IssueProvider capability, the GitLab implementation, conformance coverage. Exit: both released; no existing provider broken; a provider without the capability fails the type assertion cleanly.

Phase 4b — GitLab escalation · M · phpbotscout¶

Needs: 3, 4a released. Does: the consent modal, deduplication with subscription, project routing, mention neutralisation with its adversarial tests, back-links, and scoped relay. Exit: issues filed from Discord with full context, zero mention leakage, and a closure relayed back to the asker with its reason.

Phase 4c — gap explorer · S · phpbotscout¶

Needs: 4b. Does: cluster unanswered and low-confidence questions; browsable TUI ranked by frequency and subscription count. Exit: the report drives one real documentation change. That is the exit criterion — not that the screen renders.

Phase 5 — moderation · L · phpbotscout + upstream¶

Needs: 1b, 0.5, and the moderation capability in chat-platform-discord (R-ARCH-11a). Adversarial coverage still to add (held here rather than in the published corpus, since a list of untested vectors is more useful to an attacker than to a reader):

• multi-turn injection — a payload split across several messages, harmless individually
• retrieval-shaped injection — a question crafted so the retrieved context is itself the attack
• locale and encoding — RTL overrides, zero-width joiners, homoglyphs, in both Discord content and forge payloads
• moderation evasion — content designed to pass the ruleset while still violating it, the inverse of the existing cases

Does: the layered rule engine with bounded regex, the categories including the distinct self-harm path, the private mod queue with its report cards and action buttons, the audit trail, appeals, and the enforcement path with its hard separation from anything LLM-influenced. Exit: false-positive rate below 5% across at least 50 detections, pre-registered before measurement (Q-D3), then graduated to live with the narrowest viable permission grant. If the bar is not met, moderation remains advisory permanently — a legitimate terminal state, not a failed phase. The self-harm category is excluded from this gate and tuned for recall instead.

The most safety-critical phase, and the only one that can harm someone if it is wrong.

Phase 6 — feedback loop · M · phpbotscout¶

Needs: 4b, 4c, 5. Does: knowledge-gap reporting at scale, digests, release announcements, the MCP tool surface, and the dashboard if it is still wanted.

Also, when the public microsite goes up: run the brand-a-project pipeline for the flat house-style emblem, favicon, branding page and estate-page entries. The Discord avatar (docs/assets/brand/avatar-1024.png) is a character, deliberately — it reads as a member in a member list. It does not replace the flat emblem that logo-config and logo-forge share, which is what the docsite, favicon and GitLab project avatar want. Two marks, two jobs. Exit: the gap report is used routinely rather than admired once.

Revisit Q-I7 here: if --output json piped into something existing has been covering it, do not build the dashboard.

5. Critical path¶

0.1 ──▶ 1a-0 ──▶ 1a ──▶ 1b ─────────────┐
                                        ├──▶ 3 ──▶ 4b ──▶ 6
0.3 ──▶ 2 ──▶ 2b ───────────────────────┘        ▲
                                                 │
4a ──────────────────────────────────────────────┘

The longest chain runs through Track A: intent approval → spike → two upstream module releases → ingest → answering. Four of those five are prerequisites for a phase that has not started.

Three consequences worth acting on:

• 0.1 first, though it is quick — everything in Track A needs the app identity and token, and nothing in Track A can be tested without an install on the test guild.
• 4a whenever there is slack. It blocks Phase 4b, depends on nothing, and is the natural thing to pick up while waiting on something else.
• Track B is the hedge. If Track A stalls — intent rejected, library spike inconclusive — Track B still delivers a working ask command and answers the question that actually matters.
• Sequencing is settled by Q-D1: 0.1 immediately, 1a-0 early, then Track B carries the main effort with 1a picked up in the gaps.

6. Risks¶

7. What v1 looks like¶

v1 is complete when a member can ask a question in a public channel and get a cited answer; when a question the corpus cannot answer becomes a GitLab issue with their consent and their context, and they hear back when it resolves; when a moderator has a queue with evidence, an audit trail, and an appeal path; and when the questions the bot could not answer arrive as a ranked list of documentation to write.

Everything in §11 of the requirements — digests, announcements, FAQ synthesis, onboarding — is post-v1 and should stay there.

8. Resolved questions¶

Q-D1 — How the tracks interleave. ✅ Resolved 2026-07-27. Track B is primary, with two Track A items pulled early: 0.1 immediately (external clock), and 1a-0 soon after (small, and it produces a decision). The 1a upstream module work is then the natural thing to pick up while waiting on a benchmark run or a review.

The reasoning is not purely technical. Solo projects stall for lack of visible progress, and a working ask command is genuinely useful and completely invisible. Running the spike early buys an early concrete result without letting Discord's lead time set the pace of everything else.

Q-D2 — What gates Phase 3. ✅ Resolved 2026-07-27. The exit criterion changes from one week in shadow mode to 30 shadow answers reviewed for quality and citation accuracy.

Elapsed time was the wrong unit: it measures the calendar and assumes traffic. A count is robust in both directions — a busy server clears it in days, a quiet one takes longer but still produces a real signal rather than an expired timer. The Phase 2b benchmark remains the primary quality gate; shadow mode confirms behaviour in the wild, which is a different question from whether the answers are good.

Q-D3 — The bar for moderation enforcement. ✅ Resolved 2026-07-27. The threshold is pre-registered before measurement: a false-positive rate below 5% across at least 50 detections. If it is not met, moderation remains advisory permanently, and that is a legitimate terminal state rather than a failure.

Setting the number before there is sunk cost in the ruleset is the entire mechanism. A threshold chosen after seeing the results is a threshold chosen to be met.

The self-harm category is excluded from this gate¶

Not an exception to be argued for later — a different thing being measured.

Every other category treats a false positive as the harm to guard against. Self-harm inverts it: a false positive offers someone support they did not need, which is mildly awkward; a false negative misses someone in crisis. That category is therefore tuned for recall, accepting more false alarms — the opposite posture to the rest of the ruleset.

It also never enforces. Self-harm detections route to support resources and moderator awareness (R-MOD-4), never to delete, timeout or ban, and are excluded from the notification and appeals flow entirely (R-MOD-20). Folding it into a "can we enforce automatically" threshold would conflate two unrelated action classes and would pull the category's tuning in exactly the wrong direction.

9. Open questions¶

None. Sequencing and both graduation gates are settled; the plan's remaining unknowns are the deferred measurements already recorded in the requirements.