# Training Best Practices

R94.12 (2026-05-22) — companion to `/training.html` (the new training surface).

This is Mike's how-to-train-the-system reference. The same content lives inside
the training surface's "📚 How to train this system best" collapsible block.
This file is the standalone reference + cross-link target.

## What this is

The GFS platform learns from **two structured artifacts** that Mike (admin +
domain expert) seeds:

1. **`decision_corpus`** rows — real customer/vendor/item cases mapped to one of
   the 25 `CHAT_INTENTS` (see `src/index.ts` line 1891). R89's few-shot loop
   samples the top 4 most-relevant rows into every chat prompt.
2. **Per-pillar handbooks** + eval cases (`eval/golden-set/*.yaml`) — pinned
   behavior contracts. Eval set IS the spec (round invariant #6).

The training UI at `/training.html` is the bridge that turns Mike's domain
knowledge into both artifacts. This doc explains what makes a great entry.

## Why this matters

Today's corpus is biased (most rows are `review_spec_sheet` seeds; 21 of 25
intents have <3 entries). Bias in → bias out. The R89 few-shot sampler is only
as good as the precedents it has to choose from.

Coverage target post-R94.12:
- **5+ entries per intent** — 25 intents → 125+ minimum
- **10+ entries per pillar** — 8 pillars → 80+ minimum, ideally 200+
- **Diverse customers/items/scenarios** per intent — not 5 entries all about
  Driscoll

## 1. What makes a great training entry

- **One real case** from your 15 years of GFS experience (not synthetic).
- **Specific entities** by name: "Driscoll", "RS7245", "Echo Lake" — not
  "a customer" or "an item".
- **The query as a user would type it** — natural language, your team's
  vocabulary, the way someone actually asks.
- **The answer as it SHOULD appear** — citation pattern, format, what to
  include (price, date, source row).
- **The source evidence** — which D1 table + row is canonical
  (e.g. `pricing_master.id=12834 active 2025-2026`).
- **Confidence score** 0.5-1.0 — how sure are you this is correct?
  0.95+ for bedrock patterns, 0.65-0.80 for likely cases.

The system learns from **real signal**, not generic phrasing. Each great entry
compounds — R89 few-shot loop injects the top 4 most-relevant samples into
every chat prompt.

## 2. Best-practice template (with example)

```
INTENT:           customer_price_lookup
PILLAR:           price_admin
QUERY:            What does Driscoll pay for RS7245?
ANSWER TEMPLATE:  Driscoll's active price for RS7245 is $21.00/case as of
                  2026-01-15, from pricing_master row 12834. Source: invoice
                  INV-44721 from 2026-02-03 confirms.
ENTITY EXAMPLES:  ["Driscoll", "RS7245"]
SOURCE EVIDENCE:  pricing_master.id=12834 active 2025-2026
CONFIDENCE:       0.95
RATIONALE:        Real Driscoll pricing pattern; this is how the answer should
                  always look — quote price + date + source row + invoice
                  cross-validation.
```

Per-intent template scaffolds live at `data/training_templates.json` —
training.html pre-fills the form when you pick an intent.

## 3. Bulk-import format

Three accepted formats. Paste a block into "📥 Bulk Import" in
`/training.html`, preview parsed entries, then confirm.

### Option A — JSON array

```json
[
  {
    "intent": "customer_price_lookup",
    "query_template": "What does Driscoll pay for RS7245?",
    "answer_template": "Driscoll's active price for RS7245 is $21.00/case as of 2026-01-15, from pricing_master row 12834.",
    "entity_examples": ["Driscoll", "RS7245"],
    "source_evidence": "pricing_master.id=12834 active 2025-2026",
    "confidence_score": 0.95
  }
]
```

### Option B — Markdown blocks

```
# Entry
- intent: customer_price_lookup
- query: What does Driscoll pay for RS7245?
- answer: Driscoll's active price for RS7245 is $21.00/case as of 2026-01-15.
- entities: Driscoll, RS7245
- source: pricing_master.id=12834 active 2025-2026
- confidence: 0.95
```

### Option C — Plain text (blank-line separated)

```
intent: customer_price_lookup
query: What does Driscoll pay for RS7245?
answer: Driscoll's active price for RS7245 is $21.00/case as of 2026-01-15.
entities: Driscoll, RS7245
source: pricing_master.id=12834 active 2025-2026
confidence: 0.95
```

All three formats parse to the same schema. Preview shows per-entry parse
status before you confirm.

Behind the scenes: training.html POSTs to `/api/corpus/seed` (R102.1) in
batches of 10 (per endpoint hint). The endpoint:
- validates intent against `CHAT_INTENTS`
- enforces length bounds (10-2000 chars on query/answer)
- requires source_evidence >5 chars
- clamps confidence to 0.5-1.0
- **scrubs for PII** (SSN, credit card, email, phone) — entries with PII are
  rejected (fail-closed)
- dedupes on intent+rationale (idempotent — re-pasting is safe)
- inserts as `decision_type='pattern_rule'` with
  `tags=['intent:<id>', 'seed_source:r102_mike_admin_sprint']`

## 4. Coverage targets

| Target | What | How to hit it |
|---|---|---|
| 5+ per intent | 25 intents, 125+ entries minimum | Sidebar shows intent counts; fill 0-count intents first |
| 10+ per pillar | 8 pillars (Price/Nutrition/Finance/Relationship/Order Mgmt/Production/Bid/All) | Click a pillar in the bar to filter intents to that pillar |
| Diverse entities | mix customers (Driscoll, NYCDOE, Hempstead, NMSNC), mix items (RS7245, AIS050, GFS-prefix), mix vendors (Echo Lake, Sun-Rich, Bongards) | Resist the "5 Driscoll cases" trap |

Quick distribution checks via `wrangler d1`:

```sql
-- Per-intent counts (uses the tag scheme)
SELECT json_extract(tags, '$[0]') AS intent_tag, COUNT(*)
FROM decision_corpus
WHERE archived_at IS NULL
GROUP BY intent_tag
ORDER BY 2 DESC;

-- Per-confidence buckets
SELECT
  CASE WHEN confidence_score >= 0.95 THEN '0.95-1.0'
       WHEN confidence_score >= 0.80 THEN '0.80-0.95'
       WHEN confidence_score >= 0.65 THEN '0.65-0.80'
       ELSE '0.50-0.65' END AS bucket,
  COUNT(*)
FROM decision_corpus
WHERE archived_at IS NULL
GROUP BY bucket;
```

## 5. Bad entries to avoid

- **Synthetic / made-up cases** — the bot trains on noise and will cite fake
  rows. If you can't point at a real D1 row, don't seed it.
- **Generic templates** — "customer wants price" isn't specific enough. Use
  real customer + real item.
- **Wrong source evidence** — the bot will cite the wrong place. Verify the
  source row exists before seeding.
- **Low confidence with no rationale** — don't seed unsure cases. If <0.7,
  write a clear rationale explaining the uncertainty.
- **Duplicates of existing entries** — the endpoint dedupes by intent+rationale,
  but it wastes your time.
- **Stale data** — last year's price for a customer that re-contracted in
  January is no longer canonical.

## 6. Anti-patterns (hard rules)

- **Never include PII** — emails, phone numbers, SSNs. The endpoint scrubs PII
  patterns and will reject; human review is the safety net.
- **Never train on internal security/audit details** — per Mike's
  `no_gfs_exposure` rule. Skip anything about secrets, tokens, audit logs,
  infra.
- **Never re-seed existing entries** — the system dedupes (intent+rationale)
  but it wastes your time + clutters the log.
- **Never seed below 0.5 confidence** — endpoint clamps to 0.5-1.0; below =
  qualitative folklore, not training data.
- **Never use a Markdown header (#, ##) inside an entry value** — confuses the
  markdown parser.

## Confidence score buckets

Range: **0.5..1.0** (clamped server-side).

| Bucket | Meaning |
|---|---|
| 0.95-1.0 | Bedrock pattern, multiple corroborating rows, never seen an exception |
| 0.80-0.95 | Solid case, 1-2 confirming rows |
| 0.65-0.80 | Likely case, single anecdote |
| 0.50-0.65 | Qualitative pattern, no specific row to point at |

Default if omitted: 0.8.

## Where the data flows

```
training.html
   │
   ├── single-entry form  ─┐
   ├── bulk-import flow   ─┴─► POST /api/corpus/seed (R102.1)
   │                                │
   │                                ▼
   │                          decision_corpus
   │                          (pattern_rule rows, tagged with intent + seed_source)
   │                                │
   │                                ▼
   │                          /api/training/backfill-pattern-embeddings
   │                                │
   │                                ▼
   │                          Vectorize 'patterns' namespace
   │                                │
   ▼                                ▼
/training-legacy.html        R89 sampleDecisionCorpusFewShot()
(R60 question bank)                 │
   │                                ▼
   ▼                          <recent_precedents> block in pricing-chat
training_question_attempts   prompt → top 4 most-relevant precedents
training_sessions            injected every turn
decision_corpus (via declare-pattern)
```

## Rollback (if needed)

The R94.12 batch is stamped via `tags` JSON. To soft-archive the whole batch:

```sql
UPDATE decision_corpus
SET archived_at = datetime('now')
WHERE archived_at IS NULL
  AND tags LIKE '%seed_source:r102_mike_admin_sprint%';
```

## Cross-references

- Training surface: `/training.html` (R94.12)
- Legacy 235-question bank: `/training-legacy.html` (R60-era, preserved)
- Per-intent template scaffolds: `data/training_templates.json`
- Seeding endpoint: `src/index.ts` (search `R102 — Decision corpus seeding`)
- Few-shot sampler: `src/index.ts` `sampleDecisionCorpusFewShot()` (R89)
- Schema: `migrations/029_training_corpus.sql`
- R102 batch seeding guide: `docs/R102_SEEDING_GUIDE.md`
- Methodology: `docs/METHODOLOGY.md` (round invariants — eval set IS the spec)