v1.3.1: Exhaustive decision-tree search + UX refinements

The v1.3.0 saturation termination silently capped the search after only
the heuristic-favored part of the tree, leaving most per-set ceiling cells
stuck at "0 specs" and hiding genuinely-feasible 3-spec plans in
maximize-count mode. Replace with full exhaustive enumeration plus a
batch of UX refinements that emerged during testing.

Algorithm:

- Drop the saturation early-termination entirely. Search now runs the
  full open-set cartesian product to completion; the iteration cap is
  also removed so no scenario exits partial.
- Add mode-dependent DFS child ordering: priority-order keeps the
  priority-target-first heuristic; maximize-count orders children by
  descending count of qualifications for reachable specs (generalist
  courses tried first).
- Make the (count, priorityScore) comparator mode-aware: priority-order
  ranks by (priorityScore, count) so the user's top spec surfaces;
  maximize-count ranks by (count, priorityScore) so the highest count
  wins. The same rule drives both top-K position and per-cell ceiling
  selection (and the Recommended badge).
- Add an evaluated boolean to each ChoiceOutcome and set it on first
  leaf evaluation. Distinguishes "still searching" from "evaluated, no
  specs achieved" so the UI never shows misleading 0 specs for a cell
  the search hasn't reached yet.
- Throttled progress events (~100ms) carrying iterations / total leaf
  count, drive both the per-set spinner and the global progress bar.

UI:

- Top Plans header shows a horizontal progress bar with
  "iterations / total · NN%" while the search runs; collapses to
  "Search complete · N explored" on completion.
- Per-set spinner next to each elective set heading while any choice
  in that set is unevaluated.
- Per-cell pulsing dot + "searching" text for unevaluated cells.
- Replace the "(HCR, BNK, ...)" text labels on each course with
  color-coded SpecTag pills using a new fixed per-spec palette
  (app/src/data/specColors.ts). Same palette applied to the Top Plans
  achievement badges so the two views are visually consistent.
- "Top outcome if picked ↓" caption above the right side of each open
  elective set so the spec tags are clearly identified as decision-tree
  outcomes (not the course's own qualifications).
- Recommended badge moved inline next to the course name (instead of
  on a separate row below) to keep button heights stable.

Tests:

- Replace the saturation early-termination test with an exhaustion test
  asserting every cell ends with evaluated: true and partial: false.
- Add mode-dependent ordering test (max-count visits Climate Finance
  before Corporate Governance in fall3).
- Add evaluated-flag transition test.
- Add throttled progress-event test (>= ~100ms between consecutive
  emits).
- Performance smoke updated to a 60s budget for the exhaustive
  user-scenario search; 8-open-set typical case completes in ~7s.

Files: solver/decisionTree.ts, solver/priority.ts (already shipped),
data/specColors.ts (new), components/{TopPlans,CourseSelection}.tsx,
state/appState.ts, workers/decisionTree.worker.ts,
__tests__/searchDecisionTree.test.ts, vite.config.ts, CHANGELOG.md,
openspec/changes/decision-tree-exhaustive-search/* (full change spec).
This commit is contained in:
2026-05-09 15:47:56 -04:00
parent 4b80fac500
commit cb49123930
16 changed files with 780 additions and 110 deletions
@@ -0,0 +1,75 @@
## 1. Drop saturation, raise cap
- [x] 1.1 In `app/src/solver/decisionTree.ts`, remove the `SATURATION_LIMIT` constant and all references to `iterationsSinceTopKChange` (declaration, increment, reset, comparison)
- [x] 1.2 Raise `MAX_TREE_ITERATIONS` to `100_000`
- [x] 1.3 Confirm the only termination paths are now: (a) iteration cap fires (sets `partial = true`), or (b) DFS exhausts the cartesian product
## 2. Mode-dependent ordering
- [x] 2.1 Add `reorderByReachableQualCount(setId, upperBounds, excludedCourseIds): Course[]` — returns courses sorted by descending count of qualifications for specs whose `upperBounds[specId] >= 9`. Stable sort; ties keep declaration order; cancelled courses excluded.
- [x] 2.2 In `searchDecisionTree`, replace the `orderedCoursesPerSet` initialization to gate by `mode`: `priority-order` keeps `reorderForTarget(setId, priorityTarget, ...)`; `maximize-count` uses `reorderByReachableQualCount(setId, upperBounds, ...)`
- [x] 2.3 Unit test: assert that in maximize-count mode, the first DFS leaf for the user's reproduction scenario contains `fall3-climate-finance` (the most-generalist Fall Set 3 course) before `fall3-emerging-tech`
## 3. Evaluated/unevaluated cell state
- [x] 3.1 Add `evaluated: boolean` field to `ChoiceOutcome` (`app/src/solver/decisionTree.ts`); initialize to `false` in the per-set analysis init loop
- [x] 3.2 In `evaluateLeaf`, after running the optimizer and computing `result`, set `choice.evaluated = true` for every `(setId, courseId)` in the leaf's assignments BEFORE the comparison. (The cell is "evaluated" the moment a leaf containing it is run, regardless of whether the result improves the ceiling.)
- [x] 3.3 Confirm the `choiceUpdate` callback fires whenever either `evaluated` flips to `true` or the ceiling improves — so the UI sees the transition. Update the existing condition to fire on both
- [x] 3.4 Unit test: after one leaf evaluation containing `(spr3, spr3-analytics-ml)`, that cell has `evaluated: true`; cells in other sets that aren't in the leaf assignment remain `evaluated: false`
## 4. Throttled progress events
- [x] 4.1 Add `progress` to the `WorkerResponse` tagged union in `app/src/workers/decisionTree.worker.ts`: `{ type: 'progress'; iterations: number; iterationsTotal: number }`
- [x] 4.2 Compute `iterationsTotal` in `searchDecisionTree` before DFS starts: product of `orderedCoursesPerSet[setId].length` over all `openSetIds`. Pass it through to the progress callback
- [x] 4.3 Add `onProgress?: (iterations: number, iterationsTotal: number) => void` to `SearchCallbacks`. Track `lastProgressEmit: number` (default 0); call `onProgress` from inside `evaluateLeaf` when `Date.now() - lastProgressEmit >= 100`
- [x] 4.4 In the worker, wire `onProgress` to `postMessage({ type: 'progress', iterations, iterationsTotal })`
## 5. App state wiring
- [x] 5.1 In `app/src/state/appState.ts`, add a `searchProgress: { iterations: number; iterationsTotal: number } | null` slice (default `null`); update on `progress` events; reset to `null` on new search start and on `allComplete`
- [x] 5.2 Export `searchProgress` from `useAppState` alongside `topPlans`/`topPlansPartial`
- [x] 5.3 The `choiceUpdate` handler already updates the per-set map; verify the new `evaluated` field flows through unchanged (no code change needed; `analysis` is forwarded as-is)
## 6. Top Plans header (global progress)
- [x] 6.1 In `app/src/components/TopPlans.tsx`, accept `searchProgress` and `loading` props and render in the header:
- while `loading && searchProgress`: `Searching… {iterations.toLocaleString()} / {iterationsTotal.toLocaleString()} explored`
- after complete (`!loading && !partial`): `Search complete · {totalEvaluated} explored`
- after complete with `partial`: `Search incomplete · cap hit at {MAX_TREE_ITERATIONS}` (existing partial caption replaced/extended)
- [x] 6.2 Pass `searchProgress` from App.tsx into `<TopPlans>`
## 7. CourseSelection per-set + per-cell rendering
- [x] 7.1 In `app/src/components/CourseSelection.tsx`, in the `ElectiveSet` heading area: render a small spinner/dot next to the set name when `loading === true` AND `analysis?.choices.some(c => !c.evaluated)`. The existing "high impact" badge stays
- [x] 7.2 Replace the per-cell ceiling render branch:
- `!evaluated`: render a faint "·" or pulsing dot (use the existing skeleton pattern restyled, or a small `…` glyph) instead of "0 specs"
- `evaluated && ceilingCount === 0`: render "0 specs" in muted grey
- `evaluated && ceilingCount > 0`: render existing colored "N specs (LIST)" treatment
- [x] 7.3 Compute the recommended choice per set: pick the choice with the best `(ceilingCount desc, priorityScore desc)` — only consider choices with `evaluated === true`. Render a small `⭐ Recommended` badge on that row. Hide the badge if no choice is yet evaluated
- [x] 7.4 The recommended derivation needs `priorityScore`; import `makePriorityScorer` from `app/src/solver/priority` and memoize per-render with `state.ranking`
## 8. Tests
- [x] 8.1 Remove the saturation early-termination test from `app/src/solver/__tests__/searchDecisionTree.test.ts` (the test that asserts iteration count stays well under cap when topK converges quickly — no longer applies)
- [x] 8.2 Add an exhaustion test: small scenario (e.g., 2 open sets); after `searchDecisionTree` returns, every choice in every open set has `evaluated: true` and `partial === false`
- [x] 8.3 Add a mode-dependent ordering test: in maximize-count mode for the user's reproduction scenario, the first leaf evaluated contains `fall3-climate-finance` (verify by capturing the first `onChoiceUpdate` event for `fall3` and inspecting the assignment)
- [x] 8.4 Add an evaluated-flag transition test: assert all cells start `evaluated: false`; after one leaf evaluation, only cells with that leaf's assignments are `evaluated: true`
- [x] 8.5 Update the streaming monotonicity test if needed (still valid in concept; just verify with new termination)
- [x] 8.6 Add a progress-event throttling test: capture `onProgress` calls during a search; assert minimum interval >= 90ms between consecutive calls (small jitter tolerance)
- [x] 8.7 Update the performance smoke test to allow longer time budget (e.g., 60 seconds) since the search is now exhaustive
- [x] 8.8 Run full test suite; confirm all pass
## 9. Browser verification
- [x] 9.1 Start dev server; reproduce user's pin scenario (SP2/SP4/SP5/SE1, HCR first)
- [x] 9.2 Switch to maximize-count mode; confirm Top Plans surfaces 3-spec plans (e.g., `[BNK, CRF, FIN]` triples) if any are feasible — if not feasible for this scenario, try a less-pinned scenario to confirm 3-spec plans CAN appear
- [x] 9.3 Confirm per-set spinner appears next to "Spring Elective Set 1" while search runs and clears when complete
- [x] 9.4 Confirm per-cell rendering shows "·" or similar for unevaluated cells, then transitions to "N specs" or "0 specs" as evaluation completes
- [x] 9.5 Confirm `⭐ Recommended` appears on one course per set after at least one cell in that set is evaluated; verify it matches the best `(count, priorityScore)`
- [x] 9.6 Confirm Top Plans header shows progress text (`Searching… N / Total`) during search and `Search complete` after
- [x] 9.7 Adopt-plan still works correctly; no regression
## 10. Version + changelog
- [x] 10.1 Bump `__APP_VERSION__` to `1.3.1` and `__APP_VERSION_DATE__` in `app/vite.config.ts`
- [x] 10.2 Add `## v1.3.1` entry to `CHANGELOG.md` describing: exhaustive search (drop saturation), mode-dependent ordering, evaluated/unevaluated cell distinction, per-set + global progress indicators, Recommended marker