v1.3.1: Exhaustive decision-tree search + UX refinements
The v1.3.0 saturation termination silently capped the search after only
the heuristic-favored part of the tree, leaving most per-set ceiling cells
stuck at "0 specs" and hiding genuinely-feasible 3-spec plans in
maximize-count mode. Replace with full exhaustive enumeration plus a
batch of UX refinements that emerged during testing.
Algorithm:
- Drop the saturation early-termination entirely. Search now runs the
full open-set cartesian product to completion; the iteration cap is
also removed so no scenario exits partial.
- Add mode-dependent DFS child ordering: priority-order keeps the
priority-target-first heuristic; maximize-count orders children by
descending count of qualifications for reachable specs (generalist
courses tried first).
- Make the (count, priorityScore) comparator mode-aware: priority-order
ranks by (priorityScore, count) so the user's top spec surfaces;
maximize-count ranks by (count, priorityScore) so the highest count
wins. The same rule drives both top-K position and per-cell ceiling
selection (and the Recommended badge).
- Add an evaluated boolean to each ChoiceOutcome and set it on first
leaf evaluation. Distinguishes "still searching" from "evaluated, no
specs achieved" so the UI never shows misleading 0 specs for a cell
the search hasn't reached yet.
- Throttled progress events (~100ms) carrying iterations / total leaf
count, drive both the per-set spinner and the global progress bar.
UI:
- Top Plans header shows a horizontal progress bar with
"iterations / total · NN%" while the search runs; collapses to
"Search complete · N explored" on completion.
- Per-set spinner next to each elective set heading while any choice
in that set is unevaluated.
- Per-cell pulsing dot + "searching" text for unevaluated cells.
- Replace the "(HCR, BNK, ...)" text labels on each course with
color-coded SpecTag pills using a new fixed per-spec palette
(app/src/data/specColors.ts). Same palette applied to the Top Plans
achievement badges so the two views are visually consistent.
- "Top outcome if picked ↓" caption above the right side of each open
elective set so the spec tags are clearly identified as decision-tree
outcomes (not the course's own qualifications).
- Recommended badge moved inline next to the course name (instead of
on a separate row below) to keep button heights stable.
Tests:
- Replace the saturation early-termination test with an exhaustion test
asserting every cell ends with evaluated: true and partial: false.
- Add mode-dependent ordering test (max-count visits Climate Finance
before Corporate Governance in fall3).
- Add evaluated-flag transition test.
- Add throttled progress-event test (>= ~100ms between consecutive
emits).
- Performance smoke updated to a 60s budget for the exhaustive
user-scenario search; 8-open-set typical case completes in ~7s.
Files: solver/decisionTree.ts, solver/priority.ts (already shipped),
data/specColors.ts (new), components/{TopPlans,CourseSelection}.tsx,
state/appState.ts, workers/decisionTree.worker.ts,
__tests__/searchDecisionTree.test.ts, vite.config.ts, CHANGELOG.md,
openspec/changes/decision-tree-exhaustive-search/* (full change spec).
This commit is contained in:
@@ -0,0 +1,2 @@
|
||||
schema: spec-driven
|
||||
created: 2026-05-09
|
||||
@@ -0,0 +1,3 @@
|
||||
# decision-tree-exhaustive-search
|
||||
|
||||
Drop saturation termination; run exhaustive DFS over the open-set cartesian product. Add mode-dependent child ordering (qualifies-for-most-reachable-specs in maximize-count mode, target-first in priority-order mode). Distinguish unevaluated cells from evaluated-zero in the per-set table. Add per-set + global progress indication and a 'Recommended' marker per set.
|
||||
@@ -0,0 +1,99 @@
|
||||
## Context
|
||||
|
||||
v1.3.0 introduced a streaming decision-tree search (`searchDecisionTree`) with two early-termination criteria: a hard iteration cap (`MAX_TREE_ITERATIONS = 10000`) and a saturation criterion (`SATURATION_LIMIT = 500` iterations of no top-K change). The saturation criterion fires too eagerly because the top-K updates only when a *strictly better* outcome is inserted; many leaves produce duplicate outcome classes that don't update top-K but still increment the saturation counter.
|
||||
|
||||
Two visible consequences in the user's testing:
|
||||
|
||||
1. Per-set ceiling table shows "0 specs" for many courses. Root cause: cells are initialized with `{count: 0, specs: []}` and only updated when a leaf containing them is evaluated. The DFS, ordered by the priority-target heuristic, exhausts target-favoring branches first. Saturation fires before the DFS backtracks to non-target choices in early sets.
|
||||
2. Top Plans in maximize-count mode shows only 2-spec outcomes when 3-spec combinations exist. Root cause: the same saturation fires before the search reaches the part of the tree where 3-spec-feasible combinations live (typically generalist-course-heavy combinations).
|
||||
|
||||
Investigation in `app/src/solver/decisionTree.ts:204-225` confirmed both behaviors. A diagnostic showed the user's reproduction case (8 open sets, 49,152 leaves) saturates at ~500 iterations, leaving most cells unevaluated.
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:**
|
||||
- Per-set ceiling cells reflect the true best outcome for every (set, course) pair after search completes
|
||||
- Top-K reflects the genuinely-best plans achievable for the given pin/ranking, in either mode
|
||||
- UI distinguishes "still searching" from "search complete, this course achieves nothing"
|
||||
- Search remains responsive: high-quality results appear in the stream within the first ~hundred iterations even though full search takes seconds
|
||||
|
||||
**Non-Goals:**
|
||||
- Sub-second exhaustive search for the 8-open-set worst case (50K leaves × ~1ms LP = ~50s is acceptable in a worker)
|
||||
- Replacing the LP solver or re-architecting the optimizer
|
||||
- Caching across runs
|
||||
- Configurable iteration cap from the UI
|
||||
|
||||
## Decisions
|
||||
|
||||
### Drop saturation termination entirely (not "make it smarter")
|
||||
|
||||
The user explicitly chose Approach A: exhaustive. Smarter saturation criteria (e.g., "stable for N iters AND every cell visited at least once") add complexity without reaching demonstrated correctness — there's always a pathological combination that defeats the heuristic. Exhaustive is simpler, demonstrably correct, and feasible in a worker.
|
||||
|
||||
**Alternative considered:** Two-phase search (fast heuristic + background exhaustive sweep). Rejected — added complexity (phase transition events, partial state), and the user's preference for "exhaustive" was explicit. The progress UI absorbs the same UX pain (user sees the search running) without the implementation cost.
|
||||
|
||||
### Mode-dependent enumeration ordering
|
||||
|
||||
The current target-first heuristic biases toward priority-order mode's expected outcome (high-priority spec achieved early). For maximize-count mode, ordering by qualification breadth is a better fit because generalist courses lead to higher-count plans. Both heuristics are cheap to compute (one upper-bounds map lookup, then a per-course count) and run once per analysis.
|
||||
|
||||
For maximize-count, the score per course is `count of (specId in course.qualifications) where upperBounds[specId] >= 9`. Sorting children by this score descending puts the generalist courses first. Stable sort keeps declaration order on ties.
|
||||
|
||||
**Alternative considered:** A single unified ordering (e.g., always order by qualification breadth). Rejected — for priority-order mode, the user's priority is the meaningful signal; using breadth would suppress the priority spec early in the stream.
|
||||
|
||||
### `evaluated: boolean` on `ChoiceOutcome`
|
||||
|
||||
Adds one boolean per (set, course) cell — negligible overhead. Cleaner than a sentinel value (e.g., `ceilingCount = -1`) that consumers might forget to handle and would need to be filtered everywhere ceilingCount is read. Boolean has obvious semantics in the UI ("not yet known" vs "known, value is 0").
|
||||
|
||||
**Alternative considered:** Omit cells entirely until evaluated. Rejected — the UI needs the courseName to render the row; restructuring to fetch course names from `coursesBySet` would push more responsibility into the renderer for no clear win.
|
||||
|
||||
### Per-set "Recommended" derivation in UI, not in worker
|
||||
|
||||
The recommended choice is a function of `analysis.choices` and the comparator. Computing in the UI keeps the worker protocol simple (no new field), avoids a duplicate computation, and lets the UI re-render cheaply on each choiceUpdate.
|
||||
|
||||
The comparator: `(ceilingCount desc, priorityScore desc)`. Same as the top-K. The "Recommended" course in a set is the one whose ceiling best matches the user's overall objective.
|
||||
|
||||
### Throttled `progress` event (≈100ms)
|
||||
|
||||
Without throttling, the worker would emit a progress event per iteration — 50,000 events × 1ms = 50s of message overhead. With ~100ms throttling: ~500 events per search, each tiny. Implementation: track `lastProgressEmit` timestamp; emit if `Date.now() - lastProgressEmit >= 100`.
|
||||
|
||||
**Alternative considered:** No progress events; rely on `topKUpdate` for activity signal. Rejected — top-K updates fire only when something changes; long stretches of "exploring duplicates" would look like a frozen UI.
|
||||
|
||||
### Mode-aware comparator (emerged during implementation)
|
||||
|
||||
After dropping saturation, exhaustive search surfaces 3-spec non-HCR plans that beat 2-spec HCR plans on the original (count, priorityScore) comparator. This conflicted with v1.3.0 spec scenarios that asserted HCR appears at `topK[0]` in priority-order mode with HCR ranked first. Resolution: make both the top-K and per-cell ceiling comparators mode-dependent.
|
||||
|
||||
- `priority-order` mode: `(priorityScore desc, count desc, key asc)` — surfaces the user's top-priority spec even when higher-count alternatives exist
|
||||
- `maximize-count` mode: `(count desc, priorityScore desc, key asc)` — surfaces the maximum number of specs achievable
|
||||
|
||||
Both comparators share the deterministic `assignmentKey` tiebreaker for streaming stability. The CourseSelection "Recommended" badge uses the same mode-dependent rule so cell recommendations align with the top-K ranking.
|
||||
|
||||
**Alternative considered:** Keep the count-first comparator and let exhaustive search reveal high-count alternatives. Rejected — contradicts user-stated intent ("HCR top priority should surface") and breaks v1.3.0 spec scenarios.
|
||||
|
||||
### `MAX_TREE_ITERATIONS = 100,000`
|
||||
|
||||
Empirically, 8-open-set worst case is ~50K leaves. 100K provides 2× headroom. Larger scenarios (10+ open sets) would still be capped, with `partial: true` displayed. The fallback to "empty choices" already exists for `openSetIds.length > MAX_OPEN_SETS_FOR_ENUMERATION` (= 9), so this cap rarely fires in practice.
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
- **50s search feels slow** → Progress UI + streamed top-K make it feel active; user can adopt a plan partway through if they like what's shown
|
||||
- **Worker CPU usage during search** → Acceptable; runs in a worker thread, doesn't block UI; user can change pins to abort and restart
|
||||
- **Throttled progress means iteration count "jumps"** → Cosmetic only; UI doesn't depend on monotonic small steps
|
||||
- **`evaluated: false` initial state for every cell** → Slightly verbose payloads; choiceUpdate already sends the full set's choices array, so the change is one boolean field per cell (negligible)
|
||||
- **Mode switch mid-search** → Current behavior already terminates and restarts the worker on any pin/ranking/mode change; unchanged
|
||||
- **Tests need amendments** → Saturation tests removed (3 tests); exhaustion test added; mode-ordering test added; per-cell evaluated transition test added
|
||||
|
||||
## Migration Plan
|
||||
|
||||
Single-PR change. No data migration. Steps:
|
||||
|
||||
1. Algorithm + worker + state changes; tests updated
|
||||
2. UI updates: per-cell evaluated rendering, per-set spinner, global progress, recommended badge
|
||||
3. Browser-verify both modes against the v1.3.0 reproduction scenario; confirm exhaustive search completes and all cells are populated
|
||||
4. Bump version (`1.3.1`); CHANGELOG entry; ship
|
||||
|
||||
Rollback: revert the change; v1.3.0 behavior restored. No persistent state to migrate.
|
||||
|
||||
## Open Questions
|
||||
|
||||
- **Recommended badge for ties** — if two choices in a set have identical `(count, priorityScore)`, currently the comparator's deterministic tiebreaker (assignmentKey) picks one. UI shows just one Recommended. Acceptable for v1; could be revisited if confusing.
|
||||
- **Should "Recommended" still show before search completes** — derived from current ceilings, so it updates as the search streams. Possibly confusing if the recommendation flips mid-search. Initial behavior: show as soon as any choice has `evaluated: true`; let it update with the stream.
|
||||
- **Future: progressive `partial` flag during search** — out of scope. Today, `partial` only matters at the cap, which fires rarely.
|
||||
@@ -0,0 +1,42 @@
|
||||
## Why
|
||||
|
||||
Real-world testing of v1.3.0 surfaced two related defects in the new decision-tree streaming search:
|
||||
|
||||
1. **Many per-set choices show "0 specs"** — the saturation termination (top-K stable for 500 iterations) fires after exploring only the heuristic-favored part of the tree. Courses in early sets that don't qualify for the priority target are never the chosen course in any evaluated leaf, so their ceilings remain at the initial `{count: 0, specs: []}` and render as a misleading "0 specs".
|
||||
2. **Top plans are not exhaustive in maximize-count mode** — same root cause: saturation accepts "no improvement" too eagerly, since many leaves yield the same already-found outcome class. Higher-count plans (e.g., `[BNK, CRF, FIN]` triples) that exist deeper in the search are never reached.
|
||||
|
||||
Both behaviors mislead the user: the per-set table claims a course leads to no specs when in fact it was never evaluated, and the Top Plans panel hides plans that genuinely exist. The fix is to drop the early-termination heuristic and run an exhaustive search, with mode-dependent enumeration ordering so the most-likely-good outcomes still appear early in the stream.
|
||||
|
||||
## What Changes
|
||||
|
||||
- Remove `SATURATION_LIMIT` early-termination. Search runs the full cartesian product unless `MAX_TREE_ITERATIONS` (raised to 100,000 as safety cap) fires.
|
||||
- Add **mode-dependent child ordering** at every DFS level:
|
||||
- `priority-order` mode: keep the existing `priorityTarget`-qualifying-first heuristic.
|
||||
- `maximize-count` mode: new heuristic — order children by descending count of qualifications they hold for *reachable* specs (specs whose upper bound ≥ 9). "Generalist" courses like Climate Finance (BNK/CRF/FIN/FIM/GLB/SBI) come before specialist courses, surfacing high-count outcomes early.
|
||||
- Distinguish **unevaluated** cells from **evaluated, zero-spec** cells in `ChoiceOutcome`. New field `evaluated: boolean` (default `false`, set `true` on first leaf containing the (set, course) pair). UI renders unevaluated cells with a subtle searching indicator, not "0 specs".
|
||||
- Add **per-set progress indicator** — a small spinner next to the set name shown when `loading` is true and any choice in that set is still unevaluated; clears when every choice has been evaluated or when search completes.
|
||||
- Add **global progress indicator** in the Top Plans panel — `"Searching… N / Total explored"` with running counts, then `"Search complete · N explored"` when done. If `partial: true`, show `"Search incomplete · cap hit at N"`.
|
||||
- Add **"Recommended" marker** per set — the choice with the best `(ceilingCount, priorityScore)` per the same comparator the top-K uses; rendered as a small badge on the recommended row. Derived in the UI from `analysis.choices` (worker protocol unchanged on this front).
|
||||
- Worker emits a new `progress` event throttled to ~100ms intervals, carrying `{ iterations, iterationsTotal }`. Avoids per-iteration message flood while keeping the UI responsive.
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
|
||||
_None — this extends the existing optimization engine._
|
||||
|
||||
### Modified Capabilities
|
||||
|
||||
- `optimization-engine`: drop saturation termination requirement; require exhaustive search up to a safety cap; add mode-dependent ordering, evaluated/unevaluated cell state, per-set + global progress events, and a per-set recommended-choice derivation.
|
||||
|
||||
## Impact
|
||||
|
||||
- `app/src/solver/decisionTree.ts` — drop `SATURATION_LIMIT`; raise `MAX_TREE_ITERATIONS` to 100,000; add `reorderByReachableQualCount` helper for maximize-count mode; gate the chosen reorder strategy by `mode`; add `evaluated: boolean` to `ChoiceOutcome` and set it on first leaf containing the pair; emit throttled `progress` events; remove saturation logic
|
||||
- `app/src/workers/decisionTree.worker.ts` — add `progress` event type to the tagged union; throttle progress emission
|
||||
- `app/src/state/appState.ts` — track `searchProgress: { iterations, iterationsTotal } | null` slice; consume `progress` events
|
||||
- `app/src/components/TopPlans.tsx` — render global progress text in the header
|
||||
- `app/src/components/CourseSelection.tsx` — per-set spinner (next to set name); per-cell unevaluated rendering (skeleton/dot, not "0 specs"); "Recommended" badge on the best choice per set
|
||||
- `app/src/solver/__tests__/searchDecisionTree.test.ts` — remove saturation tests; add exhaustion test (asserts every (set, course) cell has `evaluated: true` after completion); add mode-dependent ordering test (maximize-count chooses generalist courses first); add unevaluated→evaluated transition test
|
||||
- `app/vite.config.ts` — bump to `1.3.1` (or `1.4.0` if user wants minor; default patch)
|
||||
- `CHANGELOG.md` — release entry
|
||||
- No data file changes; no schema migration
|
||||
@@ -0,0 +1,92 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Mode-dependent enumeration ordering
|
||||
The decision-tree search SHALL select its DFS child-ordering heuristic based on the optimization mode. In `priority-order` mode, children at each level SHALL be ordered with `priorityTarget`-qualifying courses first (existing behavior). In `maximize-count` mode, children SHALL be ordered by the descending count of their qualifications for *reachable* specializations (specializations whose upper-bound credit potential meets the credit threshold).
|
||||
|
||||
#### Scenario: maximize-count orders generalist courses first
|
||||
- **WHEN** maximize-count mode is selected and Fall Set 3 contains both `fall3-climate-finance` (qualifies for BNK/CRF/FIN/FIM/GLB/SBI — 6 reachable specs) and `fall3-emerging-tech` (qualifies for BRM/EMT/ENT/MTO/STR — 5)
|
||||
- **THEN** the DFS visits combinations including `fall3-climate-finance` before combinations including `fall3-emerging-tech`
|
||||
|
||||
#### Scenario: priority-order ordering is unchanged
|
||||
- **WHEN** priority-order mode is selected with HCR ranked first
|
||||
- **THEN** courses qualifying for HCR are tried before courses that do not (existing target-first behavior)
|
||||
|
||||
### Requirement: Cells distinguish unevaluated from evaluated-zero
|
||||
Each `ChoiceOutcome` SHALL carry an `evaluated: boolean` field. The field SHALL initialize to `false`. The field SHALL be set to `true` upon the first leaf evaluation that includes the corresponding `(setId, courseId)` pair. The UI SHALL render unevaluated cells with a visible "still searching" indicator distinct from cells that are evaluated and achieve zero specializations.
|
||||
|
||||
#### Scenario: New cell starts unevaluated
|
||||
- **WHEN** the search begins
|
||||
- **THEN** every choice in every set has `evaluated: false`
|
||||
|
||||
#### Scenario: First leaf marks the cell evaluated
|
||||
- **WHEN** any leaf containing `(spr3, spr3-analytics-ml)` has been evaluated
|
||||
- **THEN** the cell for `spr3-analytics-ml` has `evaluated: true`
|
||||
|
||||
#### Scenario: UI distinguishes unevaluated from evaluated-zero
|
||||
- **WHEN** a cell has `evaluated: false`
|
||||
- **THEN** the UI renders a "searching" indicator (not "0 specs")
|
||||
- **AND WHEN** a cell has `evaluated: true` and `ceilingSpecs.length === 0`
|
||||
- **THEN** the UI renders "0 specs" in muted styling
|
||||
|
||||
### Requirement: Per-set and global progress indication
|
||||
The decision-tree worker SHALL emit a `progress` event carrying `{ iterations, iterationsTotal }` at most once every 100 milliseconds during an active search. The UI SHALL display global search progress in the Top Plans panel header and SHALL display a per-set indicator next to each elective set's name while that set has at least one unevaluated choice and the search is still running.
|
||||
|
||||
#### Scenario: Progress events emitted at throttled rate
|
||||
- **WHEN** the search is running
|
||||
- **THEN** the worker emits at most one `progress` event per 100ms
|
||||
|
||||
#### Scenario: Global progress visible in header
|
||||
- **WHEN** the search is running and 15234 of 49152 leaves have been evaluated
|
||||
- **THEN** the Top Plans header shows progress text such as `Searching… 15234 / 49152 explored`
|
||||
|
||||
#### Scenario: Per-set indicator shown while choices are unevaluated
|
||||
- **WHEN** the search is running and Spring Set 1 has at least one choice with `evaluated: false`
|
||||
- **THEN** a spinner or activity indicator appears next to the Spring Set 1 heading
|
||||
- **AND WHEN** every choice in Spring Set 1 has `evaluated: true` (or the search completes)
|
||||
- **THEN** the indicator clears
|
||||
|
||||
### Requirement: Recommended choice per set
|
||||
For each open elective set, the UI SHALL identify and visually mark the choice with the best `(ceilingCount desc, priorityScore desc)` ordering as the "Recommended" choice. The marker SHALL be visible only when at least one choice in the set has `evaluated: true`. The recommendation SHALL update progressively as the search streams better outcomes for that set's choices.
|
||||
|
||||
#### Scenario: Recommended marker uses same comparator as top-K
|
||||
- **WHEN** Spring Set 3 has choices with ceilings `[HCR, BNK]` (count=2, score=29) and `[FIN, MTO]` (count=2, score=22)
|
||||
- **THEN** the choice with `[HCR, BNK]` is marked Recommended
|
||||
|
||||
#### Scenario: Higher count beats higher priority for recommendation
|
||||
- **WHEN** one choice has ceiling count 3 and another has ceiling count 2 with higher priority score
|
||||
- **THEN** the count-3 choice is Recommended
|
||||
|
||||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: Bounded search with saturation termination
|
||||
The decision-tree search SHALL terminate when the iteration count exceeds `MAX_TREE_ITERATIONS` (default 100,000). When this cap terminates the search before the cartesian product has been fully enumerated, the result SHALL include `partial: true`. The search SHALL otherwise enumerate every leaf in the cartesian product of open-set courses (the saturation-limit termination is removed).
|
||||
|
||||
#### Scenario: Search exhausts the cartesian product when within the cap
|
||||
- **WHEN** the open-set cartesian product is smaller than `MAX_TREE_ITERATIONS`
|
||||
- **THEN** the search evaluates every leaf, every cell ends with `evaluated: true`, and `partial` is `false`
|
||||
|
||||
#### Scenario: Search returns partial when cap is hit
|
||||
- **WHEN** the cartesian product exceeds `MAX_TREE_ITERATIONS`
|
||||
- **THEN** the search stops at the cap, sets `partial: true`, and returns the best top-K and ceilings found so far
|
||||
|
||||
### Requirement: Decision-tree worker protocol
|
||||
The decision-tree worker SHALL accept a `WorkerRequest` that includes optional `topK` (default 10). It SHALL emit a tagged-union `WorkerResponse` stream with four event types: `topKUpdate` (when the ranked top-K list changes), `choiceUpdate` (when a per-set ceiling cell changes), `progress` (throttled to 100ms intervals during search, carrying iteration counters), and `allComplete` (when the search terminates, carrying both final top-K and final per-set analyses, plus a `partial` flag).
|
||||
|
||||
#### Scenario: Worker emits progress events
|
||||
- **WHEN** the search runs for more than 100ms
|
||||
- **THEN** the worker emits at least one `progress` event with `iterations` and `iterationsTotal`
|
||||
|
||||
#### Scenario: Worker emits final allComplete event with partial flag
|
||||
- **WHEN** the search terminates
|
||||
- **THEN** the worker emits `{ type: 'allComplete', topK, setAnalyses, partial }`
|
||||
- **AND** `partial` is `true` only if the iteration cap fired
|
||||
|
||||
#### Scenario: Worker emits per-cell choice updates
|
||||
- **WHEN** a single combination causes a ceiling change for one course in one set
|
||||
- **THEN** the worker emits one `choiceUpdate` event identifying that set
|
||||
|
||||
## REMOVED Requirements
|
||||
|
||||
### Requirement: Saturation-limit early termination
|
||||
**Reason**: The saturation criterion (`top-K stable for SATURATION_LIMIT iterations`) terminates the search before many `(set, course)` pairs have been evaluated and before the search reaches deeper combinations that yield higher-count outcomes. The user-visible result is "0 specs" labels on un-evaluated cells and missing high-count plans in the top-K. Replaced by exhaustive enumeration up to the iteration cap.
|
||||
**Migration**: No consumer migration needed. The `searchDecisionTree` function signature is unchanged; behavior changes from "may stop early" to "always exhausts (within cap)". The `partial: true` flag remains as the only signal that the result may be incomplete.
|
||||
@@ -0,0 +1,75 @@
|
||||
## 1. Drop saturation, raise cap
|
||||
|
||||
- [x] 1.1 In `app/src/solver/decisionTree.ts`, remove the `SATURATION_LIMIT` constant and all references to `iterationsSinceTopKChange` (declaration, increment, reset, comparison)
|
||||
- [x] 1.2 Raise `MAX_TREE_ITERATIONS` to `100_000`
|
||||
- [x] 1.3 Confirm the only termination paths are now: (a) iteration cap fires (sets `partial = true`), or (b) DFS exhausts the cartesian product
|
||||
|
||||
## 2. Mode-dependent ordering
|
||||
|
||||
- [x] 2.1 Add `reorderByReachableQualCount(setId, upperBounds, excludedCourseIds): Course[]` — returns courses sorted by descending count of qualifications for specs whose `upperBounds[specId] >= 9`. Stable sort; ties keep declaration order; cancelled courses excluded.
|
||||
- [x] 2.2 In `searchDecisionTree`, replace the `orderedCoursesPerSet` initialization to gate by `mode`: `priority-order` keeps `reorderForTarget(setId, priorityTarget, ...)`; `maximize-count` uses `reorderByReachableQualCount(setId, upperBounds, ...)`
|
||||
- [x] 2.3 Unit test: assert that in maximize-count mode, the first DFS leaf for the user's reproduction scenario contains `fall3-climate-finance` (the most-generalist Fall Set 3 course) before `fall3-emerging-tech`
|
||||
|
||||
## 3. Evaluated/unevaluated cell state
|
||||
|
||||
- [x] 3.1 Add `evaluated: boolean` field to `ChoiceOutcome` (`app/src/solver/decisionTree.ts`); initialize to `false` in the per-set analysis init loop
|
||||
- [x] 3.2 In `evaluateLeaf`, after running the optimizer and computing `result`, set `choice.evaluated = true` for every `(setId, courseId)` in the leaf's assignments BEFORE the comparison. (The cell is "evaluated" the moment a leaf containing it is run, regardless of whether the result improves the ceiling.)
|
||||
- [x] 3.3 Confirm the `choiceUpdate` callback fires whenever either `evaluated` flips to `true` or the ceiling improves — so the UI sees the transition. Update the existing condition to fire on both
|
||||
- [x] 3.4 Unit test: after one leaf evaluation containing `(spr3, spr3-analytics-ml)`, that cell has `evaluated: true`; cells in other sets that aren't in the leaf assignment remain `evaluated: false`
|
||||
|
||||
## 4. Throttled progress events
|
||||
|
||||
- [x] 4.1 Add `progress` to the `WorkerResponse` tagged union in `app/src/workers/decisionTree.worker.ts`: `{ type: 'progress'; iterations: number; iterationsTotal: number }`
|
||||
- [x] 4.2 Compute `iterationsTotal` in `searchDecisionTree` before DFS starts: product of `orderedCoursesPerSet[setId].length` over all `openSetIds`. Pass it through to the progress callback
|
||||
- [x] 4.3 Add `onProgress?: (iterations: number, iterationsTotal: number) => void` to `SearchCallbacks`. Track `lastProgressEmit: number` (default 0); call `onProgress` from inside `evaluateLeaf` when `Date.now() - lastProgressEmit >= 100`
|
||||
- [x] 4.4 In the worker, wire `onProgress` to `postMessage({ type: 'progress', iterations, iterationsTotal })`
|
||||
|
||||
## 5. App state wiring
|
||||
|
||||
- [x] 5.1 In `app/src/state/appState.ts`, add a `searchProgress: { iterations: number; iterationsTotal: number } | null` slice (default `null`); update on `progress` events; reset to `null` on new search start and on `allComplete`
|
||||
- [x] 5.2 Export `searchProgress` from `useAppState` alongside `topPlans`/`topPlansPartial`
|
||||
- [x] 5.3 The `choiceUpdate` handler already updates the per-set map; verify the new `evaluated` field flows through unchanged (no code change needed; `analysis` is forwarded as-is)
|
||||
|
||||
## 6. Top Plans header (global progress)
|
||||
|
||||
- [x] 6.1 In `app/src/components/TopPlans.tsx`, accept `searchProgress` and `loading` props and render in the header:
|
||||
- while `loading && searchProgress`: `Searching… {iterations.toLocaleString()} / {iterationsTotal.toLocaleString()} explored`
|
||||
- after complete (`!loading && !partial`): `Search complete · {totalEvaluated} explored`
|
||||
- after complete with `partial`: `Search incomplete · cap hit at {MAX_TREE_ITERATIONS}` (existing partial caption replaced/extended)
|
||||
- [x] 6.2 Pass `searchProgress` from App.tsx into `<TopPlans>`
|
||||
|
||||
## 7. CourseSelection per-set + per-cell rendering
|
||||
|
||||
- [x] 7.1 In `app/src/components/CourseSelection.tsx`, in the `ElectiveSet` heading area: render a small spinner/dot next to the set name when `loading === true` AND `analysis?.choices.some(c => !c.evaluated)`. The existing "high impact" badge stays
|
||||
- [x] 7.2 Replace the per-cell ceiling render branch:
|
||||
- `!evaluated`: render a faint "·" or pulsing dot (use the existing skeleton pattern restyled, or a small `…` glyph) instead of "0 specs"
|
||||
- `evaluated && ceilingCount === 0`: render "0 specs" in muted grey
|
||||
- `evaluated && ceilingCount > 0`: render existing colored "N specs (LIST)" treatment
|
||||
- [x] 7.3 Compute the recommended choice per set: pick the choice with the best `(ceilingCount desc, priorityScore desc)` — only consider choices with `evaluated === true`. Render a small `⭐ Recommended` badge on that row. Hide the badge if no choice is yet evaluated
|
||||
- [x] 7.4 The recommended derivation needs `priorityScore`; import `makePriorityScorer` from `app/src/solver/priority` and memoize per-render with `state.ranking`
|
||||
|
||||
## 8. Tests
|
||||
|
||||
- [x] 8.1 Remove the saturation early-termination test from `app/src/solver/__tests__/searchDecisionTree.test.ts` (the test that asserts iteration count stays well under cap when topK converges quickly — no longer applies)
|
||||
- [x] 8.2 Add an exhaustion test: small scenario (e.g., 2 open sets); after `searchDecisionTree` returns, every choice in every open set has `evaluated: true` and `partial === false`
|
||||
- [x] 8.3 Add a mode-dependent ordering test: in maximize-count mode for the user's reproduction scenario, the first leaf evaluated contains `fall3-climate-finance` (verify by capturing the first `onChoiceUpdate` event for `fall3` and inspecting the assignment)
|
||||
- [x] 8.4 Add an evaluated-flag transition test: assert all cells start `evaluated: false`; after one leaf evaluation, only cells with that leaf's assignments are `evaluated: true`
|
||||
- [x] 8.5 Update the streaming monotonicity test if needed (still valid in concept; just verify with new termination)
|
||||
- [x] 8.6 Add a progress-event throttling test: capture `onProgress` calls during a search; assert minimum interval >= 90ms between consecutive calls (small jitter tolerance)
|
||||
- [x] 8.7 Update the performance smoke test to allow longer time budget (e.g., 60 seconds) since the search is now exhaustive
|
||||
- [x] 8.8 Run full test suite; confirm all pass
|
||||
|
||||
## 9. Browser verification
|
||||
|
||||
- [x] 9.1 Start dev server; reproduce user's pin scenario (SP2/SP4/SP5/SE1, HCR first)
|
||||
- [x] 9.2 Switch to maximize-count mode; confirm Top Plans surfaces 3-spec plans (e.g., `[BNK, CRF, FIN]` triples) if any are feasible — if not feasible for this scenario, try a less-pinned scenario to confirm 3-spec plans CAN appear
|
||||
- [x] 9.3 Confirm per-set spinner appears next to "Spring Elective Set 1" while search runs and clears when complete
|
||||
- [x] 9.4 Confirm per-cell rendering shows "·" or similar for unevaluated cells, then transitions to "N specs" or "0 specs" as evaluation completes
|
||||
- [x] 9.5 Confirm `⭐ Recommended` appears on one course per set after at least one cell in that set is evaluated; verify it matches the best `(count, priorityScore)`
|
||||
- [x] 9.6 Confirm Top Plans header shows progress text (`Searching… N / Total`) during search and `Search complete` after
|
||||
- [x] 9.7 Adopt-plan still works correctly; no regression
|
||||
|
||||
## 10. Version + changelog
|
||||
|
||||
- [x] 10.1 Bump `__APP_VERSION__` to `1.3.1` and `__APP_VERSION_DATE__` in `app/vite.config.ts`
|
||||
- [x] 10.2 Add `## v1.3.1` entry to `CHANGELOG.md` describing: exhaustive search (drop saturation), mode-dependent ordering, evaluated/unevaluated cell distinction, per-set + global progress indicators, Recommended marker
|
||||
Reference in New Issue
Block a user