Bill
cb49123930
The v1.3.0 saturation termination silently capped the search after only
the heuristic-favored part of the tree, leaving most per-set ceiling cells
stuck at "0 specs" and hiding genuinely-feasible 3-spec plans in
maximize-count mode. Replace with full exhaustive enumeration plus a
batch of UX refinements that emerged during testing.
Algorithm:
- Drop the saturation early-termination entirely. Search now runs the
full open-set cartesian product to completion; the iteration cap is
also removed so no scenario exits partial.
- Add mode-dependent DFS child ordering: priority-order keeps the
priority-target-first heuristic; maximize-count orders children by
descending count of qualifications for reachable specs (generalist
courses tried first).
- Make the (count, priorityScore) comparator mode-aware: priority-order
ranks by (priorityScore, count) so the user's top spec surfaces;
maximize-count ranks by (count, priorityScore) so the highest count
wins. The same rule drives both top-K position and per-cell ceiling
selection (and the Recommended badge).
- Add an evaluated boolean to each ChoiceOutcome and set it on first
leaf evaluation. Distinguishes "still searching" from "evaluated, no
specs achieved" so the UI never shows misleading 0 specs for a cell
the search hasn't reached yet.
- Throttled progress events (~100ms) carrying iterations / total leaf
count, drive both the per-set spinner and the global progress bar.
UI:
- Top Plans header shows a horizontal progress bar with
"iterations / total · NN%" while the search runs; collapses to
"Search complete · N explored" on completion.
- Per-set spinner next to each elective set heading while any choice
in that set is unevaluated.
- Per-cell pulsing dot + "searching" text for unevaluated cells.
- Replace the "(HCR, BNK, ...)" text labels on each course with
color-coded SpecTag pills using a new fixed per-spec palette
(app/src/data/specColors.ts). Same palette applied to the Top Plans
achievement badges so the two views are visually consistent.
- "Top outcome if picked ↓" caption above the right side of each open
elective set so the spec tags are clearly identified as decision-tree
outcomes (not the course's own qualifications).
- Recommended badge moved inline next to the course name (instead of
on a separate row below) to keep button heights stable.
Tests:
- Replace the saturation early-termination test with an exhaustion test
asserting every cell ends with evaluated: true and partial: false.
- Add mode-dependent ordering test (max-count visits Climate Finance
before Corporate Governance in fall3).
- Add evaluated-flag transition test.
- Add throttled progress-event test (>= ~100ms between consecutive
emits).
- Performance smoke updated to a 60s budget for the exhaustive
user-scenario search; 8-open-set typical case completes in ~7s.
Files: solver/decisionTree.ts, solver/priority.ts (already shipped),
data/specColors.ts (new), components/{TopPlans,CourseSelection}.tsx,
state/appState.ts, workers/decisionTree.worker.ts,
__tests__/searchDecisionTree.test.ts, vite.config.ts, CHANGELOG.md,
openspec/changes/decision-tree-exhaustive-search/* (full change spec).
93 lines
7.1 KiB
Markdown
93 lines
7.1 KiB
Markdown
## ADDED Requirements
|
|
|
|
### Requirement: Mode-dependent enumeration ordering
|
|
The decision-tree search SHALL select its DFS child-ordering heuristic based on the optimization mode. In `priority-order` mode, children at each level SHALL be ordered with `priorityTarget`-qualifying courses first (existing behavior). In `maximize-count` mode, children SHALL be ordered by the descending count of their qualifications for *reachable* specializations (specializations whose upper-bound credit potential meets the credit threshold).
|
|
|
|
#### Scenario: maximize-count orders generalist courses first
|
|
- **WHEN** maximize-count mode is selected and Fall Set 3 contains both `fall3-climate-finance` (qualifies for BNK/CRF/FIN/FIM/GLB/SBI — 6 reachable specs) and `fall3-emerging-tech` (qualifies for BRM/EMT/ENT/MTO/STR — 5)
|
|
- **THEN** the DFS visits combinations including `fall3-climate-finance` before combinations including `fall3-emerging-tech`
|
|
|
|
#### Scenario: priority-order ordering is unchanged
|
|
- **WHEN** priority-order mode is selected with HCR ranked first
|
|
- **THEN** courses qualifying for HCR are tried before courses that do not (existing target-first behavior)
|
|
|
|
### Requirement: Cells distinguish unevaluated from evaluated-zero
|
|
Each `ChoiceOutcome` SHALL carry an `evaluated: boolean` field. The field SHALL initialize to `false`. The field SHALL be set to `true` upon the first leaf evaluation that includes the corresponding `(setId, courseId)` pair. The UI SHALL render unevaluated cells with a visible "still searching" indicator distinct from cells that are evaluated and achieve zero specializations.
|
|
|
|
#### Scenario: New cell starts unevaluated
|
|
- **WHEN** the search begins
|
|
- **THEN** every choice in every set has `evaluated: false`
|
|
|
|
#### Scenario: First leaf marks the cell evaluated
|
|
- **WHEN** any leaf containing `(spr3, spr3-analytics-ml)` has been evaluated
|
|
- **THEN** the cell for `spr3-analytics-ml` has `evaluated: true`
|
|
|
|
#### Scenario: UI distinguishes unevaluated from evaluated-zero
|
|
- **WHEN** a cell has `evaluated: false`
|
|
- **THEN** the UI renders a "searching" indicator (not "0 specs")
|
|
- **AND WHEN** a cell has `evaluated: true` and `ceilingSpecs.length === 0`
|
|
- **THEN** the UI renders "0 specs" in muted styling
|
|
|
|
### Requirement: Per-set and global progress indication
|
|
The decision-tree worker SHALL emit a `progress` event carrying `{ iterations, iterationsTotal }` at most once every 100 milliseconds during an active search. The UI SHALL display global search progress in the Top Plans panel header and SHALL display a per-set indicator next to each elective set's name while that set has at least one unevaluated choice and the search is still running.
|
|
|
|
#### Scenario: Progress events emitted at throttled rate
|
|
- **WHEN** the search is running
|
|
- **THEN** the worker emits at most one `progress` event per 100ms
|
|
|
|
#### Scenario: Global progress visible in header
|
|
- **WHEN** the search is running and 15234 of 49152 leaves have been evaluated
|
|
- **THEN** the Top Plans header shows progress text such as `Searching… 15234 / 49152 explored`
|
|
|
|
#### Scenario: Per-set indicator shown while choices are unevaluated
|
|
- **WHEN** the search is running and Spring Set 1 has at least one choice with `evaluated: false`
|
|
- **THEN** a spinner or activity indicator appears next to the Spring Set 1 heading
|
|
- **AND WHEN** every choice in Spring Set 1 has `evaluated: true` (or the search completes)
|
|
- **THEN** the indicator clears
|
|
|
|
### Requirement: Recommended choice per set
|
|
For each open elective set, the UI SHALL identify and visually mark the choice with the best `(ceilingCount desc, priorityScore desc)` ordering as the "Recommended" choice. The marker SHALL be visible only when at least one choice in the set has `evaluated: true`. The recommendation SHALL update progressively as the search streams better outcomes for that set's choices.
|
|
|
|
#### Scenario: Recommended marker uses same comparator as top-K
|
|
- **WHEN** Spring Set 3 has choices with ceilings `[HCR, BNK]` (count=2, score=29) and `[FIN, MTO]` (count=2, score=22)
|
|
- **THEN** the choice with `[HCR, BNK]` is marked Recommended
|
|
|
|
#### Scenario: Higher count beats higher priority for recommendation
|
|
- **WHEN** one choice has ceiling count 3 and another has ceiling count 2 with higher priority score
|
|
- **THEN** the count-3 choice is Recommended
|
|
|
|
## MODIFIED Requirements
|
|
|
|
### Requirement: Bounded search with saturation termination
|
|
The decision-tree search SHALL terminate when the iteration count exceeds `MAX_TREE_ITERATIONS` (default 100,000). When this cap terminates the search before the cartesian product has been fully enumerated, the result SHALL include `partial: true`. The search SHALL otherwise enumerate every leaf in the cartesian product of open-set courses (the saturation-limit termination is removed).
|
|
|
|
#### Scenario: Search exhausts the cartesian product when within the cap
|
|
- **WHEN** the open-set cartesian product is smaller than `MAX_TREE_ITERATIONS`
|
|
- **THEN** the search evaluates every leaf, every cell ends with `evaluated: true`, and `partial` is `false`
|
|
|
|
#### Scenario: Search returns partial when cap is hit
|
|
- **WHEN** the cartesian product exceeds `MAX_TREE_ITERATIONS`
|
|
- **THEN** the search stops at the cap, sets `partial: true`, and returns the best top-K and ceilings found so far
|
|
|
|
### Requirement: Decision-tree worker protocol
|
|
The decision-tree worker SHALL accept a `WorkerRequest` that includes optional `topK` (default 10). It SHALL emit a tagged-union `WorkerResponse` stream with four event types: `topKUpdate` (when the ranked top-K list changes), `choiceUpdate` (when a per-set ceiling cell changes), `progress` (throttled to 100ms intervals during search, carrying iteration counters), and `allComplete` (when the search terminates, carrying both final top-K and final per-set analyses, plus a `partial` flag).
|
|
|
|
#### Scenario: Worker emits progress events
|
|
- **WHEN** the search runs for more than 100ms
|
|
- **THEN** the worker emits at least one `progress` event with `iterations` and `iterationsTotal`
|
|
|
|
#### Scenario: Worker emits final allComplete event with partial flag
|
|
- **WHEN** the search terminates
|
|
- **THEN** the worker emits `{ type: 'allComplete', topK, setAnalyses, partial }`
|
|
- **AND** `partial` is `true` only if the iteration cap fired
|
|
|
|
#### Scenario: Worker emits per-cell choice updates
|
|
- **WHEN** a single combination causes a ceiling change for one course in one set
|
|
- **THEN** the worker emits one `choiceUpdate` event identifying that set
|
|
|
|
## REMOVED Requirements
|
|
|
|
### Requirement: Saturation-limit early termination
|
|
**Reason**: The saturation criterion (`top-K stable for SATURATION_LIMIT iterations`) terminates the search before many `(set, course)` pairs have been evaluated and before the search reaches deeper combinations that yield higher-count outcomes. The user-visible result is "0 specs" labels on un-evaluated cells and missing high-count plans in the top-K. Replaced by exhaustive enumeration up to the iteration cap.
|
|
**Migration**: No consumer migration needed. The `searchDecisionTree` function signature is unchanged; behavior changes from "may stop early" to "always exhausts (within cap)". The `partial: true` flag remains as the only signal that the result may be incomplete.
|