Historical design or verification record. It is retained for context; use the dashboard guides and generated tool reference for current behavior.
02 — Capability probe + proactive tool gating
Status: ✅ COMPLETE — all 4 phases landed. Loop protocol, exit gate and conventions: README.md. §9 is the source of truth for a
/looprun.
0. Scope in one line
Probe each device once for version, packages, architecture and board features, cache it, and let the registry hide or annotate tools the device genuinely cannot run — so the model stops discovering incompatibility by calling and failing.
1. Why
commandUnsupported() detects an unsupported command after it has run. The model
then sees a confusing error, often hallucinates a fix, and burns two more calls. The
worst offenders:
/interface wifi(ROS 7 wifiwave2) vs/interface wireless(legacy) vs/caps-man— three stacks, mutually exclusive per device./containerneedsdevice-modeenabled and thecontainerpackage./ip servicevs ROS 6 paths;/routing/bgpdiffers wholesale between 6 and 7.- PoE, switch chip, SFP and LTE tools depend on the board, not the OS.
user-manager,wifi-qcom-ac,rose-storage,iot,zerotierare optional packages.
Cheapest real win in the whole backlog: a probe, a cache and a registry filter improve every tool at once.
2. What is probed
One batched read per device, on first use, cached with a TTL:
| Source | Yields |
|---|---|
/system/resource/print | version, board-name, architecture-name |
/system/package/print | installed + enabled optional packages |
/system/routerboard/print | model, firmware, whether it is a RouterBOARD |
/system/device-mode/print | container/scheduler/fetch permissions (7.13+) |
/interface/wifi/print count | which wireless stack answers |
Everything is READ. On a device where a probe path 404s, that simply records the
absence — a failed probe is data, not an error.
3. Architecture
src/core/capability.ts PURE — model + predicate evaluation, no I/O
src/core/capability-probe.ts fetch (executeMikrotikCommand) → normalize → model
src/core/registry.ts consults the cache when listing/serving toolsCapabilities model:
interface Capabilities {
version: { raw: string; major: number; minor: number; patch: number };
channel: "stable" | "long-term" | "testing" | "development" | "unknown";
board: string;
arch: string;
isRouterBoard: boolean;
packages: Set<string>; // enabled only
wirelessStack: "wifi" | "wireless" | "capsman-legacy" | "none";
deviceMode: { container: boolean; scheduler: boolean; fetch: boolean };
probedAt: number;
}Declaring a requirement. defineTool gains an optional requires:
requires: { minVersion: "7.13", packages: ["container"], deviceMode: "container" }
requires: { wirelessStack: "wifi" }
requires: { board: /^CRS3/ }Absent requires = universally available, which is the case for the large majority.
Only annotate tools that genuinely need it — resist the urge to fill this in
everywhere.
4. Gating behaviour (the important design call)
Three modes, configurable (capabilityGating), default annotate:
off— probe still runs for the dashboard, but nothing is filtered.annotate(default) — every tool stays listed; unsupported ones get a prefix in their description:[unavailable on this device: needs RouterOS ≥ 7.13]. The model reads descriptions to choose, so this alone prevents most bad calls.filter— unsupported tools are removed fromtools/listentirely.
Default is annotate, not filter, for two reasons: in a multi-device setup the
tool list is global while capabilities are per-device (a tool unsupported on device A
may be essential on device B), and silently vanishing tools are extremely confusing
to debug. filter is only honoured in single-device mode — document this.
Independently of the mode, the handler-side guard always runs: if a tool with
requires is invoked against a device that fails the predicate, return a clean
isError result naming the missing capability, rather than letting RouterOS produce
a parser error. That guard is the actual safety net; listing behaviour is UX.
5. Cache & invalidation
In-memory Map<deviceName, Capabilities>, TTL 6 h. Invalidate on: explicit
refresh_device_capabilities, a successful firmware upgrade tool, config reload, or
a commandUnsupported() hit (which means the model is stale — reprobe and log).
No persistence. A restart reprobing costs one batched read per device.
6. Tools (2)
| Tool | Risk | Purpose |
|---|---|---|
get_device_capabilities | READ | The probed model, rendered — version, packages, stack, board, device-mode, and what that unlocks or blocks |
refresh_device_capabilities | READ | Force a reprobe, bypassing the TTL |
Module: src/tools/capability.ts, catalog label "Device Capabilities", group
Discovery & Meta.
7. Dashboard
- Devices (
connectivity.tsx) — each device card gains a capability strip: version + channel chip, wireless stack, notable packages, device-mode flags. Detail drawer gets the full matrix and a Reprobe button. - Modules (
modules.tsx) — a device selector; supported tools render normally, unsupported ones dim with the reason on hover. This makes the whole catalog legible against a specific router, which is genuinely useful on its own. - Routes:
GET /api/capabilities(all devices),POST /api/capabilities/refresh.
8. Raycast
raycast/src/catalog.tsxandmodules.tsx— accessory icon + tooltip on unsupported tools, sourced from/api/capabilities.raycast/src/devices.tsx— version/stack in the list item subtitle.
No new command.
9. Implementation progress (loop state — source of truth)
[x] Phase 1 — Pure model + probe
-
src/core/capability.ts—Capabilities,ToolRequires,unmetReasons,satisfies,explainUnmet,channelOf,unknownCapabilities. Pure. ReusesparseVersion/compareVersionsfromfirmware-lifecycle.tsrather than defining a second comparator — that one already handles7.17beta3,7.16.2 (stable)and absent patch numbers, and two version orderings in one codebase drift apart. The blueprint'sparseVersionline was therefore not implemented; it already existed. -
src/core/capability-probe.ts—probeDevice(batched, twoPromise.allwaves) plus the pure normalisersparseSettings,parsePackages,pickWirelessStack,normalizeProbe. Every individual probe is wrapped: an absent menu records absence instead of aborting. -
tests/core/capability.spec.ts— 26 cases: version bounds incl. pre-release ordering and exclusivemaxVersion, packages (missing, pluralisation, disabled-not-counted), all three wireless stacks, board regex, RouterBOARD, device-mode, requirement combination,channelOf, and the probe parsers including a device that answers nothing. - Exit gate: ✅ global gate green —
test:typesclean, 733 tests / 67 files pass,lintexit 0. PHASE 1 COMPLETE.
Decisions taken during Phase 1 (binding on Phase 2):
- An unknown capability never blocks. Every predicate passes when the relevant
fact was not probed (no version, empty board,
probedAt === 0,wirelessStack: "none"). Failing closed on a probe that did not answer would break tools on devices that work fine — the probe is an optimisation, not an authority. Phase 2's handler-side guard inherits this and must not tighten it. unmetReasonsreturns ALL failures, not the first.explainUnmetjoins them. A tool needing 7.13 and a package should say both, or the user fixes one and hits the next.- Disabled packages are excluded from
packages. A package present but flaggedXcannot serve its menu; counting it would reintroduce the exact failure this feature removes. maxVersionis exclusive,minVersioninclusive — so{minVersion: "7.13", maxVersion: "8.0"}reads naturally as "the 7.x line from 7.13".- Wireless stack detection prefers
wifioverwirelesswhen both answer, since a v7 device can retain the legacy menu for compatibility.
[x] Phase 2 — Registry integration
-
defineToolacceptsrequires, carried ontoRegisterableTool. -
src/core/capability-cache.ts— caches the in-flight promise (the Phase 1 blocker), TTL 6 h measured from probe START,invalidateCapabilities, synchronouspeekCapabilities, andsetCapabilityClock/setCapabilityProbertest seams. A failing probe resolves to unknown and is evicted, never cached. -
capabilityGatingconfig (off|annotate|filter, defaultannotate) +--capability-gating/MIKROTIK_MCP__CAPABILITY_GATING;filterdegrades toannotatein multi-device mode with a warn-once log.schemas/config.schema.jsonregenerated (thedocs/tools-reference.mdformatter churn was reverted). - Handler-side guard in the registry callback →
isErrornaming the unmet requirement, recorded in observability like any other failure. -
withRequires(requires, module)+ applied to the three unambiguous modules: container (containerpackage + device-mode + 7.0), user-manager (user-managerpackage), PoE (routerBoard). The wireless stacks, switch and ROS7-routing were deliberately NOT annotated — see below. -
tests/core/capability-gating.spec.ts— 15 cases: all three modes, cold cache, satisfied requirement, multi-devicefilterfallback, concurrent probe sharing, TTL reuse/expiry, slow-probe freshness, failing probe not cached, invalidate, peek lifecycle. - Exit gate: ✅ global gate green —
test:typesclean, 756 tests / 69 files pass,lintexit 0. PHASE 2 COMPLETE.
Decisions taken during Phase 2:
- Listing gating needs a probe that has already resolved; the guard does not.
Tool descriptions are registered once at startup and cannot await I/O, so
gatingDecisionreads onlypeekCapabilities()and returns "no opinion" on a cold cache. In practiceannotate/filtertherefore take effect on the next registration after a probe exists (a server reload), while the call-time guard works from the first call. This limitation is inherent to MCP's static tool list — the alternative, probing every device before registering, would add a multi-second network round-trip to stdio startup and fail closed on an unreachable router. It is also whyannotateis the default: degrading to a no-op is harmless, whereas afilterthat guessed would hide working tools. - Only three modules were annotated. The blueprint listed six. The wireless
modules already auto-detect their stack per call and work across all three, so a
requiresthere would remove function; switch tools span far more boards than a single regex captures; and ROS7-only routing is already version-guarded in its handlers. Annotating those would trade a real behaviour for a cosmetic one. The three chosen are unambiguous: a missing package or absent PoE hardware cannot be worked around. withRequiresat the module boundary, rather thanrequires:on 37 individual definitions — whole modules share a requirement far more often than single tools, and one omission in 37 silently loses the guard.
[x] Phase 3 — Tools + docs
-
src/tools/capability.ts—get_device_capabilitiesandrefresh_device_capabilities, both READ. The renderer states consequences ("Container tools BLOCKED — package installed but device-mode off, enabling it needs a physical reset-button confirmation") rather than dumping fields. -
src/tools/index.ts— "Device Capabilities" catalog entry, groupDiscovery & Meta. Catalog test green; 833 → 835 tools. - Invalidation wired (the Phase 2 gap):
onConfigChangedsubscription inruntime.tsfires on everysetConfig, andfirmware_upgradeinvalidates after the install command. -
tests/core/capability-guard.spec.ts— 9 cases through the real registry (the other Phase 2 gap): guard blocks and explains, satisfied requirement runs, no-requires never guarded, unprobeable device does not block, a throwing probe does not fail the call, handler never runs when blocked, description annotated, cold cache untouched,filteromits registration entirely. -
docs/capabilities.md+ README doc-table row +docs/configuration.mdrow and paragraph forcapabilityGating. -
bun run gen— 2 new per-tool schemas,tool-catalog.json,schemas/README.mdanddocs/tools-reference.md. The tools-reference churn was kept this time, not reverted: it carries the real new module, and CI runsbun run genthengit diff --exit-code, so the committed file must match generator output byte for byte. - Exit gate: ✅ global gate green —
test:typesclean, 765 tests / 71 files pass,lintexit 0. PHASE 3 COMPLETE.
Decisions taken during Phase 3:
- Invalidation went into
setConfigvia a subscription, not into the four dashboard call sites.setConfigis the choke point every reload routes through, so one hook covers all of them. It is a subscription rather than a direct import becausecapability-cachealready importsruntimefor device resolution — importing back would be a cycle. A listener that throws is swallowed: a cache concern must never be able to break config installation. - The two tools take no
deviceparameter of their own — the registry injects the standarddeviceselector in multi-device mode, so they follow the same targeting rules as everything else.
[x] Phase 4 — Dashboard + Raycast
-
GET /api/capabilities(non-probing — reports only resolved probes) andPOST /api/capabilities/refresh(invalidate + probe one device).serializeCapabilitiesadded tocapability.tsbecausepackagesis aSet, whichJSON.stringifyrenders as{}rather than failing. -
connectivity.tsx—CapabilityStripon each device card: version, board class, wireless stack, package chips, adevice-mode blockedwarning, and a Probe/Reprobe button. An unprobed device shows a Probe button, never an empty matrix that would read as "supports nothing". -
modules.ts— per-moduleunsupported: {device, reason}[], computed from resolved probes only.modules.tsxgains a device selector that dims unavailable modules and prints the reason inline. - Raycast:
devices.tsxshows the probed version in the subtitle,modules.tsxgains anunsupported on Ntag with the per-device reasons in its tooltip,lib/types.tscarries both new shapes. - Exit gate: ✅ global gate green —
test:typesclean, 765 tests / 71 files pass,lintexit 0,bun run build:uisucceeds (8 views), Raycasttscclean insrc/. PHASE 4 COMPLETE — TASK 02 DONE.
Decisions taken during Phase 4:
GET /api/capabilitiesnever probes. It reportspeekCapabilitiesonly. Loading the dashboard must not fan a 7-command probe out to every configured router; probing is an explicit button press.- Module-level rather than tool-level dimming.
withRequiresmakes a module's requirement uniform across its tools, so the first declaring tool speaks for the module — and the Modules view is module-shaped already. Tool-level would have needed a second payload for no extra information. - The device selector only lists devices some module reports on. A device with
no resolved probe contributes no
unsupportedentries, so offering it would dim nothing and falsely imply everything is supported. Badgeingeist.tsxtakes notitleprop — tooltips are on a wrapping<span>.
Blockers / notes
- Probe cost is 7 commands per device. Phase 1 issues 4 setting prints plus 3
menu-existence checks, in two
Promise.allwaves. That is fine behind the 6 h cache, but Phase 2 must ensure the probe is triggered once and never from inside a hot path — concurrent first-use on several tools must share one in-flight promise, not fire seven commands each. Store the promise in the cache, not the resolved value. /interface wifi print count-onlyis a real command on a device without the package, and returns a parser error rather than0.menuExiststreats any error as absence, which is correct, but it means a transient failure looks like "no wireless stack". Acceptable because an unknown stack never blocks (see the Phase 1 decisions), but do not later invert that./system device-mode printoutput shape is unverified against a real 7.13+ device. The parser readscontainer:/scheduler:/fetch:keys from a settings print and defaults every absent key to permissive. Now load-bearing — the container module gates ondeviceMode: "container"as of Phase 2. The permissive default means a wrong key name degrades to "unrestricted" (tools stay available) rather than wrongly blocking, so the failure mode is safe, but confirm the real key names before Phase 4 surfaces device-mode in the dashboard as fact.Invalidation hooks not wired— done in Phase 3 (config reload + firmware upgrade). Still NOT wired: invalidating on acommandUnsupported()hit, which would prove the cached model is stale. Deferred deliberately — it needs a hook inconnector.ts, and a device that legitimately lacks a menu would then reprobe on every call to it.No end-to-end guard test— done in Phase 3 (tests/core/capability-guard.spec.ts, 9 cases through the real registry).