# AirDesk 0.9.1 architecture

Home now displays read-only active poses derived by `activeGestures(Profile.bindings)`. Shared hold/scroll poses appear once, and pointing stays visible even when its click is disabled. There is no copied home assignment state. General and Gestures are tabs in the same Settings window; only Gestures contains the editor. The separate scroll flag/checkbox is removed: bindings are the sole enabled-state authority, with legacy preference migration at load. Optional manual mapping setup appears only in Fixed distance mode, without a second reach section. Opening Settings stops capture and input, allowing camera selection immediately.

Home starts a guided PracticeLesson derived from enabled major actions in the gesture catalog, independently of the editor selection. PracticeFeed owns this small session snapshot and uses existing PracticeSession scoring. Each successful action has a readable confirmation interval before the next step; drag must finish with an explicit drop. A final open-hand rest uses fresh capture timestamps and resets on tracking gaps. Disabled actions are skipped, shortcuts are outside this introductory lesson, and mouse clicks cannot advance it. Free/targeted feed modes remain available internally for tests and later callers. Practice never enables the Windows sink.

`gesture_reference.hpp` owns the shared reference strip renderer. Home and ActionUI's optional layered desktop window use the same Profile.bindings, pose grouping, cached images and captions. The desktop reference has no controls, returns HTTRANSPARENT/MA_NOACTIVATE, uses transparent/nonactivating window styles, and fades on pointer proximity. It is visible only during explicitly enabled desktop control with another app foreground, never in practice or while paused. Its contents redraw when shown or assignments change, not for every inference frame. No camera image or extra inference is used in that overlay. The single `Profile.showGestureReference` preference is persisted alongside other settings.

`Camera → latest Frame → HandProvider / DwellController → ControlFrame / Action → SafetyPolicy → WindowsSink`

Only WindowsSink may emit OS input. Practice consumes the same normalized actions in memory. All inference is local, uses one latest frame rather than a frame backlog, and has a 250 ms freshness bound. The native app has no browser, cloud model, service, or new runtime dependency.

## Sources of truth

- `gesture_catalog.hpp`: authoritative supported poses, function IDs, labels, actions, image references and three compatible alternatives per function. Profile owns the validated bindings. Assigning a pose resolves conflicts once for recognition, UI and persistence. Open palm is reserved for rest/drop. Customization assigns known poses; it does not train new ones.
- `interaction.hpp`: coordinate transforms, action types, named pointer/delay presets, dwell settings/feedback, image sheets, hand topology and centred reach presets. Pose image references select one cached sheet/frame.
- `distance_fit.hpp`: bounded image-space adaptation, learning observed palm scale only before arming or during open-palm rest. Automatic mode adapts the base reach with stable observations and freezes the area throughout active pointing/dwell/drag. Fixed mode bypasses adaptation. This is neither optical zoom nor a measurement of physical distance.
- `preview_layout.hpp`: one aspect-fit transform for home video, mapped pointer, practice rendering, inverse mouse hit testing and practice scoring dimensions. It consumes the actual mapping area published in ControlFrame, including automatic adaptation, rather than recomputing a separate area.
- `shortcuts.hpp`: a fixed whitelist of common keyboard actions. WindowsSink rejects shortcuts while physical modifiers are held and attempts to release only keys inserted by a partial batch. Failure pauses the router. There are no arbitrary macros or scripts.
- `control.hpp`: Profile is the live settings snapshot. HandGestures recognizes finger poses, smooths pointing, delegates selection to DwellController and implements grab/drop. ControlFrame carries actual pointer position, progress, confirmation, actions and one guidance message. Rendering never recomputes gestures.
- `dwell.hpp`: modality-independent selection based on capture timestamps. A 160 ms settling interval precedes the configured fill (default 1000 ms). Hold tolerance is 48 DIP. Once settled, the pointer and click lock to the same anchor. A brief excursion up to twice the hold radius freezes progress; returning within 140 ms preserves the countdown without counting the excursion's time. Larger movement unlocks immediately; sustained departure also resets. There is no click outside the hold radius. After a click, rearming uses an independent 45 DIP departure for 140 ms, so nearby targets stay reachable but a single shake cannot arm another click; large movement still leaves immediately. A pose flicker under 100 ms freezes time without emitting input; sustained loss cancels. Invalid/stale captures never complete an action. A check mark remains until deliberate movement rearms selection.
- `pointer_filter.hpp`: one adaptive filter for pointing. A 120 ms low-pass estimate of signed velocity reduces alternating tremor's influence on the response boost. Steady uses a 0.18 base response; intentional large travel still follows promptly and Direct remains unfiltered. Hold locking is independent of filtering.
- `feedback.hpp`: presentation timing, one GuidanceStabilizer and a reusable DwellAnimation. The worker stabilizes text once before publishing to all views; it does not reset that text on safety-epoch changes. A candidate must last 600 ms, and a displayed message gets at least 1500 ms before routine replacement. Actual completed actions may interrupt. The router consumes raw eligibility before presentation, so text never delays input cancellation. The animator interpolates reported progress over 80 ms, never predicts completion, and clears cancellation immediately. Only confirmed events produce the green check and 700 ms completion pulse.
- `reach.hpp`: pure, bounded five-stage comfort calibration and validated normalized reach settings. It observes a deliberate one-finger pose, averages a stable hold, requires real movement in each direction, and never emits actions. A 900 ms hold and 1500 ms minimum step display avoid rushed instructions. Tracking gaps reset the current hold; depth changes are not mistaken for reach. Completion returns a proposal; only a fresh matching request can commit it to Profile. Cancel/timeouts preserve prior reach. The camera identity, mirror setting, normalized area, palm reference and capture aspect are persisted; transient requests are not.
- `capture_policy.hpp`: bounded native format selection, preferring extra detail at 30 fps only at the baseline aspect ratio. Camera falls back if the larger native mode or RGB conversion is unavailable. No optical/digital zoom control or resolution oscillation during pointing is introduced. Models retain fixed 192px and 224px inputs.
- `hand_framing.hpp`: image-space size/framing thresholds and concise hints. Palm size below max(24 px, 3.5% of the shorter image dimension) is too small; a landmark box over 96% of image width/height is too large; landmarks within 0.5% of an edge request whole-hand visibility. The tracker still requires 0.80 confidence. These are heuristics, not physical distance measurements. Unsafe framing cancels input.
- `practice.hpp`: one thread-safe PracticeFeed validates and scores fresh actions at arrival before redraw. PracticeSession owns targets and pickup/drop; PracticeLesson selects enabled major actions for Home's guided sequence. Hand and mouse share geometry but have separate counts. Duplicate sequences, old-session frames and stale/future actions are rejected. State storage is constant and contains no video. Practice never enables the Windows sink.
- `gesture_images.*`: WIC decodes each sheet once, caching up to 320 pixels per column. Settings image buttons and the shared read-only reference use this cache and the pose catalog. There is no guide or full-screen practice window; targets and recognition are drawn in the home preview.
- `action_ui.*`: despite the historical filename, this is now noninteractive feedback only. It creates the click-through filling circle and a conditional framing hint. **There is no toolbar, no action button window and no action-selection callback.** These ownerless, nonactivating windows remain usable when the main window is minimized. Hints appear near the bottom of the selected physical display only when the main window is not foreground.
- `ui.hpp`: shared drawing helpers, colours and filling-circle/check-mark animation for native pointer feedback and practice. A moving fill surface, bold progress arc, larger check and one outward pulse make the transition explicit. Desktop feedback uses a 30 DIP base radius; practice scales the same renderer to its target. Practice retains a scored-hit confirmation for 1100 ms and avoids drawing a duplicate pointer check on that target.
- `setup_ui.hpp`: native control IDs, captions, page membership, tab order and layout. Creation, visibility, DPI relayout, settings labels and native smoke checks consume this table. Main and Settings share one window and one set of controls; hiding a page removes its controls from keyboard navigation.
- `app.cpp`: lifecycle, settings snapshots, worker scheduling and startup. Page state is presentation only. Settings edits update the existing mutex-protected Profile and persist after the lock is released. Opening Settings pauses input; Done never resumes it. No draft settings object, web view, service, or additional dependency is introduced. Existing 0.5 preferences are preserved.

## Finger-only actions and customization

The following describes default bindings. Recognition and the three image choices consume the same catalog; reassignments update the Profile, pause input, end practice and persist together. Scroll can share a pose with a held function, but drag and scroll are mutually exclusive. Common shortcut functions start disabled. A completed special gesture followed by ordinary pointing preserves the dwell repeat block; choosing another explicit special function starts a fresh deliberate hold at the prior aim.

One extended index finger points and dwells to click. After aiming with one finger, two fingers held still right-click; moving two fingers starts scroll and latches scrolling until the pose is lowered, so stopping a scroll cannot accidentally right-click. Three fingers held still double-click. Extra-finger selection preserves the previously aimed position, cancelling the countdown if the hand moves.

A closed fist with thumb tucked may grab only after recent one-finger aiming. The pointer is frozen through finger contraction; after 350 ms the provider emits Move/Press at the prior aim. Closed-hand movement uses palm displacement from the pickup pose, mapped through the same camera region, so curling the index does not shift the object. Opening the whole hand releases and marks a completed drop. Ambiguous pose, tracking loss or safety pause releases immediately without a provider completion. Open palm otherwise rests; returning to index pointing deliberately rearms.

Practice alone handles the natural intermediate pose between a grab and an open palm. After a tracked provider drag-release without completion, it remembers the release position for at most 700 ms and accepts a fresh open palm there. It never retains a held-button state, replays a Release, or uses the later pointer position. Tracking loss, calibration, invalid coordinates, warning frames, stale data or a capture gap greater than 250 ms cancels this bounded lesson confirmation. A miss gives an explicit retry instruction. The OS provider and safety router are unchanged.

`isPracticeFunction` selects major actions for both the lesson and its reference images. The shared renderer includes the reserved open-hand transition image in Practice, highlights it during dragging/release/rest, and omits unrelated shortcut assignments. Targets use one radius function (13% of the shorter mapped preview dimension) for rendering and hit testing; the next target waits until the previous confirmation ends, avoiding overlapping success and destination circles.

The circle renderer leaves the unfilled interior untouched, preserving the camera/desktop underneath, and paints only actual progress or confirmation. It converts logical bounds with `LPtoDP` before creating its clipping region. GDI's [ExtSelectClipRgn](https://learn.microsoft.com/en-us/windows/win32/api/wingdi/nf-wingdi-extselectcliprgn) expects device coordinates; passing unscaled bounds made the fill disappear in a scaled window. `tests/render_tests.cpp` draws into an off-screen DIB and checks backdrop preservation, partial fill and confirmation at three display scales, separately from the portable gesture tests.

No input mode is selected through an onscreen bar. The shared DwellController still accepts action modes for future modalities; HandGestures derives its mode from finger poses and does not persist it as UI state.

## Coordinates and sensitivity

Landmarks use unmirrored camera pixels, independently of the moving inference crop. Profile.mappingRegion selects validated personal reach for the current capture aspect, otherwise the legacy centred reach. PointerMapping applies mirror and that area, then returns normalized [0,1] display coordinates. Pointing, palm displacement during drag and inverse preview mapping use that same transform; ControlFrame publishes the area actually used. Calibration is not recomputed during a dwell or drag. Hand filtering affects steadiness, not screen coverage. Native input supports negative monitor coordinates and virtual-desktop absolute positions. Practice consumes the actual emitted target and rejects calibration observations.

Steady, Balanced and Direct use response values from one preset catalog. Adaptive following speeds up for deliberate large travel. Dwell locking remains separate from filtering, including in Direct mode. Practice targets lie within comfortable upright-hand reach: the prior corner targets could crop the wrist or fingers and therefore never become eligible for a click.

## Safety and lifecycle

SafetyPolicy owns explicit enablement, epoch rejection, capture freshness, physical-mouse priority, partial-input failure cleanup and the held-button watchdog. The physical mouse hook has its own message thread, publishes only an atomic timestamp and never locks the router or calls SendInput. WindowsSink tracks the correct button on partial right-click failure. Practice never enables this sink.

Start control requests camera/model readiness once through StartupRequest; cancellation, timeout or failure cannot silently restart it. Stop control releases the camera. Camera/control state is never persisted. Lock/disconnect/suspend stops capture. Capture uses an asynchronous Media Foundation callback mailbox: the capture thread cancels its wait, flushes/releases the reader and shuts the source down itself. Callbacks hold no UI or Camera pointer.

Camera/model errors, human comfort, true hand accuracy and fatigue still require live testing. Synthetic tests and native startup checks cannot establish those. A future gaze provider should reuse the shared dwell/actions/safety boundary while adding its own calibrated gaze eligibility and an independently tested way to choose actions.
