HIF / Documentation / Multimodal feedback

Motion, Sound and Haptics

Status: normative multimodal design reference

Version: 0.1

This document defines temporal, auditory and tactile feedback in HIF. These modalities are functional interface channels. They MAY reinforce continuity, causality, attention, orientation and consequence, but MUST NOT become a decorative tax, an accessibility dependency or a substitute for correct state.

1. Multimodal contract

Begin with a semantic event:

event → intended human understanding/action
      → visual representation
      → programmatic/assistive representation
      → optional motion
      → optional sound
      → optional haptic

For every feedback event, specify:

trigger, owner and precondition;
meaning and urgency;
intended audience and environment;
start, update, completion, cancellation and failure;
modality mapping and timing relationship;
user and system preference;
unsupported-capability fallback;
repetition, escalation and suppression;
verification and telemetry that do not collect unnecessary sensitive data.

No modality is added until its contribution can be stated. Motion, sound and haptics MUST agree about the event. A success chime MUST NOT accompany a pending visual state; an error haptic MUST NOT fire before validation is known.

2. Functional taxonomy

2.1. Motion

Role	Human outcome	Example	Constraint
Direct response	action feels coupled to input	dragged object follows pointer	latency and geometry preserve causality
State transition	change is visible, not magical	disclosure expands	final state is available without waiting for decoration
Continuity	object identity survives transformation	selected card becomes detail	origin and destination are credible
Spatial orientation	navigation preserves a mental map	adjacent pane enters from its modelled direction	direction matches information architecture
Feedback	command or constraint is acknowledged	control settles into selected state	semantic state is independently exposed
Attention	material change is noticed	restrained highlight of updated value	not continuous, coercive or the sole cue
Progress	time and phase are represented	determinate operation advances	does not fabricate progress
Demonstration	a novel relation is explained	first-use gesture demonstration	replayable, dismissible and alternatively documented

2.2. Sound

Role	Representation	Appropriate use
Speech	language	instruction or content where speech is the selected mode
Earcon	abstract musical/timbral motif	small, learned event vocabulary
Auditory icon	sound related to a real-world cause	event with a credible acoustic analogy
Status tone	short non-speech cue	confirmation, warning or boundary
Continuous sonification	changing sound maps changing data	monitoring or data exploration with training and alternative
Spatial cue	perceived direction/distance	off-screen or spatial object where position is meaningful
Ambience	environmental state	immersive context, never essential status by itself

2.3. Haptics

Role	Example
Contact/activation	press or touch is registered
Selection/detent	discrete slider step or picker value
Alignment/snap	object reaches a guide or valid drop position
Boundary/collision	movement reaches a real limit
Completion	process or command has completed
Success/error/warning	outcome class is reinforced
Texture/continuous parameter	specialist control or accessible data representation

The number of semantic signals SHOULD remain small enough to learn and distinguish. Intensity alone MUST NOT encode an extensive vocabulary.

3. Causality and continuity

Feedback belongs to the event that caused it. The perceived source SHOULD be clear in time and space.

For direct manipulation:

the controlled object follows input with minimal avoidable latency;
animation does not fight the gesture;
release preserves current position and velocity where the physical model requires it;
snap, collision and boundary feedback occur at the actual threshold;
cancellation returns to a credible state;
interruption starts from the currently rendered state, not a stale keyframe.

For state transitions:

persistent objects retain recognisable identity;
inserted and removed content has a defined effect on neighbours;
direction follows the navigation or spatial model;
teleportation MAY be used when no spatial relationship exists;
motion MUST NOT imply that data was transferred, deleted or saved when it was not.

Shared-element transitions require stable object identity. Visual similarity alone does not establish that two elements are the same object.

4. Attention and interruption

Motion, sound and vibration compete for limited attention. Use the least interruptive channel and intensity that produces the required outcome.

Classify events:

ambient: optional awareness; no interruption;
status: relevant to current work; visible/audible without seizing focus;
action required: timely response is needed; clear persistent route;
urgent/safety critical: immediate material risk; strong multimodal cue with escalation and acknowledgement.

Marketing importance MUST NOT be classified as urgency. Repeated animation, sound or haptics MUST NOT be used to coerce engagement.

Notifications MUST define:

source and affected object;
why the person receives it;
urgency and expiry;
grouping and rate limit;
quiet hours, focus mode and channel preferences;
acknowledgement and history;
duplicate suppression across devices;
escalation only when consequence warrants it.

A notification sound or vibration MUST NOT reveal sensitive content or relationship to bystanders without the relevant privacy policy and preference.

5. Timing

5.1. Time model

Timing derives from purpose, distance, complexity, platform convention, device performance and user preference. HIF does not prescribe one universal duration.

Recommended semantic ranges are defined in the Product Profile:

immediate response: acknowledgement during or directly after input;
micro-transition: a local state change;
navigation transition: a change of region or context;
explanatory sequence: a user-controlled demonstration;
continuous/system-driven: progress, monitoring or simulation.

Each range has minimum, preferred and maximum values plus a reduced-motion equivalent. A token is not evidence that a duration is perceptually suitable.

Feedback for direct action SHOULD be delivered as close to the triggering event as the platform can reliably support. Delayed feedback weakens causality and MAY be mistaken for another event. Animation MUST NOT delay access to a result that is already ready.

5.2. Sequencing

Prefer a small number of meaningful stages. Staggering MAY reveal order, but:

it MUST NOT delay routine access to later items;
list length MUST NOT produce unbounded total duration;
focus MUST NOT land on a moving or unavailable target;
repeated navigation SHOULD reduce or omit an already understood sequence.

Sound envelopes and haptics SHOULD align with the material event, not merely with the beginning of the animation file.

6. Easing and physics

6.1. Parametric easing

Easing communicates how a change starts, travels and settles:

an entering object may decelerate as it arrives;
an exiting object may accelerate as it leaves;
a persistent object may use a symmetric or context-specific curve;
a directly manipulated object follows input rather than a decorative easing curve.

The named token MUST include the actual curve and intended role. Terms such as “smooth”, “natural” or “snappy” are not specifications.

6.2. Spring and inertial models

Physics-based motion MAY improve continuity when velocity, target or forces can change during interaction. Define:

mass or equivalent perceptual weight;
stiffness;
damping ratio;
initial and current velocity;
rest thresholds;
bounds and collision;
retargeting and cancellation.

Bounce is not synonymous with natural motion. Underdamped springs can prolong settling, distract, impair precision and affect motion-sensitive people. Use overshoot only when it expresses a real interaction property and remains controllable.

The implementation MUST have a deterministic completion condition. An oscillation that never reaches rest is not suitable for routine UI state.

7. Interruption, retargeting and cancellation

User input takes precedence over non-essential animation. Every interactive transition defines:

whether it can be interrupted;
what happens on repeated activation;
whether it reverses, retargets, completes or snaps;
how focus and accessibility state change;
how callbacks and pending operations are cancelled;
the final state after application backgrounding, route change or process interruption.

On retargeting, preserve current position and velocity where that matches the model. Restarting from the original keyframe creates discontinuity.

Visual cancellation does not cancel the underlying command. The command and presentation each need an explicit cancellation contract. A screen that has left MUST NOT later receive a stale completion animation or sound.

Rapid repeated input MUST NOT queue a long sequence of obsolete animations, tones or vibrations. Coalesce feedback around the resulting state.

8. Motion accessibility and safety

8.1. Reduced motion

The product MUST honour platform reduced-motion preferences where available and SHOULD provide an in-product control when its motion is substantial.

Reduced motion is a semantic alternative, not a global duration multiplier. Depending on the effect:

remove parallax, simulated camera travel and large-field movement;
replace spatial transition with an immediate state change or restrained dissolve;
reduce distance, scale, blur and overshoot;
stop automatic and repetitive animation;
retain essential progress and state change in a non-motion representation;
preserve direct tracking under the person's control where safer than autonomous motion.

The alternative MUST be designed and tested, not generated by setting every duration to zero. Instant removal can itself destroy orientation or cause flashing.

8.2. Vestibular risk

High-risk patterns include:

parallax and opposing foreground/background movement;
large zoom, scale and depth transitions;
simulated camera or world motion not controlled by the viewer;
movement across a large field or in peripheral vision;
repeated oscillation, spinning and auto-scroll;
rapid blur or perspective change.

These SHOULD be avoided in routine interfaces and MUST have a reduced alternative when non-essential.

8.3. Flashing and blinking

Content MUST meet the adopted WCAG flashing requirements. Merely slowing a flash does not establish safety. Evaluate frequency, area, luminance change and red flash according to the normative threshold and use official test methods.

Automatically moving, blinking, scrolling or updating content within WCAG scope MUST provide the required pause, stop or hide mechanism. Essential status must remain available when animation is paused.

9. Performance and rendering

Motion quality includes responsiveness, frame consistency, power, memory and thermal behaviour. Test on the lowest supported performance profile.

Implementation SHOULD:

perform expensive preparation before the interaction where safe;
avoid unnecessary layout and paint on every frame;
animate properties supported efficiently by the target platform;
use the platform frame scheduler;
bound particle, blur, shadow and layer costs;
avoid retaining large off-screen surfaces;
stop work when animation is invisible or the application is backgrounded;
degrade decoration before semantic feedback.

Do not assume a universal refresh rate or fixed 16.7 ms frame. Measure against the active display and platform APIs. A high average frame rate does not excuse input latency, long frames or jank at the moment of action.

Performance degradation MUST NOT skip the final state, lose a completion announcement or desynchronise modalities. When a transition cannot render reliably, substitute a simpler transition.

10. Sound design

10.1. Semantic sound

A sound belongs to a semantic family and SHOULD be distinguishable by more than volume. Define:

event and urgency;
duration, envelope, timbre and pitch range;
repetition and maximum rate;
relative gain and mix priority;
spatial behaviour;
ducking or interruption policy;
silence and visual/haptic alternative;
ownership, licence and mastering requirements.

Do not use a culturally or physically loaded auditory icon without research in the target population. An abstract earcon requires learning and consistent reuse.

10.2. Audio control

The interface MUST respect system mute, output, volume, accessibility and focus settings. User-initiated media, communication, navigation cues and notification sounds require separate policy.

Web audio that starts automatically is subject to WCAG audio-control requirements and user-agent autoplay policies. HIF SHOULD avoid automatic audio in general-purpose interfaces. A visible control MUST remain accessible when audio is unavailable.

Speech and essential listening content MUST not be masked by background sound. Transient cues SHOULD be mixed conservatively and tested through speakers, headphones, assistive listening configurations and noisy environments.

10.3. Spatial audio

Spatial audio MAY support orientation when position is part of the domain. Specify:

coordinate space and listener reference;
source ownership and persistence;
distance attenuation and occlusion;
head-tracked or screen-relative behaviour;
conflict with speech and safety signals;
stereo, mono and no-audio fallback.

Spatial position MUST correspond to the actual interface or world model. Do not place routine notifications arbitrarily around the listener. Critical information needs a non-spatial equivalent and MUST remain understandable through one channel, hearing loss or device downmix.

11. Haptic design

11.1. Semantics

Use haptics to reinforce a discrete material event. The source SHOULD be apparent through concurrent action or visual change. Haptics MUST NOT be added to every tap; saturation removes meaning and can create fatigue.

Prefer platform-defined semantic effects for standard interactions. Custom patterns MAY be used when the domain requires a distinct vocabulary and hardware support, learning, fallback and validation are documented.

11.2. Capability and limits

Actuators differ in latency, amplitude, frequency control, location and waveform support. Platforms may expose only predefined effects. Therefore:

detect support at runtime;
do not infer capability from device category;
provide a no-haptic path;
respect system intensity and disable settings;
avoid promises that a custom waveform will feel identical across hardware;
limit duration and repetition for comfort, power and thermal constraints;
synchronise with audio/visual events using measured device behaviour where precision matters.

Vibration MUST NOT be the sole carrier of an alarm, error, navigation direction or completed action. People may not feel the device, may have reduced sensitivity, may be wearing gloves or may have disabled haptics.

12. Multimodal equivalence

Equivalence means preserving outcome, not making every channel imitate every other channel.

Event	Visual/programmatic core	Optional sound	Optional haptic
Command accepted	control state and status	restrained confirmation	light activation
Operation succeeded	persistent outcome/status	success motif	success pattern
Validation failed	field relationship and error summary	error cue if appropriate	error pattern
Boundary reached	visible constraint and accessible status if needed	boundary tick	collision/detent
Off-screen urgent event	persistent notification and route	urgent tone	urgent pattern
Progress	determinate/indeterminate state and announcement policy	sonification only for justified monitoring	continuous haptic only for specialist case

Turning off any optional channel MUST leave the task understandable and operable. Where sound, motion or touch is itself the subject of the task, an appropriate description, transcript, visualisation or device-specific alternative MUST be provided as far as the domain permits.

13. Tokens and assets

Recommended token families:

motion.duration.immediate/micro/navigation/explanatory
motion.easing.enter/exit/persistent/emphasised
motion.spring.direct/settle/boundary
motion.distance.local/region
motion.reduced.*
sound.event.activation/success/warning/error/urgent
sound.gain.interface/notification/media
sound.rate.maximum
sound.spatial.*
haptic.event.activation/selection/alignment/boundary/success/warning/error
haptic.intensity.subtle/standard/strong
feedback.suppression.*

Each token records:

semantic role and forbidden uses;
value or referenced asset;
units and platform transformation;
user-preference mapping;
fallback;
version, owner and licence/provenance;
dependent components and tests.

Raw filenames such as ding2.wav, arbitrary cubic-bezier values and vibration arrays MUST NOT be the public semantic API. Assets MUST be loudness- and quality-checked, licensed, versioned and delivered within performance budgets.

14. Specification algorithm

Identify the event, desired understanding and urgency.
Confirm that persistent visual and programmatic state are correct.
Select the least interruptive sufficient modality set.
Define causality, spatial source and modality timing.
Define complete, failure, cancellation, repeated-input and background cases.
Design reduced-motion, silent and no-haptic equivalents.
Adopt platform semantic patterns where they match the event.
Tokenise roles and register licensed assets.
Prototype on representative hardware and output devices.
Test perception, distinction, comfort, accessibility and performance.
Measure field suppression, repeated notifications and unintended interruption without collecting unnecessary content.
Remove any cue whose contribution is not demonstrated.

15. QA matrix

15.1. Functional states

Test start, update, success, warning, error, cancellation, timeout, offline, background, resume and rapid repeated input. Confirm that the displayed final state—not animation completion—owns the operation result.

15.2. Preferences and capabilities

reduced motion on/off and changed while running;
audio muted, low volume, mono, different output and screen reader active;
haptics unsupported, disabled and intensity reduced;
notification focus/quiet modes;
high contrast, forced colours and magnification;
low-power and reduced-data modes where relevant.

15.3. Performance

lowest supported hardware and a thermally constrained state;
high and variable refresh rates;
interrupted frames and main-thread load;
delayed audio and haptic delivery;
application background/foreground;
multiple simultaneous cues and notification storms.

15.4. Human evaluation

Representative participants SHOULD be able to:

identify the event and source without guessing;
distinguish signals that require different action;
complete the task with each optional modality disabled;
interrupt and reverse motion without losing control;
use reduced-motion mode without loss of orientation;
tolerate repeated routine use without unacceptable distraction, discomfort or fatigue;
recognise urgent events without marketing or routine events becoming indistinguishably intense.

Include people with vestibular conditions, hearing differences, tactile sensitivity or reduced sensation and relevant assistive-technology experience. Do not expose participants to flashing or motion known to exceed safety requirements.

16. Release gates

A release is blocked when:

semantic state depends only on motion, sound or haptics;
reduced motion, silence or unsupported haptics makes a task incomplete;
animation delays or rejects input without a task reason;
a transition loses focus, state, data or command result;
flashing has not been evaluated against the normative threshold;
notification urgency is inflated or preferences are ignored;
cues are materially desynchronised from their events;
performance causes dropped final states or repeated stale feedback;
asset licence or provenance is unknown.