HIFDocs
HIF / Documentation / Evidence

HIF Evaluation and Conformance

1. Foundational principle

An interface is evaluated neither by the number of components implemented nor by its resemblance to a design mock-up. Evaluation asks:

  • whether people achieve meaningful goals;
  • whether they understand the current state and its consequences;
  • whether they can correct an error;
  • whether accessibility is preserved;
  • whether rights and safety are respected;
  • whether the solution works in its stated context.

2. Evidence ladder

Different methods provide different kinds of evidence.

E0. Declaration

The team states that it has considered the principle. This does not constitute evidence.

E1. Design review

The model, flows, states and designs have been reviewed.

E2. Static and automated verification

Types, schemas, linting, semantics, contrast, builds and contract tests have been checked.

E3. Interactive expert review

An expert performs real tasks, including keyboard and accessibility workflows.

E4. Representative user evaluation

People from the intended audience perform tasks in a context representative of actual use.

E5. Operational evidence

Production signals, support data, incidents and longitudinal research are used.

Higher-risk functionality requires a higher level of evidence.

3. Metrics

3.1. Effectiveness

  • task success;
  • correctness of the outcome;
  • critical errors;
  • recovery rate;
  • data integrity.

3.2. Efficiency

  • time;
  • number of meaningful actions;
  • waiting time;
  • repeated work;
  • expert throughput.

3.3. Understanding

  • ability to predict the outcome;
  • ability to distinguish objects from states;
  • understanding of persistence;
  • understanding of scope and authority;
  • accuracy of the mental model.

3.4. Learnability

  • success on the first attempt;
  • improvement;
  • retention;
  • transfer between representations;
  • discoverability of shortcuts.

3.5. Accessibility

  • task completion by keyboard;
  • task completion with a screen reader;
  • zoom and reflow;
  • forced colours;
  • reduced motion;
  • a non-dragging alternative;
  • error recovery with assistive technology.

3.6. Trust and agency

  • perceived control;
  • correspondence between expectation and actual effect;
  • understanding of automation;
  • ability to refuse;
  • absence of coercion;
  • confidence calibrated to the system's actual reliability.

4. Task corpus

Each product establishes a corpus of tasks:

TASK-ID
goal
starting state
actor and permissions
context/devices
success state
critical errors
applicable HIF requirements
evidence required

The task is evaluated, not a tour through the screens.

Required task classes are:

  • orientation;
  • creation;
  • modification;
  • search;
  • comparison;
  • preservation;
  • deletion;
  • recovery;
  • a permission boundary;
  • an error;
  • an interruption;
  • a cross-device or cross-context transition;
  • automation.

5. Methods

5.1. Model review

Review:

  • the object model;
  • commands;
  • the state machine;
  • permissions;
  • persistence;
  • failure modes;
  • the audit trail.

5.2. Heuristic review

The Constitution, the applicable profile and domain-specific criteria are used. A general-purpose list of heuristics does not replace a domain review.

5.3. Predictive modelling

Fitts's law, the Hick–Hyman law, KLM/GOMS and other models are used only when their assumptions are satisfied.

5.4. Prototype evaluation

Prototype fidelity must be appropriate to the question. A paper prototype cannot provide evidence of performance, focus management or screen-reader behaviour.

5.5. Usability study

Participants, tasks, success criteria, neutral moderation and the method of analysis are defined in advance. A small study can reveal problems, but does not demonstrate statistical superiority.

5.6. Accessibility evaluation

Automated and manual evaluation, assistive-technology testing and research with disabled people are combined.

5.7. Field evidence and telemetry

Telemetry is collected only with a lawful basis, minimally and transparently. A behavioural metric is not interpreted without its context.

6. Severity

  • S0 Blocker — the task is impossible, control or data is lost, or there is a critical risk.
  • S1 Critical — there is a high probability of material harm or exclusion.
  • S2 Major — the task can be completed only with serious difficulty.
  • S3 Moderate — there is a material problem of comprehension or efficiency.
  • S4 Minor — there is a local inconsistency.
  • Observation — a hypothesis or an unconfirmed defect.

An accessibility barrier is assessed according to the function and audience it excludes, not its visual prominence.

7. Conformance levels

HIF-Declared

  • a profile has been selected;
  • a requirements matrix has been compiled;
  • exceptions are known.

This is the beginning of a process, not an indication of quality.

HIF-Conformant

  • all applicable MUST requirements are satisfied;
  • exceptions to SHOULD requirements are justified;
  • automated and expert checks have passed;
  • the accessibility baseline has been confirmed.

HIF-Validated

  • HIF-Conformant has been achieved;
  • key tasks have been evaluated with representative users;
  • high-risk flows have sufficient evidence;
  • operational monitoring has been defined.

The product MUST identify the HIF version and Product Profile to which its claim applies.

8. Traceability matrix

HIF requirement
→ product requirement
→ object/command/component
→ automated check
→ manual scenario
→ research task
→ current evidence
→ exceptions

A requirement without a means of verification is incomplete.

9. Release gate

Before release:

  1. the scope of the change is defined;
  2. the affected tasks are listed;
  3. states and failure modes are implemented;
  4. all applicable MUST requirements are satisfied;
  5. type, build, contract and end-to-end checks pass;
  6. keyboard operation and accessibility have been checked manually;
  7. high-risk commands have been tested separately;
  8. content and documentation are current;
  9. each exception has an owner and an expiry date;
  10. rollback and recovery are defined.

Visual approval does not replace the release gate.

10. Design exception

An exception records:

ID
HIF version and requirement
product/profile
reason
affected users/tasks
risk
alternative
owner
expiry/review date
evidence

An expired exception prevents a new conformance claim until the exception has been reviewed.

11. Changing the standard

If a product cannot satisfy HIF:

  1. determine whether the problem lies in the local architecture;
  2. examine alternatives;
  3. register an exception;
  4. collect evidence;
  5. only then propose a change to the standard.

The popularity of a local workaround is not evidence that it is appropriate as a general rule.