HIF Test Design for Web Interfaces

Status: operational method, version 0.1

This document defines how an auditor derives a defensible set of tests from a large web interface. It supplements HIF Evaluation and the Web Audit Playbook.

Test design reduces uncertainty; it does not prove the absence of defects.

1. Test basis

Before deriving cases, collect the available test basis:

• user goals and priority tasks;
• HIF Constitution and applicable Product Profile;
• acceptance criteria and business rules;
• object, command, state and permission models;
• content and form rules;
• supported environments and accessibility target;
• analytics, support reports and previous incidents;
• architecture and dependency boundaries;
• known change and risk.

Contradictions, omissions and untestable language are findings in the test basis. Do not silently invent a requirement. Record a stated assumption and obtain confirmation where it affects consequence.

2. Risk-based selection

Assign more coverage to:

• irreversible, financial, legal, health or safety consequences;
• authentication, permission, privacy and destructive boundaries;
• primary or high-volume tasks;
• complex rules, many states or many interacting parameters;
• recent, frequent or poorly understood change;
• known incidents and complaints;
• shared components with broad reach;
• accessibility-dependent paths and recovery.

For each test condition record the selected technique, coverage target, expected oracle, environment and evidence level.

3. Case schema

TEST-ID
task/finding/risk
test basis and requirement IDs
technique and coverage item
priority
preconditions
role and permissions
environment
synthetic test data
steps
expected semantic outcome
expected state, feedback and persistence
accessibility/input expectations
cleanup or recovery
actual result
evidence ID
status and date

Expected results MUST describe consequence, not merely appearance. Include the system state, user-visible feedback, persistence and available recovery.

4. Equivalence partitioning

Use equivalence partitioning when values or conditions are expected to be handled alike.

Procedure:

1. identify the data or condition;
2. derive valid and invalid, empty and inapplicable partitions;
3. ensure partitions are non-empty and do not overlap;
4. choose at least one representative from each applicable partition;
5. include output, configuration, time, permission and state partitions, not only form inputs.

Example — invitation code:

One representative supports only the assumption that the partition behaves uniformly. If behaviour varies, refine the partition.

5. Boundary value analysis

Use boundary value analysis for ordered partitions: length, quantity, date, size, time, zoom, timeout, viewport or rate.

For each boundary b, test values immediately below, at and immediately above where the domain allows. Include minimum and maximum boundaries and clarify whether they are inclusive.

Example — allowed quantity 1–20:

0, 1, 2, 19, 20, 21

Also test representation boundaries: pasted Unicode, decimal separators, timezone transitions, leap dates, rounding, bytes versus characters and client/server disagreement where applicable.

6. Decision tables

Use a decision table when an outcome depends on combinations of conditions.

Procedure:

1. list independent conditions;
2. list possible actions/outcomes;
3. enumerate rules;
4. mark impossible combinations and the authority that makes them impossible;
5. merge rules only when the outcome is demonstrably identical;
6. derive at least one case for each feasible, consequential rule.

Example — publish command:

Decision tables expose missing and contradictory business rules. They do not model history; combine them with state transitions when sequence matters.

7. State transition testing

Use a state model where history changes available commands or consequences.

Define:

• stable states;
• events or commands;
• guards and authority;
• transitions and side effects;
• feedback;
• persistence;
• invalid transitions;
• timeout, interruption and concurrency.

Example:

Draft -> Submitted -> Approved -> Published
  |          |            |
  +->Deleted +->Withdrawn  +->Rejected

Cover:

• every state at least once;
• every valid transition;
• prohibited transitions with no unintended side effect;
• representative transition pairs or longer sequences according to risk;
• refresh, back navigation, repeated command, expiry and concurrent change;
• recovery from partial failure.

Test the semantic state and source of truth, not merely the highlighted tab.

8. Pairwise and combinatorial testing

Use combinatorial testing when many parameters interact, for example:

• browser × viewport × input × authentication state;
• locale × currency × payment method × failure response;
• role × object state × feature flag × command;
• theme × contrast preference × zoom × component variant.

Procedure:

1. model parameters and meaningful values;
2. define invalid-combination constraints;
3. choose interaction strength from risk and evidence, not convenience;
4. generate a covering array;
5. add mandatory high-risk and boundary cases;
6. record achieved coverage and untested interactions.

Pairwise means 2-way coverage. It is a compression method, not a completeness claim. NIST warns that some failures require three or more interacting factors; raise the interaction strength or add targeted cases for high-risk rules.

Never use pairwise generation before validating the input model. Missing parameters produce efficient coverage of the wrong system.

9. Scenario and acceptance-criteria testing

Describe acceptance criteria as observable examples:

Given [relevant starting state]
When [semantic command or event]
Then [outcome, state, feedback and persistence]
And [recovery/accessibility condition where relevant]

Include happy path, alternative path, error, interruption and role boundary. Examples clarify rules but do not replace the underlying general requirement.

10. Error guessing

Error guessing uses tester knowledge, defect history and common failure modes. Maintain a project-specific checklist rather than relying on memory.

Prompt categories:

• empty, whitespace-only, duplicate and extremely long values;
• pasted, composed, bidirectional and non-Latin text;
• repeated click/key, double submit and browser back;
• refresh or close during pending work;
• expired session, stale link and changed permissions;
• dependency timeout, partial response and retry;
• storage full, denied permission and offline transition;
• deep link without prerequisite state;
• content expansion and missing translation;
• focus loss, hidden error and announcement race;
• third-party block, script failure and consent refusal.

An experienced guess becomes a normal reproducible case once documented.

11. Exploratory testing

Exploratory testing combines learning, design and execution. It MUST be chartered and time-boxed for commercial reproducibility.

Charter:

Explore [target]
With [resources, roles and data]
To discover [risk/question]
Using [methods or oracles]
For [time box]
Without [prohibited actions]

Session notes record:

• coverage and route taken;
• observations and questions;
• test data and environment;
• evidence;
• interruptions;
• new risks and follow-up cases;
• time spent on setup, testing and investigation.

Debrief each session. Convert material discoveries into stable regression cases or research questions.

12. Heuristic and checklist-based testing

Use checklists for consistent recall, not mechanical pass counts. Applicable checklists include:

• HIF requirement matrix;
• accessibility manual checks;
• content and form checks;
• performance/resilience checks;
• cross-browser and responsive matrix;
• privacy/security UI-boundary checks;
• product-specific incident patterns.

Version every checklist. Add the criterion and evidence for “not applicable”. A high checklist percentage does not establish usability or conformance.

13. Non-functional test conditions

Accessibility

Derive conditions from each applicable WCAG 2.2 success criterion, HIF accessibility requirement, semantic pattern and required task. Define browser, assistive technology and input combinations. Automated rules are a subset.

Performance

Define task, page group, percentile, device class, field window and lab conditions. Keep field outcomes, lab diagnostics and synthetic uptime separate.

Responsive and compatibility

Model viewport regimes, zoom, engine, input, orientation, preferences and content expansion. Use combinatorial coverage, then add high-risk exact combinations.

Reliability and recovery

Inject only authorised failure conditions. Test timeout, interruption, retry, duplicate action, partial completion, stale state and reconnection. Confirm preservation of valid work and acknowledged results.

Privacy and security at the UI boundary

Test disclosure, choice symmetry, permission timing, session feedback, error leakage and visible/browser storage using synthetic identities. Suspected vulnerabilities are escalated, not exploited under a UI audit.

14. Oracles

An oracle determines the expected result. Use, in priority order:

1. human safety and mandatory requirements identified by the client;
2. HIF Constitution;
3. applicable normative technical standard;
4. approved product requirement and business rule;
5. platform convention or product design system;
6. consistent prior behaviour;
7. expert model or research hypothesis.

Record conflicts. A competitor’s behaviour and auditor preference are not authoritative oracles.

15. Coverage and exit

Report coverage in observable units:

• requirements covered/total applicable;
• tasks and branches covered;
• partitions and boundaries covered;
• decision rules covered;
• states and transitions covered;
• parameter interaction strength;
• environments and input methods covered;
• sample units and random-selection method;
• open risks and blocked cases.

Exit when agreed coverage and evidence objectives are met, critical findings are escalated, limitations are recorded and the remaining uncertainty is understandable. “All tests passed” means only the recorded cases under the recorded conditions.

16. Review gate

Before execution, a second reviewer SHOULD check:

• traceability to task and risk;
• missing invalid and recovery paths;
• partition overlap or gaps;
• off-by-one boundaries;
• impossible or omitted decision rules;
• missing states and unauthorised transitions;
• incorrect combinatorial constraints;
• accessibility and input alternatives;
• deterministic cleanup;
• expected results that merely repeat the implementation.

17. Official English-language sources

• ISTQB, Certified Tester Foundation Level Syllabus v4.0.1, especially test analysis and design, equivalence partitioning, boundary value analysis, decision tables, state transitions, error guessing, exploratory and checklist-based testing.
• NIST, SP 800-142: Practical Combinatorial Testing and ACTS testing guidance.
• W3C, WCAG 2.2 and WCAG-EM.
• Google, Web Vitals.
• OWASP, Web Security Testing Guide.
• UK Government Service Manual, Using moderated usability testing.