Evaluation and conformance

Version: 0.1

1. Principle

A system is not evaluated by whether it boots, nor by the number of features it contains. Evaluation answers:

• can the system be reconstructed from its own record;
• is everything in it accounted for and authenticated;
• does it hold its security properties in its delivered configuration, not in a hardened variant;
• does it protect the data of the people who run it;
• can it be updated, rolled back and eventually retired safely;
• are its claims bounded by evidence.

A pipeline that is green proves that the pipeline ran. It does not prove any of the above.

2. Evidence levels

A property may only be claimed at the level at which it has been observed. A kernel option present in a configuration file is E2 evidence about the configuration, not E4 evidence about the running system.

Required minimum levels:

3. The verification corpus

Every product maintains a verification corpus. Each entry has the form:

CHECK-ID
requirement          SCF-*
property             what must be true
method               the tool, command or procedure
environment          offline | emulated | reference hardware
expected result
evidence level
owner

The baseline corpus below is mandatory for every profile unless a profile records an exception. It is intentionally expressed as properties and methods, not as tool invocations, because tools change faster than obligations.

The baseline is a floor, not a complete mapping of the Constitution. Where a requirement has no entry here, the product MUST add one to its own corpus before claiming that requirement met; a requirement claimed without an entry is an E0 declaration.

3.1 Specification and provenance

3.2 Artefacts and attestation

3.3 Boot integrity

3.4 Cryptography and credentials

3.5 Isolation and attack surface

3.6 Data and privacy

3.7 Updates and vulnerabilities

3.8 Legal and lifecycle

3.9 Human surfaces

Verified per the applicable HIF profile, with at least: installation without data loss on a machine with existing data; first run completed with a keyboard only; first run completed with a screen reader; every destructive action correctly describing its scope; recovery entered and exited without a network.

4. Severity

• S0 Blocker — the system cannot be trusted or used: it does not boot, an update bricks it, data is lost, a secret is shared across installations, an artefact is unsigned or unaccounted for, or an active exploitation report cannot be handled.
• S1 Critical — a security or privacy property claimed by the product does not hold in the delivered configuration.
• S2 Major — a required property holds only after manual configuration, or rollback and recovery work only in some cases.
• S3 Moderate — evidence is weaker than the required level, or documentation contradicts behaviour.
• S4 Minor — a local inconsistency with no effect on the properties claimed.
• Observation — a hypothesis, not a confirmed defect.

Severity is determined by consequence for the people and organisations running the system, not by how visible the problem is.

5. Conformance levels

SCF-Declared

• profiles chosen;
• the traceability matrix exists;
• known exceptions are registered.

This marks the beginning of a process. It is not a quality signal and MUST NOT be presented as one.

SCF-Conformant

• every applicable MUST is met;
• every SHOULD exception is justified and registered;
• the whole baseline verification corpus has been executed at its required evidence level, including reference-hardware checks;
• the release is rebuildable from its lock;
• the bill of materials, provenance and signatures are complete;
• the support period, update contract and disclosure policy are published.

SCF-Validated

• SCF-Conformant, plus:
• the release has been rebuilt independently and compared;
• high-risk paths — boot integrity, update, rollback, recovery, erasure — have been demonstrated on reference hardware;
• the vulnerability process has been exercised, not only described;
• operational monitoring for the support period is defined and staffed.

A conformance statement MUST name the SCF version, the profiles, the release, the architectures and the reference hardware to which it applies. A statement without those bounds is void.

6. Traceability matrix

obligation (law, threat model, customer requirement)
→ SCF requirement
→ product requirement
→ specification field, component or policy
→ implementation
→ automated check
→ manual or hardware scenario
→ current evidence and its level
→ exceptions

A requirement with no path to evidence is incomplete and MUST NOT be counted as met.

7. Release gate

Before a release is signed:

1. the scope of change is defined and the affected profiles listed;
2. the lock is complete, and every input is authenticated and pinned;
3. the build ran in a compliant boundary and its log is retained;
4. all offline checks pass, including licence, trademark and vulnerability checks;
5. the bill of materials, manifest and provenance are complete and valid;
6. the boot chain generation is current against revocation policy;
7. emulated dynamic verification passes;
8. reference-hardware verification passes for boot, update, rollback, recovery and storage encryption;
9. accessibility checks required by the applicable HIF profiles pass;
10. the support period, channel and end-of-life date are set and published;
11. every open exception has an owner and an unexpired review date;
12. a named person approves, and the approval is recorded with the evidence.

The rollback plan and the criteria for withdrawing the release are part of the gate, not a later decision.

8. Exceptions

ID
SCF version and requirement
product and profile
reason
affected users, devices and data
risk and likelihood
compensating control
owner
expiry or review date
evidence

An expired exception blocks a new conformance statement until it is reviewed. An exception MUST NOT be used to carry a permanent architectural decision; if a requirement is wrong, propose a change to the standard.

9. After release

The support period is part of the product, so evaluation does not stop at release.

• monitor upstream for changes to the inputs of every supported release, and be able to state how far behind each release is;
• monitor vulnerability data against the bill of materials of every supported release;
• monitor revocation and signing-authority policy;
• record time from report to fix to published advisory, and review it;
• re-run the verification corpus for every update that touches boot, kernel, cryptography, update logic or defaults;
• record field failures against the corpus, and add a check for every failure the corpus did not catch.

10. Changing the standard

If a product cannot meet SCF:

1. check whether the problem is local architecture rather than the requirement;
2. check the alternatives, including a different platform family;
3. register an exception with evidence;
4. gather evidence over at least one release cycle;
5. only then propose a change to the standard.

A widespread industry practice is not evidence that a requirement is wrong. The absence of an upstream facility is a constraint to be recorded, not a justification for a weaker claim.