UNECE R155 Annex 5 Penetration Testing Checklist: Turning 69 Threat Examples into Test Cases

Every UNECE R155 assessment arrives at the same table sooner or later: Annex 5 down the left, and a column that is supposed to show what the manufacturer did about each line. Programmes that fill that column with “considered” get asked the follow-up question — considered how, and where is the evidence? This post is the checklist we use to make sure the answer is a test artefact rather than an adjective. It assumes you already have a TARA and a set of implemented mitigations; what it adds is the bridge from Annex 5 to the penetration-test plan and from the test results back to the evidence pack.

What Annex 5 Actually Asks For

Annex 5 has three parts. Part A is the threat catalogue: 69 examples of threats and vulnerabilities grouped under seven high-level categories, numbered 4.3.1 to 4.3.7. Part B lists mitigations the manufacturer should consider inside the vehicle; Part C lists mitigations for threats outside the vehicle, chiefly back-end systems. None of this is a test specification. The testing obligation sits in paragraph 7.3.6 of the regulation, which requires the manufacturer to perform appropriate and sufficient testing to verify the effectiveness of the security measures implemented.

Put the two together and the expectation is clear: the risk assessment considers every Part A example; mitigations from Part B and C are implemented where the risk warrants it; and testing demonstrates that those mitigations work. A penetration test is the most direct form of that demonstration, because it exercises the Part A threat against the Part B or C mitigation on real hardware and records the outcome. Our UNECE R155 compliance page covers the CSMS side; this checklist is about the vehicle-type side.

How to Use the Checklist

For each of the seven categories below, the pattern is the same: identify the assets on your vehicle type the category touches, pick the test cases that apply, run them, and file the artefact each produces against the Annex 5 reference. Three rules keep the result assessable.

• One artefact per test case. A bus capture, a fuzzing log, a screenshot of a rejected signature, a reproducible step list. The reviewer assesses evidence, not ticks.
• Non-applicability is argued, not assumed. If an example does not apply — no cellular interface on the variant, no third-party app platform — say so, cite the architecture document that proves it, and name the reviewer.
• Map every line to the TARA. Each tested example should point at the threat scenario and attack path in the risk assessment it exercises, and carry the attack feasibility rating the result supports. That is what lets the same report serve the ISO/SAE 21434 Cybersecurity Case.

4.3.1 — Back-End Servers Related to Vehicles in the Field

Annex 5 opens with the servers, because a compromised back end scales to the whole fleet: servers used as a means to attack a vehicle or extract data, services disrupted so vehicle functions fail, and vehicle-related data held on servers being lost or compromised.

• Authentication and authorisation on every vehicle-facing API — telematics, remote functions, diagnostics, fleet management — including token lifetime, replay and privilege separation between vehicles.
• OTA server: package signing keys, campaign authorisation, ability to target a single VIN with a crafted package.
• Certificate and key provisioning services: can a test environment issue production-trusted credentials?
• Denial-of-service resilience of services the vehicle depends on, and the vehicle’s behaviour when they are unavailable.
• Data at rest: location history, diagnostic uploads and personal data, with access logging and insider-misuse controls.

Evidence: API findings with reproduction steps, OTA server test results, a provisioning review memo. These map to Part C mitigations and to ISO/SAE 21434 Clause 10 verification.

4.3.2 — Communication Channels

The largest category, and the one most people picture when they hear “car hacking”: spoofing of messages, injection, replay, man-in-the-middle, denial of service on in-vehicle and wireless channels, and untrusted or unreliable messages accepted as genuine.

• CAN, CAN FD and Automotive Ethernet injection and spoofing across domain boundaries; gateway filtering bypass; flooding and bus-off attacks on safety-relevant buses.
• Secure Onboard Communication: freshness-value handling, MAC truncation, key-sharing mistakes, replay of synchronisation messages.
• Wireless: BLE pairing and bonding, Wi-Fi hotspot and companion-app sessions, cellular modem exposure, GNSS spoofing and jamming effects on location-dependent functions.
• V2X where fitted: certificate validation, replay protection, pseudonym handling and misbehaviour detection, not just radio-level injection.
• Diagnostic channels: UDS over CAN and DoIP reachable from outside the vehicle, session and security-level enforcement.

Evidence:captures of each injection and replay attempt with the ECU’s response, SecOC bypass attempts with outcome, wireless session tests, GNSS test results — each tied to an attack feasibility rating under Clause 15.

4.3.3 — Update Procedures

Annex 5 treats the update path as its own category: misuse or compromise of update procedures, denial of legitimate updates, and installation of unauthorised or downgraded software. This is where R155 meets R156.

• Signature verification on every update artefact, including partial and delta packages; behaviour on a tampered or re-signed package.
• Rollback and downgrade protection; version pinning against the RXSWIN the vehicle declares.
• Update-path fuzzing: malformed manifests, oversized payloads, interrupted transfers, power loss mid-flash.
• Workshop and dealer-tool flashing paths, which are often weaker than the OTA path they sit beside.

Evidence: signature and rollback test results, update-path fuzzing logs, an RXSWIN consistency check — filed under both the R155 checklist and the R156 software-update evidence.

4.3.4 — Unintended Human Actions Facilitating an Attack

A short category, and one a penetration test can only partly address: legitimate users or maintainers tricked or careless in ways that enable an attack. What a pentest can do is test the technical controls that limit the damage a human mistake causes.

• Tester and dealer-tool authentication: can an unauthenticated tool reach privileged diagnostic functions?
• Privileged-function access review: which functions are reachable from the owner’s app or the workshop tool, and what confirmation steps exist.
• Behaviour when a user pairs an unknown device or installs a third-party application.

Evidence: tester-authentication results and a privileged-function matrix. The organisational side — training, procedures — belongs to the CSMS evidence, not the test report.

4.3.5 — External Connectivity and Connections

Everything that plugs in or connects from outside: remote functions, third-party apps and devices, OBD and USB ports, charging interfaces, and external devices used as a route into the vehicle.

• OBD-II port: diagnostic reach into safety-relevant domains, aftermarket dongle scenarios, gateway enforcement.
• USB and media interfaces on the infotainment head: file parsing, firmware-update-over-USB paths, bridge to the vehicle network.
• EV charging communication: ISO 15118 handshake, Plug & Charge certificate handling, charger-to-vehicle trust boundary.
• Remote functions — unlock, climate, locate — abuse cases through the app and the backend, including account-takeover paths.

Evidence: interface test results per port and protocol, remote-function abuse cases with outcomes.

4.3.6 — Vehicle Data and Code

Extraction of software or proprietary data, manipulation of code, data or identity, erasure of logs, and compromise of personal data held in the vehicle. This is the category that lives closest to the hardware.

• Firmware extraction via debug ports, external flash, or update-package capture; key extraction from insecure storage.
• Secure Boot and Secure Flashing chains: can modified code run, and does the chain of trust hold from the Hardware Security Module up?
• Identity manipulation: VIN, calibration and configuration data writable through diagnostics without authorisation.
• Log integrity: can security or diagnostic logs be cleared or altered to hide an intrusion?
• Personal data on infotainment and telematics: storage, deletion on resale, exposure over interfaces.

Evidence: extraction attempts with outcome, Secure Boot bypass attempts, data-write tests, log-tamper results — Clause 10 verification and, for the end-to-end claim, Clause 11 validation.

4.3.7 — Potential Vulnerabilities If Not Sufficiently Protected or Hardened

The catch-all category: cryptography misused or outdated, components not hardened, debug access left open, default credentials, software vulnerabilities, network design flaws and unnecessary services exposed. Most “boring” findings land here, and they are often the ones an attacker uses first.

• Debug and JTAG access on production units; UART consoles; manufacturing modes reachable in the field.
• Cryptography review: algorithms, key sizes, random-number sources, key storage, certificate validation.
• Default or shared credentials across the fleet; hard-coded secrets in firmware.
• Software-vulnerability scan of third-party components with triage against the SBOM, and exploitation attempts where a finding is reachable.
• Network hardening: unneeded listening services, open ports on Ethernet ECUs, missing segmentation.

Evidence: hardening findings with severity, a triaged vulnerability scan — feeding Clause 8 vulnerability management as well as the test report.

The Evidence Package, Category by Category

The table below is the artefact we hand over alongside the findings report. It is the first thing a technical service reads, because it answers the coverage question before any individual finding is discussed.

Mistakes That Fail the Review

• Ticks without artefacts. A coverage table with no linked evidence is a claim, not a demonstration.
• Testing the specification instead of the vehicle. Evidence must come from the production-representative hardware and firmware, not a simulation of the design.
• Blank cells for non-applicable examples. Argue them; a blank reads as an omission.
• No link to the TARA. Findings that do not update an attack feasibility rating leave the risk assessment contradicting the test results.
• Stopping at the report. Remediation and retest evidence closes the loop; an open finding at assessment time needs a documented treatment decision.

Take-Aways

Annex 5 is not a test plan, but it is the best starting point for one. Treat each of the seven categories as a scope area, write test cases that exercise the mitigations you actually built, file one artefact per test, argue non-applicability where it is true, and map everything back to the TARA and ISO/SAE 21434 clauses. The result is a single report that the technical service, the OEM’s product-security team and your own engineers can all work from. For how we run the engagement that produces it, see automotive penetration testing in Germany and Europe; for the method itself, the six-phase pentest methodology.

Agnile Technologies delivers automotive penetration testing for OEMs and Tier-1 suppliers with findings mapped to UNECE R155 Annex 5 and ISO/SAE 21434 Clauses 10, 11 and 15, written for the technical services that assess vehicle-type cybersecurity.