medtrace

In medical device software, "we tested it" is not enough. You have to show which requirement each test verifies, which risk each requirement controls, and that those tests passed on the code you are releasing. IEC 62304 asks for traceability between software requirements, software system tests and the risk control measures implemented in software, in clauses 5.1.1 and 7.3.3. Most teams keep that in a spreadsheet, updated by hand before an audit, and it is out of date the day after.

medtrace keeps the trace in the repository, next to the code, and checks it on every commit. When a requirement has no code, no passing test, or a risk has no verified control, the build fails.

Requirements live in a YAML file in the repository: an id, a title, the text, and how the requirement is verified, by test, by inspection or by analysis. Risks live in a second file in the shape ISO 14971 uses, hazard, hazardous situation and harm, scored as severity times probability before and after controls, with the requirements that control each risk listed by id. A control can also be stated as outside the software, with the text of what that control is.

Code is tagged with a comment, # @implements SRS-003, above the definition that implements a requirement. Tests are tagged # @verifies SRS-003 directly above the test function. The tags work in any language with comments: a hash comment in Python, or /* @verifies FW-001 */ above a C test function.

The third input is the test runner's own report. Any runner that writes JUnit XML works: pytest, Maven Surefire, ctest, Jest with jest-junit. medtrace reads the result of each tagged test and attaches it to the requirement, and through the requirement to the risk it controls.

It checks twelve rules.

Rule	Fails when
TR001	a requirement has no code tagged @implements
TR002	a requirement verified by test has no test tagged @verifies
TR003	a tag names a requirement that does not exist
TR004	a requirement or risk id is defined twice
TR005	a requirement verified by inspection or analysis has no justification
TR006	a verifying test failed, was skipped, or did not run in this build
TR007	a risk has no control, in software or outside it
TR008	a risk control is not a known requirement
TR009	a risk control requirement is not verified by a passing test
TR010	a risk's residual score is above the acceptability limit
TR011	a @verifies tag is not directly above a test function
TR012	a requirement or risk entry is missing a field or out of range

In GitHub Actions the check is one step. Results from several test jobs are collected as artifacts and passed in as a glob, and the matrix is written to the job summary, so every build has its own traceability record. A second command, medtrace impact, lists the requirements, tests and risks a change touches compared with a base branch.

TR006 and TR009 are the point. A tag only says that someone intended a test to cover a requirement. It is an intention, not evidence. With the JUnit report, medtrace reads the actual test results, so a requirement counts as verified only if its tests ran and passed in this build. A skipped test or a test that was never collected is a failure, not a pass. Parametrised tests count as passed only if every case passed.

The same standard carries through to the risks. A risk is controlled only when each requirement that controls it is verified by a passing test in this build. A control that points at a requirement whose test did not run is reported as not verified, and the build fails.

Pulse to FHIR, my gateway from a pulse monitor's serial stream to FHIR R4, has 14 software requirements, 6 hazards and 36 trace tags. Its CI runs the unit tests and an integration test against a real HAPI FHIR server in separate jobs, then runs medtrace on both sets of results.

On the unit tests alone, the matrix fails, correctly. Two requirements, that each window is stored exactly once (SRS-009) and that every value links to the device that measured it (SRS-010), are verified by the integration test, which did not run. So the two hazards they control, the same reading stored more than once (HAZ-002) and a reading with unknown origin (HAZ-006), are not verified either. Twelve of fourteen test-verified requirements pass, and the check reports four errors from one test.

Rule	Where	What
TR006	SRS-009	test_replaying_twice_stores_each_window_exactly_once did not run
TR006	SRS-010	test_replaying_twice_stores_each_window_exactly_once did not run
TR009	HAZ-002	control SRS-009 is not verified by a passing test
TR009	HAZ-006	control SRS-010 is not verified by a passing test

With the integration job's results added, the same check passes.

One hazard has no control in the software at all. HAZ-005, a reading attached to the wrong patient, arises when the wrong patient identifier is given at the start, because the gateway only passes on the identifier it is given. Its control is stated as outside the software: device to patient assignment is done in the clinical workflow and checked at the bedside. The matrix shows that in the controls column, scored 8 before and 4 after, instead of pretending that code covers it.

• Not a QMS and not a substitute for one. It checks one piece of evidence: that the trace from risk to requirement to code to passing test is complete for this build.
• It does not judge whether a test actually verifies its requirement. That is design review.
• It does not validate itself. If you use it as part of a quality system, validate it for your intended use (ISO 13485 clause 4.1.6). tests/test_rules.py, one test per rule, is a starting point for that, not the validation.
• Risk scoring is a simple severity × probability matrix with one acceptability limit. Real risk management files are richer; this covers what the trace needs.