Qualityfolio: Test Management as Code & Traceability Mesh
Turn requirements and software tests into executable Markdown and queryable SQLite verification evidence. Replace disjointed web ALMs with version-controlled, Git-native assurance that a reviewer can follow from requirement to result.
The Customer Problem: Four Regulated Testing and V&V Failures
The disjointed web ALM silo
Engineers write tests in Git while QA logs results in a separate ALM tool such as Jira with Xray or a legacy test manager. The two records drift apart, and nobody can say which one is true.
The manual traceability matrix scramble
The requirements-to-test matrix is assembled by hand in a fragile spreadsheet in the weeks before an audit or notified body review, then goes stale the day after.
Unverifiable execution proof
A result is recorded as a bare pass or fail, with no environment log, run record or raw output attached that would let a reviewer check it.
Brittle test-run reconstruction
Historical test cycles cannot be re-opened or inspected because the testing platform was upgraded, migrated or its licence lapsed.
The Consequential Engineering Solution: Test Management as Code
A requirements document states what a system should do. In Qualityfolio, test artifacts play that role for quality: they declare what must be verified. Evidence records what actually happened when the test ran. Both live in the same Markdown file, under version control, so intent and result can always be compared.
- Tests are Git artifacts. Test plans, suites, cases and execution evidence are files reviewed in the same pull request as the code they verify, with the full change history Git keeps.
- Evidence is data, not a screenshot in a ticket. Every cycle compiles into a SQLite database that any SQL tool can open, and the traceability matrix is regenerated from source instead of rebuilt by hand.
- Truth gaps become visible. A test case with no evidence is an unverified expectation, not a pass. Evidence with no test case is execution without declared intent. Both show up in the data.
| Question a reviewer asks | Point-in-time ALM record | Qualityfolio evidence |
|---|---|---|
| Which requirement does this test verify? | A manual link that breaks when either side is edited. | A requirementID in the test case's META block, versioned with the test. |
| What happened in cycle 1.1? | Whatever the tool still shows after upgrades and exports. | An evidence block per cycle with status, date, assignee and attached result files. |
| Is every requirement covered? | A spreadsheet assembled before the audit. | A query against the SQLite database, run again whenever the files change. |
| Can we see last year's run? | Only if the licence and data export survived. | Check out the Git revision and rebuild the database. |
What Netspective Offers: Engineering Responsibility
Qualityfolio is the tooling. The engagement is the engineering work that makes it hold up under review for medical device software built to IEC 62304 and ISO 13485:2016.
Spry orchestration runbooks
We build and maintain the runbooks (qualityfolio.md and the SQL ETL) that turn your test artifacts and evidence into the SQLite database and dashboard, and keep them working as your repository grows.
Requirement-to-evidence schemas
We choose the hierarchy model, define the META keys and identifier scheme, and map your requirement sources so every software requirement has a place to be traced.
Qualified human verification review
Engineers who build regulated software read the traceability matrix and the evidence behind it, and flag gaps before your auditor does. Sign-off stays with your quality organization.
The Qualityfolio Engine: Architecture and Hierarchy Model
Qualityfolio installs locally on a workstation or server; it is not a web service. Everything it shows is derived from files under version control.
- Runtime and tools
- Deno 2.x or later and Homebrew, then three tools: Spry parses the Markdown and runs the runbooks (
spry rb run,spry sp spc); surveilr, a local binary, ingests the evidence and applies the SQL ETL; SQLPage serves the dashboards. - Hierarchy by heading depth
- A
doc-classifyblock in the YAML frontmatter maps each heading level to a role: project, strategy, plan, suite, case or evidence. Spry validates the structure before anything is written. - Dashboards from SQLite
- The generated
resource-surveillance.sqlite.dbis the system of record. SQLPage renders test metrics, cycle-wise execution status, case drill-downs and a requirement traceability matrix (RTM) straight from it.
- Markdown artifactsTest cases and evidence, in Git
- SpryParses and classifies the headings
- surveilrIngests evidence, runs the SQL ETL
- SQLite RSSDOne file, the system of record
- SQLPageMetrics, cycles and the RTM
spry axiom web-ui) validates structure; it does not execute tests.Four hierarchy models
Pick the model by how much structure the work needs. Heading levels are never skipped, and evidence always sits at the lowest level.
| Model | H1 | H2 | H3 | H4 | H5 | H6 |
|---|---|---|---|---|---|---|
| Small | Project | Case | Evidence | – | – | – |
| Medium | Project | Suite | Case | Evidence | – | – |
| Large | Project | Plan | Suite | Case | Evidence | – |
| Complex | Project | Strategy | Plan | Suite | Case | Evidence |
Requirements are added by overriding the strategy or plan level, so one requirement can govern many test cases. Issues are added at the evidence level, next to the failure that raised them. The Large and Complex models suit compliance work.
Metadata a test carries
| Key | Where | What it records |
|---|---|---|
@id | Under any heading | A stable identifier (for example TC-MED-001) that links a case to its evidence. |
requirementID | Case META | The requirement the case verifies. |
Priority, Tags, Scenario Type | Case META | Risk ordering, grouping, and the kind of path under test. |
cycle, cycle-date | Evidence META | Which test cycle the evidence belongs to, and when. |
assignee, status | Evidence META | Who ran it and the outcome (passed, failed, blocked). |
Ingestion & Workflow Explorer
Four steps from a Markdown test case to a traceability question an auditor can re-ask. Run the commands from the qualityfolio/ folder of your repository.
Panel 1: Markdown test specification and metadata
A test case in the Medium model. Its @id links the case to its evidence, and its requirementID links it to the requirement it verifies.
---
doc-classify:
- select: heading[depth="1"]
role: project
- select: heading[depth="2"]
role: suite
- select: heading[depth="3"]
role: case
- select: heading[depth="4"]
role: evidence
---
# Infusion Dose Calculator (SaMD)
@id samd-dose-calculator
Software system testing for an IEC 62304 Class B software item.
## Dose Alert Verification
@id dose-alert-suite
Verifies the over-limit dose alert against its software requirement.
### Verify an over-limit dose entry raises a blocking alert
@id TC-MED-001
```yaml META
requirementID: REQ-SAMD-010
Priority: High
Tags: [IEC 62304, Software System Testing]
Scenario Type: Unhappy Path
```
**Steps**
- [x] Enter a dose above the configured maximum for the selected drug.
- [x] Attempt to confirm the order.
**Expected Results**
- [x] A blocking alert names the limit that was exceeded.
- [x] The order cannot be confirmed until the dose is corrected.
#### Evidence
@id TC-MED-001
```yaml META
cycle: 1.2
cycle-date: 04-11-2026
assignee: QA engineer
status: passed
```
**Attachment**
- [Results JSON](./evidence/TC-MED-001/1.2/result.auto.json)
- [Run MD](./evidence/TC-MED-001/1.2/run.auto.md)
Synthetic illustration Follows the public Qualityfolio Medium model (Project → Suite → Case → Evidence) and the META keys in the example project files on docs.opsfolio.com. The device, IDs, steps and assignee are illustrative.
Panel 2: Spry ingestion runbook
One runbook parses the test artifacts, ingests the evidence files and applies the SQL ETL.
# Parse test-artifacts/, ingest evidence/, apply qualityfolio-json-etl.sql
spry rb run qualityfolio.md
# Inspect what was captured
surveilr shell -d resource-surveillance.sqlite.db --output table \
--cmd "SELECT nature, COUNT(*) AS files FROM uniform_resource GROUP BY nature;"
Writes: resource-surveillance.sqlite.db. The runbook command is from the Qualityfolio architecture overview; the inspection query uses the surveilr CLI's documented uniform_resource table.
Panel 3: Dashboard generation and local serving
Build the SQLPage configuration from qualityfolio.md, then serve the dashboard to this machine only.
spry sp spc --fs dev-src.auto --destroy-first --conf sqlpage/sqlpage.json --md qualityfolio.md
# Bind to localhost; the dashboard is then at http://localhost:9227/
surveilr web-ui -d ./resource-surveillance.sqlite.db --port 9227 --host localhost
Writes: sqlpage/sqlpage.json. Commands from the Qualityfolio quick start, except the host: the published sample binds 0.0.0.0, which exposes the dashboard to the network, so this one binds localhost.
Panel 4: Requirement traceability SQL query
Lists every requirement with its test cases, each cycle's status and the evidence file behind it. A requirement with no test case shows as NOT COVERED.
-- Synthetic illustration: a requirement traceability matrix from the RSSD.
-- uniform_resource is surveilr's table (it already exists in a real RSSD).
-- test_case and requirement stand in for the links Qualityfolio's ETL reads
-- from each test case's YAML META block (requirementID).
CREATE TABLE IF NOT EXISTS uniform_resource (uri TEXT, nature TEXT, content TEXT);
CREATE TABLE IF NOT EXISTS requirement (requirement_id TEXT PRIMARY KEY, title TEXT);
CREATE TABLE IF NOT EXISTS test_case (
test_case_id TEXT PRIMARY KEY,
requirement_id TEXT NOT NULL REFERENCES requirement (requirement_id)
);
WITH evidence AS (
-- .../evidence/TC-MED-001/1.1/result.auto.json -> TC-MED-001/1.1/result.auto.json
SELECT uri,
substr(uri, instr(uri, '/evidence/') + 10) AS rel,
json_extract(content, '$.status') AS status
FROM uniform_resource
WHERE uri LIKE '%/evidence/%/result.auto.json' AND json_valid(content)
), runs AS (
SELECT substr(rel, 1, instr(rel, '/') - 1) AS test_case_id,
substr(rel, instr(rel, '/') + 1,
instr(substr(rel, instr(rel, '/') + 1), '/') - 1) AS cycle,
status, uri
FROM evidence
)
SELECT r.requirement_id,
COALESCE(t.test_case_id, 'NOT COVERED') AS test_case_id,
COALESCE(x.cycle, '-') AS cycle,
COALESCE(x.status, 'no evidence') AS status,
x.uri AS evidence_path
FROM requirement r
LEFT JOIN test_case t ON t.requirement_id = r.requirement_id
LEFT JOIN runs x ON x.test_case_id = t.test_case_id
ORDER BY r.requirement_id, t.test_case_id, x.cycle;
Synthetic illustration requirement and test_case are illustrative, not Qualityfolio's ETL schema, and the query assumes each result file has a top-level status. It runs as written in SQLite 3.
Verified Proof Ledger (Engineering Ground Truth)
| Claim | Status | What it rests on |
|---|---|---|
| Markdown test artifacts ingested and classified by Spry (doc-classify) | Built and tested | The public opsfolio/qualityfolio repository, architecture overview and quick start. |
| SQLite RSSD generated from test artifacts and evidence (spry rb run) | Built and tested | Documented end-to-end execution flow: resource-surveillance.sqlite.db. |
| SQLPage dashboard with a requirement traceability matrix (RTM) | Built and tested | Quick start dashboard contents; public demo screenshots. |
| Requirement-to-test linking through @id and requirementID metadata | Built and tested | Test artifact models and example project files. |
| Cycle-wise manual evidence capture (cycle, cycle-date, assignee, status) | Built and tested | Evidence META blocks and the role-based test management workflow. |
| Automated execution capture from a CI runner | Designed | Listed as a next step in the docs; scoped per engagement, not shown in the public example. |
| Automated test generation from requirements | Designed | Described on qualityfolio.dev; not part of the documented workflow. |
| Built-in 21 CFR Part 11 cryptographic electronic signatures | Not established | Evidence is manual checkbox entries in local SQLite. Part 11 controls belong to the validated process. |
| Push-button regulatory clearance | Not established | Clearance is a regulator's decision on the sponsor's submission. |
| SaaS multi-tenant cloud test management | Not established | Qualityfolio installs locally; the docs state it is not web-based. |
Real-World Regulated Scenarios
Where the evidence goes. Qualityfolio supports Part 11-relevant record keeping when configured within a validated process; your quality organization and your regulator decide whether a record is sufficient.
IEC 62304 software verification
For Class B or C medical device software: system and software requirements linked by requirementID to unit, integration and system tests, with each cycle's result files kept as evidence.
ISO 13485:2016 §7.3 design verification records
Verified test runs compiled from the repository into the design and development file, without reconciling spreadsheets by hand before each design review.
A Git-native release quality gate
A CI step queries the database and blocks a release unless every defined requirement maps to a passed test record. Wiring the gate into your pipeline is engagement work, not a built-in feature.
Service Boundaries and a Low-Risk First Step
The 2-week V&V traceability pilot
Week 1: structure
Move your existing Word or Excel test matrices into structured Markdown, set up the Git repository and write the Spry runbooks.
Week 2: evidence
Compile the baseline SQLite evidence database, run the SQLPage traceability dashboard locally, and walk the requirement coverage gaps with your engineering leads.
Boundaries
- Qualityfolio is tooling for test management and evidence structuring. It does not replace regulatory counsel, notified body audits or your verification sign-off.
- It has no built-in electronic signatures. Evidence is recorded as manual checkbox entries in a local SQLite database.
- No outcome is promised: clearance, certification and audit results are decided by regulators, notified bodies and auditors.
Engagement detail
What does the Regulated Software Quality & V&V engagement deliver?
It closes the gap between your verification evidence and what a notified body expects under IEC 62304. The deliverables are a documented safety classification, a traceability matrix tracing 100% of software requirements to a test, and a V&V plan.
A scoped engagement to close the gap between your current verification evidence and what a notified body or auditor actually expects: safety classification, requirements-to-test traceability, and a V&V plan your quality system can stand behind — delivered by engineers who build and operate regulated software themselves, not just advise on it.
What this engagement produces
The IEC 62304 gap assessment and its exported remediation checklist are the starting point, not the deliverable. Under IEC 62304, this service closes the specific gaps that checklist surfaces for MedTech software: a documented safety classification, a working requirements-to-test traceability matrix, and a V&V plan scoped to your software's actual safety class — built by engineers who build and operate their own regulated software products, and have walked through FDA submissions, FedRAMP authorizations, and HIPAA audits directly.
- Risk-Classified
- Safety class (A/B/C) drives required rigor per IEC 62304 §4.3.
- V&V Decoupled
- Verification (built right) evidenced separately from Validation (built the right thing).
- Living Traceability
- Machine-regenerable links from requirements to test executions.
- Audit-Defensible
- Evidence structured for inclusion in your FDA submission or Notified Body technical file.
Six capability areas
IEC 62304 Software Classification & Gap Remediation
Safety class A/B/C determination grounded in an ISO 14971 risk analysis, with a remediation roadmap for the specific gaps it surfaces.
- Safety Class A/B/C Determination
- Legacy SOUP Evaluation
- Remediation Roadmap
Requirements-to-Test Traceability Matrix
Multi-directional tracing from requirement to design element to test to risk control, regenerated from source control rather than hand-maintained.
- Multi-Directional Requirement Tracing
- Design Control Linkage
- Automated CI-Generated Trace Records
Verification Protocols & Automated Test Harnesses
Unit and integration verification evidence, backed by static code analysis records a reviewer can check against the specification directly.
- Unit Verification Protocols
- Integration Boundary Testing
- Static Code Analysis Evidence
Clinical Validation & Human Factors Evidence
User-needs acceptance criteria and IEC 62366 usability testing protocols, with support through a summative validation study.
- User Needs Acceptance Criteria
- IEC 62366 Usability Testing Protocols
- Summative Study Support
ISO 14971 Risk Management Integration
Risk control verification linked to the specific hazard each one closes, with residual risk reported against the same file the design team uses.
- Hazard Mitigation Verification
- Software Risk Control Option Analysis
- Residual Risk Reports
Audit & Premarket Submission Support
FDA 510(k)/PMA software documentation and Notified Body technical files, plus direct engineering support through the audit itself.
- FDA 510(k)/PMA Software Documentation
- Notified Body Technical Files
- Audit Defense Support
Ready to Build Your Verification Evidence?
Discuss a specific V&V gap, traceability chain, or safety classification.
Engineering services description. Not a quote or a statement of work — scope and pricing are established during the initial consultation referenced from the gap-assessment tool.