REDCap · Mount Sinai internship

Automated Clinical Systems Validation

I developed automation for validating REDCap upgrades in an isolated lab during my 2026 internship at Mount Sinai. The workflow combines controlled baselines, before-and-after comparisons, negative controls, browser testing, and reviewable evidence.

See the demonstration How it works

01 · Establish a baselineRestore the lab and capture the defined comparison scope.

02 · Upgrade and challengeCompare results, exercise controls, and verify drift detection.

03 · Restore and reportReturn to baseline and package verifiable reviewer evidence.

Simplified workflow. See the actual dashboard and terminal results below.

What made this an engineering problem?

A passing smoke test does not tell a reviewer whether the system preserved project configuration, record data, access assignments, and historical logs—or whether the comparator would notice an unexpected change.

Repeatability

Control the starting state

Pair the pre-upgrade database and web baseline, isolate the validation copy, and keep restore ownership explicit so reruns start from a known state.

Detectability

Test the test itself

Introduce a deliberate, controlled change after clean comparisons. The negative challenge passes when the workflow detects the difference—not when changed data appears equal.

Traceability

Make results inspectable

Connect phase summaries to exports, browser captures, comparison reports, logs, and integrity records rather than presenting an unexplained pass/fail dashboard.

The validation workflow

The complete suite is the release-oriented workflow; smaller component commands support investigation and reruns. A standalone component pass is not the same as a successful complete suite.

01 · Prepare

Establish the disposable lab.

Check host prerequisites and configuration; restore the matching database and web baseline before validation.

02 · Capture

Freeze the comparison scope.

Capture clean pre-upgrade API and authenticated URL artifacts from a defined set of validation projects.

03 · Upgrade

Apply and verify the staged upgrade.

Run the upgrade once within the shared lifecycle, then collect and compare post-upgrade evidence using the same scope.

04 · Challenge

Prove drift detection.

Apply a controlled negative change and confirm that the comparison reports the intended difference.

05 · Exercise

Check browser and security controls.

Exercise selected authenticated workflows, data-entry behaviors, access/session controls, and applicable e-consent scenarios.

06 · Review

Restore and assemble reviewer evidence.

Retain phase results, verify final restoration, and package the qualifying run with internal and external integrity checks.

See the validation suite in action

Frames from a September 24, 2026 demonstration in a local, copied REDCap lab using synthetic test records. The terminal is executing the suite while the separate reviewer dashboard updates phase status and exposes supporting summaries. Select any image to inspect its larger version.

Local REDCap Validation Dashboard showing configured baseline and upgrade inputs with validation phases pending beside an idle terminal — **01 · Controlled start**The dashboard identifies the install, upgrade, database, and web inputs before the validation suite begins.

Terminal reports controlled negative drift proved and three initial validation phases passed while the dashboard starts its Part 11 phase — **02 · Prove drift detection**Deliberate changes are reported as expected differences; the negative-control phase passes because the comparator detects them.

Terminal shows 10 of 10 validation phases passed, final result PASS, and two evidence archive checksums OK beside the fully passing dashboard — **03 · Finish and verify**The complete suite reports 10/10 PASS after returning the lab to the baseline and verifying checksums for complete and reduced evidence archives. The dashboard lists the nine validation components; evidence packaging is the tenth suite phase.

Reviewer-facing API Negative Test summary explains the intentional drift detected across seventeen projects — **04 · Inspect the evidence**The dashboard opens the negative-test summary: 17 projects compared, one intentionally changed project detected, and four controlled-drift differences. The PASS confirms that the comparator detected the intentional drift.

What happens when a run is interrupted?

A controlled interruption should not produce a misleading successful result or permit another run to begin while the shared baseline is being restored. These separate cancelled-run captures show cleanup ownership and operator feedback while baseline restoration is in progress.

After Ctrl+C the terminal stops the active validation phase, writes interrupted-run evidence, and begins restoring the baseline database and web environment while the dashboard shows restoration in progress — **Terminal-owned cleanup**After Ctrl+C, the suite stops the active phase, writes interrupted-run evidence, and invokes the baseline-restoration path. The screenshot captures database restoration in progress.

The terminal returns to its prompt after reporting that dashboard stop cleanup owns baseline restoration; a dashboard modal says Restoring baseline and warns against starting another validation run — **Dashboard-owned cleanup**The terminal reports that dashboard stop cleanup is already active and returns to the prompt while the dashboard continues restoration. The modal tells the operator to wait for cleanup before starting another run.

An engineering decision: one shared OQ lifecycle

API, authenticated URL, and negative-control validation need distinct results, but they depend on the same controlled upgrade. Running each as an unrelated end-to-end workflow repeats expensive state changes and complicates comparison and cleanup.

Design

Share the state transition

The complete suite establishes the baseline, freezes the eligible comparison scope, captures clean API and URL evidence, and applies one upgrade within the shared OQ lifecycle. Each comparison retains its own result and supporting artifacts.

Verification

Challenge a clean result

After clean comparisons, a separate negative-control phase introduces deliberate drift and must detect it. The negative result is expected evidence that the comparator can fail for the right reason, not a failed release decision.

Ownership

Keep the release boundary explicit

Standalone commands permit investigation and focused reruns; the complete suite owns the release-oriented result, final restoration, and qualifying evidence package. A component PASS cannot substitute for the complete run.

Tradeoff: Sharing an upgrade reduces redundant work, but it also makes scope freezing, failure propagation, and cleanup ownership important. The tests check these boundaries rather than assuming that a successful upgrade means the validation passed.

What the automation checks

The checks are targeted and evidence-backed. They do not imply exhaustive functional coverage of REDCap or a formal regulatory approval.

API and authenticated URL OQ-style comparisons

Before/after project metadata, record exports, selected user rights and role/DAG assignments, and preservation of pre-upgrade audit records. Additional legitimate post-upgrade audit entries are distinguished from lost historical records.

Negative-control testing

Purposefully introduce a scoped change and verify that the comparator rejects equality. This establishes that a clean result is not simply a comparator that always says “match.”

CRF and selected Part 11-aligned controls

Browser-driven data entry, reason-for-change capture, form locking, and blocked edits with supporting database and audit evidence. These checks support validation review; they do not confer Part 11 certification.

Authentication and session baseline

Selected protected-route behavior, login, weak-password rejection, logout, and session-cookie behavior. This is a defined browser-security baseline, not a full penetration test.

Reduced e-consent lifecycle

Where an eligible disposable fixture exists, the scoped adult, in-person workflow covers participant submission, investigator confirmation, and signed-copy retrieval/hash evidence. Broader e-consent scenarios are outside this claim.

Evidence integrity and cleanup

Phase-level summaries, machine-readable results, retained supporting artifacts, a reviewer index, SHA-256 manifests, and verification of the final baseline restoration.

Related application-security work: Caddy and Coraza

In a separate WAF workstream, I reviewed external security-test findings and developed targeted Caddy/Coraza request controls in front of an existing Apache/REDCap backend. I compared direct and protected request paths during regression so a security rule could be evaluated for both blocking behavior and unintended interference.

Targeted rate limiting

Apply independent limits to selected sensitive operations, including password recovery and specific project-management actions, rather than throttling every REDCap request.

Request validation

Reject ambiguous duplicate API parameters and apply scoped length and character checks to record-renaming inputs without treating unrelated requests as equivalent.

Cross-origin policy

Replace permissive API CORS behavior with an explicit allowed-origin policy and verify expected response headers through the protected path.

Native Linux handoff

Separate controlled binary construction from installation of the prebuilt Caddy/Coraza executable, with systemd service configuration and regression checks for a Linux operator.

The WAF and upgrade-validation suite were separate engineering deliverables.

Working with the team and preparing the handoff

This was internship engineering in a team environment, not an isolated personal lab. I discussed validation priorities and security-test findings with technical stakeholders, incorporated feedback on expected evidence, and prepared setup and operator guidance so another engineer could run and review the workflows.

I prioritized repeatable runs, accurate result interpretation, and operator handoff before expanding the test suite.

How a reviewer evaluates the outcome

The deliverable is a navigable evidence record, not just an automation script. The reviewer starts with the overall suite result and follows any question down to the relevant phase and artifact.

Observed results: September 24, 2026

In the recorded run, the suite validated an upgrade from REDCap 16.0.30 to 16.0.35, then restored the copied lab to 16.0.30.

10/10: Complete suite phases reported PASS, including evidence packaging.
17: Configured validation projects included in the shared API comparison.
121: Expected authenticated URL artifacts compared successfully.
197,341: Record rows reported in each set of before/after API exports, not unique patients.

The clean comparison reported 17 identical metadata and record-export scopes and 102 passing validation CSV comparisons. The negative-control phase then found its deliberate record, historical-log, user, and role changes. Finally, the suite restored the original version and verified SHA-256 checksums for the complete and reduced review packages.

The e-consent phase used a reduced adult in-person fixture; broader lifecycle cases were outside this run.

Human-readable and machine-readable summaries

Overall and phase results, declared scope, observed differences, version information, and references into retained evidence.

Evidence package and integrity checks

Reviewer-facing artifact directories, canonical retained results, manifest checks, and an external archive checksum support integrity verification before review.

Institutional source and validation packages remain private. These recorded results do not constitute formal FDA or Part 11 certification.

Engineering skills demonstrated

Software and automation engineering

Python, Bash, SQL, JavaScript/Puppeteer, REST API testing, containerized Linux workflows, controlled state transitions, failure reporting, and documentation for another operator.

Validation discipline

Predefined scope, positive and negative controls, separate observation and interpretation, evidence integrity, and explicit distinctions between a component result and the complete run.

Technical areas

PythonBash SQL / MariaDBJavaScript / Puppeteer REDCap APIsRHEL / Podman HTTP authenticationRegression testing SHA-256 evidenceCaddy / Coraza (separate WAF work)

More engineering projects Back to top