QA Auditor - Surgical Video Annotation (Sampling, Quality Gates, Release Readiness)

Role goal: Protect dataset quality at scale by running sampling-based audits, enforcing quality gates, and ensuring every dataset release is audit-ready and meets acceptance thresholds.

Responsibilities

Protect dataset quality at scale by running sampling-based audits, enforcing quality gates, and ensuring every dataset release is audit-ready and meets acceptance thresholds; Execute sampling plans (stratified by label type, difficulty, site/surgeon/device, model confidence) for ongoing QA and release audits; Audit annotations against guidelines + ontology; score defects using standardized rubrics (severity, type, root cause); Run and validate quality gates prior to release: completeness/coverage checks, schema/constraint checks (illegal combinations, temporal consistency, boundary rules), defect density and rework thresholds; Generate rework tickets with precise instructions; track closure and verify fixes; Partner with Annotation Lead to drive calibration actions (training refreshers, guideline clarifications, examples library); Maintain QA dashboards: defect rate, rework %, gate pass rate, audit cycle time, top error patterns by annotator/team/label type; Support release readiness: ensure required artifacts are complete for the Evidence Pack (QA results, sampling logs, sign-offs); Escalate systematic issues to Adjudication/Clinical Review (definition gaps, ambiguous classes, recurring edge cases); Produce QA audit reports (sample size, findings, severity distribution, root-cause categories); Gate status + release checklist sign-off recommendations; Rework queue and closure verification; Trend insights and corrective action plan (top 3 issues + fixes)

Requirements

3–7+ years in data QA / annotation QA / operations quality; experience auditing labeled datasets (video preferred); Strong command of sampling-based QA: stratified sampling, acceptance thresholds, and audit scoring; High attention to detail; comfortable reading and enforcing detailed SOPs/guidelines; Strong documentation skills; can write clear defect descriptions and rework instructions; Comfortable with dashboards/spreadsheets; basic metric literacy (defect density, rework, throughput)

Nice to have

Video dataset auditing (preferred)