Surgical Video Annotation Program Lead

Surgical Video Annotation Program Lead (Team Lead). Location: USA (Remote/Hybrid). Type: Full-time. Reports to: Head of Delivery / Program Director. Role goal: Own end-to-end delivery of a high-throughput, clinically defensible, audit-ready surgical video annotation program—driving automation-first workflows, quality (IRR/QA), and on-time dataset releases.

Responsibilities

Program delivery (E2E): Stand up and run the annotation “factory” from intake → de-ID → task orchestration → annotation → QA/IRR → adjudication → dataset release + evidence pack; Ontology + guidelines execution: Partner with clinical SMEs to operationalize a procedure-specific ontology (phases/steps, tools, anatomy, events) and convert it into clear labeling guidelines and UI rules; Automation-first operations: Drive pre-labeling + verification workflows (not manual-from-scratch), implement routing based on model confidence/uncertainty, and continuously reduce human effort per labeled minute; Quality system: Implement and enforce multi-rater sampling strategy, IRR reporting by label type (kappa/alpha; IoU/Dice where applicable), calibration loops and retraining for annotators, QA gates + sampling plans with acceptance thresholds; Adjudication governance: Run the disagreement workflow, manage escalation to senior annotators/clinical reviewers, track ambiguity categories, and ensure guideline updates close recurring issues; Dataset release management: Own versioning, provenance, and release discipline—ensuring every dataset is reproducible and ships with an audit-ready Evidence Pack (provenance, QA, IRR, adjudication trail, sign-offs); Security + compliance coordination: Ensure labeling operations follow enterprise security requirements (access control, logging, retention, de-identification review) and support audits/vendor risk requests; Client-facing cadence: Lead weekly operating reviews, present throughput/quality metrics, manage scope changes, and ensure PoCs convert into scaled programs; Team: L1 annotators, L2 senior annotators, QA auditors, adjudicators; coordinate with clinical reviewers and ML/data engineering; Operating system: SOPs, training curriculum, calibration playbooks, quality scorecards, escalation paths, and release checklists; Metrics: Throughput, cycle time, rework rate, IRR trends, defect density, acceptance pass rate, cost per labeled hour/minute, and automation leverage (pre-label acceptance rate).

Requirements

6–10+ years in annotation operations / data operations / QA-led delivery, with at least 2+ years in a lead role managing teams and SLAs; Hands-on experience with video annotation (temporal segmentation + event labeling) and familiarity with bounding boxes/segmentation concepts; Demonstrated ability to implement multi-rater workflows, compute/interpret IRR, and run calibration to improve consistency; Strong program management skills: planning, staffing, throughput modeling, risk management, and stakeholder communication; Comfort working with tooling/APIs and structured data exports; ability to translate guidelines into tool-enforceable rules; Experience in regulated or sensitive-data environments (healthcare preferred): privacy-first mindset, audit trails, process discipline.

Nice to have

Healthcare domain familiarity: surgical workflows, OR video sources (endoscopy/robotic), common quality issues (smoke, blur, blood occlusion); Experience coordinating de-identification workflows for video/audio and supporting enterprise security reviews (SOC2/ISO-type controls); Exposure to automation/ML-assisted labeling: pre-labeling, confidence routing, active learning basics; Prior work on dataset versioning and “release” discipline (e.g., DVC-like thinking, evidence packs, reproducible builds).