Senior Network Production Engineer, Operations

Crusoe Cloud is seeking a Senior Network Production Operations Engineer to support production reliability across our global network infrastructure, including edge, backbone, data center fabric, and GPU cluster interconnects. This is a hands-on production role focused on incident response, root cause analysis, and automation-driven operational execution that keeps our hyperscale AI infrastructure running at scale. Your work will directly affect the availability of AI workloads running across thousands of GPUs worldwide.

Responsibilities

Support uptime across Crusoe's global edge, backbone, data center, and GPU cluster network, directly supporting AI workloads at scale; Write and maintain Python-based tooling to reduce toil, automate common remediation workflows, and speed up mean time to resolution across the operations org; Participate in high-severity network events, including detection, triage, mitigation, stakeholder communication, and postmortem documentation; Perform RCAs using existing and newly built tooling, and help drive remediation items to closure; Build automation on top of Crusoe's monitoring stack (streaming telemetry, SNMP, NetFlow, Kentik, Grafana, Prometheus, ThousandEyes) to reduce manual triage and surface actionable signal faster; Maintain and improve runbooks, escalation playbooks, and SOPs, flagging opportunities for scripting or automation; Help build and maintain automated dashboards and alerting that back reliability metrics, in partnership with Architecture and SRE teams; Work closely with Architecture and SRE teams to execute on operational excellence initiatives and improve day-to-day practices

Requirements

5+ years of production network engineering experience with a focus on operations, incident response, and reliability in large-scale or internet-scale environments; Strong Python and scripting proficiency, including diagnostic tooling and automation scripts; Hands-on experience with observability and monitoring tools including streaming telemetry, SNMP, NetFlow/sFlow, Grafana, Prometheus, and ThousandEyes, and the ability to script against their APIs; Experience operating RDMA/RoCE lossless fabrics for GPU or HPC workloads, including familiarity with PFC, ECN, and DCQCN tuning; Solid hands-on knowledge of BGP, EVPN-VXLAN, IS-IS, OSPF, MPLS, QoS, and TCP/IP in production data center environments; Proficiency with Arista (EOS) and Juniper (Junos) platforms in leaf-spine CLOS architectures across multi-vendor environments; Comfort operating large device fleets across multi-region environments with on-call responsibility; Bachelor's degree in Computer Science, Electrical Engineering, or a related field, or equivalent practical experience

Nice to have

Experience with NVIDIA/Mellanox networking platforms in GPU cluster environments; Familiarity with Kentik or Arbor for traffic analysis and DDoS visibility; Exposure to contributing to SLIs and SLOs alongside SRE or product teams; Exposure to operating 10K+ device fleets across hyperscale or cloud environments

Benefits

Competitive compensation; Restricted Stock Units; Paid time off & paid holidays; Comprehensive health, dental & vision insurance; Employer contributions to HSA account; Paid parental leave; Paid life insurance, short-term and long-term disability; Professional development & tuition reimbursement; Mental health & wellness support; Commuter benefits (parking & transit); Cell phone stipend; 401(k) Retirement plan with company match up to 4% of salary; Volunteer time off