Staff Software Engineer, DC Infrastructure

Crusoe’s Data Center Infrastructure Engineering (DCIE) team is fundamental to our mission of providing AI hardware and infrastructure as a service. The team provides infrastructure for Crusoe’s fleet GPU’s and data center. The team sits at the nexus of high performance computing and AI infrastructure as a service. The DCIE team owns, deployment maintenance, observability, critical environment, and automation. The team builds, and maintains GPU clusters, develops automation for logical and physical maintenance, provision systems, and observability tooling. We are seeking a highly skilled and motivated Software Engineer to join Crusoe’s Data Center Infrastructure Engineering team. This position is focused on the development of software for the management of a fleet of GPU servers as well as the data centers that house those systems. The role focuses on the developing and implementing advanced diagnostic, observability, automation and repair tooling for high-performance GPU compute clusters.

Responsibilities

Developing and implementing deep-level diagnostics and troubleshooting of hardware faults within GPU racks and high-density compute systems; Developing troubleshooting and automation tooling for GPU platforms including NVIDIA A100, H200, GB200, B200 and AMD 350X / 355X; Developing automation and AI agents for executing component-level diagnosis and remediation for failed or degraded hardware; In conjunction with data center operations develop innovative tooling and AI agents for managing the critical environment; Developing tooling for post-repair validation and testing tools such as burn-in, Pytorch, and NVIDIA NCCL to ensure system stability and performance; Own the deployment, monitoring, and operational support of developed tooling, ensuring solutions maximize GPU fleet availability and performance to drive customer success; Developing automation and operational tooling for facilities management power as well as direct liquid cooling hardware systems

Requirements

Hands-on problem solver comfortable working independently; Software engineering experience; The ability to identify a problem, rapidly develop a scalable solution and ship it; Ability to lean in and assist team members working on critical or complex technical initiatives; Ability to set the technical direction for a specific project and execute; Expertise in distributed systems, reliability, and cloud platforms (Kubernetes, IaC, GCP etc.); Strength in at least one programming language - Go, Python, Java, Rust; Strong analytical and problem-solving skills; Excellent communication and collaboration skills; Ability to work independently and within a team

Nice to have

Experience with Temporal and Kubernetes; Experience working directly with hardware vendors; Background in large-scale GPU fleet operations or hyperscale data center environments

Benefits

Industry competitive pay; Restricted Stock Units in a fast growing, well-funded technology company; Health insurance package options that include HDHP and PPO, vision, and dental for you and your dependents; Employer contributions to HSA accounts; Paid Parental Leave; Paid life insurance, short-term and long-term disability; Teladoc; 401(k) with a 100% match up to 4% of salary; Generous paid time off and holiday schedule; Cell phone reimbursement; Tuition reimbursement; Subscription to the Calm app; MetLife Legal; Company paid commuter benefit; $300 per month