DataBricks Developer (IoT sphere)

Our client is a global leader in access solutions and smart home technologies. With over a decade of innovation in the IoT space, the company is a pioneer in connected home solutions, including garage door openers, gate systems, and smart access management tools. One of their flagship innovations is the platform, which allows users to control and monitor their home access systems through connected devices. Headquartered in Oak Brook, Illinois, our client serves both residential and commercial customers worldwide. The specialist will join a team building modern cloud-based data platforms using Databricks, Python, and PySpark. The role focuses on designing and implementing scalable data pipelines based on the Medallion Architecture (Bronze, Silver, and Gold), as well as developing low-latency data services that integrate with multiple external systems through APIs.

Responsibilities

Design, build, and maintain scalable data pipelines using Databricks; Design and implement API integrations to ingest data from multiple internal and external systems; Implement and maintain data pipelines based on the Medallion Architecture (Bronze, Silver, and Gold) to support reliable and scalable analytics; Build and optimize batch and real-time data processing pipelines to meet performance, latency, and reliability requirements; Develop reusable data transformation frameworks using Python and PySpark; Optimize Spark jobs and Databricks workloads for performance and cost efficiency; Ensure data quality through validation, monitoring, and automated testing; Collaborate with architects, product owners, and engineering teams to design scalable data solutions; Troubleshoot production issues and continuously improve pipeline reliability and observability; Document data flows, architectural decisions, and operational procedures

Requirements

5+ years of experience in Data Engineering; Strong experience with Databricks; Advanced proficiency in Python, PySpark and strong SQL skills; Experience designing and implementing data pipelines based on the Medallion Architecture (Bronze, Silver, and Gold); Experience building scalable ETL/ELT data pipelines; Experience integrating data from REST APIs and third-party services; Experience developing real-time or near-real-time data pipelines; Experience optimizing Spark workloads for performance and scalability; Understanding of data quality, governance, and monitoring best practices; Strong problem-solving and communication skills, with English proficiency at B2 level or higher; Bachelor's degree in Computer Science or a related field

Nice to have

Experience with Delta Lake and Delta Live Tables (DLT); Experience with Databricks Workflows; Experience with Unity Catalog; Experience with Apache Kafka or other streaming technologies; Experience with Azure Data Factory, AWS Glue, or similar data orchestration tools; Experience implementing CI/CD processes for data pipelines; Experience working in Agile teams

Benefits

Technical and non-technical training for professional and personal growth; Internal conferences and meetups to learn from industry experts; Support and mentorship from an experienced employee to help you professional grow and development; Health insurance; Sports activities to promote a healthy lifestyle; Flexible work options, including remote and hybrid opportunities; Referral program for bringing in new talent; Work anniversary program and additional vacation days