Senior Data Engineer ( Data Collection)

We’re looking for a Senior Data Engineer in the Data Collection Ingest team to contribute to the design and coding of Similarweb’s data ingestion services & pipelines. This role involves working as part of a team that handles millions of requests per minute across multiple servers, and is responsible for a wide-range of data pipelines, processing billions of events each day.

Responsibilities

Design, code and manage end-to-end Similarweb’s data ingestion pipelines, both online and offline; Take charge of developing & maintaining modern data infrastructure, while implementing best practices for building data pipelines; Be responsible for high-scale ingestion services, solving challenges of availability, reliability, and scalability; Run the production environment by monitoring availability and taking a holistic view of system health and data quality; Own data infrastructure features from design to production using industry best practices with focus on quality and delivery; Lead design & decision-making processes of the team; Solve diverse complex problems of scale, performance and business logic; Collaborate with product managers and other team leaders to plan, nurture, and implement an efficient and effective development process; Continuously learn and evaluate new technologies in the everlasting effort to perfect our products; Perform code reviews, evaluate implementations, and provide feedback about potential improvements; Improve your skills, learn from and mentor top-notch engineers and enrich other team members; Have lots of fun

Requirements

5+ years of experience in developing code for big data infrastructure; Proficiency in technologies such as: Databricks, Spark, Airflow, Firehose, SQS, or other similar tools; Proven experience working with high scale on AWS or any other cloud provider; Experience in architecture and design of large-scale and high performance production systems; Comfortable taking challenges and learning new technologies; Excellent communication skills with the ability to provide constant dialog between teams; Ability to take business requirements and translate them to technical alternatives by performing risk management and evaluating tradeoffs

Benefits

hybrid work model; competitive compensation packages; regular team outings and happy hours; Career Week; personalized coaching; ongoing learning solutions