Kafka Administrator

Verifone is looking for a Data Platform Engineer to join our Platform Engineering team. This is a unique opportunity to own and evolve a broad data platform that spans streaming pipelines, multiple database technologies, on-prem and cloud infrastructure, and DevOps/SRE operational tooling. You will design and operate end-to-end data pipelines that power new AI and analytics initiatives, migrate streaming services to Kubernetes, and build the automation and tooling that reduces operational toil across the team.

Responsibilities

Design, build, and optimize end-to-end data pipelines—streaming and batch—that serve Verifone’s AI, machine learning, and analytics initiatives.; Work with diverse data technologies and formats: streaming via Kafka, batch via ETL processes, file formats such as Parquet, and table formats such as Iceberg or Delta Lake.; Collaborate with data engineering and development teams to enforce best practices for data flow, schema design, security, scalability, and performance.; Continuously improve data pipeline reliability, latency, and throughput across the full SDLC.; Support and integrate AWS data tooling including Athena, Glue, Redshift, and S3-based data lake patterns.; Manage and support Apache Kafka clusters (including AWS MSK), Kafka Connect, and KSQL for high-throughput, fault-tolerant event streaming.; Handle day-to-day Kafka operations: offset management, lag monitoring, consumer group management, cluster rebalancing, and partition tuning.; Perform cluster maintenance: upgrades, configuration changes, scaling events, and health checks.; Drive incident response and root cause analysis (RCA) for streaming platform issues.; Participate in the active effort to migrate Kafka and related services to Kubernetes.; Operate and support multiple database technologies in production: Redis, PostgreSQL and MySQL (indexing, replication, HA/DR, query tuning), and MongoDB.; Perform routine maintenance across database platforms: upgrades, configuration tuning, health monitoring, and capacity planning.; Support Snowflake data warehouse operations and integrations as part of the broader analytics stack.; Implement and uphold data security practices including encryption-in-transit/at-rest, secrets management, and least-privilege access across all data stores.; Develop and maintain automation scripts (Python, Bash) to reduce manual toil and improve platform reliability.; Design, implement, and manage CI/CD pipelines for patching, deployment, and configuration automation.; Build and maintain Infrastructure as Code (IaC) using Terraform, Ansible, or similar tools.; Containerize and orchestrate workloads using Docker and Kubernetes.; Manage cloud infrastructure on AWS: EC2, VPC, IAM, MSK, ElastiCache, CloudWatch, and related services.; Set up and maintain observability tooling—metrics, logs, and tracing—and participate in on-call incident response.; Support cloud migration tasks, moving workloads between on-prem and cloud environments.

Requirements

4+ years of overall engineering experience, with demonstrated work across data pipelines, database operations, and infrastructure—not limited to a single technology.; Hands-on experience with Kafka or another distributed streaming platform (Kafka, Kinesis, Pulsar) in a production environment.; Operational experience with at least two of the following database technologies: Redis, PostgreSQL, MySQL, MongoDB, Redshift or Snowflake.; Proven data pipeline experience: building or maintaining ETL/ELT processes, batch jobs, or streaming pipelines that serve analytics or operational use cases.; Strong scripting and automation skills (Python and/or Bash) with a track record of reducing manual toil.; CI/CD pipeline development and Infrastructure-as-Code experience (Terraform preferred).; Solid Linux fundamentals: networking basics, log analysis, system troubleshooting, process/memory/disk management.; Cloud engineering experience, preferably AWS (EC2, VPC, IAM, MSK/ElastiCache, CloudWatch).; Familiarity with observability practices—metrics, logs, tracing—and incident response workflows.; Understanding of distributed systems trade-offs: availability, consistency, partitions, backpressure.; Strong communication skills including the ability to present platform status and technical decisions to non-technical stakeholders.; Flexibility for on-call rotation and occasional weekend support.

Nice to have

Experience with stream processing frameworks: Kafka Streams, Apache Flink, or similar.; Familiarity with data lake table formats: Apache Iceberg, Delta Lake, or Apache Hudi.; Hands-on experience with AWS data services: Athena, Glue, Redshift, or Lake Formation.; Working knowledge of Snowflake—data loading, query optimization, or integration patterns.; Understanding of data quality, data lineage, or schema registry concepts.; Container/Kubernetes depth: deployments, StatefulSets, storage classes, Helm.; On-prem infrastructure experience: VMware/KVM, SAN/NAS storage, network configuration.; Security and compliance experience, particularly PCI-DSS (Payment Card Industry Data Security Standard).; Secrets management tooling: HashiCorp Vault, AWS Secrets Manager, or similar.

Benefits

Broad platform ownership—work across Kafka, Redis, MongoDB, PostgreSQL, MySQL, Snowflake, and more.; Direct impact on Verifone’s global payment infrastructure.; Ground-floor involvement in building data pipelines for AI, machine learning, and analytics strategy.; Hands-on participation in a Kafka-to-Kubernetes migration.; Mentorship from senior engineers and platform architects.; Clear career progression path from mid-level to senior and lead engineer.; Funded certification paths—Confluent, AWS, CKA, dbt—plus budget for courses and technical conferences.; Hybrid/remote working flexibility with a collaborative team culture.; Competitive compensation with performance-based incentives.