Data Engineer

About the role We are seeking an experienced Data Engineer to design, build, and maintain scalable data pipelines that support our analytics and machine‑learning initiatives. This on‑site position requires deep expertise in Python, Spark, and AWS to deliver reliable, high‑performance data solutions.

Key responsibilities

• Develop, test, and deploy end‑to‑end ETL workflows using Apache Spark, Python, and Scala.

• Design and maintain data models in Snowflake, Redshift, or similar cloud data warehouses.

• Orchestrate pipeline scheduling, monitoring, and error handling with Apache Airflow.

• Optimize SQL queries and Spark jobs for performance and cost efficiency on AWS.

• Collaborate with data scientists, analysts, and product teams to translate business requirements into technical specifications.

• Implement data quality checks, logging, and alerting to ensure pipeline reliability.

• Document architecture, processes, and best practices for knowledge sharing.

Required qualifications

• 5–10 years of professional experience as a Data Engineer.

• Proficiency in Python, SQL, and Scala for data processing.

• Hands‑on experience with Apache Spark and building large‑scale ETL pipelines.

• Strong knowledge of AWS services (S3, EMR, Glue, Redshift, IAM, etc.).

• Expertise in workflow orchestration using Apache Airflow.

• Solid understanding of data modeling concepts and dimensional design.

Nice to have

• Experience with containerization (Docker, Kubernetes) for data workloads.

• Familiarity with streaming technologies such as Kafka or Kinesis.

• Exposure to CI/CD pipelines for data engineering code.

• Knowledge of Terraform or CloudFormation for infrastructure as code.