Senior Data Engineer : Big Data

Description :Job Title : Senior Data EngineerExperience : 7 to 9 YearsLocations : Bengaluru / Pune / Hyderabad / NoidaJob Summary :We are looking for an experienced Senior Data Engineer with 7 to 9 years of hands-on experience in designing, building, and optimizing large-scale data pipelines and data warehouse solutions. The ideal candidate should have strong expertise in big data technologies, cloud platforms, and modern data engineering frameworks to support advanced analytics and business intelligence initiatives.Key Responsibilities :- Design, develop, and maintain scalable data pipelines (ETL/ELT) for processing large-scale datasets (multi-terabyte scale).- Build and optimize batch and real-time data processing systems.- Develop robust data architectures to support analytics, reporting, and business intelligence.- Implement data solutions using Spark, PySpark, and SQL for high-performance data transformation.- Work with cloud-based data platforms (AWS/GCP) for scalable and secure data solutions.- Develop and manage orchestration workflows using Airflow.- Integrate streaming solutions using Kafka and Spark Structured Streaming.- Collaborate with cross-functional teams including analytics, product, and engineering stakeholders.- Ensure data quality, governance, performance optimization, and reliability.Technical Skills Required :Programming Languages :- Python- SQL- Scala- JavaBig Data Technologies :- Apache Spark- PySpark- Spark SQLStreaming Technologies :- Kafka- Spark Structured StreamingCloud Platforms :- AWS or GCP- Databricks- SnowflakeData Engineering & Orchestration :- ETL/ELT Pipeline Development- Apache AirflowGood to Have :- Experience with Java-based frameworks (Spring Boot)- Familiarity with FastAPI or Django- Exposure to data warehouse modernization initiatives (ref:hirist.tech)