Sr. Data Engineer

Title: Sr. Data Engineer Location: Gurugram Exp: 5+ yrs Job Description: Required Credentials: 5 - 7 Years of Experience Required Qualifications: GCP Data Stack: Expertise in BigQuery, Cloud Composer, Cloud SQL, and Dataplex. Transformation: Proficiency in Dataform, Advanced SQL, and Python. Legacy Migration: Ability to read and interpret Oozie, Pig, Hive, and proprietary ETL configurations. DevOps: Experience with GitHub Actions, Docker, and basic Terraform knowledge. Useful Qualifications: Experience with large-scale historical data migrations from on-prem Oracle/MySQL sources. Familiarity with "Silver and Gold" data modeling layers. Contractor Scope of Work Delivery Expectations: Below covers the scope of work we anticipate the contractor supporting throughout the project timeline. Scope and Solution Expectations: Pipeline Development: Design and implement GCP-based ETL/ELT pipelines using Cloud Composer, BigQuery, and Dataform to replace legacy Hadoop-stack jobs. Data Modeling: Build core data models (Silver and Gold Layers) in BigQuery for downstream analytics and reporting. Governance Implementation: Ensure data quality and governance by implementing Dataplex, Data Validation Tools (DVT), and automated quality checks. Migration Execution: Perform large-scale historical data migrations from on-prem Oracle/MySQL to BigQuery and Cloud SQL. CI/CD: Develop and implement CI/CD pipelines for all data transformation jobs using GitHub Actions.