Data Engineer – SAS & PySpark

Job Description – Data Engineer – SAS & PySpark Project Overview We are seeking a Data Engineer to support a Data Catalogue Upgrade project, focusing on modernizing metadata, data lineage, and data governance capabilities. The role is hands-on and delivery-focused, with PySpark as the primary technology and SAS as a source system for data migration and integration. Key Responsibilities • Design, develop, and optimize PySpark-based data processing pipelines for large-scale datasets • Migrate and transform data and metadata from SAS environments into the upgraded Data Catalogue platform • Develop and maintain SQL queries for data validation, reconciliation, and quality checks • Support data lineage, metadata enrichment, and data classification as part of the catalogue upgrade • Collaborate with data architects, governance teams, and business stakeholders to ensure accurate metadata representation • Troubleshoot and resolve data processing, performance, and integration issues • Contribute to documentation, best practices, and knowledge transfer activities Required Skills & Experience • 5+ years of strong hands-on experience with PySpark (core skill) • Solid experience working with SAS (e.g., SAS datasets, procedures, or migrations) • Strong SQL skills for querying, validation, and transformation • Experience in data engineering, ETL/ELT pipelines, or data platform modernization projects • Understanding of data catalogues, metadata management, or data governance concepts Nice to Have • Experience with Data Catalogue or Metadata Management tools • Exposure to cloud-based data platforms (AWS, Azure, or GCP) • Knowledge of data lineage, data quality frameworks, or regulatory reporting • Prior experience in large-scale data migration or modernization initiatives Job Type: Full-time Pay: Up to ₹80,000.00 per month Benefits: • Health insurance • Paid sick time • Paid time off • Provident Fund • Work from home Work Location: Remote