Databricks Data Engineer Lead

🚀 We’re Hiring: Databricks Data Engineer – Lead Level 📍 Location: Hyderabad, India 🏢 Work Model: Hybrid (3 days/week in office – mandatory) 💼 Experience: 10–12+ Years Paves Technologies is looking for a highly experienced Databricks Data Engineer (Lead Level) to drive enterprise-scale data architecture, governance, and advanced analytics initiatives on the Azure Cloud. This leadership role requires deep expertise in Azure Databricks, PySpark, and distributed data systems, along with strong architectural and mentoring capabilities. 🔧 Role Overview The ideal candidate will have 10–12+ years of data engineering experience, including strong hands-on expertise in Azure Databricks, PySpark, Python, and Azure data services. You will lead the design and implementation of scalable data platforms, drive modernization initiatives, and implement Medallion Architecture (Bronze, Silver, Gold layers) to support enterprise analytics and business intelligence. 📌 Key Responsibilities • Lead end-to-end architecture and implementation of enterprise data platforms using Azure Databricks • Design and implement Medallion Architecture (Bronze, Silver, Gold layers) using Delta Lake best practices • Build scalable PySpark-based ETL/ELT pipelines across ingestion, transformation, and analytics layers • Develop advanced data transformations using Python, PySpark, Spark SQL, and advanced SQL • Architect robust data models (dimensional, star schema, normalized models) for analytics and reporting • Drive adoption of Databricks capabilities including Unity Catalog, Declarative Pipelines, and Delta Lake optimization • Implement performance optimization techniques such as partitioning strategies, file compaction, Z-ordering, caching, and broadcast joins • Define best practices, CI/CD frameworks, reusable templates, and infrastructure automation • Work with Azure ecosystem tools such as Azure Data Factory (ADF), ADLS, Azure DevOps, networking, and security services • Ensure data quality, RBAC, governance, lineage tracking, and production stability • Provide technical leadership and mentorship to data engineering teams ✅ Required Skills & Experience • 10–12+ years of experience in Data Engineering • 3+ years of hands-on experience with Databricks • Strong expertise in PySpark, Python, and distributed Spark processing • Experience implementing Medallion Architecture (Bronze/Silver/Gold) • Advanced knowledge of Data Modeling, Data Analysis, and complex SQL (CTEs, window functions, execution plan tuning) • Strong understanding of Delta Lake architecture, schema evolution, partitioning strategies, and performance optimization • Experience working with Azure Data platforms and cloud-native architectures • Strong leadership, stakeholder management, and communication skills 🎓 Mandatory Certifications • Databricks Certified Data Engineer – Associate • Databricks Certified Data Engineer – Professional 📩 How to Apply If you are interested and meet the required qualifications, please share your resume at: contracts@pavestechnologies.com hr@pavestechnologies.com Rakesh.k@pavestechnologies.com #Hiring #Databricks #DataEngineer #Azure #BigData #PySpark #DataEngineering #DeltaLake #CloudEngineering #HyderabadJobs #TechHiring #Analytics #DataArchitecture