Senior Data Engineer (Azure) – Aspiring Data Architect

Job Details Work Location Bhilai / Indore / Remote State / Region / Province Country India Domain Engineering Interest Group Techment Technology Company IT Sector Role Summary: We are seeking a senior data engineer with strong Microsoft Azure experience who is ready to grow into a data architect role. You will design and build reliable data pipelines and lakehouse/warehouse solutions while contributing to data architecture decisions (patterns, standards, governance, and non-functional requirements). This is a hands-on role with increasing ownership of solution design and technical leadership. Key Responsibilities: • Design, build, and optimize end-to-end data ingestion, transformation, and serving pipelines on Azure (batch and streaming). • Contribute to solution architecture: define target-state data platform, reference patterns, and architecture artifacts (HLD/LLD, diagrams, and NFRs). • Implement Lakehouse/warehouse solutions using proven patterns (e.g., medallion architecture, dimensional modeling, and data vault where applicable). • Partner with stakeholders to translate business requirements into scalable data models, data products, and SLAs. • Drive data quality, observability, and reliability (validation rules, reconciliation, monitoring, alerting, and runbooks). • Implement security and governance: RBAC, managed identities, Key Vault, encryption, data access policies, and lineage. • Establish CI/CD and infrastructure-as-code practices for data workloads; enforce coding and review standards. • Optimize performance and cost across storage, compute, and orchestration. • Mentor junior engineers and lead technical discussions across engineering, DevOps, and analytics teams. Required Skills & Qualifications : Azure Data Engineering • Strong hands-on experience with Azure data services such as Azure Data Factory (ADF), Azure Synapse • Analytics, Azure Databricks, and/or Microsoft Fabric (Data Engineering). • Experience with storage and lakehouse components: ADLS Gen2, Delta Lake / Parquet, and data partitioning strategies. • Experience with relational and NoSQL stores (e.g., Azure SQL, SQL Server, Cosmos DB) and data integration patterns. • Streaming and event-driven exposure: Event Hubs / Kafka, Stream Analytics, and/or Spark Structured Streaming. Programming & Data • Strong SQL skills; ability to write efficient, maintainable transformations and tune queries. • Proficiency in Python and/or Scala (preferred) for data processing and automation. • Solid understanding of data modeling (dimensional/star schema), metadata management, and data quality concepts. Platform Engineering • Experience with Git-based development and CI/CD using Azure DevOps or GitHub Actions. • Infrastructure-as-code exposure (Terraform, Bicep/ARM) and environment promotion strategies (dev/test/prod). • Observability practices: logs, metrics, lineage, and pipeline monitoring (e.g., Azure Monitor, Log Analytics). Architecture Mindset • Ability to reason about non-functional requirements: performance, scalability, resiliency, security, and cost. • Strong communication skills to explain trade-offs and guide stakeholders toward practical solutions. Nice-to-Have: • Experience with Microsoft Purview (catalog, lineage, governance) and enterprise data security policies. • Exposure to API-based integration patterns and services such as Azure Functions, Logic Apps, and API • Management. • Experience with containerization and orchestration (Docker, AKS) for data workloads. • Familiarity with analytics/BI layers (Power BI) and semantic modeling concepts. • Cloud certifications (preferred): DP-203, AZ-305 (or equivalent). What Success Looks Like (First 90 Days) • Understand the current data landscape and propose improvements aligned to business outcomes and • platform standards. • Deliver at least one production-grade pipeline/data product with monitoring, quality checks, and • documentation. • Create or refine architecture artifacts (data flows, target architecture, integration patterns, and standards). • Establish repeatable CI/CD practices and improve reliability/performance for key workloads. Education: Bachelor’s degree in Computer Science/Engineering (or equivalent practical experience). Experience: 8–10 years