Data Engineer Big Data

• Design, develop, and maintain scalable and efficient big data processing pipelines distributed computing systems. • Collaborate with cross-functional teams to understand data requirements and design appropriate data solutions. • Implement data ingestion, processing, and transformation processes to support various analytical and machine learning use cases. • Optimize and tune data pipelines for performance, scalability, and reliability. • Monitor and troubleshoot pipeline performance issues, identifying and resolving bottlenecks. • Ensure data quality and integrity throughout the pipeline, implementing data validation and error handling mechanisms. • Stay updated on emerging technologies and best practices in big data processing and analytics, incorporating them into our data engineering practices. • Document design decisions, technical specifications, and data workflows. #LI-AG1