AWS Big data Engineer

Who we are Alef Education began with a bold idea: that every learner deserves a personalised and meaningful education experience. What started in 2016 as a small pilot programme in Abu Dhabi has evolved into one of the world’s most dynamic EdTech companies—reshaping how millions of students engage with learning across the globe. Today, Alef is proudly headquartered in the UAE, working hand-in-hand with ministries of education, schools, and teachers to bring smart, data-powered platforms into classrooms in over 14,000 schools. Supporting over 1.1 million students and 50,000 teachers across the UAE, Indonesia & Morocco our AI-driven platforms generate 16+ million data points every day, helping drive smarter learning decisions. In 2024, Alef made history as the first EdTech company to list on the Abu Dhabi Securities Exchange (ADX), cementing our role as a regional innovator with global reach. As an ALEF Big Data Engineer you will have a strong understanding of big data technologies with an exceptional ability to code. You will provide technical leadership, working closely with the wider team to ensure high quality code is delivered in line with the project goals and delivery cycles. We expect familiarity with modern frameworks and languages, as well as working practices such as Clean Code, TDD, BDD, continuous integration, continuous delivery, and DevOps. Define, design, and develop services and solutions around large data ingestion, storage, and management such as withRDBMS, No SQL DBs, Log Files, Events. Define, design, and run robust data pipelines/batch jobs in a production environment. Contribute feedback to improve the quality, readability, and testability of the code base within your team. Ensure all software built is robust and scalable. Participating in sprint planning to work with developers and project teams to ensure projects are deployable and monitorable from the outside. As part of the team, you may be expected to participate in some of the 2nd line in-house support and Out-of-Hours support rotas. Degree in Computer Science, Software Engineering or related preferred Minimum of 5 years experience in a Big Data Highly Proficient in either Functional Java or Scala is a must Knowledge of Azure Big Data/Analytics services – ADLS (Azure Data Lake Storage), HDInsight, Azure Data Factory, Azure Synapse Analytics, Azure Fabric, Azure Event Hubs, Azure Stream Analytics, Azure Databricks Experience of Storing Data in systems such as Hadoop HDFS, ADLS, Event Hubs Experience of designing, setting up and running big data tech stacks such as Hadoop, Azure Databricks, Spark and distributed datastores such as Cassandra, DocumentDBs, MongoDB, Event Hubs In-depth knowledge of Hadoop technology ecosystem – HDFS, Spark, Hive, HBase, Event Hubs, Flume, Sqoop, Oozie, SPARK, Avro, Parquet Experience debugging a complex multi-server service. In depth knowledge and experience in IaaS/PaaS solutions (eg AWS Infrastructure hosting and managed services) Familiarity with network protocols - TCP/IP, SSL, etc. Knowledge of relational and non-relational database systems An ability to communicate technical concepts to a non-technical audience. Experience working knowledge of unix-like operating systems such as Linux and/or Mac OS X. Ability to quickly research and learn new programming tools and techniques.