Description
My client is a leading player in the energy sector, driving the energy transition forward with innovative solutions. Central to this mission is data-large volumes of it-and they need a robust, scalable, and powerful data platform to harness it. As part of the Datalake Platform Team, you will play a pivotal role in developing, maintaining, and making this platform accessible to the entire organization.As a Senior Datalake Platform Engineer, you will work with a cutting-edge Azure-based cloud infrastructure, supporting various teams in building self-service data pipelines. You will contribute to both the operational aspects of the platform and the strategic direction for its future development. Your expertise will enable seamless data management, empowering the teams to leverage data for analytics, data science, and decision-making.
Your Responsibilities:
- Develop and maintain the Azure-based dataplatform using tools such as Databricks, Kubernetes, and Airflow.
- Provide infrastructure-as-code (Terraform) solutions to ensure scalability and robustness.
- Support teams in building self-service data pipelines by offering an integrated platform with multiple tools and technologies.
- Troubleshoot operational issues and improve the efficiency and performance of the Datalake.
- Contribute to the strategic roadmap for the Datalake, shaping short-, medium-, and long-term plans.
Requirments:
- 5 years of experience in setting up and managing (Azure) cloud environments, with expertise in Azure Databricks, Kubernetes, ADLS, Airflow, and EventHub.
- Strong knowledge of Python, Spark, SQL, and a foundational understanding of networking.
- Hands-on experience with infrastructure-as-code and CICD processes, particularly using Terraform and Azure DevOps pipelines.
- Excellent communication skills, with the ability to share knowledge effectively in both Dutch and English.
Darwin Recruitment is acting as an Employment Business in relation to this vacancy.