Description
DevOps Spark developer with Databricks and Azure
You will operate as a team member in a DevOps team working on different challenges. The team will set-up the data platform which will service multiple Biz Dev/Ops teams in a standard way on a common platform native cloud (Azure).
You are:
- You are an enthusiastic Azure Data engineer/developer who is passionate about all things data integration, ETL, Data Warehousing, Data Factory
- You provide feedback to your colleagues to support personal development.
- You are a multi-skilled engineer;
- You are a sparring partner for the Product Owner and contribute to the roadmap- and sprint planning by defining and estimating user stories. You report on progress and impediments on a daily basis to the team and support further development of the DevOps way of working and contribute in optimizing the IT-processes;
Your role:
- As the Azure Data Engineer, you will be working in a rapidly growing Data Integration Services/Platform team
- Designing and implementing highly performant data ingestion pipelines from multiple sources using Azure Databricks
- Developing scalable and re-usable frameworks for ingesting of data sets
- Support the planning and implementation of data platform services including sizing, configuration, and needs assessment
- Implement effective metrics and monitoring processes
Your profile:
- Experience in Agile (Scrum) way of working
- Understanding of CI/CD process & way of working
- Hands on experience on Business Intelligence development or data engineering experience
- Love for automation and passion for Cloud technologies
- Demonstrated proactive approaches to problem-solving
- Good command of spoken and written English
Your Skills:
- Extensive ETL experience;
- Databricks modelling & development
- Solid experience with Datamodelling, data warehouse design and data lake concepts and practices;
- Programming skills experience eg: Phyton, SCALA
- Direct experience of building data pipelines using Azure Data Factory and Apache Spark (preferably Databricks).
- Hands on experience designing and delivering solutions using the Azure Data Analytics platform including Azure Storage, Azure SQL Data Warehouse, Azure Data Lake, Azure Cosmos DB, Azure Stream Analytics
- Azure DevOps - CI/CD
We offer:
- A challenging job, professional teams and high engineering culture;
- An organization with a dedication to DevOps and CI/CD;
- Fantastic opportunity to keep up to date and work with the newest technologies that Microsoft have to offer
Levy Associates Limited is acting as an Employment Business in relation to this vacancy.