Description
Senior Data Engineer - Azure Data Lake, Databricks, ADF, SQL, CI/CD, Spark, Python, Blob StorageSenior Data Engineer required to join an innovative insurance organisation, working within a team of data engineers that build and run the core components of an effect Azure Data Lake platform, encompassing key services such as Azure Blob Storage, Azure Data Factory, Databricks, SQL Data Warehousing and PowerBI for end-user data analytics and reporting. This role will play a senior role in the continuous utilisation, expansion, and refinement of these services, as well as the development of critical data models. The Senior Data Engineer position will encompass a mix of data engineering and BI development skills, as well as an expectation for the role holder to contribute to technical strategy development; supporting the organisation to continue its journey to develop its data platform on next-generation technology. The ideal candidate will have proven SQL/Azure SQL data warehousing exposure with solid hands-on experience with Azure Data Lake (inc. Blob Storage, Delta), Azure Data Factory and Databricks. This is a 6-month initial contract, inside IR35.
Key responsibilities:
Design, develop and hold accountability for core data engineering services, such as data orchestration and CI/CD pipelines
Design and develop models and databases both relational and dimensional to support reporting requirements
Create reporting solutions in line with standards, architectural principles and practices
Design and developer ETL processes, data lake architecture, data warehouse databases, data marts, cubes, reports and dashboards
Required skills:
Solid hands-on experience with Azure Data Lake (incl. Blob Storage, Delta) Azure Data Factory and Databricks
Proven experience of SQL/Azure SQL data warehousing
Proven experience of data engineering activities such as release management, environment controls, CICD pipeline orchestration
Experience working with Python and Spark
Proven ability of Microsoft BI software development including database/reporting development, full development cycle
Strong Datamodelling, problem solving, information analysis, attention to detail, flexibility of approach
Cornwallis Elt is an Employment Agency & Employment Business and has been listed 3 times in The Sunday Times Virgin Fast Track 100 of the UKs fastest growing private companies, as well as in the Recruitment International Top 250, Top 50 in IT and the Recruiter Fast 50 & Hot 100 reports.