10/27/2026 updated
100 % available
Senior Data Platform Engineer | Cloud Data & Analytics Specialist
Crawley, United Kingdom
Worldwide
MSc in Computer SciencePlatform EngineeringMicrosoft AzureDevOpsScrum MethodologyCloud ServicesAzure Data LakeMicrosoft FabricPySparkTerraformDatabricks
Microsoft Azure & Cloud Data Engineering
Extensive expertise in Microsoft Azure services including Azure Databricks, Data Factory, Azure SQL DB, Synapse, ADLS, Blob Storage, IAM, VNet, and VMs/VMSS. Proven track record of designing and implementing scalable, high-performance cloud data solutions in Financial Services.
Databricks & Data Lakehouse Architecture
Deep hands-on experience with Databricks, including Unity Catalog, Delta Lake, Autoloader, Spark structured streaming, and Databricks Asset Bundles (DABs). Skilled in building and optimising medallion (lakehouse) architectures and ETL pipelines for incremental and near real-time data processing.
CI/CD, DevOps & Infrastructure as Code
Strong proficiency in Azure DevOps (CI/CD, Azure Pipelines, Azure Boards, Azure Artifacts), GitHub Actions, Terraform, and Docker. Demonstrated ability to automate CI/CD pipelines, implement IaC, and enforce best practices around IAM, governance, and software delivery efficiency.
SQL & Python Development
Solid command of SQL and Python for data engineering tasks, including developing metadata-driven frameworks, fixing critical bugs, and optimising PySpark code to significantly reduce ETL pipeline runtimes.
Microsoft Fabric
Hands-on experience with Microsoft Fabric, including leading the first client project delivery using Fabric, migrating data from SQL Server to Fabric via a medallion architecture, and advising on Lakehouse and Data Warehouse architectures within Fabric.
Data Modelling & Data Warehouse Architecture
Proficiency in logical data modelling, Lakehouse/Data Warehouse architecture design, and implementing best practices for data integrity, accessibility, and governance across data estates.
Scripting (PowerShell, Bash)
Competence in scripting with PowerShell and Bash to automate operational tasks and support data platform management.
Apache Airflow & AWS
Familiarity with Apache Airflow for workflow orchestration and AWS cloud services as part of a broad cloud and data engineering skill set.
Agile Methodologies & Scrum
Certified ScrumMaster (CSM) with practical experience supporting sprint ceremonies, backlog refinement, daily standups, and retrospectives, contributing to smooth Agile project delivery.
Testing & Code Quality
Experience conducting thorough code reviews and ensuring high code quality through rigorous testing with Pytest and tSQLt, resulting in more stable and reliable data frameworks.
Extensive expertise in Microsoft Azure services including Azure Databricks, Data Factory, Azure SQL DB, Synapse, ADLS, Blob Storage, IAM, VNet, and VMs/VMSS. Proven track record of designing and implementing scalable, high-performance cloud data solutions in Financial Services.
Databricks & Data Lakehouse Architecture
Deep hands-on experience with Databricks, including Unity Catalog, Delta Lake, Autoloader, Spark structured streaming, and Databricks Asset Bundles (DABs). Skilled in building and optimising medallion (lakehouse) architectures and ETL pipelines for incremental and near real-time data processing.
CI/CD, DevOps & Infrastructure as Code
Strong proficiency in Azure DevOps (CI/CD, Azure Pipelines, Azure Boards, Azure Artifacts), GitHub Actions, Terraform, and Docker. Demonstrated ability to automate CI/CD pipelines, implement IaC, and enforce best practices around IAM, governance, and software delivery efficiency.
SQL & Python Development
Solid command of SQL and Python for data engineering tasks, including developing metadata-driven frameworks, fixing critical bugs, and optimising PySpark code to significantly reduce ETL pipeline runtimes.
Microsoft Fabric
Hands-on experience with Microsoft Fabric, including leading the first client project delivery using Fabric, migrating data from SQL Server to Fabric via a medallion architecture, and advising on Lakehouse and Data Warehouse architectures within Fabric.
Data Modelling & Data Warehouse Architecture
Proficiency in logical data modelling, Lakehouse/Data Warehouse architecture design, and implementing best practices for data integrity, accessibility, and governance across data estates.
Scripting (PowerShell, Bash)
Competence in scripting with PowerShell and Bash to automate operational tasks and support data platform management.
Apache Airflow & AWS
Familiarity with Apache Airflow for workflow orchestration and AWS cloud services as part of a broad cloud and data engineering skill set.
Agile Methodologies & Scrum
Certified ScrumMaster (CSM) with practical experience supporting sprint ceremonies, backlog refinement, daily standups, and retrospectives, contributing to smooth Agile project delivery.
Testing & Code Quality
Experience conducting thorough code reviews and ensuring high code quality through rigorous testing with Pytest and tSQLt, resulting in more stable and reliable data frameworks.
Experience
Professional experience5 - 10 years
Freelancer since2026
Languages
EnglishNative speaker
Project history
Implemented best practices around IAM, IaC, DevOps, and Governance across the Data Platform. Collaborated with Senior Leaders to gather requirements for optimisations delivering business value. Implemented workflow jobs to export Machine Learning model predictions to the internal CRM. Automated CI/CD pipelines to enhance software delivery efficiency and reduce deployment times. Acted as the first point of contact for Data Platform issues.
Enhanced and maintained a Databricks metadata-driven framework using Terraform, Python, SQL, Azure Databricks, and Azure Data Factory in a CI/CD pipeline. Conducted code reviews and ensured high code quality with Pytest and tSQLt. Facilitated sprint reviews and playback sessions with clients. Mentored junior data engineers and client developers. Collaborated with Principal Engineers on RAID logs and risk management. Guided client developers on tools including Git, Terraform, CI/CD, Pytest, Spark, GitFlow, Databricks, ADF, and Fabric. Led Databricks and Fabric workstreams including ETL migration, medallion architecture implementation, and data ingestion from APIs, Azure FTP Servers, SMTP, and Azure Blob Storage.
Constructed ETL pipelines using Azure Synapse for handling petabytes of data. Supported integration of legacy on-premises SSIS jobs to Azure. Assured smooth operation of on-premises and Azure pipeline jobs. Assisted with Proof of Concepts for data engineering tasks. Utilised Azure DevOps and Git for version control. Performed logical data modelling following best practices. Leveraged Power BI to produce comprehensive reports and visualisations.