12/03/2025 updated
100 % available
Senior Data Engineer
London, United Kingdom
Only remote
Computer Sciences EngineeringArtificial IntelligenceAmazon Web ServicesAutomationMicrosoft AzureBig DataCloud ComputingDatabasesContinuous IntegrationInformation EngineeringExtract Transform Load (ETL)Data QualityData SystemData VisualizationDevOpsDialectical Behavior Therapy
Data Pipeline Development
Expertise in building and optimizing end-to-end, high-performance data pipelines using Snowflake, dbt, SQL, and Python, including incremental models, SCD handling, and robust data quality frameworks
Cloud Data Engineering
Cloud-native data engineering professional with deep Snowflake expertise and multi-cloud experience across AWS and Azure, including security best practices, streaming pipelines, and scalable ELT/ETL architectures
DevOps and Engineering
Strong DevOps and engineering background, skilled in Git branching strategies, CI/CD pipeline automation, workflow orchestration, and monitoring/alerting for reliable deployments
Programming Languages
Proficiency in Python, Scala, and Shell Scripting for data processing and automation tasks
Database Technologies
Experience with Oracle, Cassandra, SQL Server, Elasticsearch, and Snowflake for data storage and retrieval
Big Data Frameworks
Knowledge of Prophecy AI, Hadoop, SPARK, HIVE, SQOOP, KAFKA, and dbt for large-scale data processing
Cloud Platforms
Expertise in Amazon Web Services, Google Cloud Platform, Microsoft Azure, and Databricks for cloud-based data solutions
Containerization
Experience with Docker and Kubernetes for containerized application deployment and orchestration
Data Visualization
Skills in Kibana and Datastudio for creating dashboards and data visualization solutions
Expertise in building and optimizing end-to-end, high-performance data pipelines using Snowflake, dbt, SQL, and Python, including incremental models, SCD handling, and robust data quality frameworks
Cloud Data Engineering
Cloud-native data engineering professional with deep Snowflake expertise and multi-cloud experience across AWS and Azure, including security best practices, streaming pipelines, and scalable ELT/ETL architectures
DevOps and Engineering
Strong DevOps and engineering background, skilled in Git branching strategies, CI/CD pipeline automation, workflow orchestration, and monitoring/alerting for reliable deployments
Programming Languages
Proficiency in Python, Scala, and Shell Scripting for data processing and automation tasks
Database Technologies
Experience with Oracle, Cassandra, SQL Server, Elasticsearch, and Snowflake for data storage and retrieval
Big Data Frameworks
Knowledge of Prophecy AI, Hadoop, SPARK, HIVE, SQOOP, KAFKA, and dbt for large-scale data processing
Cloud Platforms
Expertise in Amazon Web Services, Google Cloud Platform, Microsoft Azure, and Databricks for cloud-based data solutions
Containerization
Experience with Docker and Kubernetes for containerized application deployment and orchestration
Data Visualization
Skills in Kibana and Datastudio for creating dashboards and data visualization solutions
Experience
Professional experience10+ years
Freelancer since2016
Languages
EnglishNative speaker
Project history
Designed and built key risk-calculation microservices and processing engines using Python and AWS services, created Snowflake data pipelines, and established strong dbt development standards
Architected and delivered scalable ETL/ELT pipelines in Azure Data Factory, built high-performance data processing frameworks, and developed real-time data solutions
Engineered a data comparison solution using Spark, Scala, AWS Glue, and designed PySpark-based data processing pipelines achieving significant performance improvements
Certificates
Certified Databricks Professional Data Engineer
Databricks2024