01/09/2025 updated
DO
100 % available
Big Data Engineer with AWS and Cloud Expertise
london, United Kingdom
Only remote
MSc. Artificial Intelligence and Data Science, University of Hull United KingdomAlgorithmsAmazon Web ServicesAmazon Elastic Compute CloudAmazon S3Big DataCloud ComputingComputer ProgrammingDatabasesContinuous IntegrationData As A ServicesInformation EngineeringExtract Transform Load (ETL)Data VisualizationData WarehousingDevOps
Cloud Computing
Extensive experience with AWS services including S3, Glue, Lambda, CloudWatch, and EC2. Proficient in cloud resource management using Terraform.
Data Engineering
Design and implementation of end-to-end data pipelines, ETL processes, and data warehouses. Expertise in tools like Apache Spark, Redshift, and various AWS data services.
Programming
Strong programming skills in Python, SQL, and various frameworks such as Django and Flask. Experience with CI/CD pipelines and version control using Git.
Big Data Technologies
Proficiency in big data technologies including Hadoop ecosystem, Spark, and distributed databases like MongoDB.
Machine Learning
Experience in developing and deploying machine learning models, including predictive models and classification algorithms using frameworks like TensorFlow and scikit-learn.
Data Visualization
Skilled in creating data visualizations and dashboards using tools such as Tableau and AWS CloudWatch.
DevOps Practices
Familiarity with DevOps tools and practices, including Docker, Kubernetes, and Jenkins for continuous integration and deployment.
Extensive experience with AWS services including S3, Glue, Lambda, CloudWatch, and EC2. Proficient in cloud resource management using Terraform.
Data Engineering
Design and implementation of end-to-end data pipelines, ETL processes, and data warehouses. Expertise in tools like Apache Spark, Redshift, and various AWS data services.
Programming
Strong programming skills in Python, SQL, and various frameworks such as Django and Flask. Experience with CI/CD pipelines and version control using Git.
Big Data Technologies
Proficiency in big data technologies including Hadoop ecosystem, Spark, and distributed databases like MongoDB.
Machine Learning
Experience in developing and deploying machine learning models, including predictive models and classification algorithms using frameworks like TensorFlow and scikit-learn.
Data Visualization
Skilled in creating data visualizations and dashboards using tools such as Tableau and AWS CloudWatch.
DevOps Practices
Familiarity with DevOps tools and practices, including Docker, Kubernetes, and Jenkins for continuous integration and deployment.
Experience
Professional experience0 - 5 years
Freelancer since2014
Languages
EnglishNative speaker
Project history
Led end-to-end migration of legacy reporting systems to AWS, automated ETL processes, and implemented data lakes and warehouses for performance analysis.
Developed and managed AWS cloud resources, created data pipelines, and implemented various cloud functions and ETL processes using AWS services and Terraform.
Developed analytics reports using Tableau, provided consultation on data integrity, and conducted data analysis for business decision-making.