05/28/2025 updated
100 % available
Senior Data Engineer with 10+ years of experience in streaming and batch data pipelines
ANTIBES, France
Only remote
MSc. in Engineering - Optimization and Operations ResearchJava (Programming Language)Apache AirflowAmazon Web ServicesAmazon S3Bash ShellBig DataGoogle BigQueryCloud ComputingDatabasesContinuous IntegrationCouchbase ServersPython (Programming Language)Amazon Simple Notification Service (SNS)Software EngineeringSQL Databases
Cloud Data Engineering
Design and implementation of streaming pipelines on GCP and AWS, with expertise in BigQuery, Kafka, SNS, SQS, and S3 for real-time data processing and analytics.
Data Pipeline Orchestration
Leveraging Kubernetes, Airflow, and Terraform to efficiently orchestrate, scale, and manage complex batch workloads and automate infrastructure deployments.
Monitoring and Alerting
Building and maintaining monitoring solutions with Datadog and CloudWatch for live tracking of both streaming and batch pipelines to ensure system reliability.
Programming Languages
Proficiency in Python, Java, SQL, and Bash for developing robust data engineering solutions and analytics applications.
Big Data Technologies
Experience with Spark, AWS EMR, and various database technologies including BigQuery, Couchbase, and dbt for large-scale data processing.
Software Engineering Practices
Application of DRY principle, pair programming, TDD, CI/CD, and Modular Design to ensure high-quality, maintainable code and seamless deployments.
Algorithm Development
Creation of optimization algorithms for network planning, scheduling, and disruption management to improve business operations and profitability.
Design and implementation of streaming pipelines on GCP and AWS, with expertise in BigQuery, Kafka, SNS, SQS, and S3 for real-time data processing and analytics.
Data Pipeline Orchestration
Leveraging Kubernetes, Airflow, and Terraform to efficiently orchestrate, scale, and manage complex batch workloads and automate infrastructure deployments.
Monitoring and Alerting
Building and maintaining monitoring solutions with Datadog and CloudWatch for live tracking of both streaming and batch pipelines to ensure system reliability.
Programming Languages
Proficiency in Python, Java, SQL, and Bash for developing robust data engineering solutions and analytics applications.
Big Data Technologies
Experience with Spark, AWS EMR, and various database technologies including BigQuery, Couchbase, and dbt for large-scale data processing.
Software Engineering Practices
Application of DRY principle, pair programming, TDD, CI/CD, and Modular Design to ensure high-quality, maintainable code and seamless deployments.
Algorithm Development
Creation of optimization algorithms for network planning, scheduling, and disruption management to improve business operations and profitability.
Experience
Professional experience10+ years
Freelancer since2025
Languages
ArabicNative speakerEnglishNative speakerFrenchNative speaker
Project history
Leading streaming pipeline design on GCP/Kafka for real-time bus events, revamping SEO/SEA marketing batch pipelines with Kubernetes/Airflow, and implementing monitoring with Datadog while applying software engineering best practices.
Contributed to implementing Network Planning and Scheduling algorithms to build profitable schedules and built batch pipelines to ingest business metrics and automate optimization analysis.
Led disruption management algorithm design, built batch pipelines for business metrics, implemented streaming pipelines for critical airline feeds using AWS services, and created CloudWatch alarms for monitoring.