Data Engineer
Rapid Innovation GmbH
(Transport and Logistics, 10-50 employees)
Keywords
Skills
About me
I am a Data Engineer with more than 15 years of experience building reliable ETL pipelines on scalable cloud infrastructure. I help organizations designing and implementing fully automated, data driven processes to support the business case using modern technology and data infrastructure.
Building data driven backends
I thrive on building new things from scratch, contributing best practices from previous projects and experience on how to design, build, deploy and operate data processing software and infrastructure.
Requirements engineering
Understanding customer needs and business requirements and translating them into infrastructure and software is one of my core competency used throughout the lifetime of every project. My focus on the essential core value proposition provides value to the customer early on.
Enabling teams
I support and enable teams to efficiently build and operate data driven processes. Sharing knowledge and best practices on designing and implementing reliable ETL processes and required infrastructure components is part of my daily routine.
Using technology
Having worked on more than 20 projects allowed me to learn and use a broad variety of tools and services to design and build data driven backend processes. This understanding helps me in choosing the right tools and technologies for the task given.
I am a Data Engineer with more than 15 years of experience building reliable ETL pipelines on scalable cloud infrastructure. I help organizations designing and implementing fully automated, data driven processes to support the business case using modern technology and data infrastructure.
Building data driven backends
I thrive on building new things from scratch, contributing best practices from previous projects and experience on how to design, build, deploy and operate data processing software and infrastructure.
Requirements engineering
Understanding customer needs and business requirements and translating them into infrastructure and software is one of my core competency used throughout the lifetime of every project. My focus on the essential core value proposition provides value to the customer early on.
Enabling teams
I support and enable teams to efficiently build and operate data driven processes. Sharing knowledge and best practices on designing and implementing reliable ETL processes and required infrastructure components is part of my daily routine.
Using technology
Having worked on more than 20 projects allowed me to learn and use a broad variety of tools and services to design and build data driven backend processes. This understanding helps me in choosing the right tools and technologies for the task given.
Project history
10/2024
-
Present
Supply chain management application
- Support in creation of master data set for a supply chain management application from multiple heterogeneous sources
- Design and implementation of ETL and data pipelines
- Analysis and optimization of joining multiple data sources to improve data quality and coverage
- Implementation data quality dashboard for business critical KPIs
- Deployment and monitoring of various data pipelines
09/2023
-
11/2024
Data Engineer
Covestro AG
(Industry and mechanical engineering, >10.000 employees)
Support and implementation services.
- Consulting services data model, deployment strategies and process design
- Design and implementation of various ETL processes to provide access to research and production data
- Stakeholder management to manage and set expectations
- Optimizing job execution and scheduling for faster data processing
- Integration of a SAP PLM system to extract and process research data from laboratories
- Workshops and knowledge transfer on cloud and data architecture, process design
07/2021
-
10/2023
Data Engineer, Data Architect
GfK SE
(Internet and Information Technology, 5000-10.000 employees)
Concept and support of an on-premise data warehouse migration to the cloud.
- Design and implementation of a cloud-based data warehouse infrastructure based on AWS Managed Services
- Help in migrating various data processing jobs from Cloudera to AWS Managed Services
- Implementation workflow management with Apache Airflow on Kubernetes
- Design and implementation of deployment pipelines for testing and rollout
- Workshops and knowledge transfer on cloud and data architecture and process design
06/2020
-
12/2022
Data Architect, Data Engineer
RTL Deutschland
(Media and Publishers, 1000-5000 employees)
Design and implementation of a cloud-based data ware house for processing user related data.
- Implemented workflows to fetch data from various third-party providers
- Building and enabling a team to create new ETL processes
- Realtime integration of a market automation software suite
- Implemented modern ETL processing environment using Airflow, Spark and Kubernetes
- General advice on data architecture and data management
01/2019
-
03/2020
Data Engineer
Volkswagen AG
(Automotive and vehicle construction, >10.000 employees)
Design and implementation of a cloud-based data warehouse for evaluation of vehicle data. Design and implementation of a data science environment.
- Extented a prototype and put into operational readiness for a production environment
- Design and implementation of a CI / CD pipeline
- Setup of project structure and release management
- Implemented various ETL pipelines for car measurement data collection, validation and transformation
08/2019
-
12/2019
Data Architect & Data Engineer
D. Swarovski KG
(Consumer goods and retail, >10.000 employees)
Design of AWS managed infrastructure platform for sensor data processing, extension of an existing data science environment.
- Advice on design and tools for building a Kubernetes based infrastructure platform for sensor data processing
- Design and implementation of infrastructure components on Kubernetes
- Design and implementation of the ETL pipeline for data collection
- Design and implementation of CI/CD pipeline with Bamboo & Kubernetes
05/2017
-
12/2018
System Architect
aixigo AG
(Banks and financial services, 50-250 employees)
Support conception and implementation/migration of a monolith into a micro service architecture.
- Alignment and coordination of different teams regarding technology usage
- Introduction of Kafka as the central message bus for micro service communications
- Introduction of LiquiBase for database schema management
- Professional / technical support for a specific micro service
Technologies: Micro Services, Java, Docker, Kafka, Liquibase, Jenkins
10/2017
-
11/2018
Data Architect, Data Engineer
D. Swarovski KG
(Consumer goods and retail, >10.000 employees)
Design and implementation of a cloud-based data warehouse & data science environment.
- Designing data warehouse architecture & data storage strategies
- Architecture proposal of a dynamically scalable data warehouse
- Implementation of infrastructure components in Terraform & Kubernetes
- Implementation of infrastructure components in Kubernetes
- Development of ETL pipelines for data collection
01/2017
-
08/2017
Data Architect, Data Engineer
GfK SE
(Internet and Information Technology, 5000-10.000 employees)
Design and implementation of a cloud-based big data warehouse in the AWS Cloud for market research analytics.
- Technical project management
- Design of an architecture based on AWS cloud infrastructure and managed services
- Implementation of ETL data pipelines
- Development of data warehouse / workflow management
- Data preparation / process management
03/2017
-
03/2017
Requirements Engineer
Open Grid Europe GmbH
(Energy, water and environment, 1000-5000 employees)
Support in evaluating big data providers.
- Acquisition and documentation of the technical requirements for setting up and operating an Apache Hadoop based data warehouse
- Obtaining offers from various providers, preparing information for decision-making
- Implementation of a prototype for data collection
06/2016
-
12/2016
Data Engineer
Helix Leisure Pte Ltd
(Consumer goods and retail, 250-500 employees)
Architecture review and design and implementation of a realtime aggregator for machine statistics.
- Review and assessment of the existing architecture and data model design
- Implementation workshop data management/Lambda architecture
- Design and implementation of a realtime layer with Spark Streaming
10/2016
-
10/2016
Data Engineer
Universitätsspital Basel
(Public service, 5000-10.000 employees)
Workshop Big Data Technologies - Introduction and Getting Started.
- Conducting a 3-day workshop
- Introduction to Big Data / Hadoop ecosystem
- Practical exercise using big data tools in the AWS Cloud
Technologies: Hadoop, Spark, AWS Cloud, MapReduce, Hive, Pig, R, Terraform
03/2011
-
09/2016
Data Architect, Software Engineer
AltusInsight GmbH
(Internet and Information Technology, < 10 employees)
Conception and development of a web application.
- Conception of the application
- Implementation of website and backend
- Set up deployment process + hosting environment
- Setting up a fully automated Apache Hadoop Deployment process in the Amazon and OpenStack Cloud
12/2015
-
08/2016
Data Architect, Data Engineer
GfK SE
(Internet and Information Technology, 5000-10.000 employees)
Design and implementation of a continuous deployment & delivery pipeline for data-driven applications in cloud environments.
- Design and implementation of a big data infrastructure in the AWS Cloud
- Design and implementation of a continuous deployment pipeline
- Technical management of an customer internal team
02/2016
-
07/2016
Data Engineer, Software Engineer
Otto GmbH & Co KG
(Consumer goods and retail, >10.000 employees)
Support in the development of ETL processes on a Hadoop based DWH.
- Planning and implementation of a hive export module
- Implementation of a Kafka & Redis export module as part of an open source project
- Implementation of an analysis algorithm for click stream analytics
Technologies: Hadoop, Hive, Spark, Redis, Kafka, Avro, Scala, HCatalog, Schedoscope
07/2015
-
10/2015
Data Engineer
RadioOpt GmbH
(Internet and Information Technology, 10-50 employees)
Conception and implementation of a data ware house based on big data technologies - OLAP workload.
- Planning and implementation of the cluster infrastructure
- Evaluation of various input formats with regard to performance
- Preparation, execution and documentation of load tests
11/2012
-
08/2015
Data Architect, Software Engineer
Gfk SE
(Internet and Information Technology, 5000-10.000 employees)
Design and implementation of a big data architecture for evaluating telecommunications data.
- Planning and implementation of the network setup
- Planning and implementation of a medium sized Hadoop cluster
- Set up deployment process, including monitoring
- Implementation of a data integration framework for high volume data storage
07/2014
-
06/2015
Data Architect, Data Engineer
Technicolor SA
(Consumer goods and retail, >10.000 employees)
Design and implementation of a big data system for batch and real-time data processing of machine generated data.
- Planning and implementation of the deployment environment
- Evaluation of various technologies for data acquisition / data processing
- Implementation of a distributed, fail-safe high throughput messaging and analysis system for machine data (Lambda Architecture)
- Technical management of a team
03/2013
-
09/2014
Data Engineer
Ubisoft / BlueByte GmbH
(Media and Publishers, >10.000 employees)
Design and implementation of Hadoop based data warehouse for online game analytics.
- Planning and implementation of a data warehouse
- Evaluation of different approaches for data collection
- Selection of suitable technologies
- Technical management / coordination of a distributed team (GER, CN, CAN)
- Implementation of a distributed, fail-safe high throughput messaging system
02/2013
-
06/2014
DevOps Engineer
Deutsche Telekom AG
(Telecommunications, >10.000 employees)
Design and implementation of a big data infrastructure in virtualized environments.
- Planning and implementation of a big data deployment infrastructure
- Implementation deployment process for Hadoop Cluster on demand in a virtualized environment
- Prototype implementation of various algorithms with the map/reduce framework
05/2012
-
12/2012
Data Engineer
exactag GmbH
(Marketing, PR and Design, 10-50 employees)
Design and implementation of a Hadoop cluster.
- Advice and conception of a Hadoop cluster
- Selection of the suitable hardware
- Set up a deployment process and roll out the cluster
- Porting of existing statistics routines to Map / Reduce
Technologies: Apache Hadoop, Hive, Pig, Python, Java, Maven, Puppet, Debian Linux
06/2011
-
03/2012
Software Engineer, Data Engineer
Etracker GmbH
(Marketing, PR and Design, 10-50 employees)
New implementation of an analysis tool as map / reduce application.
- Analysis and integration of an existing implementation in the Map / Reduce Framework with the Hadoop Streaming API
- Installation and configuration of a Hadoop cluster including monitoring
- Set up a deployment process
Technologies: Apache Hadoop / HBase, Java, Maven, Ganglia, Chef, PHP, Debian Linux
09/2010
-
02/2011
Software Engineer
Aupeo GmbH
(Internet and Information Technology, < 10 employees)
Integration of a payment provider in the existing backend.
- Data preparation, conversion and import into database
- Mapping of the data, text matching with an existing database
- Integration of a payment provider
Technologies: Ruby / Rails, OAuth, MySQL, Git, Debian Linux
05/2010
-
09/2010
Software Engineer
OpenLimit SignCubes GmbH
(Internet and Information Technology, 10-50 employees)
Integration of a signature component in an email program.
- Set up the debug environment
- Integration of signature components in KMail
- Testing the implementation
Technologies: C ++, Qt, KDE, Ubuntu Linux
03/2010
-
05/2010
Software Engineer
Etracker GmbH
(Marketing, PR and Design, 10-50 employees)
Implementation and refactoring of an analysis tool in C ++.
- Set up a build environment for C ++ projects
- Refactoring the prototype
- Adaptation and expansion of the software to the production environment (logging , error handling, unit testing )
- Set up a deployment process
- Setting up a build server (continuous integration)
Local Availability
Open to travel worldwide
I support my customers world wide, with previous engagements in Germany, Belgium, Switzerland, Canada, China, Singapore & U.S.
Other
Für Projekte in ANÜ stehe ich nicht zur Verfügung. Bitte sehen Sie von Anfragen für Projekte in ANÜ ab. Vielen Dank!
I'm not available for projects with an ANÜ contract. Please do not contact me in that case. Thank you!
