09/06/2022 updated
HT
100 % available
Senior Big Data Engineer, Data Engineer
Nuremberg, Germany
Worldwide
Azure, Kubernetes, Hive, Spark, PostgreSQL, Camel, Knative, Big Data, Hadoop, MapReduce, Presto, Pyspark, Python, Numpy, Pandas, Hbase, AWS, cloud, IOT, S3, SQS, EMR, Redshift, Docker, Git, analytics, CloudFormation, ETL tools, Talend, Alteryx, AWS Glue, App development, data management, ETL, database, SQL-Server, MySQL, OrientDB, Excel, Microsoft Excel, VBA, PHP, Outsystems, Programming, JAVA applications, VBA applications, MS-Excel, web client, visualization, MS-Access
Languages
DeutschNative speakerEnglischGood
Project history
* Preparation, consolidation, and transformation of large (un) structured
data by using modern big data technologies such as Hadoop,
MapReduce, Presto, Pyspark, Python, Numpy, Pandas, Hive, Hbase, Livy,
Jupyterlab)
* Mainly responsible for the independent design, creation, deployment, and
management of big data pipelines within the AWS cloud infrastructure
(IOT Core, Kinesis, S3, Lambda function, SQS, Glue, EMR, Athena,
Redshift, ELK, Kubernetes, Docker, Git)
* Build data lake as a centralized repository to store structured and
unstructured data for advanced analytics - data Ingestion, big data
processing, real-time analytics (S3, Glue data catalog, Athena, EMR,
Redshift, delta lake, Databricks)
* Automatic setup for EMR clusters as well as continuous performance
improvement using AWS CloudFormation
* Develop AI Model for customer-oriented projects (focusing on xgboost)
* Responsible for 15+ POCs (Customers from automotive industry) as
Senior Data Engineer
* Automation of data preparation using ETL tools (Talend, Alteryx, AWS
Glue)
* App development using low-code framework Mendix
data by using modern big data technologies such as Hadoop,
MapReduce, Presto, Pyspark, Python, Numpy, Pandas, Hive, Hbase, Livy,
Jupyterlab)
* Mainly responsible for the independent design, creation, deployment, and
management of big data pipelines within the AWS cloud infrastructure
(IOT Core, Kinesis, S3, Lambda function, SQS, Glue, EMR, Athena,
Redshift, ELK, Kubernetes, Docker, Git)
* Build data lake as a centralized repository to store structured and
unstructured data for advanced analytics - data Ingestion, big data
processing, real-time analytics (S3, Glue data catalog, Athena, EMR,
Redshift, delta lake, Databricks)
* Automatic setup for EMR clusters as well as continuous performance
improvement using AWS CloudFormation
* Develop AI Model for customer-oriented projects (focusing on xgboost)
* Responsible for 15+ POCs (Customers from automotive industry) as
Senior Data Engineer
* Automation of data preparation using ETL tools (Talend, Alteryx, AWS
Glue)
* App development using low-code framework Mendix
"process and data management" at Cynatics Consulting GmbH /
Siemens DI S CIC (Siemens external Employee)
* Automation of ETL Process using Python, Pyspark, Presto, Hive, AWS
Glue, Talend, Alteryx
* Development and maintenance of database applications based on the
tools Microsoft SQL-Server / MySQL and OrientDB
* Maintenance and further development of the internal database application
(MM-BIB)
* Development and maintenance of terminology-based data management
templates (Excel) for the creation of structured product master data using
Microsoft Excel-VBA and SQL-Server
* Development and maintenance of a web spider for downloading and for
storing product data using PHP
* App development using low-code framework Outsystems
Siemens DI S CIC (Siemens external Employee)
* Automation of ETL Process using Python, Pyspark, Presto, Hive, AWS
Glue, Talend, Alteryx
* Development and maintenance of database applications based on the
tools Microsoft SQL-Server / MySQL and OrientDB
* Maintenance and further development of the internal database application
(MM-BIB)
* Development and maintenance of terminology-based data management
templates (Excel) for the creation of structured product master data using
Microsoft Excel-VBA and SQL-Server
* Development and maintenance of a web spider for downloading and for
storing product data using PHP
* App development using low-code framework Outsystems