Description
As Data Engineer you will interact with the business and IT people (architects, functional analysts, project managers, offshore developers) and participate in the data transformation program.As part of this program, we are moving our enterprise data platform and Machine Learning solutions to the Microsoft Azure public cloud and in this migration, you will design and develop the optimal extraction, loading and transformation of data from a wide variety of data sources.
Next to this you will be developing data models for the Integration Layer in the Enterprise Data Hub and for the Information Delivery Layer.
You will also be working on several generic frameworks e.g. framework for logging, XML/JSON parser, etc.
Skills
- You are an IT professional with a minimum of 1 year of proven experience in data modeling, business intelligence and big data engineering.
- You have experience with the Microsoft Azure data products (ADLS, ADF, EventHub, SQL DB, SQL DW, Databricks, Azure ML, Azure Data Catalog, Logic Apps, Azure API Manager, Azure DevOps, …).
- You have experience with CI/CD.
- Experience in traditional RDBMS data warehouses, SQL and ETL tools is an advantage.
- Experience with stream-processing systems (Nifi, Spark Streaming, …) is an advantage.
- Experience with ML solutions in languages like Python, Scala and libraries like Scikit, PyTorch, TensorFlow, etc. is an advantage
- Experience with ETL tools, BI Reporting tools and data modelling (Inmon, Kimball, Data Vault, …) is an advantage.