Description
I am looking to speak with Data Engineers urgently - Interviews next week!
The Role:
- Conceive and build data architectures
- Participate in the short/mid/long term vision of the overall system
- Execute ETL (extract/transform/load) processes from complex and/or large data sets
- Ensure data are easily accessible and that their exploitation is performing as requested, even in highly scalable circumstances
- Participate to the architecture and planning of the big data platform to optimize the ecosystem's performances
- Create large data warehouses fit for further reporting or advanced analytics
- Collaborate with machine learning engineers for the implementation and deployment of different solutions
- Ensure robust CI/CD processes are in place
Requirements:
We are looking for strong candidates with the following academic and professional experiences:
- A Master in Informatics, Engineering, Mathematics, or related field
- Demonstrable experience with big data platforms (Hadoop, Cloudera, EMR, Databricks)
- Technical knowledge in:
o Data pipeline management
o Cluster management
o Workflow management (Oozie, Airflow)
o Database management of SQL and NoSQL databases
o Large file storage (HDFS, Data Lake, S3, Blob storage,)
- Strong knowledge of Hadoop ecosystem: Hortonworks/Cloudera/EMR
- Strong knowledge of Java/Scala and Python
- Strong knowledge of Spark (Scala and Pyspark)
- Strong knowledge of CI/CD concepts
- Stream processing such as Kafka, Kinesis, Elisticsearch
- Good knowledge of a cloud environment
- High level understanding of data science concepts
- Knowledge of Data Visualisation framework like QlikSense is a plus
Please get in touch ASAP