Description
Data Mining Engineer - Freelancer/Contract
Marseille, South of France - Start within 3 weeks
6 month contract - Euros 500 -€600 per day Engineer - with Development oriented Data Management Data Mining, on the basis of collection of logs and metrics on Hadoop solutions
Experience between 1 to 3 years
General context of the mission:
A set up a log centralization (infrastructure, systems, midlwre, network, applications). Faced with the influx of this data, it was decided to implement a DataLake, in addition to the existing Data WareHouse, in order to collect and centralize these raw data, in order to allow their analysis for improvement of operation, incident resolution or problem, optimization of configurations or other application developments.
The service will require great skill in abstraction, organization, rationalization to guarantee the capitalization of project assets.
Services defined below:
Ensure the development and evolution of log and data processing scripts; Ensure the proper ingestion of the collected data, their storage in the Hadoop solutions; Perform analyzes within these solutions.
Deliverables defined below:
Fill in a weekly activity report indicating the progress of the scripts on which the service provider worked; Provision of a monthly performance report; Source files and scripts saved on the git or svn directories of the appropriate solution; Documentations made, including the overall presentation of the solution.
As part of this context, the provider will be required to work on:
The field of digital and web technologies Open Source tools From design and development to compiled language: Java (Scala is a popular +) Design and development in scripted language; python (JavaScript is a popular +) Linux, Windows systems
In addition, must be able to work, as an experienced, on at least 3 of the following technologies:
Apache Nifi ElasticSearch Spark Kafka YARN HDFS Jupyter Kibana d3.js
Finally, other skills are appreciated, even as a beginner, among: security (certificates, PKI, ...) industrialization tools (SALT) containerization (docker kubernates swarm) Hadoop (stack horthonworks)