Description
Big Data Engineer - Freelance/Contract - Belgium
Robson Bale are looking for a Big Data Engineer to come on board for an initial 3 month contract in Belgium.
3 months, market rate
Big Data Engineer
Job Description:
Individuals within the Big Data Engineer role ensure that data pipelines are scalable, repeatable, and secure, and can serve multiple users. They help facilitate getting data from a variety of different sources, getting it in the right formats, assuring that it adheres to data quality standards, and assuring that downstream users can get that data quickly. This role usually functions as a core member of an agile team.
These professionals are responsible for the infrastructure that provides insight from raw data and handles diverse sources of data seamlessly. They enable big data and batch/Real Time analytical solutions that leverage emerging technologies. Additional responsibilities include developing prototypes and proof of concepts for the selected solutions and implementing complex big data projects with a focus on collecting, parsing, managing, analysing and visualizing large sets of data using multiple platforms. They understand how to apply technologies to solve big data problems and to develop innovative big data solutions.
The Big Data Engineer is a technical job that requires substantial expertise in a broad range of software development and programming fields. These professionals have knowledge of data analysis, end user requirements analysis, and business requirements analysis to develop a clear understanding of the business needs and to incorporate these needs into technical solutions. They have a solid understanding of physical database design principles, and the system development life cycle. These individuals must work well in a team environment.
Primary Responsibilities
Competencies
Store:
- Datamodelling
- Data Architecture
- Big Data Framework/Hadoop: HDFS, Squid, Spark, Conda, Yarn and MapReduce
- NoSQL Databases: Cassandra, Hbase, MongoDB
Access & Transport - Connectivity:
- ETL (Extract, Transform, Load): Informatica
- Big Data Framework/Hadoop: Flume & Sqoop, Yarn, Zookeeper
Enrich:
- Real Time processing framework - Apache Spark
- Big Data Framework/Hadoop: PIG, Hive
- SQL and NoSQL
- Machine learning (nice to have): Python & algorithms
Provision:
- Workflow
- Programming: Java, Python (and optionally Scala)
Development methodologies:
- Agile: Safe or Spotify
- DataOps
Ancillary capabilities:
- Very strong communication skills
- Problem Solving
- Teamwork
- Innovation
3 months, market rate
Big Data Engineer - Freelance/Contract - Belgium