Description
Role: SC Cleared Data Engineer
Location: Newport, Gwent
IR35: Outside
Rate: Market rates
Length: 9 months
My Newport based client is looking for a Data Engineer to join them with the following skill set.
Key skills:
*Extensive proven experience of data engineering and architectural techniques, including data wrangling, data profiling, data preparation, metadata development, and data upload/download;
*Proven experience of 'big data' environments, including the Hadoop Stack (Cloudera), including data ingestion, processing and storage using HDFS, Spark, Hive and Impala;
*Extensive hands-on experience of developing ETL functionality in a cloud or on-premise environment;
*Experience of using tools such as python and SQL (in Spark) to profile, query and structure large-volume data;
*Proven experience of using Cloud Services particularly in the context of Hadoop;
*Experience of developing/utilising programming and query languages eg SQL (Hive Impala specifically), Python (through Spark), Scala.
*Understanding of data bases and applying data models.