Data Engineers - RITM

City of Newport  ‐ Onsite
This project has been archived and is not accepting more applications.
Browse open projects on our job board.

Description

SC Data Engineer

9 month contract

Must hold SC and be lives pat 2022

Inside IR35
Key Responsibilities

Collaborating with key members of the Data Engineering team to develop automated coding solutions for a range of ETL, data cleaning, structuring and validation processes.
Working with large semi-structured datasets to construct linked datasets derived from multiple underlying internal and external sources as well as supporting the wider team in delivering a range of data profiles across key strategic administrative data flows.
Working with area leads across the broader Data Architecture Division providing ad-hoc coding support on a range of projects underway in Data Architecture utilising cross-government data;
Assisting in a range of ETL and warehousing design projects in migrating data from a number of Legacy environments;
Proving training and coaching to new members of staff across the Data Engineering team.

Person Specification

Good inter-personal and communication skills;
Self-starter eg problem solving and taking initiative to sort out issues;
A quick learner, able to assimilate the complexities of unfamiliar data sources and business data requirements;
Flexibility in being able to work on several projects at the same time.
Willingness to work to the requirements of the service/deliverables and not conditioned hours.

Skills and Experience

Essential

Extensive proven experience of data engineering and architectural techniques, including data wrangling, data profiling, data preparation, metadata development, and data upload/download;
Proven experience of 'big data' environments, including the Hadoop Stack (Cloudera), including data ingestion, processing and storage using HDFS, Spark, Hive and Impala;
Extensive hands-on experience of developing ETL functionality in a cloud or on-premise environment;
Experience of using tools such as python and SQL (in Spark) to profile, query and structure large-volume data;
Proven experience of using Cloud Services particularly in the context of Hadoop;
Experience of developing/utilising programming and query languages eg SQL (Hive Impala specifically), Python (through Spark), Scala.
SC-level clearance valid for at least 1 year on commencement of the contract. PLEASE NOTE APPLICATIONS NOT MEETING THIS CRITERIA AT THE APPLICATION STAGE WILL NOT BE CONSIDERED;
Understanding of data bases and applying data models in relational database formats.

Desirable

Experience of coaching and training others in programming and ETL techniques;
Experience of UK Government Administrative Data;

Certes Computing (and all of its subsidiary companies) is committed to promoting equality and diversity in its business operations.

Start date
n.a
From
Certes Computing Ltd
Published at
28.07.2021
Project ID:
2172803
Contract type
Freelance
To apply to this project you must log in.
Register