Description
Our client in the public sector is looking for a Data Engineer to join them on this long-term contract. Working as part of a multidisciplinary agile team, you will identify, build, modify and manage data flows between complex systems. These data flows must make use of open source technologies, be maintainable and be delivered in a timely fashion. Strong collaboration across disciplines is a must.
This will be a two year contract with an initial Statement of Works for up to 2 months. At this time the role has been deemed as outside of IR35, however should the features of the engagement change, a further assessment will be made which could change the determination.
The role
- You will identify, build, modify and manage data flows between complex systems using the appropriate solution.
- You will work on the design and implementation of a data streaming service, API's to expose data to our users and their systems
- You will use your Big Data knowledge to optimise the current platform to ensure it is working efficiently as possible
- Work in an agile environment following a prioritised backlog
- Manage code versioning using Git
Skills and experience
- Designs, writes and iterates code from prototype to production-ready. Understands security, accessibility and version control. Can use a range of coding tools and languages.
- Has been involved in the introduction of a data stream service based on Java, Scala or python.
- Optimises the code to ensure processes perform optimally
- Has knowledge of specific technologies Hadoop, Flume, Sqoop, Map Reduce, Hive/Impala, Hbase, Kafka, Spark Streaming, experience with ETL tools incorporating Big Data (like Informatica, Talend etc...), Shell Scripting, Python
- Has a demonstrable understanding of how to expose data from systems (for example, through APIs), link data from multiple systems and deliver streaming services.
- Data Manipulation (SQL/No-SQL)
- DevOps (GitHub, Ansible, Continuous Integration, Test Driven Deployment)
- Produces data models and understands where to use different types of data models. Able to reverse engineer a data model from a live system.
Working arrangements
5 days a week with minimum 3 days on-site in Leeds working as part of a multi-disciplinary agile delivery team with work being specified by a work management system (JIRA)
Candidates must have SC level clearance or the ability to be cleared to SC level.
Location: Leeds
Duration: Two year contract with an initial Statement of Works for up to 2 months
Rate: up to £400 per Day