Description
Senior Data Engineer
Fuse Digital Consulting (A Division of the CPS Group) is building a Delivery Team to support one of our Key Partners. We are searching for an experienced Data Engineer to join our clients ever-growing business on a contract basis for a 3/6 month period with the possibility of extension.
Location: London - Remote
Rate: £450 - £480 per day
Duration: 3 - 6 months
IR35 Status: INSIDE
Overview
Sitting within Data Engineering function the person will Lead the team of data engineers to deliver data solutions (such as data ingestion, data enrichment, data marts/lakes, data automation) for key projects and data initiatives. This role should have holistic data management skills to contribute to evolve a data function in the areas of re-use & efficiency improvement, data assets building and creating knowledge hub. At the same time should be hands on to deliver re-usable data designs and products. The role will be reporting to Senior Data Engineering Lead (Head of Data Engineering) and requires interaction with all areas of the Business and close collaboration with other IT departments.
Responsibilities
*Analyse the business requirements and devise the solution approach for hybrid data solutions involving on-prem and cloud native environments.
*Accountability in one of the following areas:
oCreating and building re-usable assets in effort estimation, modular design & Datamodelling templates, data dictionary and definitions, service transition documents incorporating industry best practices. Organising the Wiki knowledge hub of the data function to host and keep the information current for the data solution.
oCreating and building re-usable patterns, solutions & technical components and ensuring adoption by the development teams to improve efficiency and standardisation
*Direct responsible for Design and Delivery of robust, high-performing, and scalable solutions for loading, transforming, and processing company data.
*Responsibility of building data solution in compliance to all aspects data managements such as Datamodelling, security, retention, quality, and master data management.
*Produce high quality release artefacts, service transition document and hand over the released product to operations team.
*Minimise, maintain & own the technical debt introduced by the project and have clear approach and plan to resolve those.
*Organise and participate the scrum of scrum with fellow principal engineers to brainstorm, agree and implement the resolution of technical debts or existing risks.
*Provide design assurance to data engineers to improve the quality of deliverables and ensure adherence to strategy and development practices
*Working collaboratively with:
othe Leads in data engineering to understanding strategy and put those into practice
othe data architect to define and building reporting
othe data platform team to ensure high level of performance and provide troubleshooting steps for delivered solutions
*Provide consultation to Project Managers, Business Analysts and Managers around data knowledge, current processes, best practices, and ideas to refine the business requirements.
*Performance management, coaching & mentoring of up to 4 team members.
Foundational skills:
*Stay abreast of trends and new capabilities in the domain of data engineering and identify opportunities to introduce those in data solutions.
*Understand and apply regulatory and compliance considerations to all data processing solutions, GDPR and PCI in particular.
*Contribute to Agile ALM including refinement, planning, release, and retrospective sessions
*Commits to involvement in regular performance and development conversations with their line manager and embraces a culture of in-time feedback and coaching.
*Expertise in building data solutions in following technologies:
oAzure Data ecosystem, essentially with Azure Databricks, Data Factory, Data Lake, Event hub, Cosmos DB and Synapse
oPython, Pyspark, groovy
oApache Nifi, Hive, Spark, hbase Hadoop HDFS and Kafka
oAb Initio
oMetadata Management Tools eg Glue, Atlas, Data Catalogue
oReporting Tools - Power BI, Tableau, Microstrategy
oContinuous Integration, Continuous Development, Continuous Deployment tool and DevOps model, including Git, Jira, Confluence, Jenkins.
Development-focus skills:
*Ability to multi-task including focussed projects as well as continuous improvement initiatives
*Exposure to micro services-based workloads in data environments
*Good understanding of tools and principles used in ALM
oAutomated software testing
oAgile Methodologies - Scrum/Kanban/XP
Interaction/Interpersonal Skills:
*Sharing knowledge and experience with Data Engineers.
*Liaise with Data Integration Engineers, Data Engineers and Data architect to improve solution design.
*Communicate changes through Change Management processes
*Timely escalate issues, delays and concerns to the line manager and relevant Business stakeholders
Experience
*Experience and ability to mentor and line manage small development team.
*Experience in Banking, Finance sector or sectors involving large data volumes
*Experience in AI Engineering
*8+ years direct experience in anticipating and solving complex, large-scale data challenges
*5+ years Hands-on with architecting and implementing solutions involving massive volumes of data, streaming data, relational and non-relational databases, orchestration frameworks, CI/CD pipelines, and distributed processing technologies
*Demonstrable experience in building re-usable frameworks, patterns, templates, and design artefacts in agile data development environments.
*5+ years of experience in data design and development in:
oData warehousing
oData Modelling (3NF, star schema etc)
oData lake development using big data technologies or cloud native solutions
oComplex Data integration pipelines with varied types (nested XML, JSON, Cobol, EBCIDC formats) and sources of data (direct integration, API based, ftp transfer)
oSecuring data solutions
oData Management solutions (Reference data management, master data management
If you feel you are right for this position or know a good Senior Data Engineer who would be interested, please send an updated CV with your application to (see below) Alternatively, please call and ask for Amber Wall for a confidential chat!
Senior Data Engineer
By applying to this advert you are giving CPS Group (UK) Ltd authority to hold and process your data for this specific role and any other roles we may deem suitable to you over time. We will not pass your data to any third party without your verbal or written permission to do so. All incoming and outgoing calls are recorded for training and compliance purposes. CPS Group (UK) Ltd is acting as an Employment Agency in relation to this vacancy. Our new privacy policy can be found on our website