Description
Principal Data Engineer
Contract (Initial 6 months)
London/Remote
£525pd Inside IR35
A UK based consultancy are looking for a Principal Data Engineer to join them on an initial 6-month contract. This role is within financial services, so we are looking for Data Engineers from a banking background.
As a Principal Data Engineer, you will require the following skills and experience.
Skills:
- Contribute to Agile ALM including refinement, planning, release, and retrospective sessions
- Commits to involvement in regular performance and development conversations with their line manager and embraces a culture of in-time feedback and coaching.
- Expertise in building data solutions in the following technologies:
- Azure Data ecosystem, essentially with Azure Databricks, Data Factory, Data Lake, Event hub, Cosmos DB and Synapse
- Python, Pyspark, groovy
- Apache Nifi, Hive, Spark, hbase Hadoop HDFS and Kafka
- Ab Initio
- Metadata Management Tools eg Glue, Atlas, Data Catalogue
- Reporting Tools - Power BI, Tableau, Microstrategy
- Continuous Integration, Continuous Development, Continuous Deployment tool and DevOps model, including Git, Jira, Confluence, Jenkins.
Experience:
- 8+ years direct experience in anticipating and solving complex, large-scale data challenges
- 5+ years Hands-on with architecting and implementing solutions involving massive volumes of data, streaming data, relational and non-relational databases, orchestration frameworks, CI/CD pipelines, and distributed processing technologies
- Demonstrable experience in building re-usable frameworks, patterns, templates, and design artefacts in agile data development environments.
- 5+ years of experience in data design and development in:
- Data warehousing
- Datamodelling (3NF, star schema etc)
- Data lake development using big data technologies or cloud-native solutions
- Complex Data integration pipelines with varied types (nested XML, JSON, Cobol, EBCIDC formats) and sources of data (direct integration, API based, ftp transfer)
- Securing data solutions
- Data Management solutions (Reference data management, master data management