Description
Lead Cloud Engineer (Data, DevOps)
Location: Remote
Rate: £600-£640PD
Duration: 6 months
IR35: Outside
We are looking for 2 Senior Data Engineers to join a growing team at a leading media company.
You will sit as part of a team of data engineers responsible for leading a data bus build project.
Role
- The Senior Cloud Engineer will lead on the design and implementation of a new enterprise message infrastructure.
- The message infrastructure will handle multiple data sources, transforms and downstream processor orchestrations. You will work closely with the Data Architect and business stakeholders to gain an understanding of the business data flows. This will feed into the technology, infrastructure and design pattern recommendations which, as the subject matter expert, you will make as you design, document and realise the system architecture.
- A representative high level system message flow is that signed in clients generate request messages which are synchronously processed by a recommendation engine before being acknowledged, as well as being asynchronously forwarded across a message infrastructure to downstream consumers which interact with external services and various persistence layers.
- You'll own the technical direction of Reach's new enterprise message infrastructure. This includes:
- Configuring the AWS network, services and security infrastructure
- Defining message exchange patterns and technologies
- Defining system resilience, delivery guarantees and message persistence
- Ensuring that the system is observable and can respond to changing load
- Ensuring regulatory and group data policy compliance
- Thought leadership in the use of an enterprise message infrastructure
- You'll confidently recommend and justify technology choices - but hold your views lightly and be prepared to work with other choices if discussion generates alternatives.
- You'll liaise with various technical and non-technical stakeholders to ensure that your work always contributes to achieving Reach's broader data strategy.
- You'll document and diagram your decisions where doing so will be useful to a broader audience.
The enterprise message infrastructure is a core part of Reach's long term digital strategy, so while we're looking for expertise in a number of areas, you will be supported throughout and will work within and across teams.
Skills
- Can extrapolate from the job requirements, the technical skills and personal traits required to excel in this role and can recognise those attributes in themselves.
- Can derive and assess business requirements, make technical choices and drive forward the realisation of the system - working with code, security policies and IaC as easily as Confluence and diagramming tools.
- Has worked with many data storage solutions and understands the strengths and role which each might play in a polyglot persistence environment.
- Has worked with many message exchange patterns and technologies.
- Understands security which extends to the public Internet and can use various authentication, encryption and policy methodologies and frameworks.
- Has the ability to work effectively remotely and resourcefully, with excellent remote management and communication skills.
- Has a broad palette of technical choices for consuming, validating, exchanging and storing data - including a thorough, in-depth knowledge of the AWS service and tooling landscape
- Has integrated systems using third party APIs
- While non-prescriptive as implementation choices, your experience will probably include:
- Kafka (Kinesis or EventBridge)
- REST web services
- Python (and typed JVM programming languages)
- DynamoDb, RDS and caches
- The Glue message schema registry
- Open authentication standards, JWT etc.