Description
Contract: 6 months + possible extension
Rate: £400 per day inside IR35
Location: London hybrid twice per week, 3 days remote
JD:
As a Cloudera admin/AWS admin, below activities are expected to be performed,
Deploy and configure and manage Cloudera cluster on AWS EC2 instances as well as migration to new Cloudera data platform
AWS infrastructure design and set up- Design and set up VPC, configure security groups and design Proxy and implement best practices regarding Security, cost and operational efficiency.
Good to have knowledge around Hadoop services, Spark, Scala AKKA & decision manager
Key Responsibilities - list what the person will be doing on a day to day basis:
Cloudera Admin:
- Design the cluster, deciding on the number of nodes based on the estimated amount of data the cluster is going to serve.
- Deployment and upgrade of Cloudera Hadoop distribution.
- Migrating the cluster to new Cloudera cloud platform
- Configuring and tuning Cloudera services for performance and security.
- An understanding of all the Hadoop daemons along with their roles and responsibilities in the cluster.
- Should be able to troubleshoot issues in Cloudera services and fix those.
- Adding and removing nodes in the cluster.
- Rebalancing nodes in the cluster.
- Employ security using an authentication and authorization system such as Kerberos.
- Knowledge of changes required for migrating to Cloudera cloud version CDP
- Knowledge of Cloudera data services ( CDF, Atlas, CML cloudera machine learning)
AWS infra/services:
- Architecting/operating innovative solutions built on AWS
- Design and Implement AWS architectures and environments
- Play a key role in migrating and transforming Legacy solutions to the cloud
- Design Native Cloud Application Architectures and optimize applications for AWS
- Network connectivity, Direct Connect, VPN, VPC, Security group, NACL, Route 53
- 'Hands-on keyboard' AWS implementation experience across a broad range of AWS services.
- Must have in depth AWS development experience (Containerization - Glue, Docker, Amazon EKS, Lambda, EC2, S3, Amazon DocumentDB, PostgreSQL)
- Strong knowledge of DevOps and CI/CD pipeline (GitHub, Jenkins)
- Scripting capability and the ability to develop AWS environments as code
- Hands-on AWS experience with at least 1 implementation (preferred in an Enterprise scale environment)
- Experience with core AWS platform architecture, including areas such as: Organizations, Account Design, VPC, Subnet, segmentation strategies.
- Backup and Disaster Recovery approach and design
- Environment and application automation
- CloudFormation and third-party automation approach/strategy
- AWS Cost Management and Optimization
- Experienced in developing Web Services with Python, JSON
- Familiar with Encryption, Logging, and Privacy/Security Protocols (eg TLS 1.2, ELK stack)
- Good knowledge of REST/SOAP/JSON web service API implementation.
- Presentation skills with a high degree of comfort speaking with executives, IT Management, and developers
- Demonstrated ability to adapt to new technologies and learn quickly
Job Requirements
As per the 2006 Age Discrimination Act please do not specify number of years experience. Use words like Extensive, Strong, Good, Fair
Essential Skills:
- Extensive experience with Cloudera products like Cloudera distribution Hadoop (CDH) and Cloudera cloud data platform (CDP)
- Extensive experience in designing, provisioning, deploying and configurtion of the Cloudera cluster based on customer's need
- Extensive experience in AWS service - EC2, VPC, ELB, S3, RDS, Lambda, Route 53 etc and should be able to design and deploy cloudera cluster on AWS cloud
- Strong knowledge of data processing using Cloudera services
- Strong knowledge of scurity implementation using Sentry, Ranger etc
- Strong in troubleshooting and fixing issues in intergartion of Cloudera services along with AWS services
- Good in understanding of performance tuning of the cloudera services like virtual warehouses and Spark for best user experience
Nice to Have Skills.
- Good knowledge on containerisation
- Good knowledge of AWS Glue, Step Functions etc
- Good presentation skills