Description
Our client in London have a contract Senior AWS & Big data engineer role, the ideal candidate must have:
- Experience working in a cloud environment - AWS
- Hands-on experience in AWS services S3, AWS Lambda, Athena, Step functions, AWS Glue, AWS DynamoDB (AWS Serverless Technologies)
- Understanding about ETL Pipeline.
- Experience in Requirement gathering.
- Experience in Big Data Technologies - eg Hadoop, Hive, Spark etc
- Experience of MPP (Massive Parallel Processing) databases helpful - eg Teradata, Netezza
- Must know the challenges involved in Big Data - large table sizes (eg depth/width), even distribution of data
- Experience of programming- SQL, Python, Pyspark
- Datamodelling experience/awareness - Third Normal Form, Dimensional Modelling
- Data Pipelining skills - Data blending, etc
- Visualisation experience - Tableau, PBI, etc
- Data Management experience - eg Data Quality, Security, etc
- Development/Delivery methodologies - Agile, SDLC.
- Experience working in a geographically disparate team.