Description
Big Data and Cloud Engineer (GCP)/Inside IR35/up to £700/6 months
My client a leading bank, are changing the way they govern, manage and use all of their data to allow them to reach full potential. These assets include the creation of a Data Lake for GBM and CMB, or a single virtual pool of client, transaction, product, instrument, pricing and portfolio data. Using the lake deliver solution to business requirement using Big Data as business as service
Responsible for
As a key member of the technical team, you will be expected to define and contribute at a high-level to many aspects of our collaborative agile development process:
- Expertise in software development
- Promoting development standards, code reviews, mentoring, knowledge sharing
- Adopting our DevOps culture
- Product and feature design, scrum story writing
- Data Engineering and Management
Essential experience
- Ability to build data models using Hadoop technologies.
- Programming experience in Java or Scala.
- Experience with most of the following technologies (Apache Hadoop, Kafka, Apache Spark, YARN, Hive, HBase, Apache Atlas, SQL, RESTful services).
- Experience in testing, monitoring, administering, optimizing and operating multiple Hadoop/Spark clusters across cloud providers - GCP and on premise datacenters, primarily in Python, Java and Scala
- Sound working knowledge on Unix/Linux Platform
- Experience with GCP and/or AWS is preferable and advantageous
- Knowledge of working with data is advantageous
- Hands-on experience building data pipelines using Hadoop components Hive, and Spark SQL.
- Experience with industry standard version control tools (Git, GitHub), automated deployment tools (Ansible & Jenkins) and requirements management in JIRA
- Hands on experience on cloud preferably Google Cloud platform
- Experience on Debugging the Code issues and then publishing the highlighted differences to the development team/Architects.
- Nice to have Python/Shell, ELK, Kubernetes and/GCP
Big Data and Cloud Engineer ( GCP )/Inside IR35/up to £700/6 months