Description
Hi,Hope you are doing well.
I am currently looking for suitable candidate for a role “Data Engineer” for our client.
Job Location:- London, UK
Job Role:- Contract
Job Description / Skills Required
Client is upgrading the EAP Hadoop clusters (UK and Germany) to the latest version of Cloudera CDP7. This is a major upgrade that introduced some backward incompatibilities primarily impacting the Spark 1.x users. We need a specialist team of 4 resources to support the upgrade process across RFT with Spark and Flume migration and Kafka upgrade expertise / experience in such migration
1. Kafka – to support the Kafka cluster version upgrade - work with EAP team to deploy CDP as well upgrade Kafka ensuring non-functional etc.
2. Flume Re-engineering skills to do this work – flume is not available in CDP7 - What options we have ? can we use Open source Flume , Conduct PoC and Validate etc.
3. Spark 2.4 (upgrade from 1.6) – expertise to upgrade from 1.6 to 2.4 - compile , identify errors and remediate with new dependency.
Hive 3 upgrade - this a major shift from Hive 1. Syntax changes, Underlying Hive query engine has changed from MAP/REDUCE to TEZ
Must have experience in CLOUDERA UPGRADE
Must have Strong experience in Kafka OR Flume OR Spark OR Hive
Key criteria would be the consultants should have experience in Cloudera platform upgrade projects. Any version of Cloudera but version 7 is the most preferred.
Please mention
Visa Status:-
Rate Expectation ________ (Inside IR 35)
Can join________ Days
Looking forward to hearing from you soon or earliest
Anirban Deb | Delivery Team | UBIQUE Systems
Phone : | Email : | Web : www.ubique-systems.com
Offices : London | Amsterdam | Berlin | Copenhagen | Singapore | Kolkata | Bangalore