Description
Contract Big Data/Cloud Engineer - 20,,000 AED Per/Month (£5,000 - £8,000) - Dubai - 12 months Initially - Hadoop, Azure, Cloudera
Long term contract
Dubai
Are you looking to gain commercial exposure to an international client? An opportunity to work for one of the most prestigious organisations in the world based in Dubai has arisen for a Big Data/Cloud Engineer
A Big Data engineer builds large-scale data processing systems, is an expert in Datamodelling and processing solutions and should be able to work with the latest Big Data and NoSQL database technologies. Because we work on the cutting edge of a lot of technologies, we need someone who is a creative problem solver, resourceful in getting things done, and productive working independently or collaboratively.
Key Skills:
*Having 3+ years exposure on the Cloudera Manager, Cloudera Navigator, Hue/Hive, Scala, Spark, Shell Scripting, Apache Pig, Python and Ooziee workflows.
*Having 2+ years working experience in Kafka, Flume and Spark Streaming.
*Having exposure on ADLS, Azure Data Factory and SnapLogic.
*Good understanding in XSD, XML and JSON schemas.
*Integrating Hadoop with Oracle and NoSQL databases.
*Experience in DevOps tools (CI/CD).
*Automated workflow migration from on premise to cloud experience would be added advantage
*Use technical tools or native programming languages, to find, identify and diagnose solution defects based on data models or data extracts.
*Write advanced SQL queries to verify the integrity of databases including data transformations
Required Experience:
*Minimum 5+ years of Developing and testing experience in the Apache Hadoop framework, Azure cloud, Lambda architecture and Data Lake.
*Experience in Big Data and batch/Real Time ingestion and analytical solutions leveraging transformational technologies.
*Experience in Java, spring, Elasticsearch and Cassandra.
*Understand the source systems data, gather and able to transfer the data into HDFS as RAW and Decomposed layer (including writing Oozie workflow, coordinators, Sqoop and Flume).
*Ability to identify Unit testing.
*Experience and knowledge of a wide variety of testing methods and tools covering functional, performance, regression, system integration, smoke testing of Business Intelligence, ETL, BigData ( Ingestion, Modeling, Serving layer) and Business Analytical landscape.
We are keen to build strong relationships with IT consultants based on providing a transparent and supportive service. Now in our 18th year we pay promptly and are a stable contracting partner in good financial health. Deerfoot IT: Est. 1997. REC member. ISO certified..Each time we send a CV to a recruiting client we donate £1 to The Born Free Foundation (charity no. ). We never send your CV without your email authorisation.