Description
Hadoop Administrator [HDFS/Flume/Sqoop/NiFi/Java/Python/Perl/HiveQL/Oozie/SQL]Project / Role:
Our client turns ideas into innovative networking products that connect people, power businesses, and advance the way we live. They're a performance-driven, talented and diverse team that's committed to delivering world-class products for their customers.
The Hadoop System Administrator will be part of the development of the Product Analytics platform, to ingest massive amounts of data and serve up appropriate reports using the latest Big Data technologies. If you are ready for a challenge - to build the infrastructure, work closely with the business to implement their vision, get hands-on with develop, and maintain a scalable, highly-available system - then this is the role for you.
This position will work closely with the Technology Services teams to administer the Hortonworks Hadoop cluster. You'll also take requests from the Product Engineering and Product Marketing teams to create data sets as they require them.
Main Tasks:
* Define, design and develop data ingestion, management, transformation and egress capabilities using latest toolsets, such as Kafka, Storm, Pig, Hive, Sqoop, NiFi etc from diversified data platforms
* Assign schemas and create Hive tables
* Facilitate the selection of appropriate Hadoop ecosystem tools
* Document processes and technical frameworks
* Perform data clean-up and transformation in preparation for analysis
* Troubleshoot and debug Hadoop ecosystem run-time issues
* Monitor workflows and job execution using Ambari
* Assist with commission and decommission of nodes, monitor backups and recover data as required
Requirements:
* At least 2 years of experience in IT, including working with HDFS
* Familiarity with data loading tools like Flume, Sqoop, NiFi
* Ability to write MapReduce jobs (Java, Python, Perl etc)
* Experience writing Pig Latin scripts, HiveQL queries, and Linux scripts
* Knowledge of workflow schedulers like Oozie
* Working knowledge of SQL / NoSQL DBs like Mongo, Hawq, Impala etc
* Familiarity with open source configuration management and deployment tools like Puppet or Chef
* Strong communication skills for technical and non-technical audiences
* Able to work alone or as part of a team
* Action oriented
Nice to Have(s):
* Hortonworks Hadoop Platform experience preferred
Education:
* BSc in Computer Science or related field
Reference No.: 917333FK
Role: Hadoop Administrator
Industry: Technology
Location: Co. Cork
Workload: 100%
Start Date: ASAP
Job Type: Permanent