Description
Job description:
Proficient understanding of distributed computing principles
Selecting and integrating any Big Data tools and frameworks required to provide requested capabilities
Good knowledge of Hadoop environment (Data engineering, data architecture, data transformation, automation, HDFS, YARN jobs, Spark/Tez/MapReduce, etc.)
Good knowledge of Big Data querying tools, such as Pig, Hive, and Impala
Experience with integration of data from multiple data sources
Experience with NoSQL databases, such as HBase, MongoDB, CouchBase, etc.
Experience with Python, Java, or Scala
Experience with BI Tools
Experience with automation tools in Big Data (Oozie, Luigi, etc.)
Knowledge of various ETL techniques and frameworks
Experience with various messaging systems, such as Kafka
Knowledge of Hadoop distributions: Cloudera, MapReduce, or Hortonworks
Experience with data analysis (data lineage, metadata, data quality and governance) and Data Modeling (conceptual, logical and physical) activities
Database development support activities to implement data models