Description
For a client in the Amsterdam area Global Enterprise Partners is looking for a Data Engineer starting from December'18/January'19.
Task:
You will be working on development projects related to consumer behavior, commerce, and web analytics.
Responsibilities:
- Design and implement distributed data processing pipelines using Spark, Hive, Sqoop, Python, and other tools and languages prevalent in the Hadoop ecosystem.
- Ability to design and implement end to end solution.
- Build utilities, user defined functions, and frameworks to better enable data flow patterns.
- Research, evaluate and utilize new technologies/tools/frameworks centered around Hadoop and other elements in the Big Data space.
- Define and build data acquisitions and consumption strategies
- Build and incorporate automated unit tests, participate in integration testing efforts.
- Work with teams to resolving operational & performance issues
- Work with architecture/engineering leads and other teams to ensure quality solutions are implements, and engineering best practices are defined and adhered to.
Data Engineer Skill-set:
- 6+ years' experience in large-scale software development
- 1+ year experience in Hadoop or big data technologies.
- Strong Java programming, Python, Shell Scripting, and SQL
- Strong development skills around Hadoop, Spark, Hive, and Pig
- Good understanding of file formats including JSON, Parquet, Avro, and others
- Experience with performance/scalability tuning, algorithms and computational complexity
- Ability to understand relational database schemas
- Proven ability to work cross functional teams to deliver appropriate resolution
- Experience with AWS components and services, particularly, EMR, S3, and Lambda
- Automated testing, Continuous Integration/Continuous Delivery
If your profile matches the required skill-set please reach out to me.
Lisa Isabell Stümke