Description
Senior Big Data Engineer ( Also available: Data Scientist & Big Data Solutions Architect Roles)- Develops distributed applications to solve large scale processing problems, utilizing various languages like Java, Scala , Shell etc. .
- Implements, troubleshoots, and optimizes solutions based on modern big data technologies like Hadoop, Spark, Elastic Search, Storm, Kafka, etc. in both an on premise and cloud deployment model
- Implements data architecture, including data ingress in batch and real time from a broad variety of external systems; data transformations to prepare data for analytics processing, and data egress for availability of analytics results to visualization systems, applications, or external data stores
- Supports documentation, change control, and QA processes consistent with enterprise requirements
- Establishes strong teamwork with client technical resources, and effectively communicates project status, technical issue options and resolution, and operational requirements to client stakeholders
Skills and Expereince:
- Very strong server-side Java experience, especially in an open source, data-intensive, distributed environments.
- Expert in the Hadoop Framework & programming (Spark, MapReduce, Pig, Hive, Kafka, Storm, etc.) including performance tuning.
- Implemented complex projects dealing with the considerable data size (TB/ PB) and with high complexity
- Good understanding of algorithms, data structure, and performance optimization techniques.
- Experience with agile development methodologies like Scrum
- Self motivated, and has the ability to drive technical discussions. Organized, detail oriented, able to work both independently and in a team.
- Excellent problem solver, analytical thinker and quick learner.
- Strong verbal and written communication skills
- Broad understanding of all of the following, with depth of expertise and experience in at least 3: o Hadoop security (Kerberos, Ranger, Knox) o Amazon EMR and related technologies (e.g. DynamoDB, Kinesis, S3) o Data mining, statistical modeling techniques and quantitative analysis o Data Architecture, Master Data Management and Governance o Integration with SAP HANA o Search capabilities such as Elastic Search o NoSQL DB such as Cassandra and MongoDB
- Certifications a plus: Amazon, Cloudera, Spark
- Location: Chicago or anywhere in US willing to travel 25-30%
- Base + Bonus + Bennies
- Travel Expenses Paid