Description
Spark Developer (python)I have an excellent opportunity for a Spark Developer on to join a busy government department and work on an exciting project.
The Spark Developer will have the following experience:
. Proficient in PySpark and Python
. Mathematical understanding
. Hands on experience of PySpark, Python and Cloudera Hadoop stack
. Experience of using HUE and Jupyter notebook
. Team Leadership
. Experience in transforming data, constructing and testing data pipelines, implementing workflows
. Desirable:
. implementing streamsets or similar technology
As this is government, and digital, you will need to understand software delivery in an agile team, and understand continuous delivery and have excellent communication skills.
Please apply for more information