Description
Data Engineer - 8 months - £590 per day
REMOTE WORKING CONTRACT
Inside IR35
Key skills - Azure/Pyspark/Databricks
Leading financial services client is actively recruiting for an Azure Data Engineer/Pyspark Developer concerned with using Azure Data Factory and DataBricks with the PySpark programming language to transfer transform, clean & improve raw data from source to cloud into a form that meets regulatory requirements at all stages.
Experience required for the position of Data Engineer
- Python,
- Pyspark,
- SQL
Experience and interest in Cloud platforms such as:, Azure, Databricks The ability to work across structured, semi-structured, and unstructured data, extracting information and identifying linkages across disparate data sets
Familiar with query and code performance tuning including SQL profiler and Spark optimiser
Meaningful experience in at least one database technology such as: -Distributed Processing (Spark, Hadoop) -Traditional RDBMS (MS SQL Server, Oracle, MySQL, PostgreSQL)
Understanding of Information Security principles to ensure compliant handling and management of data
Experience in traditional data warehousing/ETL tools (SSAS, SSRS)
Ability to clearly communicate complex solutions
Key skills - Azure/Pyspark/Databricks