Description
Senior Data Engineer - AWS, Python, Spark, SQL, ETL
Global Award winning consultancy seeks a senior data engineer to deliver engineering capability on a financial services data lake project.
Responsibilities:
- Responsible for the building, deployment, and maintenance of Data Lake.
- Contributes to design, code, configurations, and documentation for components that manage data ingestion, Real Time streaming, batch processing, data extraction, transformation, and loading across multiple data storages.
- Owns one or more key components of the infrastructure and works to continually improve it, identifying gaps and improving the platform's quality, robustness, maintainability, and speed
Requirements:
- multi years of experience as Data Engineer for enterprise companies
- multi years of experience in cloud data platforms (AWS)
- Strong experience of coding in Python or Scala
- Strong experience in Spark
- Knowledge of SQL and MPP databases
- Knowledge of professional software engineering best practices for the full software
- Knowledge of Data Warehousing, design, implementation and optimization
- Knowledge of Data Quality testing, automation and results visualization
- Knowledge of BI reports and dashboards design and implementation
- Experience participating in an Agile software development team, eg SCRUM
"OUTSIDE IR35" - Immediate interview and start