Description
World Leading Energy Company has an exciting opportunity for an experienced Data Engineer with strong SQL experience to work across an IoT project alongside a multi-disciplinary team.About the role:
Create and maintain optimal data pipeline architecture.
Assemble large, complex data sets that meet functional/non-functional business requirements.
Identify, design, and implement internal process improvements: automating manual processes, optimising data delivery, re-designing infrastructure for greater scalability, etc.
Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and Azure, AWS 'big data' technologies.
Keep our data separated and secure across national boundaries through multiple data centers and Azure, AWS regions.
Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.
Work with data and analytics experts to strive for greater functionality in our data systems.
Build analytics tools that utilise the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics.
Work with stakeholders to assist with data-related technical issues and support their data infrastructure needs.
About you:
AWS
Python
Data Pipelines
Big Data IoT
Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases
Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement
Strong analytic skills related to working with unstructured datasets
Build processes supporting data transformation, data structures, metadata, dependency and workload management
A successful history of manipulating, processing and extracting value from large disconnected datasets
Working knowledge of message queuing, stream processing, and highly scalable 'big data' data stores
Experience supporting and working with cross-functional teams in a dynamic environment
5+ years of experience in a Data Engineer role, who has attained a Graduate degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field
Experience with big data tools: Hadoop, Spark, Kafka, etc
Experience with relational SQL and NoSQL databases, including Postgres and Cassandra
Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc
Experience with Azure: ADLS, Databricks, SQL DW, Analysis Services, AWS cloud services: EC2, EMR, RDS, Redshift
Experience with stream-processing systems: Storm, Spark-Streaming, etc.
Experience with object-oriented/object function Scripting languages: Python, Java, C++, Scala, etc.
This needs combining with a positive attitude and an ability to work within a large, globally dispersed project team in a multi-cultural environment. You also need to be a self-starter, a logical thinker and a quick learner, with strong initiative and excellent communication, interpersonal and presentation skills, able to write clearly and concisely. We believe in equality of opportunity for all job applicants regardless of gender, marital status, race, colour, nationality, ethnic origin, creed or religion, disability, sexual orientation or age. Specialising within Energy Trading, Oil & Gas, Financial Markets and TV & Entertainment, Eaglecliff Recruitment is ISO accredited, a Member of REC and listed within the top 4% for Financial stability by Dun & Bradstreet. Please telephone for an immediate response or email your CV for a reply within one hour. Eaglecliff Ltd is acting in the capacity of an employment agency for permanent recruitment and an employment business for contractor resourcing