Description
World Leading Energy Company has an exciting opportunity for a savvy Data Engineer to join this growing team of analytics experts working within the LNG Energy Trading Team of this Global Oil and Gas company.You will be responsible for expanding and optimizing data and data pipeline architecture, as well as optimizing data flow and collection for cross functional teams.
You will need to be an experienced data pipeline builder and data wrangler who enjoys optimizing data systems and building them from the ground up.
The team goal is to deliver short-, medium- and long- term forecasts to drive competitive advantage for this company's trading teams across their global LNG, voluntary carbon markets, and other emerging markets.
These markets are expected to grow significantly in the coming years and so a strong testing capability is critical to support that growth. There are two initial workstreams in this team: Voluntary Carbon
- price and demand forecasting over a 5-15 year horizon. Understanding historical trends and a credible forward-view in carbon and associated environmental products pricing today is challenging with no single version of the truth, limited analytics and data sequestered across our organisation.
A team of data scientists, data engineers, quantitative developers, business analysts and product managers will partner with the business to create a tool that provides an outlook built on a common understanding of what is needed and flexible enough to capture the dynamic nature of the market, and a set of processes, standards and methods (framework) will be developed in conjunction with all interested parties which allows a single source of truth to be created, refreshed and maintained LNG - fundamental analysis and market data
- become a trusted partner to LNG analysts by delivering accurate, timely and highly reliable data to provide clear dependable market insights.
Ingestion of data from hundreds of API and web-based data sources, curation of that data, and partnering with the business to develop forecasts and pricing models.
Each team has a business product owner, IT product manager, data scientists and/or quant devs, business analysts, data engineers, and often a Back End developer for API development.
The two teams are supported by a project manager, solution and data architects, IRM, an experience designer and others.
The Data Engineer will support software developers, database architects, data analysts and data scientists on data initiatives and will ensure optimal data delivery architecture is consistent throughout ongoing projects.
You must be self-directed and comfortable supporting the data needs of multiple teams, systems and products. The right candidate will be excited by the prospect of optimizing or even re-designing this company's data architecture to support their next generation of products and data initiatives
Educational Requirements
Graduate degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field Work experience:
Work Experience 6+ Yrs total experience in Data Engineering projects & 2+ years of relevant experience on AZURE technology services
Skill Summary
Experience with Azure: ADLS, Databricks, Stream Analytics, SQL DW, COSMOS DB, Analysis Services, Azure Functions, Serverless Architecture,
ARM Templates
Experience with relational SQL and NoSQL databases, including Postgres and Cassandra.
Experience with object-oriented/object function Scripting languages: Python, SQL, Scala, Spark-SQL etc.
Nice to have experience with big data tools: Hadoop, Spark, Kafka,etc.
Nice to have experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.
Nice to have experience with stream-processing systems: Storm
Common mandatory Core Competencies Database: SQLDB
Programming languages: PL/SQL, Spark SQL
Data Warehousing experience
This needs combining with a positive attitude and an ability to work within a large, globally dispersed project team in a multi-cultural environment. You also need to be a self-starter, a logical thinker and a quick learner, with strong initiative and excellent communication, interpersonal and presentation skills, able to write clearly and concisely. We believe in equality of opportunity for all job applicants regardless of gender, marital status, race, colour, nationality, ethnic origin, creed or religion, disability, sexual orientation or age. Specialising within Energy Trading, Oil & Gas, Financial Markets and TV & Entertainment, Eaglecliff Recruitment is ISO accredited, a Member of REC and listed within the top 4% for Financial stability by Dun & Bradstreet. Please telephone for an immediate response or email your CV for a reply within one hour. Eaglecliff Ltd is acting in the capacity of an employment agency for permanent recruitment and an employment business for contractor resourcing