Description
Azure Cloud Data Engineer
Job Description:
We are looking for an Azure Data Engineer to join our Future Data Program. The role will be responsible for building new data pipeline architecture and optimized data flows using Azure cloud stack.
Ideal candidate is an experience data pipeline builder and data wrangler who enjoys building data products from scratch. The Data engineer will support Business Analyst and Data Architects. They must be self-directed and comfortable supporting the data needs of multiple teams, systems and products.
The right candidate will be excited by the prospect of optimizing or even re-designing our company's data architecture to support our next generation of products and data initiatives.
Required Technical and Professional Expertise
- Advance Azure Knowledge and experience working and migrating data products to on-prem to Azure.
- Experience building and optimizing big data' data pipelines, architectures and data sets using Py-Spark.
- Experience working in building Real Time data pipelines using Event-hub, storage queues and Azure stream analysis.
- Strong analytic skills related to working with unstructured datasets.
- Build processes supporting data transformation, data structures, metadata, dependency and workload management.
- Cloud Big Data Analytics in Azure Synapse Analytics, Azure Analysis Services
- Data Ingestion and Storage including Azure Data Factory, Azure Databricks, Azure Data Lake, Kafka and Spark Streaming, Azure EventHub/IoT Hub, and Azure Stream Analytics
- Experience with Bigdata Tools: Hadoop Spark, Kafka
- Experience with Stream processing systems Spark-streaming, Kafka
- Experience with Object Oriented/object function Scripting languages: Python preferred.