09/06/2022 updated
100 % available
Azure Data Engineer
Pune, India
India
Bachelor Of TechnologyPulkit is a Microsoft Certified Azure Data Engineer and Power BI developer with more than six years of experience in Data Engineering and Business Intelligence. He has designed and implemented end-to-end data solutions in Azure. Pulkit is an expert in the Azure suite of products, Azure Data Factory, Synapse, Azure Data Lake, Databricks, SQL servers, SSIS, and Power BI
Languages
EnglishNative speaker
Project history
- Created 100+ Azure Data Factory pipelines that could extract the data from many diverse sources, transform and load it to different sink destinations, and even refresh the Power BI dashboards upon completion with automated scheduled triggers.
- Archived more than 5TB of data from multiple applications based on complex business rules to cold storage.
- Created complex SQL procedures and queries to transform, clean, and process huge data as per the business logic.
- Created advance reports and real-time dashboards using Microsoft Power BI, Dax, and Power Query.
- Created data audits and data validation to check data integrity.
- Implemented transformation logic and cleansing logic in Azure Databricks using Spark SQL in SQL context only.
- Created connections inside Azure Databricks to connect with Azure Blob storage using a secret scope and secret keys.
- Migrated legacy data workloads for more than 20 complex modules from on-premise integration packages to Azure Data Factory, removed their dependencies from on-premise infrastructure and migrated every intricacy to Azure PaaS.
- Coordinated with data scientist teams to prepare the data for data science experiments and orchestrated these experiments in Azure Data Factory pipelines in the form of ML Studio Experiment and Databricks notebook.
- Optimized the degree of parallelism and computing power for the best results in Azure Data Factory Pipelines and used complex for-loops to attain the shortest runtime possible.
- Intensively worked in Azure Data Factory, Azure, Blob Storage, Azure SQL Database, Azure Synapse, and Azure Databricks.
- Used Azure DevOps for versioning and repositories in Azure Data factory. Implemented CI/CD deployment using Azure Resource Management template for Azure Data Factory multi-environment deployment.
- Successfully extracted data from SSAS cube using Azure Data Factory. Implemented Azure SSIS integration runtime and Azure SQL database catalog to store SIIS packages.
- Created and configured integration runtimes in different environments with optimized linked services and datasets.