09/28/2026 updated

**** ******** ****
100 % available

Data Architect | AI & Data Engineer | Data Lead | Data Consultant

London, United Kingdom
Only remote
Post Graduate in Microsoft Technologies, NIIT India
London, United Kingdom
Only remote
Post Graduate in Microsoft Technologies, NIIT India

Profile attachments

Jagan_Senior_Data_Engineer.docx

About me

Databricks & Microsoft Certified Gen AI | Solution Architect | Data Engineer | Power BI 15+ years of IT experience in Designing, Developing and implementing data solutions on Microsoft products such as Azure (AI Search, Data lake, Data Factory, SQL Synapse, Power BI, Fabric) & Databricks

Sql Data WarehouseData AnalysisMicrosoft AzureBusiness IntelligenceBusiness Intelligence DevelopmentBig DataData ArchitectureData IntegrationData MigrationData MiningData StructuresData Vault ModelingData VirtualizationDatabase DesignDatabase DevelopmentAzure Machine LearningAzure Data FactoryData LakePySpark
Azure Cloud Data Engineering
Extensive expertise in designing and developing ETL/ELT pipelines using Azure Data Factory, Azure Databricks, Azure Synapse, Data Lake Storage, Logic Apps, Data Key Vault, Event Hub, Stream Analytics, and Azure Functions for batch and real-time data processing across Medallion Architecture layers.

Data Architecture & Modelling
Advanced proficiency in data architecture including Medallion, Kappa, and Data Lake architectures, dimensional modelling (Star, Snowflake, Constellation schemas), OLTP and OLAP design, ETL architecture, and data warehousing methodologies such as Kimball, Inmon, Data Vault, and Agile Data Warehousing.

Business Intelligence & Reporting
Strong background in Business Intelligence development using SSIS, SSAS, SSRS, and Power BI, including creation of complex DAX functions, Power Pivot models, SSAS Tabular models, Dimension and Fact Tables, and end-to-end BI platform delivery.

Machine Learning & AI
Practical experience with Azure Machine Learning, Azure AI, Azure Open AI, LLM, and ChatGPT for developing and deploying AI models to automate business processes, achieving significant improvements in operational efficiency.

Data Migration & Integration
Comprehensive knowledge of SAP migration to Azure, on-premises to cloud migration, database migration across SQL Server, Oracle, and PostgreSQL, data transformation (ETL), schema conversion, database replication, and use of Azure Database Migration Service.

Master Data Management (MDM)
Solid experience with SQL MDS 2008 R2/2012, Profisee, and Data Quality Services (DQS) for master data and reference data management, data quality analysis, data cleansing, and data mining.

Programming & Query Languages
Proficiency in T-SQL, MDX, DAX, PostgreSQL SQL, PL/pgSQL, ANSI SQL, Python, PySpark, Spark SQL, U-SQL, SAS, C#, and VB.NET for data processing, analytics, and application development.

Configuration Management & DevOps
Hands-on experience with Git, GitHub, GitLab, Azure DevOps, Jenkins, PowerShell, and CI/CD pipelines for version control, continuous integration, and deployment of data and AI solutions.

Third-Party & Big Data Tools
Experience with Confluent, Snowflake, dbt, Palantir, Apache Spark, Apache Hadoop, and Hive for integrating structured, semi-structured, and unstructured data sources and enabling large-scale data processing.

CRM/ERP Systems
Working knowledge of SAP, Microsoft Dynamics CRM/D365, Salesforce, and Microsoft Dynamics AX for data integration and reporting across enterprise resource planning and customer relationship management platforms.

Experience

Professional experience10+ years
Freelancer since2014

Languages

EnglishNative speaker

Project history

Data Architect / Senior Data Engineer

Livestock Information Ltd
Design and develop ETL/ELT pipelines using Azure Data Factory to ingest, transform, and move data. Implement Medallion Architecture with Bronze, Silver, and Gold layers. Develop scalable data processing solutions using Azure Databricks (Apache Spark) for batch and real-time transformations. Implement Delta Lake for ACID-compliant data lake storage. Manage data access and security using Unity Catalog. Develop serverless Azure Functions for event-driven data processing. Implement RESTful APIs and optimize Databricks clusters for cost and performance.

Technical Lead / Senior Data Engineer

Ahold Delhaize
Collaborated with cross-functional teams to integrate AI solutions into enterprise applications. Developed and deployed AI models using Azure Machine Learning achieving 80% improvement in operational efficiency. Designed scalable data pipelines using Azure Data Factory and Azure Synapse Analytics. Implemented a robust metadata framework with Medallion Architecture. Managed cloud infrastructure using Azure DevOps with CI/CD. Implemented Star Schema on the Gold Layer with Synapse. Technical environment: Azure Machine Learning, Azure Databricks, Azure AI, Azure Synapse, Azure SQL Database, Azure Data Explorer, Azure Analysis Service, Event Hub, Azure Function, CI/CD, Power BI, Snowflake and dbt.

Senior Data Engineer

Cognizant Microsoft Business Group
Designed cloud-based data platforms using Azure Databricks, Azure SQL Database, Azure Data Lake Storage Gen 2, Azure Logic Apps, Azure Data Factory V2, and Azure Blob Storage. Built conceptual, logical, and physical data models for analytical use cases. Defined data architecture frameworks, standards, and principles. Mapped data from source to target and worked in an agile environment. Technical environment: Data Factory, ADLS Gen2, Databricks, Azure Synapse, Event Hub, Azure Function, and PySpark (Python).

Azure Consultant

Riverflex
Architected and developed a Retail Data Platform using Azure Data Lake, Azure Data Factory, Azure Data Brick, Synapse (SQL DWH), Azure Analysis Services, and Power BI. Delivered end-to-end ELT solutions, created ADF pipelines using Integration Runtime/Linked Services, developed Spark applications using PySpark and Spark-SQL, and managed SSAS Tabular models. Technical environment: Data Factory, ADLS Gen2, Databricks, Snowflake and dbt.

Big Data Consultant (AZURE)

Deloitte
Collaborated with cross-functional teams for business case development and identifying modelling methods. Created pipelines for data cleansing and ingestion from on-premise to Data Lake using ADF v2. Utilized Apache Spark with Python for Big Data Analytics and Machine Learning. Re-architected Azure HD Insight to Azure Data Bricks. Developed Power BI models for financial reporting. Technical environment: Data Factory, ADLS Gen2, Databricks, Azure Synapse, PySpark (Python), Azure Analysis Service, and Power BI.

Big Data Engineer

Shell
Big Data Engineering role at Shell.

Data Consultant

Prudential
Data Consultant role at Prudential.

Business Intelligence Consultant

Cabot Financial (Europe) Limited
Business Intelligence Consultant role at Cabot Financial (Europe) Limited.

Business Intelligence Consultant

Capita
Business Intelligence Consultant role at Capita.

Business Intelligence Developer

Skills Funding Agency
Business Intelligence Developer role at Skills Funding Agency.

Business Intelligence Developer

NHS North East London and City
Business Intelligence Developer role at NHS North East London and City (January 2013 - August 2013 and November 2013 - March 2015).

Business Intelligence Developer

South London and Maudsley NHS Foundation Trust
Business Intelligence Developer role at South London and Maudsley NHS Foundation Trust.

Business Intelligence Analyst

SOPHOS
Business Intelligence Analyst role at SOPHOS.

Business Intelligence Consultant

Aldermore Bank PLC
Business Intelligence Consultant role at Aldermore Bank PLC.

Business Intelligence Designer

Michael Page International
Business Intelligence Designer role at Michael Page International.

Data Engineer

Wipro
Data Engineer role at Wipro, India/UK.

Senior Software Engineer

Perot Systems
Senior Software Engineer role at Perot Systems, India.

Web Analyst

Scope e-Knowledge
Web Analyst role at Scope e-Knowledge, India.

Certificates

Building RAG Agents with Agent Bricks

Databricks

2026

Building Agentic Applications on Databricks

Databricks

2026

Databricks Certified Data Engineer Associate

Databricks

2025

Azure AI Engineer Associate

Microsoft

2024


Contact form

Log in to get in touch

You need to be logged in to use the contact form.

Sign upLog in