08/09/2026 updated
100 % available
Azure Data Engineer | ETL/ELT Pipelines | Cloud & AI Solutions
Jajpur, India
Only remote
B.Tech in Electronics & Telecommunication EngineeringdbtAutomatic Data Processing EquipmentApplication Programming Interfaces (APIs)Apache AirflowMicrosoft AzureCloud EngineeringData ArchitectureExtract Transform Load (ETL)Data WarehousingAzure SQLAzure Active DirectoryAzure Machine LearningAzure Data LakeAzure AutomationAzure Data Factory
Azure Cloud Data Engineering
Expertise in building cloud-based ETL/ELT pipelines using Azure Data Factory, Azure Databricks (PySpark), Azure Synapse Analytics, Azure Data Lake Gen2, Azure Blob Storage, and Azure Monitor for production pipeline monitoring.
Python & PySpark Development
Strong proficiency in Python and PySpark for data processing, pipeline development, SQL optimization, data validation, and reconciliation logic within large-scale data engineering workflows.
Data Warehousing & ETL/ELT Pipelines
Hands-on experience designing and implementing ETL/ELT workflows, data modeling, and data warehousing solutions, particularly within the US Healthcare domain.
SQL & Database Management
Proficiency in SQL for data querying, optimization, and management across relational and cloud-based database environments.
Full Stack Web Development
Experience building responsive production websites and applications using React.js, Node.js, Express.js, PHP, HTML5, CSS3, Bootstrap, and WordPress, including REST API integration.
AI & LLM Integration
Development of AI-powered applications, including a healthcare chatbot integrating Azure Databricks and Groq LLM, with implementation of token management and query optimization.
Programming Languages
Proficiency in multiple programming languages including Python, SQL, PySpark, JavaScript, PHP, and C++.
REST APIs & Integration
Experience working with REST API design, integration, and full-stack application architecture across various projects.
NoSQL & Firebase
Knowledge of MongoDB and Firebase for non-relational database management and real-time application development.
Expertise in building cloud-based ETL/ELT pipelines using Azure Data Factory, Azure Databricks (PySpark), Azure Synapse Analytics, Azure Data Lake Gen2, Azure Blob Storage, and Azure Monitor for production pipeline monitoring.
Python & PySpark Development
Strong proficiency in Python and PySpark for data processing, pipeline development, SQL optimization, data validation, and reconciliation logic within large-scale data engineering workflows.
Data Warehousing & ETL/ELT Pipelines
Hands-on experience designing and implementing ETL/ELT workflows, data modeling, and data warehousing solutions, particularly within the US Healthcare domain.
SQL & Database Management
Proficiency in SQL for data querying, optimization, and management across relational and cloud-based database environments.
Full Stack Web Development
Experience building responsive production websites and applications using React.js, Node.js, Express.js, PHP, HTML5, CSS3, Bootstrap, and WordPress, including REST API integration.
AI & LLM Integration
Development of AI-powered applications, including a healthcare chatbot integrating Azure Databricks and Groq LLM, with implementation of token management and query optimization.
Programming Languages
Proficiency in multiple programming languages including Python, SQL, PySpark, JavaScript, PHP, and C++.
REST APIs & Integration
Experience working with REST API design, integration, and full-stack application architecture across various projects.
NoSQL & Firebase
Knowledge of MongoDB and Firebase for non-relational database management and real-time application development.
Experience
Professional experience0 - 5 years
Freelancer since2025
Languages
EnglishFluentHindiFluent
Project history
Designed, developed, and maintained scalable Azure Data Factory (ADF) pipelines to ingest, transform, and load healthcare data from multiple enterprise systems.
? Built robust ETL/ELT workflows to process clinical, claims, provider, and member data, ensuring high data quality, consistency, and reliability.
⚡ Developed data transformation and processing solutions using Azure Databricks, PySpark, and SQL to support enterprise analytics and reporting.
☁️ Integrated data across Azure SQL Database, Azure Data Lake Storage (ADLS Gen2), and cloud-based analytics platforms for centralized data management.
? Optimized SQL queries, data models, and pipeline performance to improve processing efficiency and reduce execution time.
?️ Automated pipeline scheduling, monitoring, exception handling, and alerting to ensure reliable production data workflows.
? Collaborated with data architects, business analysts, and cross-functional teams to gather requirements and deliver scalable healthcare data solutions.
✅ Implemented data validation, reconciliation, and quality assurance processes to maintain accurate and trusted healthcare datasets.
? Followed healthcare data governance, security, and compliance best practices while handling sensitive healthcare information.
Developed Azure Data Factory pipelines to ingest healthcare data into Azure Data Lake Gen2. Built ETL/ELT workflows using Azure Databricks (PySpark) and Synapse Analytics. Improved data quality through validation, SQL optimization and reconciliation logic. Monitored production pipelines using Azure Monitor and collaborated with analysts and stakeholders.
Developed an AI-powered healthcare chatbot integrating Azure Databricks and Groq LLM. Implemented efficient token management and query optimization for long conversations. Built a modular Model Context Protocol (MCP) layer for structured healthcare data retrieval. Technologies used: Python, Azure Databricks, Groq LLM, MCP, REST API.