12/06/2025 updated
RA
100 % available
Lead Data Engineer and Architect
lucknow, India
Worldwide
MCAData AnalysisMicrosoft AzureData ManagementData SystemSnowflakeProject Delivery
A results-driven Lead Data Engineer and Architect with over 9 years (9 years and 7 months) of experience specializing in end-to-end project delivery and building scalable data solutions on Azure and Snowflake. Proven ability to lead teams and architect robust data platforms for BI and Analytics. Known for designing custom frameworks that deliver dramatic improvements in performance and reliability.
Experience
Professional experience0 - 5 years
Freelancer since2025
Languages
EnglishFluent
Project history
- Platform Leadership: Guided the Core Platform Team in designing and implementing high-performance ELT processes, leveraging a distributed processing layer for raw data before utilizing Snowflake's native capabilities for final high-speed joins and serving layers (Snowpipe, Snowpark).
- Modelling & Performance: Translated advanced Data Vault and dimensional model designs into efficient Snowflake structures; led performance tuning efforts utilizing clustering, indexing, and Iceberg tables.
- Orchestration: Developed a reusable, parameterized ADF pipeline framework to manage ingestion and orchestrate the full ELT process, including centralized control for DBT transformations.
- Led the design and delivery of two new Data Warehouses, consolidating SAP and non-SAP data into a centralized Snowflake platform.
- Achievement: Reduced development time and job runtimes by over 40%, earning a 200% performance bonus and individual recognition.
- Architected a robust data modeling strategy using Data Vault and dimensional modeling (CDC & SCD) and engineered a custom security framework in Snowflake to resolve performance issues from native policy limitations.
- Integration Framework: Upgraded the Snowflake integration framework with automated testing and profiling.
- Result: Reduced manual QA work and ad-hoc requests by 70% and led to a promotion to lead the 7+ member Consumer DW team.
- ADF Framework: Designed and implemented a reusable Azure Data Factory (ADF) pipeline framework that was metadata-driven.
- Result: Reduced ADF development effort by 70% and incremental bugs by 90%, earning an individual Kudos award.
- Data Flow: Led data analysis, discovery, and data flow design phases for the Consumer Data Warehouse.