03/12/2025 updated
TR
100 % available
Senior Data Engineering Architect specializing in Big Data and ETL Solutions
Krakow, Poland
Worldwide
Computer Science & Engineering, Jalpaiguri Government Engineering College (2007-2011)Java (Programming Language)Amazon Web ServicesData AnalysisApache HTTP ServerAutomationBig DataCloud ComputingComputer ProgrammingInformation EngineeringData GovernanceExtract Transform Load (ETL)Data ManagementData QualityData SystemData Visualization
Data Engineering
Extensive experience in building End to End DWH (Lakehouse) and Micro service based Streaming solutions.
Big Data Technologies
Proficiency in big data systems (MPP) and various big data tools including Spark, Kafka, and AWS services.
ETL and Data Pipeline Development
Expert in creating scalable ETL pipelines using YAML and various data processing frameworks.
Cloud Platforms
Proficient in AWS services, including AWS/GCP, AWS bedrock LLM, and cloud-based data solutions.
Data Governance and Quality
Experience in implementing data governance solutions, metadata management, and data quality monitoring.
Programming and Scripting
Skilled in multiple programming languages and frameworks including Java, Python, and SQL.
Data Visualization and Analytics
Competent in using tools like Grafana, Apache Superset, and creating data dashboards.
Container Orchestration
Proficient in Docker and Kubernetes for infrastructure automation and scaling.
Extensive experience in building End to End DWH (Lakehouse) and Micro service based Streaming solutions.
Big Data Technologies
Proficiency in big data systems (MPP) and various big data tools including Spark, Kafka, and AWS services.
ETL and Data Pipeline Development
Expert in creating scalable ETL pipelines using YAML and various data processing frameworks.
Cloud Platforms
Proficient in AWS services, including AWS/GCP, AWS bedrock LLM, and cloud-based data solutions.
Data Governance and Quality
Experience in implementing data governance solutions, metadata management, and data quality monitoring.
Programming and Scripting
Skilled in multiple programming languages and frameworks including Java, Python, and SQL.
Data Visualization and Analytics
Competent in using tools like Grafana, Apache Superset, and creating data dashboards.
Container Orchestration
Proficient in Docker and Kubernetes for infrastructure automation and scaling.
Experience
Professional experience10+ years
Freelancer since2020
Languages
EnglishNative speaker
Project history
Leading independent contractor work, creating automated ETL pipelines using YAML-based scalable Pyspark framework, and developing autonomous data engineering solutions.
Designed and implemented data solutions including sales material offering, data governance, and large migration projects for banking sector.
Designed new data landscape replacing DB2/Informatica system, created scalable Data Lakehouse solutions, and integrated IOT data for real-time monitoring.