06/12/2026 updated
100 % available
Data & AI Engineer | Databricks Lakehouse | PySpark/SQL | AI-Enabled Data Platforms
Warsaw, Poland
Only remote
Master's degree in Finance and Accounting - Warsaw School of Economics (2014-2016)Data LakehouseAzure DatabricksPython (Programming Language)SQL Databases
Databricks Lakehouse & Data Engineering
Expertise in designing and implementing Databricks Lakehouse architectures, including PySpark, Spark SQL, Delta Lake, Auto Loader, ETL/ELT pipelines, and Medallion architecture for scalable, production-oriented batch and streaming solutions.
AI/LLM Engineering & AI Foundations
Proficiency in LLM-assisted engineering tools, prompt engineering, AI-assisted coding, validation, and documentation. Knowledge of RAG, embeddings, vector search, PDF/Markdown ingestion, and LLM-supported data validation.
Cloud, Governance & Orchestration
Hands-on experience with Unity Catalog, data lineage, access control, Databricks SQL, Databricks Dashboards, Azure/ADLS, AWS/S3, QuickSight, Lakeflow Jobs, Declarative Automation Bundles (DABs), Airflow, and Terraform.
CI/CD & Engineering Delivery
Application of CI/CD pipelines, code review processes, documentation standards, data quality validation, migration analysis, and stakeholder requirements management in data platform delivery.
Python & SQL Development
Strong programming skills in Python and SQL, used across data engineering, data science, and analytics projects including pipeline development, modelling, and reporting.
Azure Data Factory & Cloud Integration
Experience building and scheduling ingestion and transformation pipelines using Azure Data Factory, integrating with Databricks, Airflow, and cloud storage solutions such as AWS S3 and Azure ADLS.
Machine Learning & Data Science
Background in predictive modelling using Python, scikit-learn, Azure ML for model training and deployment, and Azure Synapse for analytics. Certified in IBM Data Science Specialization and Practical Machine Learning.
ETL/ELT & Legacy Migration
Proven track record of migrating legacy ETL pipelines and notebooks into production-ready Databricks assets, including re-engineering Exasol, IBM DataStage, and Spark SQL logic into maintainable PySpark workflows.
Business Intelligence & Reporting
Experience building KPI dashboards, data marts, and analytics layers using QuickSight, Databricks Dashboards, and Exasol, supporting business reporting across retail, e-commerce, automotive, and financial domains.
Kafka & Streaming Technologies
Familiarity with Kafka and Structured Streaming for real-time data ingestion and pipeline development within Databricks Lakehouse environments.
Expertise in designing and implementing Databricks Lakehouse architectures, including PySpark, Spark SQL, Delta Lake, Auto Loader, ETL/ELT pipelines, and Medallion architecture for scalable, production-oriented batch and streaming solutions.
AI/LLM Engineering & AI Foundations
Proficiency in LLM-assisted engineering tools, prompt engineering, AI-assisted coding, validation, and documentation. Knowledge of RAG, embeddings, vector search, PDF/Markdown ingestion, and LLM-supported data validation.
Cloud, Governance & Orchestration
Hands-on experience with Unity Catalog, data lineage, access control, Databricks SQL, Databricks Dashboards, Azure/ADLS, AWS/S3, QuickSight, Lakeflow Jobs, Declarative Automation Bundles (DABs), Airflow, and Terraform.
CI/CD & Engineering Delivery
Application of CI/CD pipelines, code review processes, documentation standards, data quality validation, migration analysis, and stakeholder requirements management in data platform delivery.
Python & SQL Development
Strong programming skills in Python and SQL, used across data engineering, data science, and analytics projects including pipeline development, modelling, and reporting.
Azure Data Factory & Cloud Integration
Experience building and scheduling ingestion and transformation pipelines using Azure Data Factory, integrating with Databricks, Airflow, and cloud storage solutions such as AWS S3 and Azure ADLS.
Machine Learning & Data Science
Background in predictive modelling using Python, scikit-learn, Azure ML for model training and deployment, and Azure Synapse for analytics. Certified in IBM Data Science Specialization and Practical Machine Learning.
ETL/ELT & Legacy Migration
Proven track record of migrating legacy ETL pipelines and notebooks into production-ready Databricks assets, including re-engineering Exasol, IBM DataStage, and Spark SQL logic into maintainable PySpark workflows.
Business Intelligence & Reporting
Experience building KPI dashboards, data marts, and analytics layers using QuickSight, Databricks Dashboards, and Exasol, supporting business reporting across retail, e-commerce, automotive, and financial domains.
Kafka & Streaming Technologies
Familiarity with Kafka and Structured Streaming for real-time data ingestion and pipeline development within Databricks Lakehouse environments.
Experience
Professional experience5 - 10 years
Freelancer since2021
Languages
GermanBasic knowledgeEnglishFluent
Project history
Architected and validated a greenfield Databricks Lakehouse design for migrating analytical data from legacy systems. Engineered a reusable metadata-driven ingestion framework for 3 source systems and 11 batch/streaming pipelines with PySpark, Auto Loader, Structured Streaming, and Delta Lake. Built a RAG assistant for business-facing project and code documentation. Implemented Lakeflow Jobs orchestration and CI/CD deployment across dev/test/prod environments with DQX-based data quality rules and Databricks Dashboards. Used LLM-supported workflows for coding, review, validation, and documentation.
Migrated 50+ organizational data assets into governed Databricks Lakehouse pipelines. Refactored 50+ legacy notebooks into production-ready Databricks assets with Unity Catalog governance, access control, lineage visibility, and bundle-based delivery workflows across dev/test/prod environments. Supported wide-scope migration analysis, documentation, and validation checks across multiple stakeholder areas. Tech: Databricks, PySpark, Unity Catalog, Declarative Automation Bundles, Azure Data Factory, CosmosDB, Kafka, Terraform.
Delivered data engineering and data science work across 3 domains: retail, e-commerce, and automotive. Built a Databricks Lakehouse consolidating sales and e-commerce data into curated analytics layers supporting 4 dashboards. Migrated 30+ Exasol tables/pipelines into a Databricks Lakehouse. Developed and scheduled 30+ repeatable ingestion and transformation pipelines using Databricks, Airflow, Azure Data Factory, and PySpark. Developed a predictive tire-wear model with Python and scikit-learn using Azure ML and Azure Synapse. Tech: Databricks, PySpark, SQL, Python, Airflow, Azure Data Factory, AWS S3, Exasol, Azure ML, Azure Synapse.