08/30/2026 updated
Premium member
20 % partially availableSenior Databricks Consultant | PySpark | Unity Catalog | Azure & AWS
Orte, Italy
Only remote
Finance and Accounting - Banking specialization, SGH Warsaw School of Economics (2014-2016)About me
Databricks Certified Data Engineer Professional with 5 years of hands-on Databricks & PySpark experience. I build and optimize production data pipelines and specialize in Unity Catalog migrations, data quality and platform monitoring across Azure and AWS.
Data ArchitectureData IntegrationData TransformationData MigrationData Transformation ServicesData WarehousingDatabase TuningAzure Data LakeData StreamingBusiness Intelligence Development StudioCloud Platform SystemAzure Data FactoryDatabase OptimizationApache SparkMicrosoft Fabric
Databricks & Lakehouse Engineering
5 years of hands-on experience building, operating and optimizing enterprise data platforms with Databricks, Unity Catalog, PySpark, Apache Spark, Delta Lake and Delta Live Tables (DLT) across AWS and Azure environments.
Data Pipeline Development
Design and implementation of production-grade batch and streaming pipelines using PySpark, SQL, Kafka, dbt and ETL/ELT processes for financial and market data processing, with a focus on performance, reliability and maintainability.
Unity Catalog Migration
End-to-end experience in enterprise Unity Catalog migration initiatives, covering platform assessment, migration strategy, object and permission migration, dependency planning and coordination of downstream integrations.
Cloud Platform Engineering
Hands-on experience across Microsoft Azure and AWS, including ADLS, Amazon S3, Azure Data Factory and Microsoft Fabric for scalable data platform architectures.
Data Quality & Monitoring
Design and implementation of automated data-quality monitoring processes and validation controls for enterprise analytics platforms, including integration with Power BI reporting.
Python & SQL Development
Proficient use of Python and SQL for data transformation, modeling, pipeline development and performance optimization in complex production environments.
Data Modeling & Transformation
Experience in data modeling, data transformation and ETL/ELT design for large-scale financial data platforms with thousands of Databricks tables.
Engineering Practices
Application of Git, CI/CD, production operations, technical design, documentation and troubleshooting practices in enterprise data engineering projects.
Kafka-based Streaming
Experience working with Kafka-based streaming sources for near-real-time market-data workloads within large-scale financial data platforms.
5 years of hands-on experience building, operating and optimizing enterprise data platforms with Databricks, Unity Catalog, PySpark, Apache Spark, Delta Lake and Delta Live Tables (DLT) across AWS and Azure environments.
Data Pipeline Development
Design and implementation of production-grade batch and streaming pipelines using PySpark, SQL, Kafka, dbt and ETL/ELT processes for financial and market data processing, with a focus on performance, reliability and maintainability.
Unity Catalog Migration
End-to-end experience in enterprise Unity Catalog migration initiatives, covering platform assessment, migration strategy, object and permission migration, dependency planning and coordination of downstream integrations.
Cloud Platform Engineering
Hands-on experience across Microsoft Azure and AWS, including ADLS, Amazon S3, Azure Data Factory and Microsoft Fabric for scalable data platform architectures.
Data Quality & Monitoring
Design and implementation of automated data-quality monitoring processes and validation controls for enterprise analytics platforms, including integration with Power BI reporting.
Python & SQL Development
Proficient use of Python and SQL for data transformation, modeling, pipeline development and performance optimization in complex production environments.
Data Modeling & Transformation
Experience in data modeling, data transformation and ETL/ELT design for large-scale financial data platforms with thousands of Databricks tables.
Engineering Practices
Application of Git, CI/CD, production operations, technical design, documentation and troubleshooting practices in enterprise data engineering projects.
Kafka-based Streaming
Experience working with Kafka-based streaming sources for near-real-time market-data workloads within large-scale financial data platforms.
Experience
Professional experience5 - 10 years
Freelancer since2022
Languages
EnglishFluentItalianBasic knowledgePolishNative speaker
Project history
Design and build of production Databricks data pipelines using PySpark and SQL, including transformation and modeling of SAP-sourced data for financial reporting and Power BI. Development and optimization of Databricks jobs, transformations and queries with a focus on performance, reliability and maintainability. Design and implementation of monitoring jobs and automated controls for Databricks workloads. Designed an end-to-end data-quality monitoring process for Power BI reporting using Databricks and Microsoft Fabric. Designed the migration strategy and coordinated the transition of an enterprise Databricks platform to Unity Catalog. Implemented migration of Databricks objects and supported the transition of permissions and platform structures to Unity Catalog. Supported migration of downstream Power BI data sources and integrations as part of the platform transformation. Translated business and financial-reporting requirements into scalable technical solutions, documentation and platform improvements.
Selected project for a global financial institution / hedge fund. Developed, maintained and optimized a large-scale Databricks data platform comprising thousands of production tables and data workloads. Built and supported production batch and streaming pipelines in PySpark, SQL and Delta Lake for financial and market data processing. Worked with Kafka-based streaming sources for near-real-time market-data workloads. Improved and optimized existing ETL/ELT processes, transformations, queries and data models across a complex production environment. Participated in an enterprise migration from Hive Metastore to Unity Catalog. Contributed to end-to-end migration planning covering Databricks objects, permissions, dependencies and migration sequencing. Worked with dbt and Delta Live Tables (DLT) alongside Databricks-native data engineering workloads. Supported production operations, troubleshooting, technical design and continuous development of the platform.