09/18/2026 updated
100 % available
Data & Analytics Engineer | PySpark | Databricks | SQL | Python | Power BI
São Paulo, Brazil
Only remote
Naval Engineering, Polytechnic School - University of Sao Paulo (USP)About me
I'm a Data & Analytics Engineer with 7+ years of experience building scalable data pipelines and platforms. I specialize in Python, SQL, PySpark and Databricks. I'm passionate about solving complex problems and using AI to make engineering faster and more efficient.
Azure DatabricksData AnalysisBusiness Intelligence DevelopmentBig DataData ArchitectureData IntegrationData TransformationData WarehousingData IntelligencePython (Programming Language)SQL Server Management StudioSQL DatabasesAzure Data FactoryApache SparkPySpark
PySpark & SQL Data Pipeline Development
Design and development of large-scale PySpark and SQL pipelines for data platform migration and modernisation initiatives, rebuilding legacy data processes into scalable Databricks architectures following Medallion Architecture principles.
Databricks & Cloud Data Engineering
Extensive hands-on experience with Databricks for building, deploying, and operating data platforms, including CI/CD support, job orchestration, and infrastructure-related activities in cloud-based environments such as Azure.
Data Modelling & Analytical Datasets
Building curated data layers with standardisation, cleansing, deduplication, enrichment, and data quality rules across ingestion and transformation workflows, delivering reliable analytical datasets and KPI dashboards using Power BI and DAX.
Python & ETL/ELT Engineering
Engineering Python data pipelines for machine learning initiatives and ETL/ELT workflows, including data quality monitoring, validation, and incremental processing practices at scale.
Azure Data Factory & Cloud Integration
Delivering data solutions using Azure Data Factory, SQL Server, and Azure-based data environments, building repeatable pipelines and standardised analytical datasets for reporting and BI use cases.
GenAI & Agentic Workflows
Application of GenAI and agentic workflows across the software and data engineering lifecycle, including coding, legacy system understanding, technical documentation, task automation, and development of reusable skills and tools.
Hive/Hadoop & Big Data Processing
Building large-scale Hive pipelines integrating high-volume data from multiple sources, applying cleansing, normalisation, deduplication, and enrichment practices for reliable analytical outputs.
CI/CD & Infrastructure Automation
Supporting CI/CD pipelines, Databricks jobs and workflows, and infrastructure-related activities using tools such as Git, Terraform, Bash, and GCP to deploy and operate data platforms efficiently.
Data Quality & Monitoring
Improving pipeline reliability and performance through validation, monitoring, troubleshooting, and optimisation of data processing workloads, ensuring data quality across ingestion and transformation layers.
Design and development of large-scale PySpark and SQL pipelines for data platform migration and modernisation initiatives, rebuilding legacy data processes into scalable Databricks architectures following Medallion Architecture principles.
Databricks & Cloud Data Engineering
Extensive hands-on experience with Databricks for building, deploying, and operating data platforms, including CI/CD support, job orchestration, and infrastructure-related activities in cloud-based environments such as Azure.
Data Modelling & Analytical Datasets
Building curated data layers with standardisation, cleansing, deduplication, enrichment, and data quality rules across ingestion and transformation workflows, delivering reliable analytical datasets and KPI dashboards using Power BI and DAX.
Python & ETL/ELT Engineering
Engineering Python data pipelines for machine learning initiatives and ETL/ELT workflows, including data quality monitoring, validation, and incremental processing practices at scale.
Azure Data Factory & Cloud Integration
Delivering data solutions using Azure Data Factory, SQL Server, and Azure-based data environments, building repeatable pipelines and standardised analytical datasets for reporting and BI use cases.
GenAI & Agentic Workflows
Application of GenAI and agentic workflows across the software and data engineering lifecycle, including coding, legacy system understanding, technical documentation, task automation, and development of reusable skills and tools.
Hive/Hadoop & Big Data Processing
Building large-scale Hive pipelines integrating high-volume data from multiple sources, applying cleansing, normalisation, deduplication, and enrichment practices for reliable analytical outputs.
CI/CD & Infrastructure Automation
Supporting CI/CD pipelines, Databricks jobs and workflows, and infrastructure-related activities using tools such as Git, Terraform, Bash, and GCP to deploy and operate data platforms efficiently.
Data Quality & Monitoring
Improving pipeline reliability and performance through validation, monitoring, troubleshooting, and optimisation of data processing workloads, ensuring data quality across ingestion and transformation layers.
Experience
Professional experience5 - 10 years
Freelancer since2026
Languages
EnglishFluentPortugueseNative speakerSpanishBasic knowledge
Project history
Design and development of PySpark and SQL pipelines for a large-scale data platform migration and modernisation initiative within the insurance industry, rebuilding legacy data processes into a scalable Databricks architecture. Building curated data layers following Medallion Architecture, applying standardisation, cleansing, deduplication, enrichment, and data quality rules. Supporting CI/CD, Databricks jobs and workflows, and applying GenAI and agentic workflows across the data engineering lifecycle.
Built large-scale PySpark and Hive pipelines for a major Brazilian bank, integrating high-volume data from multiple sources. Applied cleansing, normalisation, deduplication, enrichment, and incremental processing practices to deliver reliable analytical datasets at scale. Partnered with stakeholders to deliver validated reports and KPI dashboards. Engineered Python data pipelines for a machine learning initiative focused on client default risk.
Delivered data solutions using Azure Data Factory, Databricks, SQL Server, and Power BI. Built repeatable pipelines and standardised analytical datasets for reporting and BI use cases.