07/28/2026 updated
JK
100 % available
Senior Data Engineer
Warsawa, Poland
Only remote
Master's Degree in Computer Science (Systems Design)About me
Need senior devs fast? Piligrim embeds vetted EU engineers in your workflow in 48 hours with $0 upfront fees and zero financial risk. Scale effortlessly today! ? Contact: ivankhomych@piligrim.uno
Languages & Frameworks: Python, Scala, SQL, Java, PySpark, Pandas, Flask, Ray.
Cloud Platforms & Big Data: Databricks, AWS (Glue, S3, RDS, Lambda, EMR, Step Functions, Athena), GCP (BigQuery, Vertex AI, Dataproc, Cloud Functions, Dataflow, DataStream), Azure (Data Factory, Data Lake Storage Gen2 / ADLS).
Big Data Ecosystem & Tools: Apache Spark, Spark Streaming, Apache Kafka, Hadoop, Delta Lake, dbt, Apache Airflow.
Databases & Analytics: PostgreSQL, ClickHouse, Redis, MongoDB, MySQL, DynamoDB, Oracle, MS SQL, Snowflake, Aurora, OpenSearch, Delta Table, Vector Databases, BigQuery ML, Power BI, Tableau, Grafana.
DevOps, AI & Frameworks: Docker, Kubernetes, Terraform, Git (GitHub, GitLab), CI/CD (GitHub Actions, Jenkins), Claude API, Vertex AI Search for Commerce, RAG Architectures, LangChain, LlamaIndex, Agentic AI, PyTest, JIRA, Asana.
Experience
Professional experience5 - 10 years
Freelancer since2019
Languages
EnglishFluentRussianFluentUkrainianFluent
Project history
Delivered enterprise software solutions and managed distributed cloud infrastructure components.
Built scalable backend microservices and optimized cross-functional team workflows.
Designed and optimized end-to-end ETL/ELT data pipelines utilizing Python, PySpark, Databricks, and Delta Lake.
Automated and monitored data workflows using Apache Airflow to streamline execution and minimize manual work.
Optimized Spark transformations through strategic data partitioning and chunking for accelerated processing and analytics.
Engineered distributed data pipelines with Ray to achieve scalable data processing and analytics.
Fine-tuned parallel processing mechanisms to improve overall system throughput and scalability.
Implemented PyTest for automated data quality control and continuous pipeline validation.
Certificates
Professional Cloud Database Engineer
Google2026
Professional Data Engineer Certification
Google2026
Introduction to Model Context Protocol
antropic2026