Data Engineer
Factored AI
(Internet and Information Technology, 250-500 employees)
Keywords
Java (Programming Language)
Application Programming Interfaces (APIs)
Agile Methodology
Amazon Web Services
Microsoft Azure
Big Data
Cloud Computing
Continuous Integration
Extract Transform Load (ETL)
Github
Skills
Proficient in Java, Python, Go, Scala, and experienced with cloud platforms like AWS and Azure. Expertise in building data platforms, ETL pipelines, and microservices using tools such as Spark, Kafka, Hadoop, and Azure Data Factory. Skilled in Kubernetes, Docker, and CI/CD pipelines with Jenkins, GitLab, and GitHub. Strong knowledge of SQL, Mysql, MongoDB, PostgreSQL, and big data storage solutions like Snowflake and HBase. Experienced in API integrations, Agile practices, and optimizing systems for performance and cost-efficiency.
Project history
03/2023
-
09/2024
- Developed an ad-tech data platform using Go, PostgreSQL, Snowflake, AWS, and K8s, enabling quick access to audience data for decision-making across regions
- Built a new integration system using Go, GraphQL, and REST to connect the ad platform with third-party systems.
- Designed and implemented a robust REST API to efficiently expose services both internally and publicly, ensuring seamless integration and scalability.
- Implemented Docker-based local testing environments, reducing cloud costs by emulating applications locally.
- Managed Helm charts to efficiently package, deploy, and manage k8s applications, ensuring maintainability.
- Integrated multiple applications from different projects into SonarQube, ensuring adherence to coding best practices, detecting vulnerabilities, and resolving security issues across diverse workflows and teams.
- Used PagerDuty to enhance team collaboration during critical incidents, reducing downtime and improving system reliability
02/2021
-
03/2023
Data Engineer
Inmarket
(Media and Publishers, 50-250 employees)
- Designed and developed a scalable data platform using Microservices, REST APIs, Python, MySQL, and Kubernetes, enabling seamless data access and retrieval for analysis and decision-making.
- Implemented ETL processes for distributed data pipelines using Scala, Apache Spark, and AWS EMR, ensuring efficient data processing.
- Built an interactive web-based data visualization tool with Python, React, REST APIs, and MongoDB, empowering data scientists and QA teams to analyze results effectively.
- Managed CI/CD pipelines with Jenkins, streamlining software delivery and automated testing.
- Design and implement Helm charts to efficiently package, deploy, and manage Kubernetes applications, ensuring scalability and maintainability.
- Leveraged Docker for containerized deployments and utilized AWS S3/EMR for scalable cloud-based deployment and testing solutions
- Integrated various applications into SonarQube to enforce best coding practices, identify vulnerabilities, and proactively address security issues
10/2019
-
02/2021
Data Architecture
Globant
(Auditing, taxes and law, >10.000 employees)
- Designed and implemented robust ETL processes using Azure Data Factory for one of the largest consulting firms in the world.
- Integrated Azure Data Factory with Blob Storage, Microsoft SQL, and PowerBI for seamless data processing and reporting.
- Automated ETL executions by programmatically invoking Azure Data Factory from Azure Functions.
- Automated ETL processes for large unstructured files by seamlessly integrating and orchestrating Azure Databricks jobs through Azure Data Factory.
- Developed and deployed several Azure Functions in C# to streamline data processing tasks.
- Implemented Azure Service Bus trigger integration for Azure Functions Isolated worker model, enhancing event-driven architecture.
- Managed secure integrations by handling secrets in Azure Key Vaults across multiple Azure services and environments.
- Charged with keeping the Azure Pipelines and Azure Test Plans, and involved to support data integration with internal partners in a multinational professional services partnership.
- Involved in the design of plan to ensure the continuous integration and delivery (CI/CD) through efficient pipeline management and testing practices.
- Led a cross-functional management team using Scrum and SAFe practices, driving efficient project delivery.
- Conducted comprehensive code reviews, ensuring code quality and effective management of software issues.
12/2018
-
01/2020
Software Engineer
BlueTrailSoft
(Internet and Information Technology, 50-250 employees)
- Developed microservices for IoT device control using Java, MySQL, AWS, MQTT, and K8s.
- Integrated external APIs for seamless communication between SmartThings platform components.
- Improved platform flexibility and fault tolerance by designing modular microservices architecture.
- Reviewed and optimized the code base by managing and tracking software issues, performing in-depth code reviews, and enforcing best practices to ensure robust, high-quality software delivery.
- Applied TDD to build and validate scalable backend systems with automated tests.
- Configured and managed LaunchDarkly environments to support feature toggling and real-time experimentation.
Certifications
Local Availability
Only available for remote work
