Keywords
Information Engineering
Selenium
Application Programming Interfaces (APIs)
Web Scraping
Stock Control
Docker
Amazon Web Services
Business Requirements
Ubuntu (Operating System)
Cluster Analysis
Skills
Hi, I am a Data Engineer by profession. I am very passionate about Data Engineering. Extracting, Transforming, and Loading the data is what I love to do. Creating pipelines for getting data from Data Sources to Loading it to the warehouse is what I am attracted towards.
My skill sets are Web Scraping, Data Extracting from APIs, Modeling data using Kimball's approach, and Transforming data.
Talking about the experience
- Web Scraping data from 3rd party dashboard, transforming and loading in Warehouse.
- Getting data from APIs and transforming using Pandas and loading in Warehouse.
- I have extracted data from data sources like Salesforce, Hubspot, Google Ads, and LinkedIn Ads and loaded the data in the warehouse.
- Good understanding of the business requirements along with an understanding of the expectations of the Marketing and the Products.
- Good Understanding of Marketing objects like Leads, MQLs, Hand Raiser MQLs, and SQLs.
- Created the whole Selenium project from scratch to extract data and loading in the warehouse using Prefect(data orchestration tool).
- Running Selenium in headless mode in ECS by deploying using Docker.
- Optimizing query for huge data in Snowflake using Clustering and other techniques.
- Streamline loading and Batch loading of data.
Technical Stack: Python, Selenium, DBT(Data Build Tool), Snowflake, PostgresSQL, Prefect, Fivetran, AWS services like S3 Bucket, ECR, ECS, Docker, SSH, GitHub
Preferred IDE: VSCode
OS: Linux(Ubuntu)
My skill sets are Web Scraping, Data Extracting from APIs, Modeling data using Kimball's approach, and Transforming data.
Talking about the experience
- Web Scraping data from 3rd party dashboard, transforming and loading in Warehouse.
- Getting data from APIs and transforming using Pandas and loading in Warehouse.
- I have extracted data from data sources like Salesforce, Hubspot, Google Ads, and LinkedIn Ads and loaded the data in the warehouse.
- Good understanding of the business requirements along with an understanding of the expectations of the Marketing and the Products.
- Good Understanding of Marketing objects like Leads, MQLs, Hand Raiser MQLs, and SQLs.
- Created the whole Selenium project from scratch to extract data and loading in the warehouse using Prefect(data orchestration tool).
- Running Selenium in headless mode in ECS by deploying using Docker.
- Optimizing query for huge data in Snowflake using Clustering and other techniques.
- Streamline loading and Batch loading of data.
Technical Stack: Python, Selenium, DBT(Data Build Tool), Snowflake, PostgresSQL, Prefect, Fivetran, AWS services like S3 Bucket, ECR, ECS, Docker, SSH, GitHub
Preferred IDE: VSCode
OS: Linux(Ubuntu)
Project history
Certifications
Local Availability
Only available for remote work
