Python Developer/Data Engineer
Tambo Marketplace Ltd.
(Consumer goods and retail, 10-50 employees)
Keywords
Python (Programming Language)
Information Engineering
Flask (Web Framework)
Amazon Web Services
Cloud Computing
Databases
Continuous Integration
Extract Transform Load (ETL)
PostgreSQL
SQL Databases
Skills
As a Python developer and data engineer, I bring a diverse skill set that empowers me to create efficient and scalable solutions for a wide range of data-driven challenges. With proficiency in Python, SQL, Flask, Pandas, Postgres, ETL pipelines, REST APIs, BeautifulSoup, Requests, AWS cloud technologies, Docker, CI/CD, and git, I am well-equipped to deliver top-notch projects that meet and exceed the expectations of potential clients.
My journey as a Python developer began about 4 years ago, and since then, I have honed my expertise in the language, enabling me to develop robust and elegant applications. My deep understanding of Python's libraries and frameworks, including Flask, FastAPI and Pandas allows me to architect data-driven web applications with ease. Leveraging Flask, I have developed RESTful APIs that facilitate seamless data communication, enabling clients to access and interact with their valuable datasets.
In the realm of data engineering, I have worked extensively with SQL, Postgres and Redshift to design and optimize transactional databases that efficiently store and manage large datasets. My experience in ETL (Extract, Transform, Load) pipelines has proven vital in the process of extracting raw data from various sources, transforming it into a consistent format, and loading it into databases for further analysis. I take pride in my ability to craft data pipelines that ensure data integrity, accuracy, and efficiency throughout the entire data lifecycle.
Working with cloud technologies, particularly AWS, has been a cornerstone of my career. By leveraging the power of AWS services, such as S3, Athena, Redshift, Lightsail containers and EC2, I have built scalable and cost-effective solutions, such as data warehouses and real-time data processing systems. My proficiency in Docker allows me to encapsulate applications and their dependencies, ensuring seamless deployment across different environments while promoting consistency and reproducibility.
Moreover, I am well-versed in CI/CD practices and Git, enabling me to collaborate effectively with teams and maintain version control throughout the development lifecycle. This ensures that projects are continuously integrated, tested, and deployed, streamlining the development process and enhancing overall project quality.
In my projects, web scraping has been a valuable asset, and I have utilized libraries like BeautifulSoup, Scrapy and Requests to collect data from various websites efficiently. By extracting relevant information, I empower clients to gain valuable insights and make data-driven decisions.
My passion for learning is what drives me to stay at the forefront of technological advancements, allowing me to adapt to emerging trends and incorporate cutting-edge tools and methodologies into my work.
I am excited to utilize these skills to impress potential clients and tackle complex challenges, delivering exceptional solutions that drive success and growth for their businesses.
My journey as a Python developer began about 4 years ago, and since then, I have honed my expertise in the language, enabling me to develop robust and elegant applications. My deep understanding of Python's libraries and frameworks, including Flask, FastAPI and Pandas allows me to architect data-driven web applications with ease. Leveraging Flask, I have developed RESTful APIs that facilitate seamless data communication, enabling clients to access and interact with their valuable datasets.
In the realm of data engineering, I have worked extensively with SQL, Postgres and Redshift to design and optimize transactional databases that efficiently store and manage large datasets. My experience in ETL (Extract, Transform, Load) pipelines has proven vital in the process of extracting raw data from various sources, transforming it into a consistent format, and loading it into databases for further analysis. I take pride in my ability to craft data pipelines that ensure data integrity, accuracy, and efficiency throughout the entire data lifecycle.
Working with cloud technologies, particularly AWS, has been a cornerstone of my career. By leveraging the power of AWS services, such as S3, Athena, Redshift, Lightsail containers and EC2, I have built scalable and cost-effective solutions, such as data warehouses and real-time data processing systems. My proficiency in Docker allows me to encapsulate applications and their dependencies, ensuring seamless deployment across different environments while promoting consistency and reproducibility.
Moreover, I am well-versed in CI/CD practices and Git, enabling me to collaborate effectively with teams and maintain version control throughout the development lifecycle. This ensures that projects are continuously integrated, tested, and deployed, streamlining the development process and enhancing overall project quality.
In my projects, web scraping has been a valuable asset, and I have utilized libraries like BeautifulSoup, Scrapy and Requests to collect data from various websites efficiently. By extracting relevant information, I empower clients to gain valuable insights and make data-driven decisions.
My passion for learning is what drives me to stay at the forefront of technological advancements, allowing me to adapt to emerging trends and incorporate cutting-edge tools and methodologies into my work.
I am excited to utilize these skills to impress potential clients and tackle complex challenges, delivering exceptional solutions that drive success and growth for their businesses.
Project history
03/2022
-
Present
- Developed data-intensive containerised backend services using Flask to power front end dashboards with focus on CI/CD and automated testing to ensure clients have access to the most-up-to-date data for time sensitive decision making.
- Managed automatic deployment on AWS Lightsail container service, utilising docker containers and Linux shell scripts.
- Developed a FastAPI solution to serve raw JSON data and to trigger data collection systems to acquire fresh data when needed according to a prioritisation algorithm.
- Designed and orchestrated an ETL pipeline using redis-queues to run parallel requests and perform bulk data transformation, resulting in a 40% improvement in data accuracy compared to the previous monthly data collection system.
- Built and queried relational databases (Postgres) and data warehouse (Redshift) using SQLAlchemy to provide business insight using raw SQL.
- Integrated DBT as a data transformation solution, enabling SQL query versioning, templating, and incremental table updates. This integration improved data loading on the front-end by 67%.
- Actively participated in pair coding, conducted code reviews, and wrote object-oriented code that prioritised readability,
- maintainability, and testability.
01/2020
-
03/2022
Junior Python Developer
Onalytica
(Marketing, PR and Design, 10-50 employees)
- Consumed REST APIs using the Requests library to integrate data into relational database using ORM.
- Leveraged Python and Pandas to clean, analyse, and aggregate large datasets, identifying key influencers.
- Streamlined reporting processes by leveraging Python to automate the transformation of tabular data and generate actionable strategy recommendations. This automation workflow led to a 75% time saving for junior analysts.
- Efficiently generated daily and weekly reports with trending insights sourced from live data, implementing automated data cleaning and extraction procedures.
06/2017
-
11/2019
Senior Data Analyst
Brandwatch
(Media and Publishers, 50-250 employees)
- Conducted extraction and analysis of extensive datasets to evaluate social interactions and offer strategic guidance to stakeholders.
- Created client-facing dashboards that showcase campaign trends, product mentions, and key influencers, enabling stakeholders to make timely decisions.
- Utilised advanced Brandwatch tools like Social Panels and Custom Classifiers to segment data by influencers and deliver measurable insights across relevant markets.
- Provided leadership and supervision to a team of 8 junior analysts, overseeing their involvement in quarterly projects for leading global companies.
Certifications
Amazon Marketing Cloud
Amazon
2023
SQL for Data Analysis
Udacity
2021
Introduction to PySpark
Datacamp
2021
Full Stack Web Developer Nanodegree
Udacity
2020
Local Availability
Open to travel worldwide
