The projects mainly revolve around the prevention and monitoring of advertising fraud:
+ Design and implement an application that assists in real-time automated bot prevention for online advertisements
using an external anti-fraud service (Whiteops, ETL, Airflow, Python, Pandas)
+ Migration of the team's datapipeline from Hadoop to Azure Databricks, implementation of DEV, PREPROD and PROD
environments, and implementation of dashboards and metrics to monitor the migration under Spark, Azure Databricks,
Scala, Terraform
+ Lead efforts across multiple teams to display important anti-fraud metrics on customer reports (Spark, Hive, Vertica)
+ Suggest, design and implement a better Data-pipeline architecture which improved execution time and overall
memory/CPU usage for many applications used for reporting and fraud prevention (Map-reduce, JAVA , Spark, Scala)
+ Miscellaneous tasks: Implementation and improvement of Dashboards (Grafana, Redash and PowerBI) for better
monitoring of fraud related to online advertising, writing of Runbooks for alert management
Methodology : Agile
Technical environment:
+ Airflow, Azure, Databricks, Python, Pandas, Hadoop, Map-reduce, JAVA, Spark, Hive, Scala, Grafana, Redash, Vertica,
Jenkins, Confluence, Jira, PowerBI