APPLICATION DEVELOPMENT SENIOR ANALYST
ACCENTURE
Skills
Apache Spark, Azure Cloud, Databricks, Azure Data Factory, Azure Synapse, Pyspark, Hadoop, Hive, ETL,Flask API, Azure App Services
Project history
05/2019
-
02/2020
Self Service: worked as technical lead to this project. The data is retrieved using azure data
factory which then later triggers databricks jobs which performs transformation on this data
This data is later stored in azure SQL database.
Responsibility:
* Worked on ADF to create pipelines.
* Worked on Azure databricks, created Pyspark transformation jobs.
* Worked as a point of contact for the project.
* Direct interaction with client- Status update ETA.
factory which then later triggers databricks jobs which performs transformation on this data
This data is later stored in azure SQL database.
Responsibility:
* Worked on ADF to create pipelines.
* Worked on Azure databricks, created Pyspark transformation jobs.
* Worked as a point of contact for the project.
* Direct interaction with client- Status update ETA.
09/2018
-
04/2019
APPLICATION DEVELOPMENT SENIOR ANALYST
ACCENTURE
Crunch Project: The scope of the project was to extract files from Ambari, performing
transformation and storing those transformed data into respective hive table
Responsibility:
* Worked as an independent resource in the project.
* Worked on spark transformation jobs .
transformation and storing those transformed data into respective hive table
Responsibility:
* Worked as an independent resource in the project.
* Worked on spark transformation jobs .
07/2014
-
08/2018
SENIOR PROGRAMMER
BITWISE SOLUTIONS
Hadoop analytics - Worked on Hadoop system, migrated all the existing processes in google
cloud platform. As a primary tool we used dataproc cluster. Processes were written in apache
spark. For real time streaming POC was done on pubsub and apache kafka. Scheduling was done
through Apache airflow
Responsibility:
* Working on existing architecture of the Hadoop ecosystem.
* POC on google cloud platform tools- Dataproc, Google cloud storage, Bigquery, apache
beam. Apache Airflow
* Migrated processes into google cloud platform using Dataproc, Apache spark,
Bigquery,pubsub.
cloud platform. As a primary tool we used dataproc cluster. Processes were written in apache
spark. For real time streaming POC was done on pubsub and apache kafka. Scheduling was done
through Apache airflow
Responsibility:
* Working on existing architecture of the Hadoop ecosystem.
* POC on google cloud platform tools- Dataproc, Google cloud storage, Bigquery, apache
beam. Apache Airflow
* Migrated processes into google cloud platform using Dataproc, Apache spark,
Bigquery,pubsub.
Local Availability
Only available in these countries:
India
Other
IT Profressional with 6+ years of experience in Data Engineering. Exposure to Google cloud Platform, Cerftified on AWS and currently working on Azure Cloud.
