Profileimage by sushmita pandit Data Engineer| Microsoft Azure| Databricks | Azure Datas Factory| PySpark| Python| from Berlin

sushmita pandit

available

Last update: 14.09.2021

Data Engineer| Microsoft Azure| Databricks | Azure Datas Factory| PySpark| Python|

Graduation: computer science and engineering
Hourly-/Daily rates: show
Languages: German (Full Professional) | English (Native or Bilingual)

Skills

Experience in implementation of modern data warehouse on Azure.
  • Proficient with creating etl pipelines with Azure data factory(can connect to different data sources  such as mysql server, microsoft sql server, salesforce, ip2location, s3 buckets. configure source, sink, datasets, linked services, azure key pass)
  • Proficient with utilising ADLS and Blob storage both as sink and source.
  • Proficient with utilising aws s3 bucket as source to extract streaming data.
  • Proficient with real time Apache kafka datastreams processing.
  • Databricks:
    Proficient with pyspark programming (can easily perform all types of transformations). Mounting storage, creating functions, creating dimensions and facts notebooks. Experience with optimising spark cluster, optimising complex notebooks. Experience dealing with Terra bytes of data.
  • creating gold storage as source for Tableau, PowerBI. experience dealing with json, parquet, csv, Avro files .
  • Excellent undertsanding of Apache spark data processing engine( How it processes data under the hood).
  • experience with Machine learning, Python, pandas, numpy, sci kit learn
  • proficient with SSIS, sql server, sql scripting.

Project history

·     Worked for ETL process of data loading from different sources and data validation process from staging area to catalog database.
 ·    Data analysis- Manipulating, cleansing & processing data using Excel, Access and SQL. Responsible for loading, extracting and validation of client data. Liaising with end-users and 3rd party suppliers. Analyzing raw data, drawing conclusions & developing recommendations Writing T-SQL scripts to manipulate data for data loads and extracts. Developing data analytical databases from complex financial source data. Performing daily system checks. Data entry, data auditing, creating data reports & monitoring all data for accuracy. Designing, developing and implementing new functionality. Advising on the suitability of methodologies and suggesting improvements. Carrying out specified data processing. Supplying qualitative and quantitative data to colleagues & stakeholders. Used QLIKVIEW for Reporting purposes.
 

Local Availability

Only available in these countries: Germany
Looking for Remote work
Profileimage by sushmita pandit Data Engineer| Microsoft Azure| Databricks | Azure Datas Factory| PySpark| Python| from Berlin Data Engineer| Microsoft Azure| Databricks | Azure Datas Factory| PySpark| Python|
Register