09/06/2022 updated
PA
100 % available
Data Engineer
Hyderabad, India
India
BTech in Electronics and Communication EngineeringMicrosoft ExcelAlgorithmsData AnalysisCA Workload Automation AeMicrosoft AzureBig DataUnixCloud ComputingComputer ProgrammingData ProcessingDatabasesInformation EngineeringExtract Transform Load (ETL)Data ProfilingDatabase Tuning
Programming, Python, PLSQL, Scala, Oracle, NO-SQL, MySQL, Pyspark, Hadoop, Hive, Spark Steaming, Tableau, Autosys, Control-M, Git, UNIX, Cloud, Azure Databricks, Microsoft Azure SQL Database, Data Lake, Data Factory, Oracle cloud infrastructure architect associate, Oracle cloud foundation associate, Spark, Sql, JSONs, Excel, Text, RDD, Spark Streaming, Data Analysis, large data sets, Python programming, SQL queries, Pandas, data manipulation, algorithms, Regression, Query optimization, DB scripts, ETL, Big data, Sqoop, Azure, Azure Data Factory, SQL database, coding, debugging, Oracle 11G, stored procedures, database, materialized views, DB, data profiling, shell scripting, Teradata, Database tuning
Experience
Professional experience0 - 5 years
Languages
EnglishFluentHindiNative speaker
Project history
Worked on Pyspark and spark streaming for higher level data driven extraction.Expertise in using Spark-Sql
with various data sources like JSONs, Excel,Text
● In-depth understanding of Spark Architecture including Spark core, SparkSQL, DataFrames, RDD, Spark Streaming hands on experience on
in Analysis and design
● Data Analysis with large data sets of Structured and Unstructured data. Extracted, transformed and loaded to generate CSV data files with
Python programming and SQL queries
● Extensively used SQL, Pandas, Pypark for Data Analysis and Model building.Developed codes on python modules,decorators and functions
for data analyzing ,data manipulation and analytical operations
● Gained exposure to algorithms using Python and its concepts as data-preprocessing, Regression, classification and appropriate model
selection techniques
● Expertise in data base programming (SQL, PLSQL) and Query optimization. Prepared DB scripts for releases
● Performed ETL operations with Big data technologies like Sqoop and data pipelines. Also performed complex transformations and action on
the datasets using Hive, Spark, Azure Databricks and Data flow
● Took leverage of Azure Data Factory for cloud based ETL operations. Extracted data from SQL database and loaded to create Data lakes
* Extensive experience coding and debugging Oracle 11G stored procedures and various other objects. Involved in creating database
objects like tables,views,procedures,collections,packages,Triggers,Indexes,Synonyms, Sequences,materialized views and functions
using SQL to provide definition, structure and to maintain data efficiently
* Worked on DB-fit defect tracking tool for testing. Experience using scheduling tools such as Control-M
* Expertise in performing data analysis and data profiling using complex SQL on various sources systems including Oracle
* Prepared the consolidated deployment scripts for each source system using shell scripting and developed Tableau reports and
Dashboards
* Gained expertise in data base programming (SQL, PLSQL), Teradata, Database tuning and Query optimization.Prepared DB scripts
for releases
* Worked on VPD(Oracle Virtual Private Database) policies for Data hiding
objects like tables,views,procedures,collections,packages,Triggers,Indexes,Synonyms, Sequences,materialized views and functions
using SQL to provide definition, structure and to maintain data efficiently
* Worked on DB-fit defect tracking tool for testing. Experience using scheduling tools such as Control-M
* Expertise in performing data analysis and data profiling using complex SQL on various sources systems including Oracle
* Prepared the consolidated deployment scripts for each source system using shell scripting and developed Tableau reports and
Dashboards
* Gained expertise in data base programming (SQL, PLSQL), Teradata, Database tuning and Query optimization.Prepared DB scripts
for releases
* Worked on VPD(Oracle Virtual Private Database) policies for Data hiding