11/06/2024 updated
TJ
100 % available
Experienced Big Data Engineer with Hadoop Ecosystem Expertise
shakopee, USA
Worldwide
Bachelors in Information TechnologyC (Programming Language)Java (Programming Language)Amazon Web ServicesAmazon Elastic Compute CloudAmazon S3Data AnalysisApache AntBig DataApache ImpalaDatabasesContinuous IntegrationExtract Transform Load (ETL)Data WarehousingIBM DB2DevOps
Hadoop Ecosystem
Extensive experience with Hadoop ecosystem components including HDFS, MapReduce, HBase, Spark, Yarn, Kafka, Zookeeper, Pig, Hive, Sqoop, Storm, Oozie, Impala, and NiFiand Flume.
Big Data Technologies
Proficiency in deploying and testing Big Data technologies, including Data Warehousing (ETL/BI) Testing, Big Data Testing, and web services testing.
Cloud Services
Expertise in working with AWS cloud services like EC2, S3, Redshift, EMR, Lambda, DynamoDB, RDS, SNS, SQS, Glue, Data Pipeline, and Athena for big data development.
Programming Languages
Proficiency in various programming languages and scripting tools including Java, Scala, PL/SQL, HiveQL, shell scripting, Python, and C.
Database Technologies
Experience with SQL and NoSQL databases including HBase, Cassandra, MongoDB, Oracle, MySQL, MS SQL Server, DB2, and Teradata.
Data Processing and Analytics
Skills in data ingestion, transformation, and analysis using tools like Spark, Pig, Hive, and various ETL processes.
DevOps and CI/CD
Familiarity with version control systems, build tools, and continuous integration/deployment pipelines using Git, SVN, Maven, Jenkins, and Ant.
Extensive experience with Hadoop ecosystem components including HDFS, MapReduce, HBase, Spark, Yarn, Kafka, Zookeeper, Pig, Hive, Sqoop, Storm, Oozie, Impala, and NiFiand Flume.
Big Data Technologies
Proficiency in deploying and testing Big Data technologies, including Data Warehousing (ETL/BI) Testing, Big Data Testing, and web services testing.
Cloud Services
Expertise in working with AWS cloud services like EC2, S3, Redshift, EMR, Lambda, DynamoDB, RDS, SNS, SQS, Glue, Data Pipeline, and Athena for big data development.
Programming Languages
Proficiency in various programming languages and scripting tools including Java, Scala, PL/SQL, HiveQL, shell scripting, Python, and C.
Database Technologies
Experience with SQL and NoSQL databases including HBase, Cassandra, MongoDB, Oracle, MySQL, MS SQL Server, DB2, and Teradata.
Data Processing and Analytics
Skills in data ingestion, transformation, and analysis using tools like Spark, Pig, Hive, and various ETL processes.
DevOps and CI/CD
Familiarity with version control systems, build tools, and continuous integration/deployment pipelines using Git, SVN, Maven, Jenkins, and Ant.
Project history
Supporting internal teams with Azure Data Factory, maintaining SQL queries, troubleshooting data processes, and working on AWS services for data management and analysis.
Developed data products using cloud-based technologies, migrated data to AWS, and designed ETL processes for scientific data migration.
Led the design and development of scalable Big Data platform software, worked on data migration to Snowflake, and established best practices for Data Engineering & Analytics.