Big Data Developer
TRG RESEARCH & DEVELOPMENT
(Internet and Information Technology, 50-250 employees)
Skills
SKILLS
PROGRAMMING: SCALA JAVA PYTHON JAVASCRIPT
BIG DATA: SPARK KAFKA HBASE CASSANDRA ACCUMULO AIRFLOW
CLOUD: AZURE DATABRICKS CLOUDERA AWS HDINSIGHT
OTHERS: MONGODB ELASTICSEARCH SOLR DOCKER KUBERNETES
PROGRAMMING: SCALA JAVA PYTHON JAVASCRIPT
BIG DATA: SPARK KAFKA HBASE CASSANDRA ACCUMULO AIRFLOW
CLOUD: AZURE DATABRICKS CLOUDERA AWS HDINSIGHT
OTHERS: MONGODB ELASTICSEARCH SOLR DOCKER KUBERNETES
Project history
02/2020
-
Present
● Create from scratch a big data solution that processed around 900 millions
spatial-temporal records per day
● Create the cloud infrastructure and pipelines to support the solution using Azure as
cloud provide: Hbase cluster, Spark cluster, Kafka cluster , Azure Blob, Datafactory,
Databricks
● Used GEOMESA as spatial-temporal framework solution (after researching into other
solutions)
● Ingestion Spark pipelines from different sources: raw files, Kafka, Parquet and
databases synchronizations with: MongoDB, ArangoDB, ElasticSearch
● Complex query and analytics using Spark, SparkML and Kafka
● Create knowledge graph using Spark and ArangoDB
● Image and text analysis pipelines using ML, Spark and Kafka
● Poc’s around other graph databases: Neo4J, JanusGraph & Gaffer
● Create custom docker infrastructure to support development on all aspects
● Backend and Front-end development to support POC’s or small changes in
production: Python, TypeScript, NodeJS, Springboot, Javascript and Angular
● Monitoring using: Prometheus, Grafana and Jaeger
spatial-temporal records per day
● Create the cloud infrastructure and pipelines to support the solution using Azure as
cloud provide: Hbase cluster, Spark cluster, Kafka cluster , Azure Blob, Datafactory,
Databricks
● Used GEOMESA as spatial-temporal framework solution (after researching into other
solutions)
● Ingestion Spark pipelines from different sources: raw files, Kafka, Parquet and
databases synchronizations with: MongoDB, ArangoDB, ElasticSearch
● Complex query and analytics using Spark, SparkML and Kafka
● Create knowledge graph using Spark and ArangoDB
● Image and text analysis pipelines using ML, Spark and Kafka
● Poc’s around other graph databases: Neo4J, JanusGraph & Gaffer
● Create custom docker infrastructure to support development on all aspects
● Backend and Front-end development to support POC’s or small changes in
production: Python, TypeScript, NodeJS, Springboot, Javascript and Angular
● Monitoring using: Prometheus, Grafana and Jaeger
05/2021
-
05/2021
Big Data Developer
Deloitte
(Internet and Information Technology, 250-500 employees)
● Setup and administrate Cloudera and Hortonworks cluster
● Migrate Hive queries to Spark
● Optimise Spark Pipelines
● Migrate Hive queries to Spark
● Optimise Spark Pipelines
Local Availability
Only available in these countries:
Romania
