AWS Data Architect

Project description: Data Platform modernization (migration to AWS cloud) for one of the biggest Pharmaceutical company.
Goal is to migrate existing on-premises Cloudera big data platform into AWS cloud for:
• to enable future platform development by refactoring existing code and leveraging latest technologies
• to optimize costs (e.g., using serverless technologies to pay only for time of actual processing, not idle time).
• to decrease maintenance efforts and improve platform's SLAs
• to conform with company's IT strategy for cloud movement
Responsibilities:
• Provide expert knowledge (in area of both, AWS Cloud platform and Big Data Processing)
• Review existing code and suggest improvement (performance optimization, increasing code' maintainability)
• AWS cost optimization (e.g. changes to enable Glue Jobs scaling)
• Designed and implemented Alerts & Monitoring solution (which is designed to be easily reused in other company's projects)
• Designed and implemented CI/CD process to fully automate AWS Resource deployment (using Azure DevOps pipelines)
Technology stack:
AWS CloudFormation (Infrastructure as a Code)
Azure DevOps, git (CI/CD Process)
AWS Glue Jobs (data ingestion and processing using Spark)
AWS Glue Workflows and Triggers (for orchestration)
AWS Athena (data consumption layer)
AWS S3, S3 event notifications
AWS Lambda
AWS EventBridge (alerts and monitoring, including cross-account alerts/monitoring)
AWS Simple EMail Services

Customer: Worldwide Electronics manufacturer
Role: Data Integration Developer
Technology stack: AWS (Redshift, MWAA, Glue, Lambda, Cloudformation, S3), Azure DevOps, Snowflake, MS SQL, SAP
• Project was a platform for data integration and processing (from various company data sources) to provide reports/metrics to external customers
• Conducted PoCs to define technological stack and prove proposed architecture
• Implemented Data Ingestion Layer (from Snowflake, MS SQL, SAP [via SAP views], Excel files) using Glue Jobs
• Implemented ETL transformations (using Redshift stored procedures), orchestration - AWS Managed Airflow
• Developed Data Synchronization checks (verifying data is in-sync with sources)
• Designed & Implemented IaaC approach (AWS CloudFormation + Azure DevOps CI/CD)
• Implemented Schema changes approach (Liquibase)
• Conducted Code Reviews
--------------------------
Customer: Healthcare Company Role: Solution Architect (for Project Discovery phase)
Technology stack: Oracle ODI, Airflow, Snowflake
• Project is assess migration of existing customer’s warehouse and ETL processes from Oracle stack into cloud (Azure + Snowflake)
• Proposed target architecture, detailed phase-by-phase migration approach
• Analyzed ODI pipelines source code to estimate migration efforts needed; define high-level timeline and team composition for migration project
--------------------------
Customer: World-wide pharmaceutical company
Role: Data Engineering Team Lead
Technology stack: AWS (Glue, Lambda, Aurora, CloudFormation), Python, Big Data Stack (Hive, Impala)
• Project was a platform integrating data from various data sources over the whole company
• Defined data model, data flow
• Implemented coding standards
• Developed Cloud formation templates for AWS infrastructure provisioning
​​​​​​​--------------------------
Customer / My Role: World leading sports equipment manufacturer
Role: Solution Architect
Technology stack: AWS (S3, EC2, VPC, Cloudwatch), Jmeter, Exasol, Python
• Designed a performance testing framework for customer’s data warehouse (build on top of Jmeter core)
• Worked as a team lead and developer during the implementation of performance testing framework
• Designed and conducted series of tests to assess customer’s migration to newer cloud version of Exasol (including, but not limited to, performance testing using developed framework)
​​​​​​​--------------------------
Customer: World-wide telecom provider
Role: Solution Architect
Technology stack: AWS (Redshift, S3, VPC, Cloudwatch, Cloudtrail, CodeCommit), Snowflake, Exasol
Ran a series of PoC to help customer choose the most suitable cloud-based warehouse technology (considered options were Snowflake, Exasol and AWS Redshift).
• Worked with stakeholders to identify scope and important aspects for PoCs
• Handled majority of technical tasks (such as customer-specific synthetic dataset preparation, re-implement one of customer’s existing ETL processes for all 3 cloud WH options, etc.)
• Conducted multiple review sessions with company’s technical management to present and discuss research results
• Participated in beta-testing for AWS Lake Formation
​​​​​​​--------------------------
Customer: Worldwide e-commerce company
Role: DBA
Technology stack: AWS Aurora, SAP
• Investigated DB performance issues, identified bottlenecks, provided recommendations to resolve
• Reviewed and validated DB configuration, find optimal settings for product's workload, leveraging AWS Aurora cloud-native features for optimization
• Developed a strategy of efficient DB monitoring
• Worked closely with development team to help them write SQL queries efficiently, suggest an optimal indexing strategy etc.
​​​​​​​--------------------------
Customer: World-wide informational agency
Role: Data Architect
Technology stack: AWS (Aurora, Elasticache, ElasticSearch), Oracle v11, v12
In a team of solution architects assessed and provided recommendations for transition of customer’s on-premise solution into AWS Cloud.
My primary areas of responsibility were:
• Data Layer Architecture,
• Data transfer strategy and required data structure changes (AWS
Conversion tool, AWS Data Migration Service)
• Projected cost-estimates
Non-project activities:
• Participated in a AWS Preview testing of AWS Aurora Serverless and AWS Lake Formation
Knowledge sharing:
• Developed and conducted AWS Redshift training
• Conducted AWS Immersion Day [Redshift] on behalf of Amazon for their customer
• 5 EPAM internal talks, 2 talks at independent conferences (AzureDays, IT Week)
• Graduated from Solution Architecture School (EPAM internal)

The product is an OLTP information system working 24/7(including Ordering, Logistics, Manufacturing and Accounting subsystems).
• Oracle architect, responsible for: Database design, key infrastructure decisions, coding guidelines, performance tuning and 3rd party products integration
• PL/SQL Developer for the company's most complex and important projects
• Team Lead (12 people: software engineers, business analysts, QA): performing reviews of source code, software requirements and test cases.