Data Engineer | AWS, Snowflake, Pyspark, Sql, Python,ETL,Stored Proc

Cloud Data Engineering: Extensive experience with AWS services including Glue, Lambda, DynamoDB, S3, API Gateway, and Step Functions. I automate workloads and orchestrate complex data workflows to streamline business operations. Data Pipeline Optimization: Proven ability to optimize SQL code, even with massive datasets (up to 4.5 billion rows), and shift processes from monthly to weekly runs, enhancing performance and reducing processing time by up to 50%. Data Migration and Integration: Expert in migrating complex systems to the cloud using AWS Data Migration Service, Snowpipe, and Snowflake. I’ve successfully transitioned hundreds of tables and critical business modules, ensuring seamless data validation and integrity. Automation and CI/CD: Skilled in automating data workflows using Bitbucket, Jenkins, and Apache Airflow, including dynamic DAG creation for new business logic deployments. Big Data and Spark: Developed robust Spark applications and wrote complex business logic in PySpark to handle large-scale data processing tasks efficiently. Role-Based Access Control (RBAC): Created custom RBAC models in Snowflake for secure and compliant data access management tailored to client needs. Advanced SQL and Data Validation: Expertise in writing and validating SQL for data transformation, including CDC processes and ensuring data consistency across platforms like Oracle, Snowflake, and AWS.