Data Engineer | Databricks, GCP, Azure, Snowflake | SQL & Python Expert

With a strong background in data engineering, I specialize in designing, building, and optimizing scalable data pipelines that transform raw data into valuable insights. My expertise spans across big data processing, cloud platforms, and workflow automation, making me a valuable asset for organizations seeking robust data solutions.

• Big Data & Databricks: I have hands-on experience working with Databricks, where I leverage Apache Spark to efficiently process and analyze large datasets. My focus is on optimizing Spark jobs for performance and cost efficiency, ensuring that data transformations are both scalable and maintainable.
• SQL & Data Warehousing: I am highly proficient in SQL, crafting complex queries, optimizing database performance, and managing ETL processes for data warehousing. I specialize in transforming raw data into aggregated/flat tables that support business intelligence and reporting.
• Python & Automation: I use Python extensively for data ingestion, transformation, and automation. My experience includes working with async API calls in Streamlit, ensuring efficient data retrieval and user interaction in web applications.
• Data Ingestion & Pipeline Monitoring: I have deep experience in building and maintaining data pipelines, ensuring smooth data flow from various sources while handling schema evolution, data quality, and error handling.

Tech Stack and Libraries:
Python, SQL, Databricks, GCP, Snowflake, Power BI, FastAPI, Docker, MongoDB, Airflow, Pentaho Data Integration, RDBMS