Data Engineer Intern
Prefeitura Municipal de Pato Branco
Keywords
Information Engineering
Microsoft Azure
Extract Transform Load (ETL)
Query Performance
Amazon Web Services
Data Analysis
Big Data
Cloud Computing
Computer Engineering
Data Validation
Skills
AWS, Apache Spark, Azure Data Factory, Azure Data Lake, large volumes of data, cloud-based, Computer Engineering, Analytics, data inconsistencies, data integration, data pipelines, data processing, data retrieval, data storage, data solutions, data validation, data warehousing, Databricks, extract, transform, and load, ETL, Git, indexing, Data Engineer, data engineering, Azure, Azure DevOps, Azure services, SQL Server, Power BI, PySpark, Python, query performance, SQL, SSAS, SSIS, version control.
Project history
05/2022
-
10/2022
* Developed and maintained data pipelines using Databricks and Azure Data Factory, extracting, transforming, and
loading large-scale datasets for analysis and reporting.
* Leveraged PySpark and SQL to perform data transformations, ensuring data accuracy, consistency, and completeness.
* Collaborated with a team of data engineers and data scientists to optimize data processing workflows, reducing processing
time by 30%.
* Implemented data validation checks, reducing data inconsistencies by 20% and enhancing overall data quality.
* Assisted in the development and maintenance of data warehousing solutions, optimizing storage and retrieval processes.
* Utilized Azure services and technologies to enhance data engineering capabilities and support efficient data retrieval.
* Documented data engineering processes and created reports to communicate project progress and findings.
* Overall, gained hands-on experience in Databricks, PySpark, Azure Data Factory, and SQL, along with strong teamwork
and communication skills in a fast-paced environment.
loading large-scale datasets for analysis and reporting.
* Leveraged PySpark and SQL to perform data transformations, ensuring data accuracy, consistency, and completeness.
* Collaborated with a team of data engineers and data scientists to optimize data processing workflows, reducing processing
time by 30%.
* Implemented data validation checks, reducing data inconsistencies by 20% and enhancing overall data quality.
* Assisted in the development and maintenance of data warehousing solutions, optimizing storage and retrieval processes.
* Utilized Azure services and technologies to enhance data engineering capabilities and support efficient data retrieval.
* Documented data engineering processes and created reports to communicate project progress and findings.
* Overall, gained hands-on experience in Databricks, PySpark, Azure Data Factory, and SQL, along with strong teamwork
and communication skills in a fast-paced environment.
05/2022
-
10/2022
Data Engineer Intern
MG Info
* Collaborated with a cross-functional team at MG Info Company to design, develop, and maintain data pipelines utilizing
Databricks and Azure Data Factory.
* Implemented data transformations using PySpark and SQL to ensure data accuracy, consistency, and completeness in
large-scale datasets.
* Worked closely with senior data engineers to optimize data processing workflows, resulting in a 30% reduction in
processing time.
* Assisted in the development and maintenance of data warehousing solutions, streamlining storage and retrieval processes.
* Leveraged Azure services and technologies to enhance data engineering capabilities and facilitate efficient data retrieval.
* Maintained comprehensive documentation of data engineering processes and produced insightful reports to communicate
project progress and findings.
* During my internship at MG Info Company, I gained valuable hands-on experience with Databricks, PySpark, Azure Data
Factory, and SQL. I developed strong collaboration, problem-solving, and communication skills while working in a
dynamic and fast-paced environment.
Databricks and Azure Data Factory.
* Implemented data transformations using PySpark and SQL to ensure data accuracy, consistency, and completeness in
large-scale datasets.
* Worked closely with senior data engineers to optimize data processing workflows, resulting in a 30% reduction in
processing time.
* Assisted in the development and maintenance of data warehousing solutions, streamlining storage and retrieval processes.
* Leveraged Azure services and technologies to enhance data engineering capabilities and facilitate efficient data retrieval.
* Maintained comprehensive documentation of data engineering processes and produced insightful reports to communicate
project progress and findings.
* During my internship at MG Info Company, I gained valuable hands-on experience with Databricks, PySpark, Azure Data
Factory, and SQL. I developed strong collaboration, problem-solving, and communication skills while working in a
dynamic and fast-paced environment.
Certifications
Local Availability
Only available for remote work
