10/10/2024 updated
100 % available
Senior Data Engineer
Biot, France
Worldwide
Java (Programming Language)Artificial IntelligenceAmazon Web ServicesData AnalysisAtlassian ConfluenceTest AutomationMicrosoft AzureBatch FilesBig DataCloud ComputingCloud StorageDatabasesContinuous IntegrationData CleansingInformation Engineering
AWS, artificial intelligence, automation test, test automation, batch files, big data, cloud, cloud storage, confluence, CI/CD, analytics, data analysis, data cleansing, Data pipeline, data processing, data transformation, data warehouses, data warehouse, databases, database, deep learning, ETL, github, GCP, Data Engineer, Data Engineering, Java, makefile, Azure, Python, regression test, SAP, SQL, SQL query, stored procedure, Salesforce, sharding, software quality assurance, test strategy, system integration test, TIBCO, terraform, test case, test execution, Shell
Experience
Professional experience10+ years
Freelancer since2020
Languages
ChineseNative speakerEnglishFluentFrenchFluent
Project history
2023 Performed data engineering with worldwide financial data for cashflow and
capex reporting and forecasting. Built fully automated pipeline in GCP to
simplify the process (90% time reduced)
Built ELT pipeline in GCP with tools: SAP SLT, cloud storage, pub/sub,
big query, workflows, cloud function, cloud run.
Used stack for Data: SQL, python; Build: makefile, terraform; Cicd:
cloud build, github
Optimized the cost and performance using sharding, partitioning and
clustering
capex reporting and forecasting. Built fully automated pipeline in GCP to
simplify the process (90% time reduced)
Built ELT pipeline in GCP with tools: SAP SLT, cloud storage, pub/sub,
big query, workflows, cloud function, cloud run.
Used stack for Data: SQL, python; Build: makefile, terraform; Cicd:
cloud build, github
Optimized the cost and performance using sharding, partitioning and
clustering
Worked with an on-prem data warehouse to collect, organize and classify
~1.5 millions datasets per day from ~200 databases and batch files for
testing purpose. Offered ready-to-use datasets for all kinds of test in
transactional environment.
Developed and maintained data processing script in shell including
automated data cleansing process
Done data transformation in stored procedure with SQL
Offered insights by using analytic and ranking function in SQL
~1.5 millions datasets per day from ~200 databases and batch files for
testing purpose. Offered ready-to-use datasets for all kinds of test in
transactional environment.
Developed and maintained data processing script in shell including
automated data cleansing process
Done data transformation in stored procedure with SQL
Offered insights by using analytic and ranking function in SQL
Worked on various digital transformation projects. supervised a group of
testers to deliver high quality projects. Analysed the business and
technical requirements and elaborated test strategy. Perform critical test
and troubleshooting. Managed and share the test asset across teams.
Consolidated test efficiency KPI. Planed and coordinated testing activities.
Identified as key player in various projects.
Worked with cross-functional teams as the test referent during CRM
(Salesforce) and Debt collection (Codix IMX) projects covering the end to
end business processes. Reviewed and analysed system specifications.
Developed and applied testing strategy. Build thorough test WIKI in
confluence.
Delivered solid results with helping and coaching new team member.
Improved critical thinking skills by problem analysis
testers to deliver high quality projects. Analysed the business and
technical requirements and elaborated test strategy. Perform critical test
and troubleshooting. Managed and share the test asset across teams.
Consolidated test efficiency KPI. Planed and coordinated testing activities.
Identified as key player in various projects.
Worked with cross-functional teams as the test referent during CRM
(Salesforce) and Debt collection (Codix IMX) projects covering the end to
end business processes. Reviewed and analysed system specifications.
Developed and applied testing strategy. Build thorough test WIKI in
confluence.
Delivered solid results with helping and coaching new team member.
Improved critical thinking skills by problem analysis