Description
Trust in Soda are partnered with a Global Leader in Research & Development. As a Data Engineer, you will be part of a cross-functional team that is responsible for continuously developing and deploying tools for the data & Machine Learning pipeline. The technology stack includes Spark, Go, Python, R and different database solutions (SQL and NoSQL) running in the cloud. They use a microservices and containerization (Docker, Kubernetes) approach to develop new solutions. You will develop, maintain, and improve the data pipeline within the scope of:
We expect a data engineer to bring a broad software engineering experience:
- Backend development in any programming language of your choice.
- Design of web services.
- Algorithms and complexity analysis.
- Linux system administration, development, and production environments.
- Cloud, container and microservices infrastructures.
- Software security.
- Development workflow automation
About You:
And a strong focus on data processing:
- Databases, theory and practice.
- Distributed data processing.
- Real Time event processing.
- Concepts of functional programming.
- Data privacy and anonymization techniques.
- Enterprise data warehousing, Business Intelligence and ETL principles.
- Statistics and analytics.
- Machine learning.
Preferred
- Experience with developing Big Data ETL pipelines
- Experience with Cloud Platforms, like GCP, AWS or Azure
- Experience with CI/CD tools
- Foundation of Machine Learning and basic algorithms.