remote

Big Data Cloud Senior Engineer/Architect

Flemish Brabant Province

Keywords

Amazon Web Services Big Data Data Science Team-working Adaptabilidad Amazon Elastic Compute Cloud Amazon S3 Data Analysis Authentications Cloud Computing Communication English Identity Management Infrastructure Management Machine Learning Product Family Engineering Product Lifecycle Workflows Cloud Platform System Apache Spark Tooling Assembly and Dismantling Amazon Virtual Private Cloud (VPC) Cloudformation Kubernetes Cloudwatch Terraform User Administration

Description

My client are looking for an experienced Cloud platform engineer in a data analytics context

Job Description:

CLIENT group is currently creating a new big data platform in AWS. On top of the platform, we are building a platform that focuses on supporting data science and other technical computing workflows. You will
participate in this effort, as a valuable team player, to analyse, design, and implement the platform in close collaboration with data scientists.

As a platform engineer for data science, you are responsible for engineering a well architected data science platform with particular emphasis in interactive computing and highly automated model life cycle management based on MLOps. Main considerations for the design include security, maintainability, flexibility, user friendliness and integration between components on the platform.
You work closely with data scientists to understand their functional needs and translate them in an actual solution. Then, you rapidly iterate over the solution based on frequent feedback from data scientists.
Due to how the platform is used in the group, you are also responsible for documenting guidelines for future platform operators.

Skills Required:
An ideal candidate should have:

  • Good working knowledge of cloud computing in general, and AWS specifically (AWS IAM, EC2, S3,VPC)
  • Other AWS services such as Cognito, Lambda, CloudWatch, ECR, Athena etc
  • Basic understanding of AWS products related to machine learning (AWS SageMaker, MLFlow)
  • Experience with Infrastructure-as-Code (IAC) tools such as Terraform, Pulumi, AWS CloudFormation
  • Kubernetes/Helm
  • Good knowledge of user management (authentication and authorization) with AWS products and OSS solutions
  • Good grasp of big data ecosystem and tooling especially Apache Spark and related technologies
  • Good verbal and written communication skill in English
  • Willingness to work in close collaboration with data scientists and other engineers from different teams
  • An ability to work in a highly dynamic environment
  • A can-do attitude to cope with non-technical constraints

You can work 2 days per week remotely with 3 days per week required in Leuven Belgium.

Start date
ASAP
Duration
12 months
From
ComTech Europe Limited
Published at
23.11.2022
Project ID:
2503281
To apply to this project you must log in.
Register