About the client

Our client is a North American company that strives to revolutionize the way we think about food and health by leveraging deep research. To realize this bold vision, the customer applies cutting-edge data science & machine learning techniques powered by modern tools and cloud-native services to analyze the millions of molecules that exist across human food ration.

Challenge

The client needed to strengthen collaboration within the data scientists team by moving research capabilities into cloud workloads. Besides, they wanted to automate and unify the deployment process of AWS resources. That, in turn, would decrease the time and effort required for a team of data scientists to build and test their models.

Solution

Delivering the solution, the team used Terraform framework for infrastructure as a code implementation. Terraform code itself is stored in the GitLab repository, and the Terraform state – in the S3 bucket of the specific environment.

GitLab CI automation pipeline was used to enable the deployment of AWS resources on different environments. In turn, it allowed to do a one-click deployment of the AWS infrastructure to the development/production environment.

All the secrets are encrypted and securely stored on GitLab. GitLab CI provides customers with a single control point for the deployment process – for example, for reviewing Terraform code and deploying it on approving the merge request. It also enables control access in the AWS environment and restricts it for regular users, which means that all infrastructure changes can be deployed only with pipelines.

High-level solution diagram

ECS deployment example

Technologies

Result

The developed solution has allowed the client to use Terraform code and GitLab pipelines instead of manual AWS infrastructure and lambdas deployment. The data scientists team can now deploy AWS resources and test models in cloud workloads faster and more efficiently, which means they can focus on key scientific objectives rather than cloud engineering.