Estimate project
← All posts

MLOps – what it is and why it is important

This article explained MLOps, the advantages it brings and its practices, and walked through a stock prediction model evolving from a basic setup with no MLOps into a complete MLOps system.

7 min read
Machine LearningMLOps

Along with the growth of AI and ML, a new acronym gained popularity – MLOps. MLOps is short for Machine Learning Operations. The idea centres on making the path of ML models into production smooth, safe, and well organised.

Back in 2019, Gap executives stated that 87% of data science projects never reach production. You might wonder, “How can that be?”. The main reason is that, back then, good practices were lacking and deployment in most companies was chaotic. MLOps practices appeared as a response to that.

MLOps sits where 3 concepts meet: Data Engineering, Machine Learning, and DevOps.

In this article, I will describe in plain language what MLOps is, how it applies to Machine Learning pipelines, and why it is important. Next, I will go through MLOps practices and their connection to DevOps. Lastly, I will demonstrate how a system differs when built with MLOps and without it.

What is MLOps?

MLOps is a collection of practices that support cooperation between data scientists and operations professionals (that is, between those who build models and those who use models as software components). Following these practices:

  1. raises the quality of the solution,
  2. makes management simpler,
  3. automates model deployment to a production environment,
  4. makes it easy to stay in line with business goals and regulations.

The chart above illustrates how the various parts of the Machine Learning cycle relate to one another. It begins with Data Sourcing and Data Labelling, followed by Experiment tracking, Model Versioning, and Model Deployment. At the end, we monitor predictions to keep track of how our model is being used. To guarantee high quality, MLOps oversees every segment of the Machine Learning process. Let’s now discuss why it is necessary.

Why is it important?

MLOps offers a framework that makes training and deploying models easier while reducing operational costs.

Because it comes with monitoring tools, it can considerably lower risk. Once monitoring is set up, you can detect many issues, for example data drift (when incoming data departs from the training data and the model’s performance drops sharply; during the COVID-19 pandemic, the majority of Machine Learning models broke down after exactly this kind of unexpected anomaly).

With MLOps in place, you can also scale your solution when you collect more data or need to serve 10x more users. It lets you soft-release new model versions, test them (e.g., A/B testing) or scale dynamically as required (which cuts operating costs when the volume of requests fluctuates). On top of that, you keep control over changes to datasets and experimental setups.

‍

MLOps practices

The entire MLOps paradigm revolves around its practices. Below, I will describe at which points of the Machine Learning pipeline these practices come into play:

  • Exploratory data analysis – investigate and prepare data for machine learning by building reproducible datasets, tables, and visualisations. The goal is to gain a better understanding of the data you intend to use in later steps.
  • Data preparation and Feature Engineering – build features from transformed data in an iterative way. At this stage, you want to set up a feature store, i.e. shared, easily accessible storage holding pre-computed features. A component like this lets teams share features with each other. Versioning the dataset or features is also essential for following good dataset practices and tracking how it changes.
  • Model training and hyperparameters tuning – look into open-source libraries such as Scikit Learn, PyTorch, or TensorFlow. If you need something simpler, AutoML is an option.
  • Model evaluation and governance – look into platforms such as MLFlow for managing model artifacts, versions, and transitions across the model’s lifecycle.
  • Model inference and serving – Keep track of inference times, request delays, and a host of other production-related QA concerns. Automate production-related pipelines with CI/CD tools such as orchestration and repositories.
  • Model deployment and monitoring – Automate permissions, route traffic, expose the model through REST API endpoints, and scale the service dynamically whenever required.
  • Automated model retraining – Once all the building blocks and monitoring tools are in place, you can introduce automatic retraining of the model whenever you notice a shift in the data, the model’s performance drops, or a new batch of data becomes available.

MLOps vs DevOps

MLOps drew heavy inspiration from DevOps, from its success, and from the positive impact it had on Software Engineering. Just as DevOps principles promote a fast and interactive way of building applications, MLOps applies similar rules to delivering machine learning models to production.

At the same time, MLOps has to deal with additional challenges. To begin with, MLOps is more experimental, since the resulting model is not probabilistic by definition. Given this, reproducibility is harder to ensure in Machine Learning than in Software.

Furthermore, Software teams consist of less technical roles such as UI/UX and PMs, plus more technical, code-oriented roles such as Software engineers. ML teams include all of these, with the technical staff further expanded by Data Scientists, Machine Learning Researchers, and Machine Learning Engineers, each concentrating on a different part of the solution while working closely with the rest.

Testing models is much harder and less predictable than testing Software. There is still no formal approach to designing ML tests, whereas in software it is a mature field split into distinct scopes (unit, integration, system and acceptance). ML systems additionally need a model validation step on top of all the software tests.

Automating deployment is harder in the ML world than in Software because of the many small choices Data Scientists make while training a model. To automate the whole process, you need to replicate their reasoning and add some kind of monitoring so that you don’t release models of doubtful quality.

ML systems may be affected by seasonal changes or by the gradual evolution of the incoming data, which rarely happens in Software systems. Model performance can deteriorate in more ways than in software systems, so this has to be taken into account when designing and running them. Such issues with shifting incoming data may result from:

  1. treating training data and production data differently.
  2. a seasonal mismatch between the time the model was trained and the time it was deployed. (e.g., a model trained on summer data may perform badly when winter snow arrives)
  3. incorrect assumptions made when collecting data again, which introduce bias into a retrained model.

Monitoring is harder than in classic software, since you also need to track all the statistics describing the model’s behaviour, set up alerts for when its performance drops, enable rollbacks to earlier versions, and define triggers for training a new model.

Zrzut%20ekranu%202024-06-13%20o%2012.25.17

MLOps or no MLOps

To better illustrate the impact of MLOps on a project, along with its issues and limitations, let’s take an example in which you want to forecast the price of a tech stock with a stock price prediction model.

With a simple model trained on the data at hand, we tune it against a fixed set of test data. ML projects typically look like this: you have a pipeline running in your Jupyter Notebook. In our case, we trained a well-performing model able to forecast the price of Tesla stock in 2021-2022 from historical data. Sadly, it cannot be used outside your Notebook, since it hasn’t been deployed to the cloud yet.

With MLOps, the process is more complex, but it is also far more useful.

  1. You want versioned datasets, because you know your system will behave like a living organism and evolve over time, so you version your data. This matters a great deal for the reproducibility of our ML solution: we may want to train model 122 on data up to the end of 08.2023, and the following month extend it to the end of 09.2023 and train model 123. Without data versioning, reproducing model 122 would be difficult.
  2. You rely on an experiment tracking tool, because you test many different hyperparameter combinations. In our example, hyperparameters cover every training-related decision: which architecture (XGBoost, SVR, or autoregressive network), which learning rate, how long to train, which augmentation to apply, etc. In the end, you want to pick the best-performing, stable solution out of many experiments and keep it under control, which requires experiment tracking.
  3. Users or other developers need access to your model, so you deploy your stock forecasting model in the cloud (e.g. Google Cloud or AWS) as a REST API.
  4. You need monitoring for when your model’s performance drops sharply, since bad trading calls could cost you money. In that case, you need to roll back to the previous version or fall back to a rule-based system.
  5. If the model performs badly, think about retraining it on more recent data. For that, you add monitoring, automated dataset creation, model training, and deployment. Done right, this lets the model keep performing very well over a long period.

‍

Summary

This article explained MLOps, the advantages it brings and its practices, and walked through a stock prediction model evolving from a basic setup with no MLOps into a complete MLOps system.