Reinhard Lindner

Aug 16, 20234 min read

MLOps vs DevOps

You may have heard tons of information concerning DevOps practices. The latter has raised great interest among businesses. Google Trends show 100% user interest in DevOps for the past couple of years. Sencury also posted articles based on DevOps culture and Cloud-specific DevOps. However, do you know that DevOps has given insights to deliver better and faster within other engineering technologies? For example, MLOps, where ML stands for Machine Learning. It has a part of DevOps (Operations) in it.

Where do we use MLOps?What are its components and benefits for your business? When do you need MLOpsand what is its difference from DevOps practices? Let’s find out with Sencury!

What is MLOps?

MLOps is the shortened term for Machine Learning Operations. It stands at the core of Machine Learning engineering. The main goal of MLOps is to streamline machine learning models to production. And, also, track and maintain ML models throughout the whole process.

MLOps = Design +Model Development +Operations

MLOps Usage

Quality ML and AI solutions require using an MLOps approach. Basically, it adopts CI/CD practices. Here, ML models are being monitored, validated, and governed. Who adopts it? Data scientists and DevOps engineers, who work together to achieve success.

MLOps Components

There's no defined MLOps timeline in projects dedicated to machine learning technology. Moreover, it can cover the process from data pipeline to model production. Or any part of the process the project requires. E.g., model deployment only.

MLOps principles can be beneficial and are applicable in the following cases:

Exploratory Data Analysis (EDA)

Exploration, sharing, and data preparation in iterations for an ML lifecycle. This can be achieved via the creation of datasets, tables, and visualizations that can be reproduced, edited, and shared.

Data Preparation & Feature Engineering

Refined features require iterative data transformation, aggregation, and removal of repetitive information.

Model Training & Fine-tuning, RLHF (Reinforcement Learning from Human Feedback)

There are two options: via open-source libraries that can perfectly train and improve model performance. Or via automated ML tools including those available in major clouds (e.g. AWS SageMaker JumpStart). The latter can perform trial runs and create code that can be reviewed and deployed afterward.

Model Evaluation

Perform model evaluation in experimental and production environments. Consider:

Evaluation datasets to validate model performance
Multiple continuous training runs to track prediction performance across them
Performance comparison and visualization between different models
Using AI techniques (interpretable) for model output interpretation

Model Governance

ML lifecycle requires end-to-end tracking of model origins and versions and managing model artifacts and transitions.

Model Inference & Serving

Manage both how often a model can be refreshed and the inference request times. To automate the pre-production pipeline, use CI/CD tools such as repositories and orchestrators.

Model Optimization for Deployment

There are several options for a model to be optimized before deployment. For example, by

Data quantization. This is the process of compressing an AI model by reducing its high computational, storage, and energy requirements. In other words, every ML model has numerical representations that are being lowered to drastically decrease the model’s size. This allows computations to run more quickly and requires less memory. There are two ways to perform quantization: Quantification of weight and activation.
Model pruning. Pruning an ML model requires setting a 0 value to certain weights. This makes a model unable to overfit. There are various ways to prune a model. E.g., at the start prune a random number of weights. Or at the end of the training process to make a model lighter. The main idea of pruning is to keep a complex model architecture and increase the model's capacity (with as many interactions between features as possible), due to limiting its size.

Model Deployment & Monitoring

Put the actual model to the test by defining an actual use case. These may be Single-sample inference deployment, Batch deployment, and Deploying models onto edge devices. To get registered models into production faster, automate permissions and cluster creation. Then, enable REST API model endpoints.

Model Retraining (Automated)

Create alerts and automation to be able to correct any data deviations between model training and model inference.

There's no defined MLOps timeline in projects dedicated to machine learning technology. Moreover, it can cover the process from data pipeline to model production. Or any part of the process the project requires. E.g., model deployment only. MLOps principles can be beneficial and are applicable in the following cases: Exploratory data analysis (EDA) Exploration, sharing, data preparation in iterations for an ML lifecycle. This can be achieved via the creation of datasets, tables and visualizations that can be reproduced, edited and shared. Data preparation & feature engineering Refined features require iterative data transformation, aggregation, and removal of repetitive information. Model training & fine-tuning, RLHF (Reinforcement learning from human feedback) There are two options: via open-source libraries that can perfectly train and improve model performance. Or via automated ML tools including those available in major clouds (e.g. AWS SageMaker JumpStart). The latter can perform trial runs and create code that can be reviewed and deployed afterward. Model evaluation Perform model evaluation in experimental and production environments. Consider: Evaluation datasets to validate model performance Multiple continuous training runs to track prediction performance across them Performance comparison and visualization between different models Using AI techniques (interpretable) for model output interpretation Model governance ML lifecycle requires end-to-end tracking of model origins and versions and managing model artifacts and transitions. Model inference & serving Manage both how often a model can be refreshed and the inference request times. To automate the pre-production pipeline, use CI/CD tools such as repositories and orchestrators. Model optimization for deployment There are several options for a model to be optimized before deployment. For example, by Data quantization. This is the process of compressing an AI model by reducing its high computational, storage, and energy requirements. In other words, every ML model has numerical representations that are being lowered to drastically decrease the model’s size. This allows computations to run more quickly and requires less memory. There are two ways to perform quantization: Quantification of weight and activation. Model pruning. Pruning an ML model requires setting a 0 value to certain weights. This makes a model unable to overfit. There are various ways to prune a model. E.g., at the start prune a random number of weights. Or at the end of the training process to make a model lighter. The main idea of pruning is to keep a complex model architecture and increase the model's capacity (with as many interactions between features as possible), due to limiting its size. Model deployment & monitoring Put the actual model into the test by defining an actual use case. These may be Single-sample inference deployment, Batch deployment, and Deploying models onto edge devices. To get registered models into production faster, automate permissions and cluster creation. Then, enable REST API model endpoints. Model retraining (automated) Create alerts and automation to be able to correct any data deviations between model training and model inference.

Why is There a Need for MLOps?

Getting machine learning models into production is not an easy task. Mostly, due to the complex components of the ML lifecycle (see the picture above). Often, it requires hundreds of GPUs and weeks of time which constitute a serious cost constraint. It is a great challenge to keep the processes synchronized and work together to reach a goal. It requires extra accuracy and precision to do the task. Collaboration of DevOps, Data engineers, Data scientists, and ML engineers becomes critical as well. As MLOps includes experimentations, iterations, and continuous improvements of the ML lifecycle.

The biggest benefits of MLOps are its efficiency, scalability, and risk reduction.

Efficiency: faster results; faster production
Scalability: vast scalability and management of thousands of models at once
Risk reduction: greater transparency and faster response to risk events; policy compliance

Differences Between MLOps vs DevOps

MLOps focuses on data management and model versioning. In its turn, DevOps prioritizes overall application performance and reliability. Let’s see the full comparison of MLOps vs DevOps in the table below.

MLOps DevOps Goal Manageability of ML lifecycle SDLC process automation Objectives Model infrastructure setup and automation Business goals Motivation Continuous training and fine-tuning CI/CD, release management tooling Deliverables Model, training dataset, training configuration Code and integration, installation packages, binaries

Sencury on MLOps vs DevOps

MLOps is an important service provided by Sencury. It was shaped by leveraging the strengths and capabilities of our Data Science, Data Engineering, and DevOps specialists. It leverages an out-of-the-box consulting package for the cloud, on-premise, edge computing, or hybrid ML ecosystem. Contact us today to receive more details!