• Home
  • Archive
  • Tools
  • Contact Us

The Customize Windows

Technology Journal

  • Cloud Computing
  • Computer
  • Digital Photography
  • Windows 7
  • Archive
  • Cloud Computing
  • Virtualization
  • Computer and Internet
  • Digital Photography
  • Android
  • Sysadmin
  • Electronics
  • Big Data
  • Virtualization
  • Downloads
  • Web Development
  • Apple
  • Android
Advertisement
You are here: Home » MLOps Lifecycle and Evaluation of Model Performance

By Abhishek Ghosh November 16, 2021 7:11 pm Updated on November 17, 2021

MLOps Lifecycle and Evaluation of Model Performance

Advertisement

Data science, machine learning (ML) and artificial intelligence have triggered a real hype in recent years and received a lot of attention in the industry. Machine learning methods are used to increase either the productivity of the users or the interactivity of the application. Numerous data science teams spend their time training machine learning models.
However, we observe two types of problems that arise in practice.
In this article, we have tried to explain how continuous delivery works for machine learning models and the basics of evaluating model performance. Above all, how to accommodate data engineering, model engineering and software engineering pipelines in a CI/CD pipeline. In addition, we want to show how to automate the manual process of deploying ML models using DevOps practices.

Table of Contents

  • 1 Introduction
  • 2 MLOps Lifecycle
  • 3 Operationalization of Machine Learning
  • 4 Model-performance Measures
  • 5 Starting a Project
  • 6 Discussion

Listen to the audio of this article :

Advertisement

---

Abhishek Ghosh 15 · MLOps Lifecycle and Evaluation of Model Performance

Either the majority of ML models do not manage to be integrated into a software product, or the model deployment takes too much time. We have identified several reasons that explain this problem:

  • ML Model Deployment is a complex process. In general, this involves the management of the three pipelines Data Engineering, Model Engineering and Software Engineering.
  • There are no standardized processes to bring an ML model into the production environment. Machine Learning Model Operation Management (MLOps) is still in its early stages.
  • Defining the right infrastructure stack to automate machine learning deployments currently requires a trial-and-error process. Many tools and systems for machine learning and AI model monitoring are in an active commercial offering phase.
  •  

    MLOps Lifecycle

     

    The implementation of machine learning in software systems takes place in few phases.

    Design (Problem Definition)

    Even the best ML solution will fail if it does not solve usage or business problems. Therefore, in this first phase, we try to find out who the users of our software are. What problems do they have? Which of these problems are best solved with ML? And most importantly, can we accept a nondeterministic solution through ML? Let’s take as the example software for billing professional travel expenses. This software could automatically detect some metadata fields, such as the category of a document, and thus facilitate the process of travel expense reporting.
    This phase is crucial for the success of an ML project. However, it may take some time until the right problem is found.

    Model Development (Data Science Research)

    After we have analyzed the workflows in software usage and identified use cases (tasks) suitable for ML, we can convert these use cases into ML projects. You then start the so-called Data Science Research. In this second phase, you go through a series of processes iteratively:
    The mapping of the user problem to the ML algorithms. You analyze and identify which ML algorithm is best suited to solve the problem. In our example, we chose unsupervised ML because category recognition is a classification task.

    • Data engineering, in which you first analyze whether the necessary data is available or what effort we have to make to obtain the data and provide it with labels (for training algorithms). In addition, data engineering can include other steps such as data integration, preparation, cleaning and validation.
    • Model engineering, in which different classification algorithms are trained on the data and thus different ML models are produced. By evaluating the performance, it is decided which ML model is best suited for our problem.

     

    Operationalization of Machine Learning

     

    Training the ML models can take a lot of time and be challenging, but the real challenge is integrating an ML system into the production environment, i.e. into the software product that interacts with users. An ML system consists of three main elements: training data, ML model and code for model training. We use the DevOps principles for ML systems (MLOps) to combine ML development and operations. As an extension of DevOps, MLOps is dedicated to automating and monitoring (in all steps) the integration of ML systems into software projects.

     

    Model-performance Measures

     

    The measures may be applied for several purposes, including:

    1. Model evaluation: a developer may want to know how good the model is, i.e., how reliable are the model’s predictions or how frequent the errors or the expected magnitude of errors;
    2. Model comparison: a developer may want to compare two or more models to offer a client option to choose between them;
    3. Out-of-sample and out-of-time comparisons: a developer may want to check a model’s performance when applied to new data from the client to evaluate whether performance has been worsened.

    Most of the commercially available model-performance measures are based on the comparison of the model’s predictions with the values in a provided dataset. In real life predictions and the dependent-variable values are not equal and hence for professional service delivery, and we want to quantify the disagreement.

    MLOps Lifecycle and Evaluation of Model Performance

    Click here to view the full size image

     

    Discussion

     

    The goal of this article was to direct the self-employed professionals towards the upcoming measures to offer better service to better clients.

    Normally, we use rapid application development (RAD) tools for the experimental phase. The main focus of RAD tools is on creating ML prototypes and functional requirements. Very popular are e.g. Jupyter notebook or cloud-based environments such as Colab from Google. There are several advantages and disadvantages to using RAD tools. The data was prepared, new features were derived and various machine learning models were trained. But how a developer will evaluate the performance of these models? For this purpose, so-called model performance model monitoring is being introduced in the machine learning area.

This Article Has Been Shared 804 Times!

Facebook Twitter Pinterest

Abhishek Ghosh

About Abhishek Ghosh

Abhishek Ghosh is a Businessman, Surgeon, Author and Blogger. You can keep touch with him on Twitter - @AbhishekCTRL.

Here’s what we’ve got for you which might like :

Articles Related to MLOps Lifecycle and Evaluation of Model Performance

  • Social Impact of Big Data : Part 1

    It is needless to say that discussions, thought on social impact of Big Data has some practical need by both the IT & non-IT individuals.

  • How To Learn Big Data For Beginners

    Many Are Interested to Learn Big Data Analytics and Other Works But Clueless Where From to Start. Here is How To Learn Big Data For Beginners.

  • How to Install Apache Ignite on Ubuntu Server

    How to Install Apache Ignite on Ubuntu Depends on the Purpose. Apache Ignite is a distributed database, also is a caching and processing platform.

  • Why Can Industries Benefit From Blockchain?

    A blockchain makes it possible to transmit information in a tamper-proof manner using a decentralized database shared by many participants so that copies are excluded. The database is also known as a distributed ledger. It is stored on many computers in a peer-to-peer network, with each new node taking over a full copy of the […]

  • What is Data Fusion?

    Data fusion is the process of merging and completing incomplete data sets. It is an important part of information integration. Data in a recipient record is supplemented with the help of a donor record. The donor record consists of variables and the recipient record from variables. The variables are therefore present in both data sets. […]

Additionally, performing a search on this website can help you. Also, we have YouTube Videos.

Take The Conversation Further ...

We'd love to know your thoughts on this article.
Meet the Author over on Twitter to join the conversation right now!

If you want to Advertise on our Article or want a Sponsored Article, you are invited to Contact us.

Contact Us

Subscribe To Our Free Newsletter

Get new posts by email:

Please Confirm the Subscription When Approval Email Will Arrive in Your Email Inbox as Second Step.

Search this website…

 

Popular Articles

Our Homepage is best place to find popular articles!

Here Are Some Good to Read Articles :

  • Cloud Computing Service Models
  • What is Cloud Computing?
  • Cloud Computing and Social Networks in Mobile Space
  • ARM Processor Architecture
  • What Camera Mode to Choose
  • Indispensable MySQL queries for custom fields in WordPress
  • Windows 7 Speech Recognition Scripting Related Tutorials

Social Networks

  • Pinterest (24.3K Followers)
  • Twitter (5.8k Followers)
  • Facebook (5.7k Followers)
  • LinkedIn (3.7k Followers)
  • YouTube (1.3k Followers)
  • GitHub (Repository)
  • GitHub (Gists)
Looking to publish sponsored article on our website?

Contact us

Recent Posts

  • Projector Screen Basics February 6, 2023
  • What is Configuration Management February 5, 2023
  • What is ChatGPT? February 3, 2023
  • Zebronics Pixaplay 16 : Entry Level Movie Projector Review February 2, 2023
  • What is Voice User Interface (VUI) January 31, 2023

About This Article

Cite this article as: Abhishek Ghosh, "MLOps Lifecycle and Evaluation of Model Performance," in The Customize Windows, November 16, 2021, February 6, 2023, https://thecustomizewindows.com/2021/11/mlops-lifecycle-and-evaluation-of-model-performance/.

Source:The Customize Windows, JiMA.in

PC users can consult Corrine Chorney for Security.

Want to know more about us? Read Notability and Mentions & Our Setup.

Copyright © 2023 - The Customize Windows | dESIGNed by The Customize Windows

Copyright  · Privacy Policy  · Advertising Policy  · Terms of Service  · Refund Policy

We use cookies on our website to give you the most relevant experience by remembering your preferences and repeat visits. By clicking “Accept”, you consent to the use of ALL the cookies.
Do not sell my personal information.
Cookie SettingsAccept
Manage consent

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
Necessary
Always Enabled
Necessary cookies are absolutely essential for the website to function properly. These cookies ensure basic functionalities and security features of the website, anonymously.
CookieDurationDescription
cookielawinfo-checkbox-analytics11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Analytics".
cookielawinfo-checkbox-functional11 monthsThe cookie is set by GDPR cookie consent to record the user consent for the cookies in the category "Functional".
cookielawinfo-checkbox-necessary11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookies is used to store the user consent for the cookies in the category "Necessary".
cookielawinfo-checkbox-others11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Other.
cookielawinfo-checkbox-performance11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Performance".
viewed_cookie_policy11 monthsThe cookie is set by the GDPR Cookie Consent plugin and is used to store whether or not user has consented to the use of cookies. It does not store any personal data.
Functional
Functional cookies help to perform certain functionalities like sharing the content of the website on social media platforms, collect feedbacks, and other third-party features.
Performance
Performance cookies are used to understand and analyze the key performance indexes of the website which helps in delivering a better user experience for the visitors.
Analytics
Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics the number of visitors, bounce rate, traffic source, etc.
Advertisement
Advertisement cookies are used to provide visitors with relevant ads and marketing campaigns. These cookies track visitors across websites and collect information to provide customized ads.
Others
Other uncategorized cookies are those that are being analyzed and have not been classified into a category as yet.
SAVE & ACCEPT