AI News, QA – Metamorphic Testing for Machine Learning Models

QA – Metamorphic Testing for Machine Learning Models

In this post, you will learn about how metamorphic testing could be used for performing quality control checks/testing on machine learning models. The post is primarily meant for data science (QA) specialists to plan the test cases to test the machine learning (ML) model implementation from QA perspective.

Given the example discussed in the above diagram, a set of test cases can be planned (test plan) in the following manner: For coming up with the metamorphic test plan for doing quality checks on ML models, the following would be needed: Let’s try and understand as to why we need to perform metamorphic testing for doing quality checks on ML models.

or “test oracle”.  The presence of test oracle in software application development is a frequently invoked assumption saying that that output of an implementation could be verified against expected values by testers or testing mechanisms such as automated test programs.

expected values, there needs to be some mechanism to test the output of the ML models. However, when “testing” the model is spoken about, it is often referred with the scenario during the development (model building) phase when data scientists test the model performance by comparing the model outputs (predicted values) with the actual values.  This is not same as testing the model for any input where the expected value is not known.

As described in the earlier section, once one or more metamorphic relations are determined, it becomes simpler to come up with test cases which could be used to verify the correctness of ML models predictions.  These test cases could be made part of the automation based on the following: The automation could be achieved with scripting and programming languages.

DEF CON 24 - Amro Abdelgawad - The Remote Metamorphic Engine

As a matter of fact, it is all about time to reverse engineer the most complex piece of code. Code complicity techniques are usually used just to increase the time ...

DEF CON 24 - Joe FitzPatrick, Joe Grand - 101 Ways to Brick your Hardware

Spend some time hacking hardware and you'll eventually render a piece of equipment unusable either by accident or intentionally. Between us, we've got ...

DEF CON 24 - Amro Abdelgawad - The Remote Metamorphic Engine

As a matter of fact, it is all about time to reverse engineer the most complex piece of code. Code complicity techniques are usually used just to increase the time ...

Delta Debugging - Software Testing

This video is part of an online course, Software Testing. Check out the course here:

System Security

Cryptography and Network Security by Prof. D. Mukhopadhyay, Department of Computer Science and Engineering, IIT Kharagpur. For more details on NPTEL ...

Potential and Kinetic Energy Lesson for kids

A simple cartoon film ,consisting simple explanation on the difference between Potential and Kinetic energy.Potential energy is the stored energy in an object ...

Catena (soil) - Video Learning - WizScience.com

A "Catena" in soil science is a sequence of types of soil down a hill slope. Each soil type or "facet" differs somewhat from its neighbours, but all occur in the same ...

Symposium on Architecture: “Anachronometrics”

Anachronometrics” is a neologism denoting an act of temporal displacement in which one seizes on the future or past as a point of comparison, to emphasize ...

DEF CON 24 - Tim Estell and Katea Murray - NPRE: Eavesdropping on the Machines

After the Rise of the Machines they'll need to communicate. And we'll need to listen in. The problem is that proprietary protocols are hard to break. If Wireshark ...