Title: Quantitative Evaluation of Neural Networks for NDE Applications Using The Roc Curve
Authors: Mackay A. E. Okure and Michael A. Peshkin
Keywords: performance evaluation, ROC method, probability of detection, neural networks, fundamentals of QNDE methods.
Issue Date: 1994
Abstract: The relative operating characteristic (ROC) method is applied to performance evaluation of neural networks. The study was motivated by the need to objectively evaluate neural networks for flaw waveform identification in NDE equipment, and to compare neural network performance with other methods. NDE applications are characterized by noisy real- world data, less-than-perfect detection and a serious problem of false alarm indications. The ROC method is explained by modeling neural network output as exponential probability distributions with two peaks, one near 1 (flaw) and one near 0 (no flaw). 100% POD (probability of detection) can only be achieved when the POFA (probability of false alarm) is also 100%, and if a POFA of 0% is required, the POD also falls to 0%. The ROC curve presents all intermediate performance information in an objective form and depicts the inevitable trade-off in every interpreter, human, neural, or otherwise. The ROC method is applied to the comparison of the performance of a neural network and a threshold-based scheme in classifying real-world eddy current data collected from an aircraft wheel NDE system.
Presented at:

Attachments

Attachment Name Attachment Type
Quantitative Evaluation of Neural Networks DOC PDF PS