Home → Techniques and Tips → NeuralTools → Interpreting Incorrect% in Testing Report
15.24. Interpreting Incorrect% in Testing Report
My question is about the Detailed Report that NeuralTools produces when testing category predictions in a PNN. With a Good result, the Incorrect% is 100 minus the Prediction%, which makes sense to me. But with a Bad result, there doesn't seem to be a relationship between the Incorrect% and the Prediction%. What does the Incorrect% mean?
The Incorrect% is the sum of the probabilities for all the incorrect categories.
This is easier to understand if you "change Detailed Report settings to show the probabilities assigned by a Probabilistic Neural Net to every possible category for the dependent variable." Click Utilities » Application Settings. In the Reports section, click the drop-down menu at the end of the Columns in Detailed Reports row. A dialog box titled NeuralTools — Columns to Display in Detailed Reports will open. "In that dialog, select Probabilities of All Categories (for PNN) for Testing. Then train a PN Net on a data set with at least 3 categories in the dependent variable." (Source: NeuralTools 7 user manual, page 51, or topic "Training Reports" in the help file.)
Attached is part of the results from our standard example "Auto Loans 1a" (Help » Example Spreadsheets) with that application setting in effect. (Click the image to open it full size, in a new tab.)
Row 5 is probably the clearest: The correct answer was "late payments", but NeuralTools had computed there was only a 43.12% chance it was correct. NeuralTools picked "timely payments" because it thought there was a 54.69% chance that was correct. It computed a 2.19% probability for the third category, "default". The Incorrect% is 100% minus the probability of the correct answer, 100–43.12 = 56.88%. That same 56.88% is the sum of the probabilities of the incorrect answers, 54.69+2.19%. By contrast, for a Good result like row 6, Incorrect% is always 100% minus Prediction%, since the predicted answer was correct and all the incorrect answers were also not predicted.
For Good and Bad answers alike, Incorrect% is the total probability of all incorrect answers, which equals 100 minus the correct%. However, NeuralTools displays the Prediction%, not the correct%. If the answer was Good, the predicted answer was the correct answer, so the probability shown in Prediction% is the correct%. But for Bad answers, the prediction was not the correct answer; therefore, the correct% is not shown, the Prediction% is the probability of the one incorrect answer that had highest probability, and the Incorrect% is the total probability of all the incorrect answers.
Last edited: 2018-06-07