Confusion Matrix & Cyber Crime
The confusion matrix is a matrix used to determine the performance of the classification models for a given set of test data. It can only be determined if the true values for test data are known. The matrix itself can be easily understood, but the related terminologies may be confusing. Since it shows the errors in the model performance in the form of a matrix, hence also known as an error matrix. Some features of the Confusion matrix are given below:
- For the 2 prediction classes of classifiers, the matrix is of 2*2 table, for 3 classes, it is 3*3 table, and so on.
- The matrix is divided into two dimensions: predicted values and actual values along with the total number of predictions.
- Predicted values are those values predicted by the model, and actual values are the true values for the given observations.
- It looks like the below table:
The above table has the following cases:
- True Negative: Model has given prediction No, and the real or actual value was also No.
- True Positive: The model has predicted yes, and the actual value was also true.
- False Negative: The model has predicted no, but the actual value was Yes, it is also called a Type-II error.
- False Positive: The model has predicted Yes, but the actual value was No. It is also called a Type-I error.
Interpretation: You predicted positive and it’s true.
You predicted that a woman is pregnant and she actually is.
Interpretation: You predicted negative and it’s true.
You predicted that a man is not pregnant and he actually is not.
False Positive: (Type 1 Error)
Interpretation: You predicted positive and it’s false.
You predicted that a man is pregnant but he actually is not.
False Negative: (Type 2 Error)
Interpretation: You predicted negative and it’s false.
You predicted that a woman is not pregnant but she actually is.
Just Remember, We describe predicted values as Positive and Negative and actual values as True and False.
Calculations using Confusion Matrix:
We can perform various calculations for the model, such as the model’s accuracy, using this matrix. These calculations are given below:
Classification Accuracy: It is one of the important parameters to determine the accuracy of classification problems. It defines how often the model predicts the correct output. It can be calculated as the ratio of the number of correct predictions made by the classifier to all number of predictions made by the classifiers. The formula is given below:
- Misclassification rate: It is also termed as Error rate, and it defines how often the model gives the wrong predictions. The value of the error rate can be calculated as the number of incorrect predictions to all number of the predictions made by the classifier. The formula is given below:
- Precision: It can be defined as the number of correct outputs provided by the model or out of all positive classes that have been predicted correctly by the model and how many of them were actually true. It can be calculated using the below formula:
- Recall: It is defined as the out-of-total positive classes, how our model predicted correctly. The recall must be as high as possible.
- F-measure: If two models have low precision and high recall or vice versa, it is difficult to compare these models. So, for this purpose, we can use F-score. This score helps us to evaluate the recall and precision at the same time. The F-score is maximum if the recall is equal to the precision. It can be calculated using the below formula:
Other important terms used in Confusion Matrix:
- Null Error rate: It defines how often our model would be incorrect if it always predicted the majority class. As per the accuracy paradox, it is said that “the best classifier has a higher error rate than the null error rate.”
- ROC Curve: The ROC is a graph displaying a classifier’s performance for all possible thresholds. The graph is plotted between the true positive rate (on the Y-axis) and the false Positive rate (on the x-axis).