Breast Cancer classifier using the K-Nearest Neighbors (KNN) algorithm

Hi there! You probably know that October is Breast Cancer Awareness Month, and following my learning path I decided to write a simple-yet-powerful classifier that predicts whether a test result indicates a benign or malignant tumor using the K-Nearest Neighbors algorithm.

Introduction

I think the beauty of KNN is its simplicity. In Brazil we have a popular saying: “tell me who you’re with and I’ll tell you who you are.” The same happens in KNN: given a set of points in an n-dimensional space and a point X, we predict that the class of X will be the most predominant class among X’s K-Nearest Neighbors.

Being k the number of nearest neighbors by the euclidean distance of the point X: when K = 1 the predicted class for X might be a triangle; when K = 3 the predicted class for X might be a square (because two of the three nearest neighbors of X are squares).

Okay, but what happens when K = 2 and we have a draw? Square or triangle? Well, what would you do as a human? I guess I would just choose randomly. But that is something to consider when we use even values for K. We also need to fine-tune the K parameter well: small values of K (like k = 1) may lead to a very narrow prediction, while a large value may be too broad — and in both cases the algorithm may not generalize well.

Project repository

You can check out the project at marciojmo/breast-cancer-classifier. I’m using a breast cancer dataset from Kaggle (many thanks to Merishna for sharing).

Getting started

Let’s begin by loading and taking a look at the data. We have some characteristics of a tumor and its diagnosis, with 0 meaning a benign tumor and 1 meaning a malignant one.

We can also see that the feature values vary a lot in scale — for example, mean_area rounds around 10³ and mean_smoothness rounds around 10⁻¹. If these features were not normalized, then when calculating the euclidean distance the mean_area would be considered more important than the mean_smoothness, leading KNN to rely more on the former. Normalization ensures all features are mapped to the same range of values and therefore have the same importance.

Normalizing data

Let’s proceed by putting our features and our target classes into variables X and Y, and then normalize our features using the fit_transform method of the MinMaxScaler from the sklearn.preprocessing module. The MinMaxScaler ensures all features range between 0 and 1, therefore having the same relevance when calculating the euclidean distance.

Finding the best K

How do we find the best K? Let’s just try every value from 1 to 100 and see which one performs best on our dataset. But first we need to split our data into training and testing, using the train_test_split method from the sklearn.model_selection module.

Now let’s create a list of values ranging from 1 to 100 and iterate over it. We use each of these values as the number of nearest neighbors (k) and train a new KNN classifier on the training data. At the end of each iteration we calculate the accuracy of the model using the test data and store it in a list called accuracies. We can then plot the accuracies for each value of k to get a visual sense of what is happening, and grab the best value of k.

Making a prediction

What is the use of technology if it is not (well) applied? Let’s create a randomized test case and make a prediction!

We begin by retraining the classifier with the best value of K and storing the accuracy for later use. Then I generate a random test case using the features’ min and max values as reference: for every feature that is not the “diagnosis” (our target class), we sample from a uniform distribution between the feature’s min and max value.

Important: remember that we trained our classifier using normalized data, so we also need to normalize this test case before making any prediction. We use the same feature scaler to normalize our test case and then predict and print out the result.

Conclusion

KNN is a classifier simple to understand, and even so it achieves very good results. We just created a classifier using KNN that can tell whether a tumor is benign or malignant with 92% accuracy using a dataset containing only around 600 examples. We also understood the importance of feature scaling in this algorithm and a way to fine-tune the best value of k.

May the code be with you!

The original article includes code screenshots and plots — check the Medium version or the repository for the full code.