Can AI Diagnose Breast Cancer?

This project follows the Scientific Method. Review the steps before you begin.

Setting Up the Google Colab Environment

1. You will need a Google account. If you do not have one, make one when prompted.
2. Download the breastcancer.ipynb file from Science Buddies.
3. Upload the file to Google Colaboratory (you will need to sign in to your Google account at this point or make an account).
4. Read the Troubleshooting Tips and How to Use This Notebook sections. Follow the instructions you find there.
5. Run the code block under Importing Libraries, to bring in all of the functions.

Preprocessing the Dataset

Preprocessing a dataset is a critical step in the machine learning process. It involves preparing the raw data before feeding it into a machine learning algorithm. Preprocessing serves several important purposes, and we will explain and demonstrate one at a time.

1. Dropping Features: First, we will drop features that we think will be uninformative for modeling. In this case, we will be dropping the ID column. The patient ID is usually a unique identifier assigned to each patient. It doesn't carry any inherent predictive value and is simply an arbitrary identifier. We have provided the code to delete certain columns from our Pandas DataFrame—a two-dimensional and highly flexible data structure provided by the Pandas library in Python. Run the code and confirm that the ID column has been deleted from the dataframe.
2. Normalizing/Scaling Features: In KNN, the algorithm relies on measuring distances between data points to make predictions. To ensure accurate results, it is important that all features contribute fairly to these distance calculations.If features are not scaled or normalized, those with larger numerical ranges can unfairly dominate the distances.

   Watch this video to learn more about scaling data:

   

   https://www.youtube.com/watch?v=n9KeJLGwW0U

   For instance, if one feature spans ages from 0 to 100, and another spans salary from 10k to 50k, the salary's larger range could overshadow the age feature in distance calculations.

   1. Normalization: This is the process of scaling all the values in a dataset to a similar range. The goal is to bring the values of different features or variables to a common scale so that they are directly comparable and do not introduce bias or distortion in the analysis. There are several techniques used to normalize data, and we'll be using Min-Max scaling to bring the values between 0 and 1.
   2. Scaling: In essence, scaling ensures that no single feature overwhelms the distance calculations. This enhances the algorithm's fairness in considering all features, resulting in more accurate predictions.

   As with the previous step, we have provided the code to normalize certain columns from our Pandas DataFrame. Add in the names of the columns that we will be normalizing. We will be normalizing our numerical variables, which in our case would be everything but Diagnosis. Add in these names to the list specified by the comment in the code. Then, run the code in the cell.

3. Encoding Categorical Variables: Encoding categorical variables is essential when working with machine learning algorithms, because most algorithms, including KNN, require numerical input. Categorical variables, which represent qualitative attributes like gender or group, cannot be used directly in their original form because they lack a numerical representation that algorithms can process.
   1. Label encoding: This involves converting categorical values into integers. Integers are whole numbers, both positive and negative, without any decimal or fraction components (for example, -1, 5, and 42). Each category is assigned a unique integer value.

      In our case, the only variable that requires encoding is Diagnosis. Because there are only two possible outcomes for diagnosis (B for benign and M for malignant), label encoding would assign 0 to benign and 1 to malignant.

   We have provided the code to label encode our Diagnosis feature. Run the code in the cell and pay attention to how that changes the values in our dataframe!

4. Splitting the Training and Testing Data: Splitting data into training and testing sets is crucial in machine learning. By doing so, you assess your model's performance on new, unseen data, ensuring its ability to generalize beyond training examples.

   Watch this video to learn more about why we split datasets.

   

   https://www.youtube.com/watch?v=dSCFk168vmo

   We have provided the code to split the dataset into training and testing portions. Take note of how X and y look following this step, as well as the dimensions of X_train, X_test, y_train, and y_test. The numbers inside the parentheses represent the number of rows and the number of features in each set.

Training the Model

1. We have provided the code to make a KNN classifier with a specified number of neighbors, where we have chosen 5 as the default number. This code then trains the classifier using the training data. Afterward, it calculates how accurate its guesses are.
   1. Accuracy is a measure used to evaluate the performance of a machine learning model. It represents the proportion of correctly predicted outcomes or labels compared to the total number of instances in the dataset. In simpler terms, accuracy tells you how often the model's predictions are correct.

      Mathematically, accuracy is calculated as:

      $$accuracy = \frac{number\ \:of \:correct \:predictions}{total\: number\: of\: predictions}$$
   2. Precision is another measurement used to evaluate the performance of a machine learning model, and it focuses on the accuracy of positive predictions by the model. It answers the question: Of the instances the model predicted as positive, how many are actually positive?

      Mathematically, precision is calculated as:

      $$precision = \frac{true\: positives}{true\: positives\: +\: false\: positives}$$
   3. Recall is yet another measurement used to evaluate the performance of a machine learning model. Recall measures the ability of the model to correctly identify all positive instances. It answers the question: Of all the actual positive instances, how many did the model predict correctly?

      Mathematically, recall is calculated as:

      $$recall = \frac{true\: positives}{true\: positives\: +\: false\: negatives}$$

   In these formulas:

   • True Positives (TP) are the cases where the model correctly predicted the positive class
   • True Negatives (TN) are the cases where the model correctly predicted the negative class
   • False Positives (FP) are the cases where the model incorrectly predicted the positive class when it was actually negative
   • False Negatives (FN) are the cases where the model incorrectly predicted the negative class when it was actually positive

Visualize the Model

1. Before graphing, we must reduce the dimensionality of the data. Dimensionality refers to the number of different features used to describe each item in the dataset. There are many methods for reducing dimensionality, and here we will use a technique called Principal Component Analysis (PCA).

   Watch this video to learn more about PCA:

   

   https://www.youtube.com/watch?v=HMOI_lkzW08
   1. In the dataset, there are multiple features (dimensions) that describe each instance. In our breast cancer dataset, we have features like radius, texture, perimeter, etc. Visualizing with more than three dimensions is challenging because we cannot easily represent such high-dimensional spaces on a two-dimensional surface like a graph.
   2. PCA helps address this issue by transforming the original features into a new set of features called principal components. These principal components are a linear combination of the original features and capture most of the variability in the data. By using just the first few principal components, we can effectively reduce the dimensionality of the data while retaining as much meaningful information as possible.
   3. In the provided code, PCA is used to reduce the dataset to just two principal components. This means that instead of visualizing the data in its original high-dimensional space, we're now plotting the data points in a two-dimensional space defined by the two most important principal components. A decision boundary is the dividing line or surface that separates different classes in a classification model. This makes it possible to create a clear and interpretable graph of the decision boundary of the KNN classifier. 
2. The graph generated by the provided code visualizes the KNN classifier's decision boundary in two-dimensional space. This decision boundary represents how the KNN classifier separates different classes based on the data features.
   1. The color blue is Class 0, corresponding to the samples that are benign, and the color orange is Class 1, corresponding to the samples that are malignant.
   2. The blue and orange background mean that if a new point is in blue, then the model will predict that the new point is a benign tumor. If a new point is in orange, the model will predict it is malignant.
   3. If you see an orange point in the blue background, or a blue point in an orange background, the model has misclassified those points. 
3. We will be comparing different neighbor sizes using a loop that spans from 1 to 21 neighbors. We have the starter code under the graph of the decision boundary, and under each comment we have provided the pseudocode for each step in the loop. (Hint: The code will be very similar to the code in the Training the Model section.)

Evaluating the Model

1. Can you explain the difference between accuracy, precision, and recall?
   1. Which metric do you think is the most important in the case of diagnosing breast cancer patients, when there are high costs for false positives (a patient who has a benign tumor gets unnecessary treatments or interventions) and false negatives (a patient has a malignant tumor but is not diagnosed and therefore does not receive treatment)?
2. Describe what the graph predicting the KNN decision boundary using PCA-reduced data is showing.
3. Can you explain the relationship between the regions on the graph and the predictions made by the KNN model?
4. How does the graph help us understand the performance and behavior of the KNN classifier?
5. Based on the graph illustrating the contrast in performance metrics across varying neighbor counts, which value of k seems to optimize accuracy, precision, and recall? What do you think is the reason this neighbor size is optimal?

Experimenting with Weighted KNN (Optional)

Weighted KNN is a variation of the K-Nearest Neighbors algorithm used for classification tasks in machine learning. In standard KNN, all neighboring data points have an equal say in the classification decision. However, in weighted KNN, different neighbors contribute differently to the decision-making process based on their proximity or other factors.

1. Feature Selection: We can select a subset of features that we would like to assign more weight to. There are many feature selection techniques, but we will be using Univariate Feature Selection, which uses statistical tests to select the most relevant features.
   1. ANOVA (Analysis of Variance) F-statistic is a statistical test used to compare the means of two or more groups to determine whether there are statistically significant differences among them. In the context of feature selection in machine learning, the ANOVA F-statistic is often used to assess the relationship between individual features (variables) and a categorical target variable.

   We have provided the code to perform feature selection. Run these code cells, then complete the project as we did for regular KNN. How do regular KNN and weighted KNN compare?