Top Machine Learning Interview Questions and Answers

Machine Learning is a branch of Artificial Intelligence that allows computers to learn patterns from data and make predictions or decisions without being explicitly programmed. Instead of following hard-coded instructions, Machine Learning enables the computer system to adapt through experience and make improvements over time.

 Types:

Supervised learning: Learn from labeled data. For example, in spam detection, the model learns from emails labelled as “spam” or “not spam” to detect new spam emails.

Unsupervised learning: Find patterns in unlabeled data. For example, a business groups customers by behaviour (e.g., spending habits) to improve marketing.

Reinforcement learning: Learn from trial and error, using rewards and penalties. For example, a machine learns to drive a car by staying in lanes and avoiding obstacles, improving over time.
 

Overfitting in machine learning happens when a model learns the training data too well, covering the noise and details. Therefore, it performs well with training data but poorly on new, real-time (unseen) data.

Example: Imagine a model that memorizes the training data but does not do well predicting new data.
Assume you’re teaching a child to identify dogs. If you show only pictures of small black dogs, the child may think that only small black animals are dogs. So when you show a big brown dog later, the child might not identify it. That’s overfitting. Too focused on specific training examples.

Prevention:

• Increase the amount of training data
• Use Regularization techniques (L1/L2)
• Prune decision trees
• Do Cross-Validation
• Do early stopping in training

The bias-variance tradeoff is about balancing the two types of errors:
Bias: error due to incorrect assumptions (model is too simple)
Variance: error due to being sensitive to training data (model is too complex)

Example:
When predicting house prices, a simple linear model (like a straight line) might ignore important factors and make wrong guesses. This is high bias. On the other hand, a very complex model might fit every small detail in the training data, even the noise. This is high variance. The key is to find a middle ground where the model is just right. Not too simple, not too sensitive. So it works well on new data too.

A confusion matrix is a performance measurement tool used in machine learning classification to assess a model’s ability to correctly predict an outcome. It displays counts of correct positives (true positives), correct negatives (true negatives), incorrect positives (false positives), and incorrect negatives (false negatives).
A confusion matrix is useful because it provides a picture of a model’s accuracy, precision, recall, and performance, especially in cases of imbalanced datasets.

Example: 
Suppose we test a model on 10 fruits. Let there be 6 Apples and 4 Oranges. Here’s the confusion matrix with the prediction.

So, among 6 Apples, the model predicted 5 Apples correctly and missed 1.
Out of 4 Oranges, the model predicted 3 Oranges correctly and missed 1.
Hence, correct predictions = 5 Apples + 3 Oranges = 8 Correct out of 10.
Wrong predictions = 1 Apple + 1 Orange = 2 wrong out of 10.
A confusion matrix helps to identify exactly where the model is making mistakes, as the model sometimes confuses Oranges with Apples and vice versa.
Use: Helps in calculating accuracy, precision, recall, etc. 

Cross-validation is a supervised machine learning process in which we can assess how well the model performs on unseen data. Cross-validation involves splitting the data into several parts, training the model on several parts, and testing on the others. 
Cross-validation can improve model accuracy, help identify overfitting, and make the model generalize better for new data.

Example:
In 5-fold cross-validation, you split the data into 5 parts (folds), train the data on 4 folds, and test on the remaining fold 5 times.
 

Classification: Make predictions about which category/Label something belongs to.
Example: Predict if an email is spam or not. 
Algorithms: Logistic Regression, Decision Trees, SVM (classification), etc.
Regression: Make predictions about numeric values.
Example: Predict the price of a house. 
Algorithms: Linear Regression, Decision Trees, SVR, etc.
 

Feature scaling is an important data preprocessing technique in machine learning that involves normalizing or standardizing the input features of the dataset so the features are processed on the same scale (meaning one feature doesn't dominate others because of its range or units).
Feature scaling is crucial for improving the performance and accuracy of models, especially for algorithms such as SVM, KNN, and gradient descent-based models.

Common Techniques:

• Min-Max Scaling (Normalization): Scales data to a fixed range, usually [0, 1].
• Standardization (Z-score Scaling): Centers the data to have a mean of 0 and a standard deviation of 1.

Example 1 

Imagine two features.
Height (cm) ranges from 150 to 200
Weight (kg) ranges from 50 to 100
If feature-scaling is not done, height would dominate as its values are larger. After scaling (Min-Max scaling or Standardisation), both features will be in the same range from 0 to 1 or mean 0 to standard deviation 1, which makes them equally important to the model.

Example 2
If age is a feature (20-60) and income is a feature (10,000-100,000), scaling the age and income features will put both on a similar scale.

Accuracy: Correct predictions / Total predictions
Precision: Correct positive predictions / Total positive predictions
Recall: Correct positive predictions / Actual positive predictions
F1-Score: Trade-off between precision and recall
Confusion Matrix: A table that shows correct and incorrect predictions for each class.

Example:
Consider a model that classifies emails as Spam or Not Spam.
If the model predicts 90 emails correctly out of 100, the Accuracy = 90%
If the model flags 50 emails as Spam, but only 40 were the Spams actually, Precision = 40/50 = 80%.
If there were 45 Spam emails in total, and the model catches 40 Spam emails, Recall = 40/45, nearly 89%.
F1 Score balances 80% and 89%.

The Curse of dimensionality refers to issues related to dimensionality in machine learning and is an issue that occurs in high-dimensional spaces. The more features you have, the sparser the data is going to be. Sparse data means less chance to find patterns and generalize.
This can result in: overfitting, increased computation time, and lower performance in your model, especially with algorithms like KNN that rely on distance.
For example, consider finding your friend in a small room (2D space). It’s easy. Now think about finding them in a huge building with many floors and rooms (higher dimensions). This will be more difficult.
In Machine Learning, more dimensions mean the data spreads out too much and makes the models less accurate and harder to train.

Training Data: Used to train the model and adjust weights/parameters. 
Testing Data: Used to evaluate the performance of the model on unseen data.

Example 1
Train a spam detector on 80% of emails, then test on the other 20%.

Example 2
If you have 100 photos of cats and dogs, use 80 photos to train the model (training data). Use 20 new photos to test the model (testing data) and find out if the model can identify them correctly. 

The Gradient Descent algorithm is an optimization algorithm to find the best values for a model by reducing errors step by step.
The model starts with random values(weight) and then calculates the error (difference between the prediction and the actual result). Next, it adjusts the values a little to reduce the error. This process repeats many times until the error is minimized.

Example 1
Consider rolling a ball down a hill to reach the lowest point (minimum error). With each step, the ball moves in the direction that lowers the height (error) the most. Similarly, Gradient Descent updates the model in small steps to reduce error.

Example 2 
Consider walking down a hill. You can feel the slope beneath your feet, and you take steps downhill until the slope is completely flat (minimum error). 

Bagging (Bootstrap Aggregating): constructs multiple models in parallel on random data samples. It primarily aims to reduce variance.
Example: Random Forest.  
Boosting: constructs sequentially, whereby each new model aims to correct the errors of the previous. It reduces bias and variance.    
Example: AdaBoost, XGBoost.
Practical example: If you are guessing the winner of a game,
Bagging: Asks 10 friends separately and takes a majority vote.
Boosting: First, ask one friend, then ask another friend to correct the first friend’s mistakes, and continue improving with each friend.
 

Regularization prevents overfitting by adding a penalty to the model for being too complex (having too many large weights). This compels the model to keep everything simple and focus on important patterns instead of memorizing the training data.

Types

L1 Regularization (Lasso)
It adds the absolute value of weights as a penalty. This can make some weights exactly zero, helping in feature selection (ignoring unimportant features).

L2 Regularization (Ridge)
It adds the square of weights as a penalty. This keeps all weights small but rarely makes them zero. It helps reduce the impact of less important features.
For example, consider a model predicting house prices. Without regularization, it uses all features (even noisy ones like “color of the door”) and overfits.
With L1, it may completely ignore “color of the door” by making its weight zero.
With L2, it will reduce the influence of “color of the door” by making its weight small, but not zero. 

Parametric: A parametric model has a fixed number of parameters that work well with less data. For example, Linear Regression models data with straight-line equations. 

Non-parametric: A non-parametric model has no fixed form. It expands as you add more data. For example, the K-nearest neighbors (KNN) model predicts based on actual data points.

Assume you have a mix of red and blue dots on a graph. When a new point comes up, KNN checks the closest ‘K’ dots around it like its nearest neighbors and assigns the new point the most common colour among them. There’s no fixed formula. The model just depends on the data’s original distribution to make decisions. 

• Resample (oversample the minority class or undersample the majority class)
• Use different metrics (e.g., F1-Score or AUC)
• Experiment with algorithms that perform well on imbalanced datasets (e.g., Random Forest with class weights

Example 1
In fraud detection (where fraud cases are rare), oversample the fraud examples.

Example 2
You have 900 cat photos and 100 dog photos. You can make more dog copies (oversample) or reduce cat photos (undersample) so the model treats both more fairly.

PCA is a technique to reduce the number of features (columns) in the dataset while keeping the most important information. It finds new features called principal components that capture most of the variation or patterns in the data.
It is used to simplify complex datasets, to make data visualization easier (like converting from many dimensions to 2D or 3D), and to speed up machine learning models by reducing features.

Example 1
From 100 features, PCA may reduce it to 10, making the model faster and less prone to overfitting.

Example 2
We have students' data with 10 exam scores. PCA can reduce this to 2 or 3 new features that still represent their overall performance. Therefore, it becomes easy to analyze or visualize. 

The Kernel Trick helps SVM handle complex data by moving it to a higher dimension, where the data points can be separated with a straight line.
For example, two rings in 2D (points inside the ring and points outside) can’t be separated with a straight line. But after applying the Kernel Trick and moving the data to 3D, they become easy to separate.

Batch GD: Uses the entire data set for each stage of the optimization. Slow, but stable. 
Stochastic GD (SGD): Uses one data point at a time. Fast, but has "noisy" updates.
Mini-Batch GD: It processes small chunks of data at a time, finding a good balance between fast updates and stable learning.

Decision Trees separate the data by conditions on features before coming to a decision. 

Example: In a decision tree for loan approval: 
First, check income > threshold → then check credit score → then make a decision.
 

The three categories of feature selection procedures are.

• Filter-based approaches (e.g., correlation),
• Wrapper-based approaches (e.g., Recursive Feature Elimination),
• Embedded techniques (e.g., Lasso regularization).

For example, removing features that are of low importance before training a model. 

CNNs are deep learning model frameworks that are able to distinguish patterns in images by using filters (convolutions) to capture edges, textures, and shapes.

Example: Facial recognition apps, image classification (cats vs dogs).

RNNs process data sequentially, keeping track of previous input via hidden states. 
Example: RNNs are commonly used for language translation or predicting the next word in a sentence. 
RNNs are designed to handle time-series data and sequences effectively. 

The vanishing gradient problem is a common problem when training deep neural networks, where the gradients become very small during backpropagation. As a result, the earlier layers learn very slowly or not at all.

For example, imagine you're playing a game where you whisper a message from one person to the next. If the message is passed through many people, it gets softer each time. By the time it reaches the first person, it’s almost silent. In deep neural networks, the learning signal can fade the same way, so the early parts of the network don’t learn much.
It can impact model performance by making it inefficient to train and unstable, which is more of a serious issue in deep networks with sigmoid or tanh activations. To reduce this, apply the following techniques.

• ReLU activation: By keeping positive values and setting negatives to zero, it introduces non-linearity.
• Batch Normalization: To stabilize and speed up training, normalizes layer inputs.
• Use LSTM or GRU (in the case of RNNs): Helps RNNs remember long-term dependencies and reduce vanishing gradient issues. 

Transformers apply self-attention to evaluate the importance of words in a sentence with one another. 

For example: 
In the sentence “The ball was kicked by John,” the model determines that “John” is the kicker.
Used in models such as ChatGPT, many language tasks utilize Transformers for performance. 

Coefficients in logistic regression tell us how each feature influences the final prediction.
If a feature has a positive coefficient, it makes the model more likely to predict "yes" or 1. If it’s negative, it pushes the prediction towards "no" or 0.
The bigger the number (in either direction), the stronger the feature’s impact.
For example, if age is positively correlated, it says that as age increases, the log-odds of the target class (e.g., likelihood of having a disease) also increase. 

XGBoost builds a lot of small decision trees serially. Each tree tries to fix the errors of the previous tree.

Example: XGBoost is a favorite tool in Kaggle competitions when dealing with data in table form (like spreadsheets).
Companies use it to predict if a customer is about to leave (churn) based on factors like how often they log in, how much they spend, or how long they’ve been a customer. 

GANs (Generative Adversarial Networks) work like a game between two players:

• The Generator tries to create fake data that looks real (like fake photos).
• The Discriminator tries to spot the difference between real and fake data.

They keep competing. The generator continuously improves its ability to create fake data that increasingly resembles real data, until the discriminator finds it challenging to distinguish between the two.

Example: Generating fake human faces. 

• Drop rows or columns that have excessive null values
• Fill missing data with mean/median/mode
• Use algorithms that naturally handle missing values
   

Example: If you were missing the age from a customer data set, you would just fill in the average age. 

Batch Normalization: Normalizes through all of the samples simultaneously along the batch dimension (beneficial for deep Convolutional Neural Networks).
Layer Normalization: Normalizes through the features of each sample (good for Recurrent Neural Networks). 
Batch Normalization is for image models, and Layer Normalization is for text models. 

Steps:

• Export the model (e.g., using pickle, ONNX, etc.)
• Serve the model using an API (Flask, FastAPI)
• Deploy to a cloud instance or your server

Challenges:

• Handling a high volume of traffic
• Model drift (data can change over time)
• Latency issues
• Example: Deploy a fraud detection model for a banking app.