This review groups frequent machine learning and deep learning interview topics into a practical map. It is not a list of trivia; each section connects a concept to why it matters and how it is used. The order moves from data preparation and evaluation to classical models, then deep learning, natural language processing, computer vision, and a set of applied problems.

Machine learning: feature engineering and evaluation

Normalization

Normalization maps numeric features into comparable ranges. Without it, features with larger units can dominate distance or gradient-based learning. For example, height in meters (1.6–1.8) and weight in kilograms (50–100) are not on the same scale; a model may appear to weight weight more strongly simply because its values are larger. Min-max scaling maps values linearly to [0, 1]. Z-score normalization centers the mean at 0 and scales to unit standard deviation. Gradient-descent-trained models usually need normalization: linear regression, logistic regression, SVM, and neural networks. Tree-based models such as decision trees generally do not.

Image augmentation

When image training data is limited, the model sees less information and relies more on prior assumptions; the usual symptom is overfitting, with good training performance and weak test generalization. Augmentation creates plausible variations: random rotation, translation, scaling, cropping, padding, horizontal flips, pixel noise such as salt-and-pepper or Gaussian noise, color changes, and adjustments to brightness, sharpness, contrast, or saturation.

Supervised and unsupervised learning

Supervised learning uses labeled examples to learn a mapping from input to label. Unsupervised learning works with unlabeled data and tries to infer internal structure. Typical supervised tasks are regression and classification; typical unsupervised tasks include clustering.

Regression metrics

Common regression metrics include mean absolute error, mean squared error, median absolute deviation, and R-squared. R-squared closer to 1 indicates a better fit; closer to 0 indicates a worse fit.

Classification metrics

Precision is the fraction of predicted positives that are correct; it answers whether the classification is accurate. Recall is the fraction of actual positives that are found; it answers whether the classification is complete. F1 is the harmonic mean of precision and recall and balances the two.

A/B testing

A/B testing runs two models or variants in a real environment. Offline evaluation cannot fully remove overfitting, cannot reproduce production engineering conditions such as latency, data loss, or missing labels, and cannot compute some business metrics. Online A/B testing usually buckets users into experiment and control groups. The bucketing must keep samples independent and unbiased; a user should always fall into the same bucket, often by hashing a stable user_id.

Hyperparameters

Hyperparameters are set before training rather than learned from data. Grid search checks all points in a range; it can find the global optimum if the range is wide and the step is small, but it is expensive. A practical strategy is coarse-to-fine search. Random search samples points from the range; with enough samples it can find the global optimum or a good approximation, and it is often faster. Validation curves compare hyperparameters by evaluation score; learning curves compare algorithms as training data size changes. Learning rate too large prevents convergence, too small slows convergence. Batch size too large slows training, too small makes convergence unstable. Dropout rate, kernel size, and model depth also affect capacity and cost.

Underfitting and overfitting

Underfitting means poor performance on both training and prediction. Overfitting means the model fits training data too closely: training metrics are good, but test and new-data metrics are poor. To reduce overfitting: collect more data, reduce model complexity, add regularization, use ensemble methods, prune trees, reduce training iterations, add dropout, or inject noise. To reduce underfitting: add features, increase model complexity, or reduce regularization strength.

Confidence probability, cross-validation, class imbalance

Confidence probability measures how certain an inference is; larger values mean more certainty. Cross-validation splits N samples into K folds, uses one fold as test and K-1 as train, rotates the test fold, and averages generalization error. This gives a more stable estimate. Class imbalance can be handled by expanding data, undersampling the majority class, using a different metric, trying a different algorithm, or weighting classes so the minority class matters more.

L1 and L2 regularization

L1 and L2 are penalty terms added to a loss. L1 is the sum of absolute values; L2 is the Euclidean norm, the square root of the sum of squares. Regularization reduces model complexity and overfitting by discouraging large coefficients. From a Bayesian view, it adds a prior that guides optimization toward constrained solutions.

Weight initialization

Weights should not all start at zero. With zero initialization, neurons in a layer receive the same signal and update symmetrically, so the network cannot learn distinct features. For sigmoid, large random weights can saturate activations near 0 or 1, causing vanishing gradients; very small weights with standard deviation 0.01 concentrate activations near 0.5 and reduce expressiveness. Xavier initialization uses standard deviation about 1/sqrt(n) for the previous layer size n, keeping activation spread across layers. For ReLU, He initialization uses standard deviation sqrt(2/n), roughly doubling the scale because ReLU zeroes negative values.

Regression and classification models

Linear regression

Linear regression finds a best-fit linear model. In one variable, y = w0 + w1x; w0 and w1 are learned. It suits data that are roughly linear and models assumed to be linear. Its strengths are simplicity, fast modeling, interpretability, and use as a building block for nonlinear models. Its weaknesses are poor handling of nonlinear data and highly complex relationships. Multiple regression extends this to several variables: y = w0 + w1x1 + ... + wnxn.

Polynomial regression

Polynomial regression uses higher-order terms, for example y = w0 + w1x + w2x^2 + ... + wnxn. It can capture nonlinear curves, and adding terms increases capacity. The cost is that the best degree often needs prior knowledge; a poor choice can overfit.

Decision tree regression and random forest

A decision tree sends samples down branches to leaves. Its core idea is that similar inputs should produce similar outputs. For regression, a leaf predicts the mean of its samples. Entropy measures disorder; as a split makes child nodes purer, entropy falls. Decision trees are interpretable and work for classification and regression, but they overfit easily, are unstable to small data changes, and use greedy split selection, so they do not guarantee a global optimum. Random forest samples both data and features to build many trees, reducing the influence of strong samples and strong features and improving generalization.

Binary classification and logistic regression

Binary classification places samples into two classes. Logistic regression is a generalized linear model: it computes a linear score and passes it through the sigmoid function y = 1 / (1 + e^(-t)), compressing the real line into (0, 1). A threshold, often 0.5, turns the probability into a class.

Naive Bayes

Bayes theorem is P(A|B) = P(A)P(B|A) / P(B). P(A) is the prior, P(B|A) is the likelihood, and P(A|B) is the posterior. Naive Bayes assumes features are conditionally independent given the class. This assumption is often violated, but the method can still work well when features are relatively independent. Its strengths are simple logic and stable behavior across data characteristics; its weakness is that the independence assumption often fails.

Decision tree classification

Decision tree classification follows the same branching idea as regression, but leaves output discrete classes, usually by voting among samples in the leaf.

Support vector machine

SVM is a binary classifier that finds a separating hyperplane with maximum margin. Support vectors are the closest points to the boundary. A good boundary should classify most samples correctly, keep the margin to support vectors as large as possible, keep distances from support vectors equal, and remain simple, preferably a linear hyperplane. If the data are not linearly separable in the original space, a kernel maps them to a higher-dimensional space. Linear kernels are fast and suit many features relative to samples. Polynomial kernels add higher-order terms and can become complex. RBF kernels are flexible, have fewer parameters than polynomial kernels, and are a good default to test. SVM handles high-dimensional complex classification well and generalizes well, but it is less efficient on very large samples and sensitive to missing data.

Clustering

Clustering groups samples by similarity, often distance, and is unsupervised. A good clustering algorithm scales to large data, handles different data types and noise, is insensitive to sample order, works under constraints, and is easy to interpret and use. The silhouette coefficient measures internal compactness and external separation. For point i, a(i) is its average distance to points in its own cluster, and b(i) is the minimum average distance to another cluster. The silhouette is (b - a) / max(a, b). Values near 1 are good; near -1 are poor; near 0 indicate overlap.

K-means randomly picks initial centers, assigns each sample to the nearest center by Euclidean distance, recomputes each cluster mean, and repeats until centers stabilize. It is simple and fast and often gives good clusters, but it needs K in advance, is sensitive to initialization, converges to local optima, and is sensitive to noise and outliers. Mean shift selects a point, draws a radius-r neighborhood, moves the center toward the mean of points in the neighborhood, and repeats. It does not need K and is stable to initialization, but it assumes the sample space follows some probability distribution. DBSCAN grows clusters from core points within a radius, marks unreachable points as noise, and can find arbitrary-shaped clusters without specifying K. It is fast and handles noise, but large data need memory, uneven density hurts results, and distance and MinPts tuning is not trivial. Agglomerative clustering starts with each sample as a cluster and merges nearest clusters until the desired count. Ward minimizes variance increase; average uses average distance; complete uses maximum distance. It has no cluster center and is stable for unclear centers, but it still needs a target number of clusters.

Deep learning fundamentals

A feedforward neural network, or multilayer perceptron, has an input layer, hidden layers, and an output layer. Signals move in one direction, so the network is a directed acyclic graph. A simple composition is f(x) = f3(f2(f1(x))). Deep learning tends to perform better with large data and can avoid manual feature engineering, but it can underperform classical ML on small data, is complex, and is often hard to interpret.

Activation functions turn the summed input into an output and add nonlinearity. Sigmoid maps to (0, 1), is smooth and differentiable, but is expensive and causes vanishing gradients. Tanh maps to (-1, 1), has zero-centered output and faster convergence than sigmoid, but still suffers from vanishing gradients. ReLU returns x for x > 0 and 0 otherwise; it is simple and avoids vanishing and exploding gradients in positive regions, but it has no output for non-positive values. Softmax turns multi-class scores into relative probabilities that sum to 1 and is common in classification output layers.

A loss function measures the gap between prediction and target. Cross entropy measures the difference between probability distributions and is used for classification. A gradient is a vector pointing in the direction of steepest increase; gradient descent adjusts weights opposite the gradient. The loop is: compute loss, stop if small enough, compute gradient, take a small step, repeat. If gradients vanish, optimization stalls; solutions include ReLU or LeakyReLU, batch normalization, and residual connections. Gradient explosion is the opposite: gradients become too large. Gradient clipping limits the step norm, and weight regularization can help.

Batch gradient descent uses all samples per update and converges stably but is slow on large data. Stochastic gradient descent uses one sample per update, is fast per step, but converges unstably. Learning rate controls step size. Momentum adds a velocity term to smooth zigzag paths. AdaGrad adapts the learning rate per parameter by accumulating squared gradients, shrinking updates for frequently changing parameters. Adam combines Momentum and AdaGrad ideas and is often an efficient default.

Convolution applies a filter over an input. In discrete form, s(t) = sum f(a)g(t-a). With input height H, width W, kernel size FH and FW, padding P, and stride S, output height is (H + 2P - FH)/S + 1 and width is (W + 2P - FW)/S + 1. Pooling reduces spatial size, lowering computation and improving generalization. Max pooling takes the maximum in a region; average pooling takes the mean. Pooling has no learnable parameters, keeps channel count unchanged, and is robust to small changes.

Normalization makes sample statistics consistent. In neural networks it helps optimization and avoids saturation. Batch normalization normalizes intermediate activations, reduces sensitivity to initialization and learning rate, can reduce the need for dropout and L2, and speeds convergence. It also slightly perturbs data distribution, which can regularize. AlexNet used ReLU, dropout, overlapping max pooling, local response normalization, and GPU acceleration. Dropout randomly zeroes a proportion of hidden units during training, reducing co-adaptation and forcing robust features. At inference, the full network is used.

Convolution and pooling differ in structure and purpose. Convolution changes channel count, extracts local correlated features, and its parameters depend on kernel size and number. Pooling usually reduces spatial dimension, keeps channels, extracts more generic features, and adds no parameters. For kernel size, early networks used large kernels because depth was limited. Stacking two 3x3 convolutions gives the same receptive field as one 5x5 but with fewer parameters, so 3x3 stacks are usually more efficient. To improve image classification accuracy: increase data and augment it; use batch normalization, learning rate schedules, and weight decay; increase depth or use a stronger model.

Natural language processing

Tokenization splits text into atomic units such as words or phrases; it directly affects semantic understanding. Chinese segmentation includes forward maximum matching, backward maximum matching, and bidirectional maximum matching. Forward maximum matching takes the longest dictionary word length m, matches from left to right, cuts on success, and drops the last character on failure. Backward maximum matching does the same from the right. Bidirectional matching compares both results and chooses the one with fewer tokens; if token counts tie, it prefers fewer single-character tokens. Studies cited in the source report that about 90 percent of Chinese sentences are segmented identically and correctly by forward and backward matching, about 9 percent differ with one correct result, and under 1 percent are either identically wrong or differently wrong.

POS tagging assigns parts of speech. Stemming reduces a word to a stem or root, which may not be a full word. Methods include lookup tables, suffix stripping, lemmatization, and hybrid approaches. Lemmatization uses a dictionary to convert a word to its base form, such as drove to drive.

Bag of words represents a sentence by word counts and ignores word order and context. Term frequency is the count of a word divided by total words. Document frequency is the fraction of documents containing a word. Inverse document frequency is log(n / (docs(w, D) + 1)), measuring how rare and informative a word is. TF-IDF multiplies TF and IDF; a word that appears often in one document but rarely across documents gets a high score. One-hot encoding gives each word a long vector with a single 1, but it loses word relations and creates a high-dimensional sparse space. N-gram models use sequences of N words; for example, in 'I love deep learning', bi-grams are I-love, love-deep, deep-learning, and tri-grams are I-love-deep and love-deep-learning.

A corpus is a collection of language material, often stored and annotated on a computer. Corpora can be heterogeneous or homogeneous, monolingual or multilingual, raw or annotated, synchronic or diachronic. A co-occurrence matrix counts words appearing in a window around a target word and uses those counts as a vector. It improves on one-hot similarity but remains sparse and high-dimensional. Word2vec learns word representations by predicting context words from a target word or the target from context. Its vectors have size independent of vocabulary and provide distributed representations. In a tiny example, 'was rich king he had beautiful queen she was kind' gives context pairs such as was -> rich and rich -> was, king. The well-known vector relation queen = king - he + she is an illustration of learned semantic structure, not a hand calculation.

Computer vision

Image sampling converts a continuous image into discrete points. Resolution is the size of the sampled image and the smallest detail it can represent; a 640x480 image has 307,200 points. RGB stores red, green, and blue intensities. HSV stores hue, saturation, and value. Grayscale conversion can use one channel, the maximum channel, the average channel, or a weighted average; a common weighting is 0.30R + 0.59G + 0.11B because human vision is more sensitive to green and less to blue. Image addition can average multiple images to reduce noise or overlay a watermark. Image subtraction enhances differences and supports background removal and motion detection. Interpolation for enlargement includes nearest neighbor, which is fast but blocky, and bilinear interpolation, which uses four neighboring pixels and two linear interpolations.

Histogram equalization reshapes the gray-level distribution to be more uniform, improving overall contrast, especially for dark, bright, or low-detail images. Mean filtering averages a neighborhood and smooths noise but blurs as the template grows. Gaussian filtering weights the center more than the edges and reduces blurring compared with a large mean filter. Binarization sets pixels to 0 or 255 by a threshold, reducing data and highlighting contours; it supports connected-component extraction, morphology, and region segmentation. Dilation expands shapes; erosion shrinks them. Opening is erosion followed by dilation and removes small noise and thin connections. Closing is dilation followed by erosion and fills small holes and connects small breaks. Top-hat is the original image minus its opening and reveals noise or bright details. Black-hat is the closing minus the original and reveals small holes or dark details. An image pyramid repeatedly downsamples an image; each level usually halves width and height, giving multiple scales for processing and feature extraction. Hough transform detects lines and simple shapes by voting; it can be extended to circles and ellipses.

Edge detectors include Sobel, Prewitt, Laplacian, and Canny. Sobel uses first derivatives and local averaging, so it smooths noise; it has horizontal and vertical templates. Prewitt is similar but with different weights. Laplacian is a second-derivative isotropic operator; it is sensitive to isolated pixels and needs smoothing before use on noisy images. Canny is a multi-stage method: Gaussian smoothing, gradient magnitude and direction, non-maximum suppression, and double-threshold edge connection. Denoising options include mean filtering, median filtering, Gaussian filtering, and bilateral filtering. Median filtering is good for salt-and-pepper noise but can lose detail; bilateral filtering reduces noise while preserving edges. Common color spaces include RGB, HSV, and YUV.

Fourier transform converts a time-domain signal to the frequency domain and can separate high-frequency and low-frequency image components; high frequencies relate to edges and fine detail, low frequencies to smooth regions. Segmentation methods include watershed, FCN, U-Net, Mask R-CNN, and DeepLab. FCN removes fully connected layers and predicts per-pixel classes. U-Net is simple and effective for small samples. Mask R-CNN adds a segmentation branch to Faster R-CNN. DeepLab uses dilated convolution and conditional random fields. Face detection includes Haar cascades and deep models such as MTCNN and Cascade CNN. Face recognition includes LBPH and EigenFaces, and deep models such as DeepFace, FaceNet, and DeepID. Liveness detection can be action-based or silent. Face attributes include gender, age, race, and expression. Text detection includes traditional morphology and MSER+NMS, and deep methods such as CTPN, SegLink, and EAST. Text recognition includes CRNN+CTC and CRNN+Attention.

Applied interview problems

Changing the threshold in a binary classifier usually raises precision and lowers recall; AUC is threshold-independent for a fixed score ranking, so it should not change, although some interview answers describe it differently. Batch size and learning rate both affect convergence speed. LSTM improves on RNN by handling vanishing gradients and longer sequences; bi-LSTM adds reverse-sequence features. Age is continuous; for models that need discrete input, bin it; for gradient-based models, normalize it; for models that accept continuous features without gradient descent, input it directly. A training set with 1:9 positive-negative ratio and 90 percent accuracy, plus a balanced validation set with 50 percent accuracy, suggests imbalance-driven underfitting rather than simple overfitting; undersample the majority and oversample the minority. Receptive field grows with convolution and pooling; for 3x3 conv, 3x3 conv, 2x2 max pool, 3x3 conv with stride 1 and no padding, a 32x32 input gives feature maps 30x30, 28x28, 14x14, and the receptive field becomes 10x10 before the last convolution. Acceleration can come from hardware such as GPU and multi-GPU, model compression such as lightweight models, pruning, and distillation, normalization and batch norm, and code-level choices such as learning rate schedules and adaptive optimizers. BN reduces vanishing gradients, regularizes, speeds convergence, and stabilizes training. Hard-margin SVM loss enforces y(i)(theta^T x(i) + b) >= 1 and maximizes 1/||theta||. TF-IDF weights rare discriminative words more than raw document frequency. For classical bag-of-words, TF-IDF, and N-gram models, word vectors often beat character vectors because segmentation adds lexical meaning; for deep learning, character vectors can be competitive and sometimes better because they preserve fine-grained semantics while keeping computational performance. Discretization reduces dimension, adds nonlinearity, makes features easier to add or remove, and reduces noise; methods include interval binning, chi-square splitting, and information-gain splitting. For gradient descent on ln(wx) with w=2, x=10, and learning rate 0.1, the gradient with respect to w is 1/w, so updates give about 1.95, 1.8987, and 1.846 after the third round. A 32x32x3 input with ten 5x5 kernels, stride 1, and padding 2 gives 32x32 output per kernel and 10x32x32 = 10,240 features. ROI pooling divides a candidate region into a fixed grid and max-pools each cell, producing a fixed-size output from different-sized regions. TensorFlow while_loop runs on the TensorFlow graph, while a Python for loop runs in the interpreter; if the loop contains no TensorFlow operations, Python can be faster.

Sigmoid derivative is f(x)(1 - f(x)); it vanishes for large positive or negative inputs and is largest at x=0. ReLU has gradient 1 for positive inputs, which helps with vanishing gradients. SSD and YOLO differ in speed and accuracy trade-offs; YOLOv3 uses Darknet53, logistic classifiers, nine anchors, and three scales, while YOLOv2 uses Darknet19, softmax, five anchors, and two scales. For the 1000-sample confusion matrix with 600 A and 400 B, class B precision is 200/(200+100)=2/3 and recall is 200/(200+200)=0.5. Imbalanced classes can be handled by resampling; overfitting is diagnosed by comparing training and test accuracy. Depthwise separable convolution splits convolution into depthwise and pointwise steps; for a 5x5 three-channel input with four 3x3 three-channel kernels, ordinary convolution has 4333=108 parameters, while depthwise separable has 333=27 plus 1134=12, for 39 total. Two-stage detectors first propose regions and then classify and regress, giving higher accuracy but slower speed; one-stage detectors classify and regress directly on feature maps, giving faster speed but often lower accuracy. GRU simplifies LSTM with two gates, update and reset, while LSTM has three gates, forget, input, and output; GRU has fewer parameters and often converges more easily, while LSTM can represent more with very large data. Bilinear interpolation performs linear interpolation in x and then in y. SIFT is scale-invariant and robust to rotation, scaling, and brightness; HOG counts gradient orientations in local cells and blocks; LBP compares a center pixel with neighbors and encodes the pattern. Common image segmentation, face detection, face recognition, liveness detection, and text detection methods follow the same pattern: traditional pipelines rely on handcrafted features and morphology, while deep models learn features end to end.