Preparing to Model

🔄 2.1.1 — Machine Learning Activities

Building an ML system is not just about choosing an algorithm — it follows a structured lifecycle of activities that ensure the model is effective and reliable.

The ML Lifecycle

Key ML Activities Explained

Activity	Description	Key Task
Problem Definition	Clearly define what you want the model to achieve	Is this classification, regression, or clustering?
Data Collection	Gather relevant data from various sources	Databases, APIs, web scraping, surveys
Data Preparation	Clean, transform, and format data	Handle missing values, encoding, normalization
Exploratory Data Analysis	Understand data patterns and distributions	Visualizations, statistics, correlation analysis
Feature Engineering	Create or select the most informative features	Feature creation, selection, transformation
Model Training	Train algorithms on prepared data	Choose algorithm, set hyperparameters
Evaluation	Measure model performance	Accuracy, precision, recall, F1-score
Deployment	Put model into production	APIs, integration, monitoring

📊 2.1.2 — Types of Data in Machine Learning

Understanding data types is fundamental because different ML algorithms require or perform better with specific types of data.

Scales of Measurement

Scale	Properties	Examples	Operations
Nominal	Categories, no order	Gender, Country, Color	= , ≠
Ordinal	Categories with order	Education level, Customer satisfaction	= , ≠ , < ,>
Interval	Equal intervals, no true zero	Temperature (°C), Calendar dates	= , ≠ , < ,> , + , −
Ratio	Equal intervals, true zero	Height, Weight, Income	= , ≠ , < ,> , + , − , × , ÷

🗃️ 2.1.3 — Structures of Data

✅ 2.1.4 — Data Quality and Remediation

The quality of your data directly determines the quality of your ML model. "Garbage in, garbage out" is a fundamental principle.

Common Data Quality Issues

Remediation Techniques

Issue	Technique	Description
Missing Values	Imputation	Replace with mean, median, mode, or predicted values
Missing Values	Deletion	Remove rows/columns with missing data (if small %)
Noisy Data	Smoothing	Binning, regression-based smoothing, clustering
Outliers	Detection & Removal	IQR method, Z-score, or domain-specific thresholds
Duplicates	Deduplication	Identify and remove duplicate records
Inconsistency	Standardization	Uniform formats, consistent naming conventions

⚙️ 2.1.5 — Data Pre-Processing

Data preprocessing transforms raw data into a clean, usable format suitable for ML algorithms. This is the most time-consuming but critical step.

Key Preprocessing Steps

1. Data Cleaning

Handling missing values, removing duplicates, correcting inconsistencies, and filtering noise.

2. Data Transformation

• Normalization (Min-Max Scaling): Scales features to a fixed range, typically [0, 1]

• Standardization (Z-score Normalization): Transforms data to have mean = 0 and standard deviation = 1

3. Data Encoding

• Label Encoding: Assigns integer values to categories (e.g., Red=0, Blue=1, Green=2)
• One-Hot Encoding: Creates binary columns for each category (e.g., is_Red=[1,0,0], is_Blue=[0,1,0])

4. Feature Scaling

Ensures all features are on comparable scales. Especially important for distance-based algorithms (KNN, SVM) and gradient descent optimization.

5. Data Splitting

Dividing data into training set (typically 70–80%) and test set (20–30%) to evaluate model generalization.

📉 Dimensionality Reduction

Dimensionality reduction is the process of reducing the number of features (dimensions) in a dataset while preserving as much information as possible.

Why Reduce Dimensions?

• Curse of Dimensionality: As dimensions increase, data becomes sparse, making patterns harder to find and models less reliable.
• Reduced Computation: Fewer features mean faster training and prediction times.
• Noise Reduction: Removing irrelevant features can improve model accuracy.
• Visualization: Reducing to 2D or 3D enables visual exploration of high-dimensional data.

Approaches

PCA — Principal Component Analysis

PCA is the most widely used dimensionality reduction technique. It works by:

1. Standardize the data (mean=0, variance=1)
2. Compute the covariance matrix
3. Find the eigenvectors (principal components) and eigenvalues
4. Sort eigenvectors by eigenvalues in descending order
5. Select top-k eigenvectors as the new feature space
6. Project data onto the new k-dimensional space

🎯 Feature Subset Selection

Feature subset selection aims to identify the most relevant features for your ML model, removing redundant or irrelevant ones.

Methods of Feature Selection

Method	Approach	Pros	Cons
Filter Methods	Rank features using statistical tests independent of any ML model	Fast, scalable, model-independent	Ignores feature interactions
Wrapper Methods	Use an ML model to evaluate subsets of features	Considers feature interactions	Computationally expensive
Embedded Methods	Feature selection is built into the model training process	Balance of speed and accuracy	Model-specific

Common Filter Techniques

• Correlation Coefficient: Remove features highly correlated with each other (multicollinearity)
• Chi-Square Test: Measures dependence between categorical features and target variable
• Information Gain: Measures how much a feature reduces uncertainty about the target
• Variance Threshold: Remove features with very low variance (near-constant values)

Forward Selection vs Backward Elimination

🃏 Quick Revision — Flashcards

🧠 Unit 2 Quiz