Ai Activation Functions — Free AI & ML Tutorial
Learn Ai Activation Functions in AI & ML with a free, beginner-friendly tutorial, examples and practice for Indian students on Syllab.in.
TL;DR: Learn Ai Activation Functions in AI & ML with a free, beginner-friendly tutorial, examples and practice for Indian students on Syllab.in.
Written & reviewed by the Syllab.in Academic Team (CBSE/NCERT subject experts) · Updated
Ai Activation Functions in AI & ML
Activation functions introduce non-linearity into neural networks. Without them, stacking multiple layers is equivalent to a single linear transformation — no matter how deep the network, it would only learn linear patterns. Activation functions let networks learn complex, curved decision boundaries.
Sigmoid: σ(x) = 1/(1+e⁻ˣ). Outputs 0–1. Used in binary classification output layers. Problem: vanishing gradients for very large or small inputs — gradients become near-zero, stopping learning in deep networks.
ReLU (Rectified Linear Unit): max(0, x). Simple, fast, solves vanishing gradient for positive values. Default choice for hidden layers in most networks. Problem: "dying ReLU" — neurons can get stuck at zero permanently.
Softmax: converts a vector of raw scores into probabilities summing to 1. Used in the output layer for multi-class classification. Leaky ReLU, ELU, GELU are improved variants that fix dying ReLU.
Ai Activation Functions — Syntax
# Common activation functions: # Sigmoid: f(x) = 1 / (1 + exp(-x)) → output: (0, 1) # Tanh: f(x) = (exp(x)-exp(-x))/(exp(x)+exp(-x)) → output: (-1, 1) # ReLU: f(x) = max(0, x) → output: [0, ∞) # Leaky ReLU: f(x) = x if x>0 else 0.01*x # Softmax: f(xᵢ) = exp(xᵢ) / Σe
Learn Ai Activation Functions step by step with Syllab's free interactive AI & ML tutorial — runnable code examples, practice exercises and instant AI feedback, all free with no signup. Explore the full AI & ML course →