Multilayer Perceptron · Module 1

Neural Network Architecture

1. What are we training?

Trainable connections glow subtly.

Input, hidden, and output layers — top to bottomFive input features feed hidden layers of four, three, and two neurons, then one output.Input layer · 5 featuresHidden layers · 4 → 3 → 2 neuronsOutput layer · 1 neuronFeatures go inProbability

How many weights and biases will be trained?

Every connection has one weight, and every neuron after the input layer has one bias. Count them one pair of adjacent layers at a time:

Input → hidden layer 1
5 × 4 = 20 weights
4 biases
Hidden layer 1 → 2
4 × 3 = 12 weights
3 biases
Hidden layer 2 → 3
3 × 2 = 6 weights
2 biases
Hidden layer 3 → output
2 × 1 = 2 weights
1 bias
Parameters to train
40 weights
10 biases
Total trainable parameters
50

2. What does one neuron do?

The first hidden neuron receives all five inputs. Each input is multiplied by its connection weight, then the results are added with one bias term.

Input, hidden, and output layers — top to bottomFive input features feed hidden layers of four, three, and two neurons, then one output.x1w1x2w2x3w3x4w4x5w5Input layer · 5 featuresHidden layers · 4 → 3 → 2 neuronsOutput layer · 1 neuron+ b↓z→f(z)→aFeatures go inProbability

Its calculation is:

z=w1x1+w2x2+⋯+w5x5+ba=f(z)\begin{aligned}z&=w_1x_1+w_2x_2+\cdots+w_5x_5+b\\a&=f(z)\end{aligned}
x1,…,x5x_1,\ldots,x_5
the five input features.
w1,…,w5w_1,\ldots,w_5
the weights for those inputs.
bb
the bias added to the weighted inputs.
zz
the weighted inputs plus the bias.
ff
the activation function.
aa
the value sent forward after activation.

The same calculation happens at every neuron. Each neuron in hidden layer 2 receives the activation values from all neurons in hidden layer 1 as its inputs.

3. How the values flow in the network

Highlighted forward pass: a fully connected 5–4–3–2–1 networkx₃ → a₁⁽¹⁾ → a₁⁽²⁾ → a₁⁽³⁾ → z. Highlighted lines trace one route; all other connections remain visible. The output logit z passes through sigmoid to produce the prediction.Input layer5 featuresHidden layer 14 neuronsHidden layer 23 neuronsHidden layer 32 neuronsOutput layer1 neuronw₁₁⁽¹⁾w₁₂⁽¹⁾w₁₃⁽¹⁾w₁₄⁽¹⁾w₁₅⁽¹⁾w₁₁⁽²⁾w₁₂⁽²⁾w₁₃⁽²⁾w₁₄⁽²⁾w₁₁⁽³⁾w₁₂⁽³⁾w₁₃⁽³⁾w₁₁⁽⁴⁾w₁₂⁽⁴⁾x₁x₂x₃x₄x₅a₁⁽¹⁾a₂⁽¹⁾a₃⁽¹⁾a₄⁽¹⁾a₁⁽²⁾a₂⁽²⁾a₃⁽²⁾a₁⁽³⁾a₂⁽³⁾zz = logitσ(z)σ(z) = p
z1(1)=w11(1)x1+w12(1)x2+w13(1)x3+w14(1)x4+w15(1)x5+b1(1)a1(1)=f ⁣(z1(1))z1(2)=w11(2)a1(1)+w12(2)a2(1)+w13(2)a3(1)+w14(2)a4(1)+b1(2)a1(2)=f ⁣(z1(2))z1(3)=w11(3)a1(2)+w12(3)a2(2)+w13(3)a3(2)+b1(3)a1(3)=f ⁣(z1(3))z=w11(4)a1(3)+w12(4)a2(3)+b1(4)p=σ(z)=11+e−z,0<p<1\begin{aligned}z_1^{(1)}&=w_{11}^{(1)}x_1+w_{12}^{(1)}x_2+w_{13}^{(1)}x_3+w_{14}^{(1)}x_4+w_{15}^{(1)}x_5+b_1^{(1)}\\a_1^{(1)}&=f\!\left(z_1^{(1)}\right)\\[12pt]z_1^{(2)}&=w_{11}^{(2)}a_1^{(1)}+w_{12}^{(2)}a_2^{(1)}+w_{13}^{(2)}a_3^{(1)}+w_{14}^{(2)}a_4^{(1)}+b_1^{(2)}\\a_1^{(2)}&=f\!\left(z_1^{(2)}\right)\\[12pt]z_1^{(3)}&=w_{11}^{(3)}a_1^{(2)}+w_{12}^{(3)}a_2^{(2)}+w_{13}^{(3)}a_3^{(2)}+b_1^{(3)}\\a_1^{(3)}&=f\!\left(z_1^{(3)}\right)\\[12pt]z&=w_{11}^{(4)}a_1^{(3)}+w_{12}^{(4)}a_2^{(3)}+b_1^{(4)}\\p&=\sigma(z)=\frac{1}{1+e^{-z}},\qquad 0<p<1\end{aligned}