Learning Goal - Building Blocks: Sigmoids as Step Functions
3 important questions on Learning Goal - Building Blocks: Sigmoids as Step Functions
How does a large weight w turn a sigmoid into a step function? Give the mathematical argument.
A hidden neuron computes σ(wx + b). As w → ∞:
- If wx + b > 0 (even slightly): σ(wx + b) → 1
- If wx + b < 0 (even slightly): σ(wx + b) → 0
Where exactly does the step occur? Derive the step position s.
The step occurs at the midpoint of the sigmoid's transition — where its input is exactly zero:
wx + b = 0 → x = −b/w ≡ s
So the step position s = −b/w. To place a step at any desired position s, set the bias b = −ws. The large weight $w$ is fixed; only b needs adjusting to move the step anywhere on the x-axis.
wx + b = 0 → x = −b/w ≡ s
So the step position s = −b/w. To place a step at any desired position s, set the bias b = −ws. The large weight $w$ is fixed; only b needs adjusting to move the step anywhere on the x-axis.
What two parameters fully control a step neuron, and what does each control?
- w (weight): controls the sharpness of the step — larger |w| = more vertical = closer to ideal step function
- b (bias): controls the position of the step via s = -b/w — adjusting b slides the step left or right
In practice, w is set very large and fixed; b is tuned to place the step precisely.
- b (bias): controls the position of the step via s = -b/w — adjusting b slides the step left or right
In practice, w is set very large and fixed; b is tuned to place the step precisely.
The question on the page originate from the summary of the following study material:
- A unique study and practice tool
- Never study anything twice again
- Get the grades you hope for
- 100% sure, 100% understanding
Remember faster, study better. Scientifically proven.
