Learning Goal - Extension to Multiple Inputs
3 important questions on Learning Goal - Extension to Multiple Inputs
How does the step-function construction extend to a 2D input (x, y)?
A hidden neuron with two inputs can be made to ignore one input by setting that input's weight to zero. For example, setting the y-weight to zero and the x-weight very large creates a "sheet" — a step function that fires based only on x, extending uniformly in the y-direction. Combining two such neurons (with weights +h and −h) creates a 3D bump — a ridge of height h at a fixed x position, extending along the entire y-axis.
Why is the output bias b ≈ −3h/2 in the tower construction?
After summing the two bumps, the output signal has three distinct levels: 0, h (plateau), and 2h (tower). We need a threshold that lies strictly between h and 2h — the midpoint is (h + 2h)/2 = 3h/2.
Setting b = −3h/2 in the output neuron means the neuron fires only when the input exceeds 3h/2. Only the tower regions (height 2h) clear this threshold; the plateau regions (height h) do not. The very large output weight then drives the tower to 1 and leaves the plateau at 0.
Setting b = −3h/2 in the output neuron means the neuron fires only when the input exceeds 3h/2. Only the tower regions (height 2h) clear this threshold; the plateau regions (height h) do not. The very large output weight then drives the tower to 1 and leaves the plateau at 0.
How are arbitrary multi-input functions approximated using tower functions?
Tile the input space with many thin towers at different locations (xᵢ, yᵢ), each with a different height vᵢ matching f(xᵢ, yᵢ). By summing towers of varying heights across the input domain, the network builds a piecewise-constant approximation to any target function f(x, y) — just as bumps approximate 1D functions.
More towers → finer grid → better approximation. The same principle extends to any number of input dimensions.
More towers → finer grid → better approximation. The same principle extends to any number of input dimensions.
The question on the page originate from the summary of the following study material:
- A unique study and practice tool
- Never study anything twice again
- Get the grades you hope for
- 100% sure, 100% understanding
Remember faster, study better. Scientifically proven.
