LIMA
With God's help
עברית

Gradient

Imagine you are standing on a mountainside with your eyes closed, and your goal is to reach the summit by the fastest route. To do so, you feel around with your feet, find the direction in which the sense of the uphill slope is strongest, and take a step there. The gradient is exactly this "compass" - a mathematical arrow that always points in the direction of the steepest and fastest climb in the local surroundings, and the length of the arrow tells you how steep that slope really is.

The theory in brief

The gradient is an operator that acts on a scalar field (a function that returns a numerical value, such as pressure, temperature or height) and maps it to a vector field. For a three-dimensional function \(f(x,y,z)\), the gradient is denoted by the nabla symbol \((\nabla)\) and is defined as the vector of all the partial derivatives of the function: \[\nabla f = \left( \frac{\partial f}{\partial x}, \frac{\partial f}{\partial y}, \frac{\partial f}{\partial z} \right)\]

The gradient has two geometric meanings that make it a vector: its direction points toward the direction of maximum increase (the fastest climb) of the function at that point, and its magnitude (its norm, \(|\nabla f|\)) equals that maximum rate of change. Moreover, the gradient allows immediate computation of the "directional derivative" – the rate of change of the function in any arbitrary direction \(\hat{u}\) – by computing the dot product \(\nabla f \cdot \hat{u}\). Another essential property, widely used in physics and optimization, is that the gradient vector is always perpendicular (orthogonal) to the contour lines or level surfaces of the function (the places where the value of the function is constant).

∇f Steep: long arrow Gentle: short arrow The green arrow is perpendicular to the contour and points up the steepest climb Dense lines = steep slope = long ∇f

How to read the lab

The mountain range = the scalar field \(f(x,y)\)

The height of every point of the holographic mesh is the value of the function. Dragging the background rotates the table, and the wheel zooms in and out.

The plasma ball

Drag it to any point on the mountain. The white trail shows its path.

The shadow and the laser beam

Under the ball, on the glass of the table (the \(xy\) plane), its shadow is projected, and a laser beam connects the two.

The green arrow = the gradient \(\nabla f\)

It starts at the shadow on the glass. Its length follows \(|\nabla f|\).

Contour Slicer

Orange neon rings where \(f(x,y) = C\). The ring through the ball is yellow, and a white square marks the angle between it and the arrow.

Water drops

A rain of blue light particles that flow down the mountain. One bright drop leaves from the ball itself.

Pure 3D space \(f(x,y,z)\)

A temperature fog: a hot red core and cold blue edges. The ball floats in the air and the arrow is 3D. The "temperature bubbles" are the level surfaces.

90°

The readings in the control column

The angle between the arrow and the contour line and the direction to the summit. While dragging: the angle between the direction of motion and the arrow, and the directional derivative \(D_{\hat u}f\) along the motion.

Playback rate and pause

They change only the pace of the drops and the fog, not the function and not the gradient.

To the lab ↓
Loading the lab...
1

Experiments to try in the lab

  1. Finding the Summit and the Plain (direction and magnitude). Place the plasma ball on a perfectly flat horizontal plain on the mountain, and then slowly drag it toward a steep cliff wall leading to the summit. Watch the length and direction of the green arrow (the gradient) along the way.
    When you stand on the plain, you will notice that the green arrow shrinks to a point and vanishes – there is no slope, so the derivatives are zero (this is a "critical point"). As you approach the cliff, the arrow will leap in length and grow considerably, always pointing, like a compass needle, straight toward the steepest route to the summit. The experiment shows sharply the dual nature of the gradient: its magnitude indicates the strength of the slope (the rate of change), and its direction is the direction of maximum climb.
  2. Walking the Edge (orthogonality to contour lines). Turn on the "Contour Slicer" button so that the orange neon rings appear on the mountain. Now, try to drag the ball exactly along one of the contour rings (that is, without going up or down). Look at the angle formed between your walking path and the green arrow.
    You will see right away that the green arrow sticks out at exactly 90 degrees (perpendicular) to your path, at every single point on the ring. The reason for this is fascinating: a contour line represents a direction in which the function does not change at all. If the rate of change in the direction tangent to the ring is zero, then the maximum change (the gradient) must occur exactly in the direction perpendicular to it. This is the visual demonstration of the most important theorem in vector analysis – the gradient is always perpendicular to the level lines.
  3. The Path of Least Resistance (the negative gradient). Place the ball in the middle of a steep slope and turn on "Water drops" mode. Follow the flow path of the blue light particles sliding down to the valley, and compare their path with the direction the green gradient arrow points to.
    Water, as physics dictates, will always choose the steepest and fastest path down. You will immediately notice that the path of the water points exactly 180 degrees against the green arrow (which points to the steepest climb). This experiment is the basis of the "Gradient Descent" algorithm used in artificial intelligence – to find the minimum most efficiently, we must step against the direction of the gradient, exactly like water flowing to the valley.
  4. The Blind Climber (the directional derivative). Turn off the contour lines. Pick in your mind a random direction that does not coincide with the direction of the green arrow (for example, an angle of 60 degrees from it), and drag the ball in that direction. Notice the rate at which you gain height compared with dragging directly along the arrow.
    You will discover that when you deviate from the direction of the gradient, your climb is gentler and slower. If you deviate by 90 degrees, you will not climb at all (you will walk along a contour line). This is the heart of the concept of the "Directional Derivative": the rate of change in any random direction you choose is simply the projection (the dot product) of the gradient vector onto your direction. The gradient contains within it the information about all possible climbing directions.
  5. Into the Fog (the gradient in 3D space). Switch to "pure 3D space" mode \((f(x,y,z))\). The mountain will disappear and be replaced by a holographic fog of temperature, in which the center is blazing hot (red) and the edges are cold (blue). Float with the ball through free space, turn on the "temperature bubbles" (spherical level surfaces), and watch the gradient arrow, which has now become three-dimensional.
    Now there is no mountain to climb and no physical notion of "height". Even so, the green arrow will keep behaving in exactly the same way: it will point straight toward the strongest heat source in the room, and it will be perfectly perpendicular to the temperature bubbles. The experiment frees the student from depending on the two-dimensional topographic image, and proves that the gradient is a universal concept – it always points in the direction of maximum change of the local property, in any number of dimensions you choose.