A New Least-Squares Regression Model

KEYWORDS:
Teaching;
Regression line;
Classroom apparatus.

Chris du Feu

Queen Elizabeth's High School, Gainsborough,

England.
e-mail: chris@beckingham0.demon.co.uk
Summary
This article describes simple apparatus for
classroom demonstration of tting a least-squares
regression line.

^ INTRODUCTION ^

lthough it is easy enough to prove the

formula for the least-squares regression line
by dierentiating the expression for the sum of the
squares of the deviations from the line, this proof
is beyond the scope of most practical statistics
syllabuses. It is also beyond the mathematical
reach of many who study statistics as a service
subject. Even amongst those who have the
necessary algebraic skills to carry out the manipulations required in the proof, its apparent
complexity may obscure understanding. A proof
by completing the square will avoid calculus but is
equally daunting algebraically. Further, it may
seem quite arbitrary that we choose to minimize
the sum of squares ^ why not the sum of absolute
deviations, or sum of cubes of deviations or
product of deviations? What is needed is a
practical demonstration which appeals to the
intuition and illuminates the mathematical proof.
Read on ^ help is at hand. Below is a description
of simple apparatus which can not only give the
line of best t, but also show why the least-squares
concept is an intuitive one.

^ BUILDING THE MODEL ^

First, some pinboard is required. For a `personal'
model, this need be little larger than an A4 sheet of
paper, but for a classroom demonstration model
it should be as large as is convenient. The pinboard
must be of rm material, at least as rm as
medium-density breboard. For a permanent
model, it will be useful to attach beading to two of
the edges. This beading will be parallel to the xand y-axes, and must stand slightly proud of the
pinboard. It is important to inlay a magnetic
compass in the board ^ the top right-hand corner
is probably the most convenient place. Plot your

scatter diagram on a sheet of graph paper and lay

the sheet on the board so that the axes lie parallel
to the beading (gure 1).

Fig 1. The working model

Next you need a collection of hard steel pins. Each

pin must have a thin steel `ag' attached to the
upper part of the shaft (gure 2). A convenient size
for the ags is 2 by 3 cm for the personal model,
but larger for a classroom display board. Place the
pins rmly in the graph paper which now rests on
the pinboard, one pin pushed in at each point on
the diagram. Push the pins down until the lower
edge of the ag touches the graph paper, ensuring
that the ag is aligned exactly with the y-axis. It
will be important that the ags point away from
where the regression line will be. However, if you
align them incorrectly, the apparatus is forgiving.
It will let you know of the problem and you can
adjust the oending ags. For classroom demonstration it does help if things work well, so do
practise rst.
Two more pieces of equipment are needed ^ elastic
bands and a rod which will be the regression line. I
borrowed a smooth, light, rigid rod of zero diameter
from the A-level mechanics resource box. Ensure
that the rod is not of a magnetic material. Steel is
Teaching Statistics.

Volume 23, Number 1, Spring 2001 .

17

Fig 3. Inserting the rod

friction in the elastic bands does allow the

apparatus to come to rest fairly quickly. If any
ags were wrongly aligned, it will now be obvious
because the rod will foul them. Simply turn them
through 1808 and all will be well. Once the position
of the line has been found, it can be marked on
the graph paper. The apparatus is then ready for
the next scatter diagram.

Fig 2. Pin with ag

out, but carbon-bre rods have most of the

properties required. The elastic bands must have
natural length equal to the length of the ags on the
pins ^ it is best to make your pins to match the bands
rather than vice versa ^ and the bands must also
be strongly magnetic. (The technology department
will be able to help here provided they have the
equipment to impregnate ordinary elastic bands
with nely ground strong iron lings.) A relatively
low modulus of elasticity is also important.

^ CLASSROOM USE ^
All is now ready. Using the on-board compass,
align the apparatus with the y-axis north/south.
(Incidentally, the classroom demonstration model
cannot be used vertically against a blackboard,
which is a pity. It is always awkward having a
large class gathering round a demonstration on a
table top. Perhaps, if transparent pinboard were
available, the small model could be placed on an
overhead projector.) Place one elastic band over
each ag. Then slide the rod through each band in
the obvious order (gure 3). This task is easier
than it seems because the rod, being smooth, slides
easily through even the most extended band.
However, you do have to grip the end of the rod
very rmly. Release the rod gently and let it settle
into position. Although it is light, the internal
18

. Teaching Statistics.

Volume 23, Number 1, Spring 2001

^ THEORY ^
It is easy to see why the device works. Because
the natural length of the elastic band equals the
length of the ag, it has zero extension when at rest
on the ag. When the rod is threaded through the
band, the extension equals the distance between
the rod and the pin end of the ag (gure 2). The
strength of the Earth's magnetic eld and the very
low modulus of elasticity of the bands ensures that
they all lie aligned north/south, which is parallel
to the y-axis. The whole system must settle in a
position of minimum stored energy. Now, the
energy stored is given by
E

X lx2i
l
i

where xi are the extensions of the bands, and l

and l are the common length and modulus of
elasticity. This equation reduces to
lX 2
E
xi
l
Thus, because l=l is constant, E depends solely on
Sx2i . The natural position of minimum stored
energy is also the position which minimizes the
sum of squares.
The apparatus is also useful for demonstrating the

regression line of x on y. Arrange things as before,

except that the pins must be aligned with the ags
east/west and the board turned through 908 so that
the x-axis is now aligned with the Earth's magnetic
eld. If you have already drawn the regression line
of y on x it will now be clear (in most cases) that
the two lines are dierent.

^ POTENTIAL ENHANCEMENTS ^
Diligent readers will now have anticipated how to
demonstrate the third regression line, where the
sum of the squares of the perpendicular distances
from the line is minimized. An obvious modication is to substitute non-magnetic elastic bands
(which, it must be said, are rather easier to obtain
than the strongly magnetic variety). However,
there is a problem. The pins must be arranged with
their ags perpendicular to the regression line,
and this alignment is not known until the line is
found. There are a number of solutions which
spring to mind, although I have not yet tested
them.
First would be to modify the pins so that the band
is held in a tube attached along the top of the ag.
The tube would be of a length equal to the natural
length of the band, the band hooked over a
retaining pin at the distal end, and the opening at
the proximal end would have smooth edges. This
would allow the band to bend as it came out of the
tube so that the ag need no longer be in exact
alignment with respect to the regression line.
A second approach would be easier but only
approximate. Smooth circular pins of small but
nite diameter would be used. The bands would be
a snug t on the pins so that extension began as
soon as they were pulled in any direction. The
greater the diameter of the pins, the easier to nd
bands of the right size, but the worse would be the
match between extension and distance from the
regression line (gure 4).
A third approach would involve two parallel

Fig 4. The problem with thick pins

horizontal boards. The lower board would have a

grid of hooks and the upper board, which would
be smooth, a matching array of holes. The
distance between the boards would equal the
natural length of the elastic bands. Elastic bands
would be attached to the appropriate hooks on
the lower board and pulled, from above, through
the holes of the upper board using a crochet
hook.

^ FOOTNOTE ^
Dynamic visual aids can be crucial in giving
students understanding of concepts. When I rst
used this device, one morning at the very beginning
of April a few years ago, I was astounded by how
rapidly the concept of the least-squares regression
line was grasped. You will be amazed too. Sadly,
the apparatus was borrowed by an ex-colleague
who left without returning it. Since then, I have
been forced merely to describe the apparatus to
students, as I have done to you. Curiously, this
seems to be more eective than using the
apparatus itself.
Acknowledgements
Thanks to Chris Frost, Georey Lynn and
Richard du Feu for help with the apparatus and
photography, and to the anonymous referee for
comments and suggestions.

Teaching Statistics.

Volume 23, Number 1, Spring 2001 .

19