Other: Footfall Detector


Footfall Detector (1992 – 1995)

A person walks silently past the window of an office. Inside the office, a PC beeps to tell that someone is near. The Footfall Detector, completely invisible to the person who has just passed by, did its job. In 1992, TNO Waalsdorp developed the core of what they believed to be a very useful technology for intruder detection systems that are based on analysing sound and seismic waves. 

Footfall detector
Footfall detector

The team that developed the footfall detector specialised in analysing acoustic and seismic waves. The waves are picked up by a geophone – a device most commonly used by oil companies for seismic sensing. Buried just under the earth’s surface, the geophone can produce a signal so rich in information that, when analysed properly, a distinction can be made between different moving objects.

The sound pattern generated by a moving object – the presence of certain frequencies, and the fluctuations in their intensities over time is often so distinct that you could say there is a “signature tune” played as it passes by. Whether a vehicle, a helicopter, or a person, each moving object produces a signature.

Our aim was simply to prove that an intruder detection system based on a Geophone is a sound idea.

A major problem was how to distil an object’s signature from the background noise and other interference. Isolating a “target” sound from background noise is not new. In the past, various laboratories have developed algorithms for processing seismic signals, mostly in the course of defence research. The team at TNO, using their experience in helicopter classification, applied their signal processing expertise to extract the footfall signature from a geophone signal.

The first step towards distilling this signature was writing a computer program which had the task of deconstructing the digitized version of the rich geophone signal, and then seeing whether a footfall signature could be constructed from its constituent parts.

Getting the Footfall Detector program to account for the many possible variations within a footfall signature, however, was no small task. A Footfall Detector, however, would have major advantages over other intruder detection systems. The Geophone is passive, for a start. Therefore, it cannot be detected easily. Besides, it is very robust and invisible after installation.

To reap the full benefit of the wealth of data, the team aimed to program the Footfall Detector with enough “intelligence” to distinguish a walking person from other targets. When the team began the Footfall Detection project, a Geophone was buried underneath the path outside an office window of the Laboratory, and the recordings of passers-by were used as a starting point.
But to be commercially viable, the Footfall Detector needed more experience than this set-up could provide. Person-walking signatures are affected by the way a person walks, the body weight, the seismic properties of the soil and the coupling of the energy into the ground (e.g. the passage distance). Packed soil, for example, transmits the waves more readily than loose soil.

Footsteps outside the office window, recorded through the moist Dutch soil, didn’t present the biggest challenge to the algorithm. The team needed worst-case examples to train their Detector with, in other words, seismic data gathered through a variety of soil types. Another challenge was ensuring that footsteps could be detected through background noise. To solve this, the team recorded a person walking past the office window as cars were passing by (effectively masking the sound of the footsteps).

So: how do you train an algorithm to make intelligent decisions – to alert you when it “hears” a footstep, but to keep quiet when a branch or other object falls on the ground with equal impact?

In February 1992, over 300 recordings were made of pedestrians in Sassenheim that:

  • moved with an ordinary stride, a ‘careful’ stride or a hefty stride,
  • wore ordinary shoes or sneakers,  
  • moved over grass, tiles, or a sidewalk,
  • passed the Geophone at various distances from zero to four metres, and
  • without traffic noise or with cars passing at a distance of ten to 25 metres in the neighbouring street.

Other sets of recordings were used as well. One set from the USA included recordings of two and three persons, people walking by, running people, and people crawling. In addition, recordings of three well-trained heavy guard dogs walking and running were made. Those recordings were used repeatedly as input for the detector’s algorithm to be ‘trained’.

Various recordings
Various recordings used for training the algorithm

The detector was optimised by real-time visualisation of its performance at each stage of its development. “It was a matter of increasing, say, one parameter very slightly and seeing whether it improved the detection rate; and if so, then increasing the same parameter a bit more; and so on,” recalls a team member. “A very tedious exercise“.
The parameters that were being adjusted generally controlled the detector’s sensitivity by establishing the relative importance of particular parameters. Only by weighing the individual parameters as more or less important could the “signature” of a footstep be brought into focus.

As with any intrusion detector system, there were two measures of the detector’s success; its detection rate, and its “false alarm rate”. The former had to be maximised; the latter, minimised. Two continuous hours of seismic recordings of hundreds of vehicles on the Landscheidingsweg, as well as recordings of a military exercise with jeeps, trucks, armoured personnel carriers, helicopters, and other aircraft, were processed with the new settings to ensure a low false alarm rate.

What most impressed us was the detector’s ability to distinguish between footsteps and other movements” – in other words, the winning combination of a high detection rate and low false-alarm rate. This was undoubtedly because the Footfall Detector had been trained to recognise patterns of data. More intelligent decisions were possible than common intruder detection systems could make.
The results were so encouraging that TNO decided to develop a hardware prototype. The initial core of the Footfall Detector – initially a PC program – was reduced to a couple of chips: an Intel 8051 microcontroller and some analogue amplification and filtering. An external 8-bit Analog-to-Digital Converter (ADC) completed the demonstrator. The algorithm was adapted to suit the 8051 properties. Two geophones could be monitored simultaneously by a single processor. 

Prototype characteristics:

  • 5V power supply
  • power consumption analogue circuits: < 2 mA
  • power consumption digital circuits: approx. 100 mA
  • sample rate: approx. 250 Hz
  • 8-bit word size

In early 1995, the 8-bit ADC was replaced by a 12-bit version which increased the sensitivity of the system by a factor of 16. Moreover, the Intel 8051 microcontroller was replaced by a secure, low-power compatible microprocessor. That increased the maximum number of geophones to be monitored by a single processor, whilst decreasing the power consumption. The flexibility of the design allowed good adaptation capabilities to different soil types, and easy interfacing to existing intruder detection systems.