Test facility for outdoor intrusion detection systems (1990 – 2005)
To secure objects with a high risk, it is sometimes necessary to use electronic intrusion detection systems (IDS) to secure the object’s perimeter to realise the minimum required delay time for reacting to alarms. In order to minimise disappointments after purchasing such a system, it is important to make the right choice among the offers. After all, the suitability of these IDSs is highly dependent on the specific outside conditions under which these systems are used. Indoor detection systems have this disadvantage much less because the operating conditions inside are more controlled. The specifications for outdoor sensors and IDSs are often not very concrete. They are often stated in terms of a “high probability of detection”, a “low probability of false alarms”, “not to be misled”, etc., often based on the most optimal environmental conditions. Comparison of products for a specific application is not possible on this ground.
To be able to make an independent assessment of IDSs on matters such as the usability of such a system for specific operational situations, ease of use, detection probability, chance of undesired alarms, ability to mislead the system, etc. TNO-FEL built an outdoor IDS test facility on the Waalsdorpervlakte in the early 90s. The test facility offered the possibility to answer the question of which of the IDSs on the market could be used as most suitable and cost-effective in a specific environment.
The test site is approximately 50 meters wide and 100 meters long. The facility was equipped with a number of cameras that covered the entire site. Infrared lighting was installed on the camera masts so that the whole area could be viewed via monitors at night. A number of video recorders and a digital “time-lapse” recorder system allowed the recording of images. Because the functioning of all outdoor IDSs is influenced by the weather conditions, the test site was equipped with a weather station with which all relevant meteorological parameters were continuously measured and recorded. This allowed us to quantitatively determine the behaviour of IDSs in different weather conditions during a long test series.
When testing an IDS, the alarm output of the system was connected to a data logger in the measuring booth that registered all alarms including a date-time group. The data logger was equipped with a number of channels so that it was possible to automatically compare more IDSs under the same test conditions.
In most cases, the following IDS properties are considered important:
- detection probability
- false positive rate
- influences of the environmental conditions
- detection volume and detection range
- misleading possibilities
- set-up time and settling time (mobile IDS)
For testing a specific IDS or comparing different IDSs, test scenarios can be devised that depend on the properties to be investigated. The probability of detection is a statistical quantity that can be determined with a certain reliability. This reliability increases with the number of repetitions of the same measurement. The detection probabilities for an intruder who tries to outwit an IDS running, rolling, crawling, creeping, or running, however, are all different and also depend on the environmental conditions. Determining the detection probability of an IDS for all intrusion scenarios and all environmental conditions with high reliability is therefore practically impracticable since the number of measurements needed would become astronomically high. However, a good indication can be obtained of the IDS behaviour for different scenarios, and IDSs can be compared objectively when identical tests are performed under the same environmental conditions.
It was sometimes relatively easy to design a ‘simulation intruder’ for certain IDSs in order to reduce the labour-intensive use of test persons. The advantage was that the tests became test-person independent and therefore guaranteed a high degree of reproducibility. For an IDS that functioned for example with relatively low radio frequencies, a certain amount of salt water in a PVC container could simulate a person. For an IDS that functions with high (radar) frequencies, an object could be used that represents the ‘radar cross-section’ of a person.
Below is a photo of the test facility with some IDSs functioning on microwave frequencies (radar-IDS).
For this type of IDS sensor, a ‘worst case scenario’ considered was an intruder who tries to pass the IDS rolling with the body axis in parallel to the viewing direction of the radar antenna. A metal sphere with a diameter of about 30 cm has an equally large ‘radar cross-section’ and could represent such an intruder for this type of IDS. Below is a photograph showing a plastic rail arranged at right angles to the antenna view, over which such a sphere can be rolled back and forth. With this ‘worst case’ simulated intruder it was relatively easy to establish the detection probability with good reliability without a test person having to roll back and forth an exhaustive number of times on the test site. Light gates in the plastic rail trajectory automatically measured the speed of the sphere. This also made it easy to determine whether and to what extent the detection probability depended on the speed of the intruder and whether the IDS could be deceived, for example, by a fast passage or a very slow passage.
Knowledge of the precise operation of the IDS also offered the possibility to devise specific scenarios with which the system could be misled. These scenarios could be carried out on the test site with the IDS.
Just as the detection probability cannot be determined in absolute terms, it is easy to determine the likelihood of unwanted alarms. This property is also highly dependent on the specific environmental conditions and the principal operation of the IDS. With long-duration tests, it was possible to determine how certain factors (e.g. weather influences) quantitatively influence the probability of false alarms. For example, the correlation of undesired alarm behaviour with the meteorological parameters could lead to the conclusion that a certain IDS could only be used properly (on the basis of a quantitative criterion) up to wind force X and/or that it would not work in the case of frost formation. This also could lead to recommendations for the use of a combination of IDS that would fit the specific user requirements.
The basis for this contribution was derived from a 1999 article by ir. H.A. van Hoof.