Camera Surveillance (1995 – 2005)
At the end of the 1990s, TNO Den Haag Waalsdorp obtained its first civil contracts in the field of camera surveillance from municipalities, Schiphol Airport and the Dutch Railways. Much knowledge about the use of electronic cameras was developed in a military context, such as research into infrared equipment and (infrared) cameras in aeroplanes. However, camera surveillance research encompasses much more than technical aspects of cameras. It also concerns the design of central observation areas, the positioning of the cameras, the analysis of suspicious human behaviour, the interpretation of group behaviour and data protection (privacy; pre-GDPR). Existing knowledge available at TNO Human Factors, currently TNO location Soesterberg, was also used.
After the first contracts for researching camera surveillance in public spaces, a stepwise plan was drawn up to optimise camera surveillance using the available knowledge:
- The improvement of the camera image.
The images had to be sharpened to better recognise people, objects and vehicles.
- The analysis of the camera images.
This required programming of the cameras to select certain image parts and behaviours. For example, when people walk to the right, the camera can indicate this. More complicated movements followed later. Automating such analysis required a lot of research.
The camera must send signals to the observer about certain behaviours and objects. The observer plays a crucial role. He/she must identify certain issues of interest, such as deviant behaviour, and act accordingly. The camera can support him/her in this task by providing a visual or auditory signal in the event of deviant behaviour or behaviour that can be linked to crime or threats. The observer must then interpret the observed behaviour. The latter is something that humans are good at, but part of that skill is intended to be transferred to the software of the camera by applicable algorithms. When successful, this should create a division of labour between humans and computers. Observers can then concentrate on smaller details while the computer provides a good overview and recognises patterns.
TNO developed a list of 196 deviant behaviours that deserve the attention of the observer. These behaviours range from nervous walking around to an elevated temperature of the observed person (certain types of advanced cameras can also measure temperature at a distance). TNO developed software for a camera that could recognise 48 deviant behaviours via image analysis, such as throwing objects, running, hitting, kicking and the like. If the camera detects such movements, it emits a signal. Similarly, certain types of sounds could be detected, such as shouting or screaming. For signalling, it is even possible to combine the detection of certain sounds with fast gestures or deviant behaviour.
To achieve optimal security use with Camera Surveillance, a collaboration of various parties is necessary. In the beginning, this primarily concerns communities, police, fire brigade, railways, railway police, etc. These parties often had no idea what they wanted to achieve with camera surveillance. TNO drew up an operational plan with them that could be translated into a functional design. This resulted in the technical requirements. Apart from the users of camera surveillance, the security industry, i.e. producers of security systems, suppliers of observation rooms, developers of video content, and other stakeholders, plays an important role. Essential are the guards who must learn to work together with the cameras. Proper training of these employees is a prerequisite for the effective use of camera surveillance.
Camera surveillance often raises the question of its effectiveness. To what extent do cameras contribute to the safety of public spaces? That question can only be answered in conjunction with the entire security system. Camera surveillance also raises completely different questions. The relationship to privacy is a matter that regularly engages politics and the media, or the use of cameras in demonstrations and protests. This also applies to the possibility of abuse by totalitarian regimes. Such themes were at that time also on the agenda of TNO.
The camera plan
To assess the effectiveness of cameras, the camera plan is examined. The camera plan consists of a map on which, among other things, the camera positions and fields of view are drawn. In addition, the camera plan includes a description per camera which states the other functional requirements. A certain surveillance location (spot of attention) can be shown either continuously or non-continuously. If an image of this location can only be obtained by operating a movable camera, this is referred to as non-continuous surveillance. Therefore, more surveillance locations cannot be shown or recorded simultaneously (continuously) in the control centre with one movable camera. Therefore, there is a chance that an incident at one of these locations will not be observed and/or recorded. If surveillance at such a location were to take place with a fixed camera, an image of this location would be continuously available and could be recorded.
Even more important is what you want to use the recorded image for.
For observation purposes, the starting point is that a person of average height (approx. 1.8 metres) must have a height on the monitor of at least 10% of the total image height at a standard monitor size and line resolution (at that time monitors were still cathode ray tube monitors). With this measure as a starting point, the maximum height of the camera’s field of view is fixed (and thus also the maximum width, depending on the camera’s aspect ratio, e.g. 4:3). Observation is sufficient to detect the presence of one or more persons.
The starting point for recognition purposes is that a person of average height on the monitor must have a height of at least 50% of the total image height at a standard line resolution. Recognition allows an operator to recognise a person as being the same as the one he saw shortly before. Recognition is necessary to be able to follow a person in the surveillance area or to pass on a description.
For identification purposes, it is assumed that a person of average height on the monitor must have a height of at least 120% of the total image height at a standard line resolution. This means that a person from the knees fills the image. Identification is necessary to distinguish a person from all other persons.
An example of a person’s size on a monitor for observation, recognition and identification is shown below.
The focal length of a camera lens to be chosen determines up to which (maximum) distance the requirement of observation, recognition or identification is met. There is a logical relationship between the parameters of camera height, camera range, focal length, resolution, CCD chip size and resolution. This relationship is always used for the design of the camera plan.
The figure below shows an example of how the field of view of a fixed ‘observation’ camera is represented in the camera plan. This fixed camera is used to observe people in a narrow street. In the indicated field of view, a person will occupy at least 10% of the monitor height. The drawn 50% limit is only indicated as an indication. But in practice, when the camera is placed at a height of approximately five metres, this distance also roughly corresponds to the beginning of the camera’s field of view, i.e. the end of the blind area below the camera.
The figure below shows an example of the total non-continuous field of view of a movable camera. If the movable camera is controlled by the operator, the limits of observation, recognition and identification are determined by the camera’s maximum zoom range.
The figure above is for one camera. For complex environments, such as stations, software is used to compare the camera setups already present and to be planned with the intended objectives (identification, recognition, observation).