Computer history: Scope 3.4 and NOS/BE permanent files

Scope 3.4 en NOS/BE permanente files

Although all file I/O was accomplished through local files, there was a separate system that kept track of permanent files (PFs). PFs were disk files that persisted between jobs. The relationship between local files and PFs was not intuitive, and was a frequent source of confusion and complaint from users. One had to request before writing a single byte that the file was allocated to a permanent file disk resource, otherwise the data ended in the bit bucket after the job or terminal session was finished. In principle, this was easy:

PFs were managed by the permanent file system, which was largely distinct from the portions of the operating system that did I/O to and from disk files. The PF system gave each permanent file a unique name containing 1-40 characters. Each permanent file was associated as well with a 1-9 character user ID or PUBLIC (no ID specified). Unlike local filenames, PF names could contain any Display Code character, though most filenames contained only alphanumeric characters.

Cataloging files

Local files were deleted from the system at job termination unless they were first entered into the PF system. This was done via the statement:
CATALOG,local_filename, permanent_filename, ID=username [,optional parameters].

This made the local_filename permanent by creating an entry in the PF system under the name permanent_filename. CATALOG did not make a copy of the data; the data still resided in the same sectors on the same disk drive. CATALOG called the PP-program PFC (permanent file catalog) that simply stored the file information and disk sector pointers into two special system files, the PF-Catalog (PFC) and the PF-Directory (PFD). The first one contained per permanent file the volume, segment and disk sectors where the data resided. The PFD contained the directory with per permanent file name PFC-pointer(s) to the (max 5) cycles. The PFC and PFD consistency was verified at each system deadstart (“boot”). Bit drops in the early 6×00 series of systems (no parity checks in memory) caused many times crossed files which the RECOVER process tried to resolve.

To access an existing permanent file, a user issued an ATTACH statement of the form:
ATTACH,local_filename, permanent_filename [,optional parameters].

This created a local file which was associated with the named permanent file. For the remainder of this job, I/O to local_filename was done to the permanent file previously cataloged under permanent_filename.

Permanente file attributen

In addition to the 40-character file name, permanent files had these attributes:

  • A 9-character ID. This field was pretty worthless. Its main function was to help group files; you could list permanent files with a given ID.
  • A cycle number, from 1-63. The default was 1. File cycles were like versions in DEC’s VMS operating system (which post-dated SCOPE and NOS/BE, of course). You could have up to five different permanent files with the same file name, with each file having a different cycle number. When you attached a permanent file, you accessed the highest-numbered cycle in the system unless you explicitly specified a specific cycle in the ATTACH statement.The contents of the different cycles need not be related, but in practise different cycles typically were different versions of the same program or data file. The concept of cycles was intended to allow the easy maintenance of commonly-used files. If you had a program that many people used and you wanted to provide a new version of it, you might not be able to modify the file if it was currently in use. So, you would create a file with the same name but a higher cycle number. People which currently used the older cycle would not be affected. Those people subsequently attaching the file would get the new version. The problem came when you got to cycle 63.
  • Retention period. This was the number of days before the file could automatically be purged from the system. 999 meant the system should never purge the file.
  • Permissions. There were five different access permissions associated with a permanent file. Each permission was controlled by a password with a maximum length of nine characters. Therefore, the use of passwords on files was much more common in SCOPE and NOS/BE than on other operating systems.By default, permanent files were cataloged with no special permission control, which meant that anyone on the system could do anything to the file. Nevertheless, it was rare for users to mess with each other’s files, and when they did, it was usually accidental.
    The following password-protected permissions were available, listed with their two-character CATALOG and ATTACH parameter name:

    • Read permission (RD). This allowed the user to read the file.
    • Extend permission (EX). This allowed the user to permanently add data to the end of a permanent file. Oddly, you could append to a file without EX permission. However, the appended data would not be a permanent part of the permanent file until an Extend operation was performed on the file. You needed EX permission to perform this Extend.
    • Modify permission (MD). This allowed the user to overwrite or rewrite portions of the permanent file that were already part of the permanent file system. I don’t think you needed MD permission to rewrite data that had been appended to a file but not Extended. I believe that Modify permission also allowed a file to be truncated.
    • Control permission (CN). This allowed the user to purge (delete) a file cycle. When set, this permission was also necessary to create a new cycle of an existing file.
    • Turnkey permission (TK). This redundant feature was a separate permission that essentially combined all the features of RD, EX, MD, and CN. Actually the permanent file catalog PP program PFC internally called a subroutine five times, once for each of the permissions, to set these passwords.
      If there was a TK password on a file, you needed to submit the TK password to attach the file. TK could be seen as the “master lock”. Even if you knew the RD, EX, MD, and CN passwords, or if there weren’t these passwords on a file, you needed to provide the TK password to gain any access to the file.

    Passwords were used for more than just security reasons. If a file was attached with only Read permission and no other user currently had the file attached with anything more than Read permission, then multiple users could attach the file. This was called Multi-Read access. Putting CN, EX, and MD passwords on a file, but no RD password was therefore common. This allowed multiple users to use the file simultaneously. Also, when multiple jobs/users could be active, one could allow multiple access using “higher permissions” by specifying the multi-read (MR=1) parameter.

PRUs and disk allocation units

Disk data on disks was given to the user in multiple chunks of Physical Recording Units or PRUs. Each PRU contained 64 words of 60 bits. The disk allocation unit was the minimum disk unit that could be allocated to a permanent file. The disk allocation unit size had a minimum that depended upon the physical device characteristics. During the disk initialisation process, larger allocation units could be specified. Larger allocation units required less system overhead in reserving and assigning data space and reading it back. The drawback was a larger disk space waste as the last block contained much more unused bytes.

Indirect Permanent Files (IPF)

For small files, the disk space waste could be enormous, especially when a large disk allocation unit setting was used. Wasted disk space of 30-40% could occur. The (KRO)NOS ‘sister’ operating system main intent was to support interactive work in contrast to the batch oriented NOS/BE had a separate permanent file concept targeted on small files. This was the Indirect Permanent File concept.
The University of Adelaide implemented in Pascal this Indirect Permanent File concept for NOS/BE. Small files could be SAVEd in, or GET from, a single permanent file which acted as container for a set of indirect permanent files (IPF). The total disk waste size of many small permanent files was reduced to that of one single file. As the IPF-system was fast and required easy short command names, it became popular at many sites as well, including at TNO-FEL.

(with special thanks to Mark Riordan, MSU who provided the basis for this page)