Data organization

Input from the task’s keyboard and output to its screen (terminal or monitor) have no real data organization. The data is anything from a single byte to a stream of bytes with no real concept for records or files. Data stored on storage media such as disk or tape, however, can be organized in three different ways based on how the data is to be stored and retrieved:

Sequential access, where data records are read or written one after the other in a specific sequence. That sequence might be based on chronology (when the record was written) or on the value of a sequencing key (index number or key field).

Random access, where data records are read or written individually based on some value associated with the record. This value is normally referred to as the key of the record.

Indexed sequential access, in which records are arranged in logical sequence by key. Indexes to these keys permit direct access to individual records.

Each access method has its advantages and disadvantages: sequential is normally faster than random, providing you want to process all records in their order; random allows unordered processing and is essential for interactive, unsorted transaction processing. Thoroughbred Basic provides different file types for each access method, with differing characteristics, to provide the programmer with tools to efficiently store and retrieve data.

Sequential data access

There are three file types that are designed for sequential access, although all file types allow the access of data records sequentially:

INDEXED: Sequential files with fixed record lengths.
SERIAL: Sequential files with variable record lengths.
TEXT: Sequential files with no records, only a sequence of bytes.

The first two file types can be opened and read (using the READ directive) starting with the first data record in the file. Each successive read accesses the next sequential data record in the file until you attempt to read beyond the last record, which generates an ERR=02 signifying end of file. Thoroughbred Basic offers one additional feature for these sequentially organized files. When each record is written, it is assigned a unique number and its physical position in the file is linked to that number. This permits access to an individual record within a sequential file by its number (referred to as its IND value), and eliminates the necessity of reading every record from the front of the file up to the record desired.

Since TEXT files do not recognize the concept of records, IND is used to refer to the starting byte number in the file, and the READ uses SIZ= to determine how many bytes to read.

You cannot remove a record from a sequentially organized file, just as you cannot remove a piece out of the middle of a sequential magnetic tape. With INDEXED or TEXT files, you may write over any individual record or bytes since each record is of the same size. SERIAL files, with their varying record sizes, do not permit overwrite. The only way to write into an existing SERIAL file is to open the file and issue a LOCK directive, which prohibits anyone else from writing to the file while you have it locked.
The first write adds a data record just after the last record in the file, and all other records are sequentially written from there. SERIAL files are very useful for spooling printouts where each printed line is logically sequential in order but may vary significantly in length from one printed line to another. TEXT files are very useful when interfacing to system-generated, flat files such as those produced by the UNIX editors vi and ed.

Object libraries

Starting with release level 8.2, programs can be stored in files called Thoroughbred Basic object libraries. A Thoroughbred Basic object library is a collection of Thoroughbred Basic programs in one file, with a table of contents. Opening the file loads the table of contents into memory, which allows for faster program loading and increases the speed of applications that use many different program modules.

When Thoroughbred Basic tries to load a program into memory using directives such as LOAD, RUN, CALL, or ADDR, it searches the table of contents before it searches the disks.

The object library is a TEXT file in the following format:

Bytes: Description
01 - 50: Reserved for Thoroughbred Basic internal use.
51 - 52: $5441$ (TA). This is the only number that shows that this TEXT file is an object library.
53 - 54: Maximum length of program names in the table of contents (NL below).
55 - 58: Offset to start table of contents.
59 - 60: Length of table of contents.

Table of contents entry

01 - NL: Program name (null or space filled).
NL + 1 - 4: Starting byte address of program, from beginning of the file.
NL + 5 - 6: Program length.

Thoroughbred Basic expects a sorted table of contents.

To open an object library and load its table of contents into memory, use the following syntax:

OPEN (channel,OPT=“OLIB”) libfile$

Random data access

Although it is normally not logical for a file to exist in any order other than some form of sequential, it is often desirable to access each data record in non-sequential order. The IND record number used in sequential data access is usually not sufficient for the purpose of distinguishing one record from another. That normally requires differentiation by some name, field, or group of fields; and Thoroughbred Basic refers to that differentiator as a key.

Just as the IND record number pointed to a specific record in the file, there is a KEY function that points to a specific record in a random data access file structure. Unlike the IND record number, however, this KEY is a string of characters and keys are kept in sorted order (collating sequence). That sorted order is referred to as the sequential order of the file, even though it has no correlation with the actual order in which the data records were written.

In reality, random data access is accomplished through the use of two files for each data file: one that holds the KEY values and points to the data record in the file for each key, and one that actually holds the data records. There are three Thoroughbred Basic file types that support random data access:

SORT: Composed of single dimension keys only and no associated data records. These are normally used as cross-keys into another file or can be used to provide the Thoroughbred Basic programmer a one-pass sorting capability for any data file.
DIRECT: Composed of keys and associated data records. Each data record has a single, unique key. One key, points to only one record and one record has only one key. The actual key value need not be part of the data in the record.
MSORT: Composed of multiple keys and associated data records. Unlike DIRECT files, MSORT files have multiple levels of keys and permit access to data records by primary key or by any of the defined secondary keys. With MSORT files, the keys must be part of the data in the record. The primary key must be unique within the file, but there may exist duplicates among secondary keys.

Each file type has its specific uses and advantages. The time needed to get a key in a SORT file is generally less than the time needed to get a data record in a DIRECT file, which is generally less time than in an MSORT file. MSORT files, however, offer access to data records in multiple keyed orders, with Thoroughbred Basic maintaining the integrity of keys and data. A comparable effect could be accomplished through the use of a DIRECT file and one or more associated SORT files, but the programmer then assumes the responsibility for maintaining the integrity between the DIRECT file data records and the SORT file pointers. This 2-file approach normally takes more time overall than MSORT alone in reading and writing data records.

Indexed sequential access

Thoroughbred Basic offers one file type that uses indexed sequential data organization: TISAM. It should be noted that the MSORT file type, discussed earlier in random data access, actually uses a limited sequential access data organization (a separate index file with pointers into the actual data file), but was implemented primarily to provide multiple-keyed access to a random file as opposed to only single-keyed access provided by the DIRECT file type.

TISAM files are comprised of multiple keys and associated data records with multiple levels of keys, permitting access to data records by primary key or by any of the defined secondary keys. As with MSORT files, TISAM requires that the keys must be part of the actual data in each record. The primary key must be unique within the file, but there may exist duplicates among secondary keys unless the secondary key is designated as a unique key sequence.

TISAM files use a two-file structure that is comparable to C-ISAM files, commonly found in the UNIX environment. Thoroughbred Basic READs and WRITEs records from or to a C-ISAM file provided that the data structures used within the file are compatible with Thoroughbred Basic data structures. It should be noted that the concept of fields in a file, delimited by the field separator character $8A$, are not recognized in TISAM files. Each record is treated as a single field, made up of bytes or groups of bytes as defined by use. All other Thoroughbred Basic file types (except TEXT, where only bytes are recognized), accept the concept of fields and field separators, allowing for a difference between READ and READ RECORD or WRITE and WRITE RECORD.

From the Thoroughbred Basic Developer Guide, version 8.9.1. Open this page in the PDF