April 1980 - 1 - _ ____________ 1 Introduction There are two programs, which run as separate tasks under the DEIMOS operating system [1], to control the transmission of information on the ring. The programs are written in IMP77 [2]. The first is a device driver for the ring interface, it provides for _____ basic _____ [3] transfers on the ring, initialisation of the interface and block notification of power failure on the ring. The ring driver is designed to be used by several tasks each of which specifies to the ring driver a range of input ports for which it will accept input. The second program provides an implementation of the byte stream protocol BSP [4], using the ring driver to perform the data transfers. The BSP handler, on initialisation, notifies the ring handler that all input to ports in the range 1 up to the maximum number of byte stream ports allowed (currently 15) should be routed to itself. As both these programs were developed for a PDP11/34 acting as a front-end for the EMAS operating system they assume that two other tasks are concurrently running. These are a buffer manager (task number 17) and a task to buffer output to the control terminal (task number 21, device .T3). The external interfaces to the ring driver and the byte stream protocol handler are described in the next two sections. Some of the features of the ring driver were designed for use solely by the BSP handler, they should be ignored by other users. Throughout this document the terms packet and data packet are used to refer to the smallest unit of information which may be transferred on the ring. _ ______ ____ ______ 2 DEIMOS ring driver The ring driver links in as task number 10 with interrupts on numbers -6 and -7, the task name is RING. It assumes that the buffer manager is task 17, ring driver segment 4 is mapped onto buffer manager segment 5. Output on stream 0 is sent to task 21 which is normally a task to buffer output onto the control terminal. Stream 1 is used to output the trace buffer of data transferred to and from the ring. _._ _____ _________ 2 1 Input buffering The mechamism for allocating input buffers is initially very simple, it may need to be extended as experience is gained of the performance of the system. The mechanism is as follows :- the ring driver keeps a pool of input buffers (initially 8 in number) of length 256 bytes. Four bytes are reserved by the buffer manager and 2 are needed to hold the source address, this leaves 250 bytes for the incoming data (including the port number). When a basic block is input to one of the buffers and the checksum is verified the port number is compared with the list of port number ranges notified by user tasks to the ring driver. If the port number does not lie within one of these ranges the input is ignored and the message "RING:INPUT REJECTED port address" is output to the control terminal. Otherwise the buffer is passed to the user task and the ring driver requests another buffer from the buffer manager. If an input buffer is not available the input is ignored and the message RING:NO BUFFERS is output to the control terminal. The assumption is that higher level protocols will limit the rate at which input is received, and be able to recover from an occasional missed transfer. This should apply to the byte stream protocol handler. The April 1980 - 2 - advantage of this mechanism is it's simplicity and low overhead both in terms of the number of dedicated input buffers required and the number of messages required to administer it. _._ ______ _________ 2 2 Output buffering The output buffering is explicit at the user level. The ring driver allows user requests for a buffer to be returned to the buffer manager at the end of a transfer and/or the notification to the initiating task that an output transfer has been completed. _._ ________ 2 3 Timeouts The ring driver enforces a timeout of 2/5 of a second on a complete input or output transfer. If a timeout occurs a message is output to the control terminal giving the source station address, also in the case of an output transfer a message is sent to the task which initiated the output. _._ ___________ _______ 2 4 Compilation options Within the source of the ring driver there are three sets of optional statements to give debugging and monitoring facilities. Such statements are each marked with a comment of the form !#letter where letter is S, E or T. The editor may be used to systematically comment in or out particular options. _._._ ___ _ (_____) ______ 2 4 1 The T trace option This option keeps a cyclic trace of data transmitted to and from the ring interface. An INT T or a PON message is used to cause the current trace buffer to be output to stream 1 (see 2.5.1 and 2.6.2.5). The contents of the trace buffer are a type flag (a letter) and a 16 bit data item. When the buffer is output, consecutive data items of the same type are output on a new line preceded by ********** type-letter. The type letters are I,O,S,X and E. I and O refer to 16 bits of data input or output to the ring interface. S is a clock tick entry, currently this is every 2/5 of a second , the data item is the number of clock ticks since the start of the rng driver. X is a timeout event, it means that an input or output transfer was not completed in a minimum of one clock tick. The E event is when a data packet input or output is altered by having 1 added to test error handling by higher level protocols. The data altered will be the preceding item in the trace buffer, see the E option below. _._._ ___ _ (_____) ______ 2 4 2 The S stats option The S option is to keep statistics of the number of data transfers on the ring. An INT P is used to enable ( INT O to disable) the output of information to the control terminal. If the output is enabled a line of 11 numbers is output every minute referring to the previous one minute period. The meanings of the numbers is given in the table below. Note that the values are accumulated as 16 bit quantities so that some values may overflow during heavy traffic periods. April 1980 - 3 - 1 : Total number of retries of output packets. 2 : 3 : 4 : number of output transfers requiring 0,1,2,3,4,5 5 : retries of the data. 6 : 7 : 8 : number of output transfers requiring >5 retries. 9 : total number of data packets input. 10 : total number of data packets output. 11 : number of interrupt task activations. _._._ ___ _ (______) ______ 2 4 3 The E errror option The E option allows deliberate corruption of every 1000th data packet input or output to the ring so that the error recovery mechanisms of higher level protocols may be tested. The facility is initially disabled, it may be enabled and subsequently disabled using INT E. As this facility is intended for testing it assumes that the T option is also included. _._ ___ __________ __________ 2 5 INT characters recognised _._._ ___ _ 2 5 1 INT T This causes the current contents of the trace buffer (see 2.4.1) to be output to stream 1. The message DONE is output to the control terminal when this has been done. Note that if this output is sent to a file the ring driver will need to be stopped (using HOLD RING, KILL RING) so that the file is closed in a sanitary fashion. However at the end of the trace several lines of asterisks are printed to ensure that the last block of trace information is written to the disc. If the system is rebooted the temporary file (#name) thus obtained may be renamed and listed. _._._ ___ _ 2 5 2 INT E The E option (see 2.4.3) is initially set to be not active (EFLAG=0), EFLAG may be toggled on/off using INT E to enable and disable this facility.The new value of EFLAG is echoed when the INT E is recognised. _._._ ___ _ 2 5 3 INT P This enables the output of traffic statistics (see 2.4.2). _._._ ___ _ 2 5 4 INT O This disables the output of statistics (see 2.4.2). _._ ___ ________ __ ___ ____ ___ ____ ______ 2 6 PON messages to and from the ring driver April 1980 - 4 - _._._ ______ ______ 2 6 1 buffer format The format of buffers for input and output is as follows. ____________ ringbuff( ______ (ringbuff) ____ link, ___________ length, type, recordformat record name byteinteger _______ address, ____________ data(0:124) ) integer integerarray link may be used to chain buffers together within user programs. length contains the length as used in the header packet of a ring basic block, in the case of single packet output length should be the number of elements of data to be output. type is used by the buffer manager, it should not be changed. address this is the ring address of the source of input and the destination of output. data The first length+2 elements of data are the port number (in data(0) ) and the data part of a basic block. On output a checksum is generated and sent. On input the checksum may be zero (and is checked to be so) if the corresponding flag bit is set in the header packet, if the checksum fails the input is ignored and a message put on the control terminal. Values are input and output as 16 bit quantities. _._._ ________ __ ___ ____ ______ 2 6 2 commands to the ring driver The format of the PON record depends on the function required. Bytes 0 and 1 of the PON record are the task number of the ring driver (=10) and the number of the task requiring the service. These are omitted in the table below. byte 2 3 4 5 6 7 function enable port 0 - lower port limit higher port limit *output buffer 1 port no. buffer pointer - - *output sp buf 2 - buffer pointer - - xrdy 3 port no. destination port address sequence xnotrdy 4 " " " " xnodata 5 " " " " xclose 6 " " " " xreset 7 " " " " xclosereq 8 " " " " output trace 16 " " " " * These function codes may have the following flag bits added if the additional actions are required. release flag (=128) The ring driver will release the output buffer back to the buffer manager after it has been output. April 1980 - 5 - tell flag (=64) At the end of output a PON message will be sent from the ring driver to say that the buffer has been output. The port number field of the record is not used by the ring driver but is echoed in any PON sent in reply, transfer failed for example. The commands between 3 and 8 are intended for use by the BSP handler. The other functions are described below. _._._._ ______ ____ 2 6 2 1 Enable port This command is used to signal that a particular task will take input with port numbers in the specified range. No check is made on the consistency or overlap of port ranges. The current version of the ring driver will store 8 such port ranges, any further enable port entries will be ignored and the message "RING:TOO MANY PORTS" sent to the control terminal. _._._._ _______ ____ 2 6 2 2 Disable port Nullifies a previous "Enable port" call from this task, the port limits should be the same as on the "Enable port" call. _._._._ ______ ______ 2 6 2 3 Output buffer This command causes a basic block to be sent to the address specified in the buffer record. The count part of the header packet is as specified in the length field of the buffer. If the transfer fails, that is the whole transfer is not completed within the timeout period of 2/5 of a second then a transfer failed message is sent to the originator of the request. Normally the buffer is not released back to the buffer manager so that higher level protocol software may request retransmission. The options of releasing the buffer at the end of output and signaling the originator at the end of output are indicated by adding flag bits to the function code. _._._._ ______ ______ ______ ______ 2 6 2 4 Output single packet buffer This function is used to output a block of single packet transfers. This is intended mainly for booting machines on the ring. No header packet or checksum are sent just the contents of the data array as 16 bit values. If the transfer fails, that is the whole transfer is not completed within the timeout period of 2/5 of a second then a transfer failed message is sent to the originator of the request. The options of releasing the buffer at the end of output and signaling the originator at the end of output are indicated by adding flag bits to the function code. _._._._ ______ _____ 2 6 2 5 Output trace This command forces the output of the trace buffer (see 2.4.1) for debugging. _._._ ________ ____ ___ ____ ______ 2 6 3 Messages from the ring driver Bytes 0 and 1, the destination and reply task numbers, of the PON record are omitted in the table below. April 1980 - 6 - byte 2 3 4 5 function output done 0 port no. - output error 1 port no. - input done 2 port no. buffer pointer input error 3 port no. - ring offline 4 - - The port no. is an echo of that sent by the originator of the output request. The port number of an input transfer is in the first element of the data array in the input buffer, the input buffer should be released back to the buffer manager after being processed. If the ring goes offline a message is sent to the control terminal and a PON message sent to all tasks which have enabled ports to the ring driver. Input being subsequently received by such tasks signals that the ring is online again. _._ _______ __ ________ 2 7 Summary of messages The messages output by the ring driver to the control terminal are summarised below, with explanations where necessary. Output from the statistics option is covered in section 2.4.2 . RING OFFLINE nn This message is output when the first 'enable port' command is received , nn is the ring address of this machine. *************** RING SWITCHED OFF RING:TOO MANY PORTS >a Too many enable port commands have been received, currently only data about 8 port pairs can be stored. RING:QUEUE FULL Output commands sent to the ring are queued until the output can be performed, currently a maximum of 32 requests can be held. RING:CHECKSUM FAIL FROM nnn RING:NO BUFFERS There is no input buffer in which to store available input data, the input data is input and ignored. RING:NO I BUF An output transfer to this machine address has been received but there is no input buffer available to copy it to. April 1980 - 7 - RING:INPUT REJECTED ppp nnn Input from address nnn to port ppp has been received but the port has not been enabled. RING:OUTPUT TIMEOUT TO nnn RING:INPUT TIMEOUT FROM nnn _ ____ ______ ________ _______ 3 Byte Stream Protocol Handler The BSP handler was designed as a replacement for the Nodes Standard Interface (NSI) protocol handler (called GATE) which provides a functionally similar task for the Edinburgh Regional Computing Centre Network (RCONET). The interface to the BSP handler has been made as similar as possible to that for GATE. The BSP handler has the task name BSPS, it links in as task number 16. It assumes that the buffer manager is task 17, BSPS segment 4 is mapped onto buffer manager segment 5. Output on stream 0 is sent to task 21 which is normally a task to buffer output onto the control terminal. _._ _________ 3 1 Buffering Buffers used for input and output should have space allocated in them for the BSP overhead. A suitable format is: ____________ bspbuff( ______ (bspbuff) ____ link, recordformat record name ___________ length,type, byteinteger _______ address, port, rcommand, tcommand, integer ________________ data(0 : 243) ) byteintegerarray link may be used to chain buffers within the user task. length contains the number of bytes of data in the array. type is used by the buffer manager, it should not be changed. address this is the ring address of the source of input. rcommand and tcommand these are the byte stream protocol commands, they should not need to be accessed except possibly in the case of input when tcommand has bit 2 set (=4) if this is a "close request" i.e. te last block of data. Although this is signalled anyway by BSPS see 3.2.2. April 1980 - 8 - data this is the data part of the buffer. Buffers used for output should be obtained from the buffer manager and passed to BSPS. When the data has been output and acknowledged the buffer is released by BSPS back to the buffer manager. Input buffers are obtained by the ring driver and should be released by the user task back to the buffer manager after they have been processed. _._ ________ __ ____ ______ ____________ 3 2 examples of byte stream transactions Normally a "conversation" will ensue between BSPS and a user task in order to set up and transfer data along a "byte stream". The examples following illustrate the special cases of a byte stream used only for input and a byte stream used only for output. In general a byte stream is full duplex. The various messages to and from BSPS are listed and their parameters described fully in a later section. In the examples the names of commmands and responses are written in upper case. _._._ _______ _ ____ ______ ___ ____________ ____ 3 2 1 Opening a byte stream and transmitting data User:OPENCALL specifying the ring address of the destination and the function code to be used on the open. The user also specifies an internal stream number which is echoed by BSPS in any replies to this open. This enables a user task to send multiple OPENCALLs and to distinguish between the replies. Note that the address of the ring destination is required. The user task may have obtained a physical ring address by accessing the ring name server using the ring driver directly.This is beyond the scope of the byte stream protocol. BSPS:OPEN REPLY A this response from BSPS indicates the byte stream port number which has been allocated in response to the OPENCALL. This port number is used to identify the byte stream in all further messages in this "conversation". The user stream number is echoed to identify this response. BSPS:OPEN REPLY B this response indicates whether the destination has accepted or rejected the open. If the latter then the conversation is finished otherwise a byte stream is now available. User:PUT OUTPUT specifying a buffer BSPS:OUTPUT DONE signifying that the data has been accepted at the destination. Further calls of PUT OUTPUT to BSPS may be made before the OUTPUT DONE is received by the user task; the output requests will be queued and an OUTPUT DONE sent as each is completed. April 1980 - 9 - (repetitions of PUT OUTPUT/OUTPUT DONE) User:CLOSE CALL this is implemented by BSPS as a transmission of null data with close request specified. This is the normal termination procedure. Calls of PUT OUTPUT after the CLOSE CALL will not be acknowledged with OUTPUT DONEs and the buffer will be immmediately released back to the buffer manager. BSPS:CALL CLOSED when the close command is received from the destination. At any time the byte stream may be aborted: User:ABORT CALL any outstanding PUT OUTPUTs will not be acknowledged. BSPS:CALL ABORTED the byte stream has been closed. If the destination fails to respond properly to the byte stream transfers or the ring goes down then BSPS will abort the byte stream. BSPS:CALL ABORTED User:ABORT CALL _._._ _________ ____ __ _ ____ ______ 3 2 2 Receiving data on a byte stream In this case, assuming that the open will be received by BSPS, the user task needs to specify that opens with particular function codes should be directed to itself. Currently incoming function codes should be between 0 and 25. These correspond directly with the facility numbers used on RCONET. User:ENABLE FACILITY specifying a function code (facility number) between 0 and 25. BSPS:INCOMING CALL the byte stream port number, function code and source address of the open are given. April 1980 - 10 - User:CALL REPLY indicating whether the open is rejected or accepted. If the latter then input is enabled on the byte stream. BSPS:INPUT DONE an input buffer is passed to the user task. User:ENABLE INPUT allows another input buffer to be received. (repetitions of INPUT DONE and ENABLE INPUT) BSPS:INPUT DONE the last data block will contain bit 2 (close request=4) set in the tcommand field of the input buffer. The data may be null i.e. length=0. User:ENABLE INPUT optional. BSPS:CALL CLOSED explicit indication of the close request. User:CLOSE CALL causes initiation of the byte stream close. As in the previous example the byte stream may be aborted by the user task or by BSPS in the case of failure. The above examples illustrate all the byte steam facilities and commands. A byte stream port is inherently full duplex and may be used in that fashion. In the single duplex case there is no requirement for it to be the data transmitter that initiates the opening of the byte stream as is done for simplicity in the examples above. _._ ____ _______ ________ 3 3 Send Message Commands A set of commands have been implemented in BSPS in addition to the byte stream commands. These enable operator messages to be sent on the ring. They are sent as "single shot transfers" [5] with no error correction or flow control, they correspond to the "send message" facility available on RCONET. A byte stream port is allocated temporarily to control the transfer and this port number is specified in reponses. The buffer format is as follows: ____________ messagebuff( ______ (messagebuff) ____ link, recordformat record name ___________ length,type, byteinteger _______ address,port,sscommand,reply port,function code, integer ________________ message(0:241) ) byteintegerarray link and type are as before. April 1980 - 11 - length is the length in bytes of the message. The integer fields of the record should not need to be accessed. The message is in ascii code in the byte array. _._._ _______ _ _______ 3 3 1 Sending a message User:OPEN MESSAGE specifying the ring address, function code and a buffer containing the message. BSPS:MESSAGE REPLY indicating whether the message was accepted or not and giving the buffer containing the reply. If BSPS does not receive a response to the message within the timeout period of 1 to 2 seconds then the byte stream port is freed with no indication to the user task. _._._ _________ _ _______ 3 3 2 Receiving a message User:ENABLE FACILITY giving the function code for which messages are to be accepted. BSPS:MESSAGE RECEIVED giving a pointer to the buffer containing the message, this buffer should not be released by the user task as it is used to send the reply by BSPS. User:CALL REPLY indicating that the message is accepted or rejected. In either case the byte stream port is deallocated. _._ ________ __ ____ 3 4 Messages to BSPS This section lists and specifies the parameters of the PON commands to BSPS. Bytes 0 and 1 of the PON record are the task number of BSPS (=16) and the number of the task sending the message. These are omitted in the table below. byte 2 3 4 and 5 6 7 function buffer pointer ENABLE FACILITY 1 - - - fn code DISABLE FACILITY 2 - - - fn code CALL REPLY 3 port no. - - result* ENABLE INPUT 4 port no. - - - PUT OUTPUT 5 port no. pointer - - CLOSE CALL 6 port no. - - - ABORT CALL 7 port no. - - - OPEN MESSAGE 8 - pointer address fn code OPEN CALL 9 stream no. - address fn code Notes * result should be 128 if this is a response to a message, otherwise a non-zero value implies OK and a zero implies rejection. April 1980 - 12 - _._ ________ ____ ____ 3 5 Messages from BSPS Byte 1 of the PON record will be the task number of BSPS (=16). byte 2 3 4 5 6 7 function INCOMING CALL 2 port no. - - address fn code INPUT DONE 3 port no. buffer pointer - - OUTPUT DONE 4 port no. - - - - CALL CLOSED 5 port no. - - - - CALL ABORTED 6 port no. - - - - OPEN REPLY A 7 stream no+ port no.@ - address fn code OPEN REPLY B 8 port no. - - address =0 => OK MESSAGE RECEIVED 9 port no. buffer pointer* address fn code MESSAGE REPLY 10 - buffer pointer - =0 => OK Notes + the stream no. is that specified on the OPEN CALL to BSPS. * this buffer should not be released back to the buffer manager. @ a value for the port no. of 0 means that all ports are in use. _._ ________ _______ __ ____ 3 6 Messages printed by BSPS BSPS outputs information about certain errors to help in developing byte stream protocol implementations. Byte stream protocol errors give a message of the form: ********** ERROR- n address where address is the ring address of the other end of the byte stream. Also a PON is sent to the ring driver task to cause it to output it's trace buffer (see 2.4.1). The table below gives the meanings of the various values of n. error number 1,2 A RDY command has been received with the correct sequence number but the byte stream is not in the correct state (ESENT or NESENT). 3 A RDY command has been received with the wrong sequence number. 4 A DATA command has been received with the wrong sequence number. 5 A DATA command has been received but the byte stream is in the wrong state (not ESENT or NESENT). 6 OPEN ACK has been received with the byte stream in the wrong state. If there is an error in a PON message to BSPS the message: BSPS:PERROR- n (5 byte values) is output, the byte values are bytes 1,2,3,6,7 of the PON record (see the table in section 3.4). The error numbers are: April 1980 - 13 - error number 1 Invalid function. 3 Function code too big on ENABLE/DISABLE FACILITY. 4 CALL REPLY function with the byte stream not in the correct state. The other messages output by BSPS are reasonably self-explanatory. BSP:ILLEGAL INPUT FROM nnn BSP:TIMEOUT DEST=nnn An essential element of the protocol has not been acknowledged. BSP:RETRANSMIT TO nnn Data retransmission has been requested with a repeated RDY. This may be an "idle timeout" request. _ __________. 4 References [1] DEIMOS: User Manual B. A. C. Gilmore ERCC [2] IMP77 Computer Science Dept. Univ. of Edinburgh. [3] Basic Ring Transport Protocol (version 2) 27:10:78 R. D. H. Walker, Computer laboratory, Cambridge University. [4] Ring Byte Stream Protocol Specification April 1980 M. A. Johnson, Computer Laboratory, Cambridge University. [5] A Protocol for "Single Shot" ring transaction. 19.4.79 N. A. Ody, Systems Research Group, Computer Laboratory,Cambridge University of Kent Computing Laboratory at Canterbury RING programs for the DEIMOS operating system S.E.Binns Date April 1980 ________ ____ Contents Page 1 Introduction 1 2 DEIMOS ring driver 1 2.1 Input buffering 1 2.2 Output buffering 2 2.3 Timeouts 2 2.4 Compilation options 2 2.5 INT characters recognised 3 2.6 PON messages to and from the ring driver 3 2.7 Summary of messages 6 3 Byte Stream Protocol Handler 7 3.1 Buffering 7 3.2 examples of byte stream transactions 8 3.3 Send Message Commands 10 3.4 Messages to BSPS 11 3.5 Messages from BSPS 12 3.6 Messages printed by BSPS 12 4 References. 13 Copyright University of Kent at Canterbury 1980