Subject: Re: Procurement Procedures From: CCTA Date: Fri, 19 Jun 87 12:15 BST To: P.E.Williams@UK.AC.EDINBURGH In reply to: <17 Jun 87 15:17:33 bst 110889@BUSH> Via: UK.AC.EDINBURGH.EMAS-A ; (to uk.ac.edinburgh.bush) 19 Jun 87 12:40:18 bst Via: UK.AC.RUTHERFORD.GEC-B ; (to uk.ac.edinburgh.emas-a) 19 Jun 87 12:16:52 bst Msg ID: <19 JUN 1987 12:15:41 NJIN03@UK.AC.RUTHERFORD.GEC-B> Guide to the Procurement process for University systems funded by the Computer Board for Universities and Research Councils Part I The Operational Requirement 1. Introduction This guide gives advice on the preparation of an Operational Requirement for a university procurement. It is intended mainly for open-tender procurements of time-sharing systems, although parts of it may be relevant to other types of procurement. Part II discusses the procurement process after issue of the OR. 2. The purpose of an Operational Requirement (OR) The OR sets out for potential suppliers the details of the University's needs for which they are invited to submit proposals. It is an important document which must enable them to determine whether or not they should invest effort in competing for business. The OR must accurately state the requirement and clearly separate its essential aspects from those extras which are considered highly desirable and desirable. It is important that the requirement be agreed before the OR is issued. It cannot be evolved during the procurement stage as suppliers must have a firm statement on which to base their proposals. The OR also forms the basis on which suppliers' proposals must be evaluated. 3. Responsibility for OR preparation The University is responsible for preparing the OR, with advice from CCTA (CT1 Division, Riverwalk House and CT5, Norwich), the Joint Network Team, the Computer Board Secretariat and the Godfather. It is important to ensure that the OR: a) accurately and completely reflects the requirement, is technically sound, and internally consistent; b) describes a requirement rather than a solution to a requirement; c) is expressed in terms suitable for issue to the computing trade; d) does not unfairly favour any potential suppliers; e) will not require an unreasonable amount of equipment in relation to the cash limit; f) reflects Computer Board policy, particularly with regard to networking. 4. Content of the OR The OR must provide the background to the project, detailed requirements, and instructions to suppliers; the aim is to produce a document to which suppliers will respond with comprehensive proposals that are easy to evaluate. Investment of effort at the OR stage to arrive at a meticulously clear statement of the requirement will save effort all round in later stages in the procurement. The detailed requirements must specify the minimum set of facilities that must be provided, together with further facilities classified into two categories of "highly desirable" on the one hand and "desirable" on the other. Requests for information about proposed products may also be included in this section, but should be clearly identified as such, and not classified as requirements of the solution. Suppliers will be expected to offer a system which meets the complete minimum set of requirements; the real competition amongst suppliers will be based on facilities offered over and above the minimum. It is important that the OR indicates the basis on which proposals from suppliers will be evaluated. By definition, proposals which do not meet the full minimum set of requirements cannot be considered. In framing the content of the OR and in sub-classifying the minimum, highly desirable, and desirable facilities the university will have to have regard to the detail of the final evaluation exercise. Many suppliers are keen to talk to universities at an early stage during the preparation of the OR. While this can be a very useful source of information about what products are available, and what is likely to be a reasonable requirement in relation to the budget, great care should be taken that suppliers are not allowed to influence the university's view of its own requirements. Suppliers also find such discussions useful because they receive early information on the likely content of the OR. This is desirable because it means that suppliers have longer to consider the content of their proposals and they should therefore be able to make better submissions. However, it is important that all likely suppliers are allowed equal opportunity for such discussions so that the procurement can be seen to be being conducted fairly. In a letter covering the OR the cash available (which may combine Computer Board and University funds) will be stated. (A draft covering letter is included as Annex A.) This will be applied as a cash limit. Associated with this cash limit will be a related upper limit for annual recurrent costs of the system; the Board will normally meet annual recurrent costs of up to 10% of its capital contribution. Given the cash limit approach it becomes vital for the hitherto "mandatory" requirements to be scaled down to a realistic minimum such that all likely suppliers can meet that minimum. The structure of the OR and suggested wording is given in Annex B. The text is intended to be the basis for university ORs for general time-sharing systems. It does not provide a complete draft text which can be issued with only minor amendments, but a set of building blocks which can (and must) be tailored to meet individual requirements. Some of the text (particularly in Sections 1, 6 and 7) may be copied directly, but it is particularly important that Section 4 is studied closely to ensure that any features that are made mandatory are really essential, and that any special features that are peculiar to individual universities are included. Text in brackets ([...]) is comment and not intended for inclusion in the OR. The text is available via JANET or on 3.5" or 5.25" floppy disk (IBM PC format); send requests to: Rob Whetnall CCTA CT1 Riverwalk House 157-161 Millbank LONDON SW1P 4RT Telephone: 01-217-3096 JANET address: Whetnall@RL.GB 5. Issue of OR A list of suppliers with suitable equipment should be agreed with CCTA CT1; the number of suppliers who are invited to submit proposals will vary with the nature of the requirement, but a list of 10 - 15 companies is typical for most university procurements (more may be appropriate if the OR contains more than one major functional area and may best be met by a number of different suppliers). Responsibility for issuing the OR rests with the University. Annex A Draft covering letter for issue of OR to suppliers Dear Sirs University of : Procurement of Computing Service Equipment The University will shortly be replacing its current central service computer equipment in accordance with the procurement procedures of the Computer Board for Universities and Research Councils and invites proposals for its replacement based on the enclosed operational requirement. The central service provides facilities to all members and departments of the University. The procurement will be conducted on an open tender basis and it is a requirement of the Computer Board that a response to the operational requirement will provide enough technical and contractual information to allow the University to draw up a shortlist of suitable suppliers for approval by the Board. Those suppliers are expected to be in a position to submit satisfactory final tenders based on the responses to the operational requirement. You should therefore offer a viable solution to the University's requirement and specify costs, including educational discount and indicating whether VAT has been included, delivery dates (subject to an order being placed by a given deadline) and any other conditions to which you are prepared to commit or may wish to include in contract. For a number of reasons, suppliers may not be able to offer systems which seem likely to be acceptable by the University without further consultation with and clarification by the suppliers. Such cases should ideally be dealt with before shortlisting, but if the overall viability of the proposal is not in doubt, it may be acceptable to delay discussions until after shortlisting. The University will not proceed with any proposal that does not provide sufficient information to allow a considered view to be taken. I have to emphasise that your proposal must, at the least, satisfy all the minimum requirements listed but, in addition, the University will evaluate the number of highly desirable and desirable requirements which you propose to meet. It may be a requirement of evaluation that performance tests, including benchmarking, will be carried out where a supplier cannot demonstrate to the University's satisfaction that a proposed system meets the terms of the operational requirement. The University will be advised in this procurement by the CCTA on technical and contractual matters and by the JNT on communications. The capital sum available for this procurement is # ?.?? M including VAT at the current rate. This is the full and final figure and must cover all costs such as delivery and installation. There is also an upper limit of # ??? on annual recurrent costs which must cover all direct costs such as hardware and software maintenance (including third party software). Between now and the closing date, appointments may be made to see me and other staff of the Computing Service for any clarification of points from the OR which may be necessary. If your company can offer suitable equipment to meet the University's requirement within the cash limit, please send ? copies of your proposal to me at the above address to arrive not later than 12.00 noon on ...... Additional single copies of the proposal should be sent to each of the following by first class mail, postmarked no later than [date as above]: [Insert name] [Insert name] Computer Board for Universities CCTA and Research Councils Riverwalk House Elizabeth House 157-161 Millbank York Road LONDON SW1P 4RT LONDON SE1 7PH [Insert name] [Insert name] Joint Network Team CCTA Atlas Centre Gildengate House Rutherford Appleton Laboratory Upper Green Lane Didcot, OXON OX11 0QX NORWICH NR3 1DW Yours faithfully, Annex B Draft text for a University Operational Requirement University of Operational requirement for the supply of New computer equipment 1. Introduction 2. Background to the requirement 3. Outline of required system 4. Details of requirements 5. Communications requirements 6. Further information requested from suppliers 7. Instructions to suppliers Appendices [Acknowledgements: this OR contains ideas based on experience gained in many university and government procurements over the past few years. Particular thanks are due to Oxford University Computing Service and SWURCC for permission to use text from the ORs for their recent procurements.] [It is intended to update this document periodically to reflect changes in the requirements of universities, and the equipment available from suppliers. Universities should contact CCTA before starting to write an OR to check that they have the latest version of the text; universities should also contact CCTA at an early stage for advice on how to express their requirements if they differ significantly from those outlined in the text.] 1. Introduction [This section should provide a brief introduction to the OR and stress any important features of the following sections.] The University of shortly expects to install new computing equipment which will form the basis of its central academic computing service for a number of years. This document is the Operational Requirement for the supply of that equipment. Section 2 gives the general background to the procurement. Section 3 describes in outline the functions which the new equipment is expected to fulfil. Section 4 describes the detailed requirements which are divided into 4 categories: Minimum requirements: [previously known as mandatory requirements] Suppliers who are unable to meet all of these requirements should not submit proposals as the facilities requested here form an absolute minimum without which the University would not be able to mount a service: shortlisting of such a proposal would therefore be impossible. While it is strongly preferred that proposed solutions can demonstrably meet all the minimum requirements at the time of the proposal, in some circumstances it may be possible to accept proposals where requirements are met either by products that are already under development, or by commitments to special developments on an acceptable timescale. Highly desirable and desirable requirements: None of these facilities is absolutely essential, but suppliers should bear in mind that the highly desirable requirements are very important to the University, so it is unlikely that tenders that do not include a high proportion of these features within the budget will be successful. Information requirements Information requirements are used where the university has no specific requirement to be met, but requires answers to specific questions about the proposed system. It is essential that all information requirements are fully answered; failure to do so may mean that the university's evaluation cannot be conducted properly, and hence automatic exclusion of the supplier's proposal. Evaluation of tenders will be based on capital and recurrent costs and the extent to which the highly desirable and desirable requirements have been met, along with other criteria discussed in Appendix A. Suppliers should particularly note that the 7-year life over which the systems will be costed means that recurrent costs are an important part of the overall evaluation, and also that non-tendered costs (such as extra staff costs) can play a major part in the final decision. The procurement is subject to detailed communications requirements. Recent advances in data communications have led to the specification of non-proprietary network protocols for which support is now available on a variety of computer systems. Computer centres in universities and Research Council institutes use these protocols as the mechanism for giving access to their computer resources. For equipment already installed, the protocols have often been implemented by means of development projects. However, new equipment purchased is expected to support the protocols on delivery or from an early date. Wherever possible, internationally agreed standards are being adopted: for those functions where stable international standards do not yet exist, interim standards have been chosen. The communications requirements are given in Section 5. Because of the need for the new system to interwork immediately with existing equipment both on and off site, the communications requirements are mandatory except where specific indications are given to the contrary. Subsequent sections deal with additional issues, including the timetable and contractual conditions which apply to the procurement. The procurement is subject to a fixed limit for capital and recurrent expenditure which is given in the covering letter to this document. Costs to be covered by this sum are for hardware and software to meet all the minimum requirements, including any charges for delivery and installation. Proposals will be considered from manufacturers acting either as sole supplier of all hardware and systems software, or as supplier taking prime contractual responsibility for all hardware and systems software, whether directly supplied or subcontracted to third parties. In principle, a final contract placed with a single vendor on either basis will be strongly preferred. However, the University reserves the right to discuss with manufacturers alternative sources of supply for any part of the system (hardware or software) if it appears that there could be significant performance and/or cost advantages in doing so. This is particularly significant for applications software where educational licences may be available only to the University. All such costs must still be met from within the overall budget. Discussion on this subject would normally be expected to take place after shortlisting, but they may be conducted earlier if necessary. [In general, CCTA advise that a prime contractor should be sought to meet all aspects of the OR. However, many university requirements are so complex that no supplier can fully meet them from their own product range, and supplier's profit margins for university procurements are so low (or non-existent) that they will nearly always charge a very high handling charge on any third-party equipment for which they are expected to take responsibility. Therefore, it is often necessary in practice to accept multiple contracts in order to meet the requirement within the budget. This is acceptable, but acceptance conditions should be very carefully specified to ensure that the total system will function correctly.] Technical queries should be directed to: [Insert name, address and telephone number here] Contractual queries should be directed to: [Insert name] CCTA Gildengate House Upper Green Lane NORWICH NR3 1DW Telephone: Norwich (0603) 69[insert number] (direct line) or 0603 660181 2. Background to the requirement [This section should briefly discuss the following topics: the university, including student numbers and important departments; the role of computing within the university, including any significant facilities outside the computer centre; the computer centre, and its role within the university; areas of particular expertise within the computer centre; existing facilities and systems run by the computer centre; typical uses of the existing computer centre equipment, and major users either in terms of departments or applications; use of the National Centres, and other off-campus systems; use of JANET; strategic plans for development of Information Technology within university; in addition to plans for the computer centre, this may include network infrastructure, administrative computing and library automation, and the relationship of such systems to the computer centre.] 3. Outline of required system [This should give an overview of what is required in supplier-independent terms, and give a broad outline of the typical uses of the new system. It is not a section that suppliers are supposed to write a response to, and therefore must not include any requirements that are not detailed in Sections 4 or 5. If some of the computer centre facilites discussed in Section 2 are being retained, it is important to make clear which systems the new equipment will replace. The aim should be to produce an easily-readable 1 - 2 page summary of the requirement, as a background to the detail of the following sections.] 4. Minimum, highly desirable and desirable requirements [The following is a selection of paragraphs which may be used in section 4 of the OR. They do not represent a complete set of requirements, but a set of building blocks which individual universities can tailor to reflect their own needs. The paragraphs are intended to represent a 'core' set of typical university requirements, but some of them will doubtless be irrelevant, while others will need considerable amendment or addition.] [The paragraphs have been classified into minimum requirements, highly desirable and desirable requirements, but it is particularly important that this aspect of any text used is checked carefully to ensure that only the genuine set of minimum requirements is classified as such.] [It is permissible for the OR to contain several distinct areas of functionality which may be met by either a single supplier, or by separate suppliers for each area of functionality, with no prime contractor. If this approach is adopted, it is important that requirements are classified to make it clear which area(s) of functionality they apply to.] This section specifies the minimum requirements which are essential if the system is to be acceptable to the University, and other requirements classified as highly desirable, desirable, and information requirements. For each requirement suppliers must comment in detail on the features of the solution proposed, stating how and why the system is considered to satisfy the requirement in question. A checklist for each category of requirements is included at Appendix C. The different categories of requirement in this section are indicated by the letters MR for minimum requirement, HDR for highly desirable requirement, DR for desirable requirement and IR for information requirement. 4.1 Timetable [If there are any mandatory features of the timetable (such as a latest date by which the system must be installed, or restrictions on installation to vacations) they should be clearly stated.] 4.2 Workload and performance 4.2.1 Workload MR1 The system must be capable of supporting ??? concurrent active users via the network working at terminal speeds of ???. Details of the network connection methods are given in the Network Service item in the Communications section (5.??). [Information on the numbers of simultaneous users to be supported over the various network connection methods can also be given if appropriate.] A typical breakdown of the terminal workload will be: ??% screen-editing ??% filestore management or other undemanding system tasks ??% compilations of moderate size (about 200 lines of source) ??% compilation of large programs (> 1000 lines) ??% execution of moderate-sized programs ??% execution of large programs ??% execution of standard packages. [If a significant proportion of the work is not in single precision, this should be clearly stated.] MR2 While supporting the terminal workload, the system must be able to run any background tasks to support coloured book mail, file and job-transfer and a batch workload equivalent to ??% of the system. HDR1 It is highly desirable that the system can support more than the above workload. MR3 If a multi-machine solution is proposed, there must be at least one CPU capable of supporting ??% of the workload, and the smallest machine must be capable of supporting at least ??% of the workload. 4.2.2 Performance MR4 With the above workload running, the system must be capable of providing a response of 0.5s for simple editing commands and 1.0s for commands causing the screen to be rewritten. Responses to simple system commands must be less than 2s. The time for a user to log in must not exceed 20s. (Response times are defined as being measured at the interface to the proposed equipment.) HDR2 The system should be appropriately configured to support the above workload, and should be balanced in terms of CPU power, memory and I/O capacity. DR1 It is desirable that response times are better than this. [If computationally intensive batch forms an important part of the requirement, it may be appropriate to specify a minimum performance level for this work, possibly in terms of a Whetstone rating, or a LINPACK MFLOP rating.] [If a full interactive benchmark is being considered to validate performance, a paragraph should be inserted saying that the University reserves the right to do this. An alternative is to make it mandatory that suppliers provide reference sites supporting similar workloads either on machines the same as the ones proposed, or similar enough to allow sensible extrapolation] 4.3 Hardware features 4.3.1 Memory MR5 A virtual memory system is required with a virtual address space of at least 16MB available to a single user process. DR2 A larger address space is desirable. 4.3.2 Floating-point arithmetic MR6 The system must be capable of performing floating point operations in hardware or microcode to give results of at least ? and ?? decimal digits of precision. IR1 Suppliers must state the processing rate in KWIPS (Kilo Whetstone Instructions per second) or MFLOPS for each precision offered. IR2 Suppliers must provide details of the methods used for floating-point operations, including mantissa and exponent base and length, and the rounding method used. This information must be provided for all precisions offered. Compliance with the requirements for accuracy will be validated using the NAG FPV (Floating Point Validation) package in Fortran-77. (These tests are available from NAG, 256, Banbury Road, Oxford, OX2 7DE.) [It is CCTA's recommendation that the accuracy of all systems used to perform significant amounts of floating-point calculations is verified. (Around two-thirds of such implementations have been found to contain serious design flaws.) Universities having their own equivalents to the NAG package may wish to use these instead. If the university does not wish to purchase the NAG package, an alternative is to make it mandatory that suppliers run the test and provide results for the proposed system.] HDR3 It is desirable that if more than one processor is offered, the accuracy on all processors is identical. 4.3.3 Filestore MR7 The system must provide at least ??GB of formatted user file space in addition to space required by the operating system. ??% of the filestore may be on optical disc, or any equivalent write-once/ready-many media. DR3 It is desirable that a mechanism for automatically archiving users files, with recovery on demand. If such mechanism is proposed in an acceptable form and with sufficient archiving hardware, ??% of the user filestore may be stored on tape. HDR4 More user filestore is highly desirable. DR4 The provision of hardware (and any associated software) to assist the searching of large database files is desirable. It is important that this should include the ability to search serial files as well as indexed files. MR8 Sufficient hardware must be provided to backup the online filestore. IR3 The time taken to back-up the on-line filestore concurrently with normal work must be stated, as must any restrictions on running of normal work during back-up. IR4 The University expects difficulty in backing up the large filestore required, especially with limited operator coverage. Suppliers are asked to comment on any planned development of solutions using high-volume cassette tapes, optical discs, etc. [Any requirements for exchangeable magnetic disks should be stated here. However, many suppliers no longer offer this technology, so it is unwise to make it mandatory.] 4.3.4 Tape interchange MR9 At least ? tapedecks capable of reading and writing tapes at 1600 bpi PE and 6250 bpi GCR must be provided. These decks are in addition to any provided to meet the filestore backup requirement. If a multi-machine solution is proposed, more tape decks may be required to allow all users sufficient access to tape decks. DR5 It is desirable that a tapedeck capable of supporting 800 bpi NRZI operation is provided. DR6 It is desirable that utilities are provided to write tapes for and read tapes from other systems in a wide range of formats including DEC, IBM, ICL,... 4.3.5 Printing capacity MR10 Print capacity equivalent to ???? lpm must be provided; this speed must be sustainable while printing lines of 132 characters using a 96-character set. The UK ASCII character set must be provided on all printers. DR7 For reasons of resilience, it is desirable that this capacity should be provided by more than 1 printer. DR8 It is desirable that a printer capable of printing graphical output and user-defined character sets is provided. This should have a capacity of at least ? A4 pages per minute. [Requirements for distributed printing capability should be included here.] 4.4 Operating System features 4.4.1 General MR11 Great importance will be attached to the quality of the user interface. The system must be easy to use for the infrequent or inexperienced user, providing a consistent and uncomplicated command syntax with intelligently-chosen defaults for optional parameters. It must also offer access to the full range of facilities when used in more sophisticated manner. The system must however offer features (such as short forms of command names and user-defined abbreviations) to ensure that the interface remains efficient for the experienced user. HDR5 It is highly desirable that users can type ahead. HDR6 It is highly desirable that input and output can be redirected between files and the terminal without the need to recompile programs. DR9 It is desirable that users should be able to produce a log of their terminal sessions. DR10 The provision of software to facilitate interworking of the system with common personal computer or workstation operating systems (eg MSDOS, Unix) is desirable. DR11 It is desirable that the source of the operating system should be made available for reference purposes, and the University should be able to make minor changes. The supplier should state his attitude to the maintenance of systems containing such modification. IR5 Suppliers should comment on facilities offered for including special purpose routines in areas such as scheduling, accounting, and device handling. DR12 The ability to test a new version of the operating system without major impact on the user service is desirable. DR13 It is desirable that frequently used software can be prepared in such a way as to minimise the resources required and time taken to invoke it when called interactively. IR6 Any special licence conditions for use of any proposed software (such as restrictions on the sale of compiled code) must be clearly stated. 4.4.2 Scheduler HDR7 The system should include scheduling algorithms that are designed to give priority to interactive users whose usage between interactions is small (in terms of mill-time and working set). 4.4.3 Screen editor MR12 A screen editor is an essential component of the user interface and must work over all terminal access methods. A line editor is also essential. DR14 It is highly desirable that the screen editor utilises the Simple Screen Management Protocol (SSMP) referred to in the screen-oriented functions item in the Communications Section (5.??). DR15 It is desirable to provide an editor with the following features: multiple windows on a single screen, word awareness, reformatting paragraphs, cut and paste, source language awareness. 4.4.4 Command language MR13 Except for utilities intended to exploit the capabilities of screen mode terminals, the user interface must be similar on both screen and scroll mode terminals, and all normal facilities must be usable from both types of terminal. HDR8 It is highly desirable that the command language provides the following features: ability to tailor and extend the language, both by users and the system manager; short forms of command names and parameters (with a consistent scheme for devising the short form); user-defined abbreviations; "wild-card" facilities; consistent defaults for parameters; prompting for missing mandatory parameters; prompting for optional parameters on demand; lack of case sensitivity; programming language like features, with facilities for storing and manipulating string, integer and boolean variables, repetition and conditional execution. 4.4.5 Help system MR14 The system must provide a help facility. HDR9 This should be structured and interactive, and provide summary information on operating system commands, compilers and applications. HDR10 It is highly desirable that facilities exist to provide local enhancements to the help system. DR16 It is desirable that the system logs all help requests made. 4.4.6 Batch system MR15 The system must provide facilities for users to submit batch jobs. DR17 It is desirable that users should be able to monitor the progress of their batch jobs. HDR11 It is highly desirable that batch jobs can be scheduled to ensure that small jobs are not seriously delayed by large ones. MR16 The user interface must be identical for interactive and batch jobs, apart from any differences intrinsic to the two different ways of working. 4.4.7 Filestore structure MR17 If a multi-machine solution is proposed, it must behave as a single entity so that a user's work and filestore are not bound to one of several processors. MR18 The system must be capable of handling at least ??? thousand files and ???? usernames. A hierarchical file structure for user files is desirable. HDR12 Filenames should be at least ?? characters. It is desirable that null characters such as underline can be inserted to improve readability. MR19 The system must contain privacy mechanisms that prevent unauthorised access by one user to another's files, but which allow the owning user to permit access to files and sets of files by other users and groups of users without undue complication. DR18 Access permissions should include execute, read, append, write and delete. DR19 It is desirable that inexperienced users should not need to be aware of where their files are placed, or how the data in them is organised. MR20 It must be possible to limit the filestore available to a user or group of users in such a way that the amount of space allocated can easily be changed and the user(s) warned if they are approaching the limit. 4.4.8 System management MR21 The system must record all significant use of resources (CPU, filestore and i/o activity) and utilities should be provided to analyse the data by user and groups of users. HDR13 If a multi-machine solution is proposed, the system should contain facilities for automatic balancing of both the interactive and batch workloads. DR20 The system should include monitoring tools that enable response time and turnround to be calculated and the setting of scheduling parameters to be adjusted. DR21 It should be possible to record usage of all or selected commands, utilities and packages. MR22 The system manager must be able to allocate budgets to users, and the system must use these budgets to control either the rate of use of resources or the absolute use of resources. DR22 This mechanism should preferably operate in a hierarchical manner to allow privileged users, acting for departments, to adjust the budgets for users in their department (within an overall limit), using simple commands. MR23 Following corruption or damage to a disc or smaller contiguous section of the filestore it must be possible to restore that part of the filestore from backup without affecting the rest of the filestore. HDR14 If a single track on a disc is affected, it should be possible to reformat or bypass that track and restore only the files affected. Suppliers must describe their proposed recovery mechanisms in detail and indicate the tools available to provide filestore integrity checks. MR24 It is essential that it is possible to retrieve any copy of a damaged or deleted file that exists in the back-up system without undue difficulty and without disruption to the normal operation of the machine. DR23 The system should contain means for the system manager to permit or restrict access to chosen facilites for particular users or groups of users. IR7 Suppliers should detail the amount of effort required to maintain and update the system software, the frequency of software releases, and the methods of update and error correction. 4.4.9 Security MR25 A high level of security must be provided against unauthorised access to or alteration or destruction of information held within the system. There must be security within processes so that one user cannot functionally affect other users. Suppliers must give details of the security features offered by the system and their application in practical working environments. 4.5 Languages [Specify any requirements in the following areas in terms of minimum, highly desirable and desirable requirements: languages; standards to which compilers must conform; mixed-language programming; optimisation tools; debugging tools; libraries.] DR24 It is desirable that different languages provide a common interface to the user. DR25 It is desirable that compilers flag language extensions. 4.6 Applications [Specify the following in terms of minimum, highly desirable and desirable requirements: any general applications areas; any named applications that are required; any areas where functional equivalents to a named package are acceptable.] 4.7 Unix DR26 The ability to offer Unix as a controlled subsystem is desirable. If Unix is available, suppliers should comment on the relationship between their Unix implementation and the system's own operating system, particularly with respect to filestore. 4.8 Operations MR26 The system must be capable of running satisfactorily with no operator intervention for long periods up to a maximum of ??? hours. MR27 The system must be capable of identifying jobs requiring operator intervention and preventing them becoming active during unattended periods. DR27 It is desirable that the system is capable of automatically restarting after a power failure. 4.9 Enhanceability MR28 It must be possible to enhance the proposed system to support a workload at least ??% greater than that specified earlier. IR8 The supplier must state how the proposed configuration can be upgraded to increase its processing power, main store, terminal connectivity and disc capacity after it has been installed and indicate the costs of doing so. HDR15 It is highly desirable that it should be possible to upgrade the system incrementally. DR28 It is desirable that suppliers will commit to the same level of discount for future upgrades as for this contract. 4.10 Reliability MR29 The system must provide an extremely high level of reliability. The supplier must give figures for the expected MTBSI of both hardware and software, and a list of relevant reference sites. A target figure for MTBSI is 500 hours, with 98% serviceability. Definitions relating to serviceability assessments are given in Appendix B. [The suggested target figures may be changed as appropriate. Specific MTBSI and serviceability figures can be made mandatory in the OR, but this is often counter-productive because it may mean the exclusion of good suppliers who are honest about the reliability of their products and the shortlisting of dishonest suppliers who know that any claims they make at this stage are very difficult to verify.] IR9 Suppliers will be required to give contractual commitments on reliability at MoA stage, and must state the serviceability and MTBSI figures to which they are prepared to commit for contractual purposes. It should be noted that these are minimum values for contractual liability and that higher levels of serviceability will be expected in practice. IR10 Details of hardware and software maintenance arrangements must be provided, including the requirements and suggested schedules for preventive hardware maintenance and the mechanism and response times for reaction to hardware and software fault reports. Suppliers should note that maintenance cover is only required for prime shift, but the system will normally be operated for 24 hours per day, seven days per week. MR30 If the system requires preventive maintenance, the maximum allowable within prime shift (09.00-18.00) is 2 hours per month. DR29 It is desirable that the system should provide a degree of resilience, with as few single points of failure as possible. 4.11 Environment [Any environmental constraints on the system such as power consumption, floor space, height, weight, heat output, noise output, etc. must be stated.] IR12 The environmental requirements of the proposed equipment must be given. These should include dimensions and weight of each item, with access requirements for delivery and service clearances. They should also include power supply, cabling and earthing requirements, air-conditioning requirements (including any special arrangements such as water-cooling or chilled underfloor air supply), and the noise levels associated with the proposed equipment. Short-listed suppliers will be asked to inspect the computer suite to confirm that the environment is suitable for their equipment. 4.12 Documentation MR31 The supplier's documentation on all aspects of the system must be comprehensive and comprehensible. MR32 At least one copy of all relevant documentation must be included in the budget, and the cost of further copies must be itemised. DR30 It is desirable that documentation can be made available in machine-readable form. IR11 Suppliers should indicate their policy towards arrangements for the bulk purchase of introductory user documentation. DR31 It is desirable that copies can be made of suppliers' copyright documentation for the university's internal use without payment of any fee; suppliers should state their policy in this area. DR32 It is desirable that tutorial materials such as on-line self-teaching aids for learning the editor or other commonly-used parts of the system are available. 4.13 Training IR13 Details of courses available for operational and software support staff must be specified, including content, duration, cost and location. Suppliers should also state how much training is recommended for satisfactory installation and operation of the proposed system, and how much is included within the budget. 4.14 Conversion HDR16 It is highly desirable that transition to the new system can be effected with minimum effect on the user population. Suppliers should comment on their ability to provide utilities to assist this process. As a minimum, utilities should be provided to read filestore tapes from the existing system. HDR17 Specific aids to assist the migration of the existing production database load are highly desirable. 4.15 Support IR14 Suppliers should state how much support (if any) they are prepared to offer within the budget to assist with the installation of the software and conversion from the existing system, and also to support the university in the longer term with problem resolution and development of the system. [If particular difficulties with installation are expected, it may be appropriate to make provision of some support a desirable requirement.] 5. Communications Requirements [Consult the JNT for the latest standard text for this section.] 6. Further information requested from suppliers Suppliers are reminded of their obligations under the Health and Safety at Work Act 1974. Particular attention is drawn to the requirements of the Act relating to equipment design and manufacture and to the installation, operation and maintenance of equipment. The supplier must report any particular requirements or special instructions relating to the proposed equipment and its consumable supplies necessary to ensure they they will be safe and without risks to health. A nil return is required if this is appropriate. Suppliers in doubt as to the technical assessment of their equipment for safety purposes are advised to refer to BS 6204: 1982, Safety of Data Processing Equipment or the relevant IEC 435 standard. The CCTA offers no technical advice in this area but reserves the right to question any aspects of the equipment and its operating instructions related to safety. The above is without prejudice to the operation of the Health and Safety at Work Act 1974 and does not relieve the suppliers of any obligation they may have under the Act as designer, manufacturer, supplier or otherwise. The University is seeking a supplier with an established commitment to the education market, offering a system which is proven in all major respects. Suppliers must provide a list of reference sites running similar equipment to that proposed, preferably in a similar environment. 7. Instructions to Suppliers Proposals should be structured in the following way: 1. Management summary Summarise the proposal, highlighting the most important features. Describe the approach and plans towards satisfying and supporting the requirements of the project. Summarise the total cost of the proposal. 2. Description of the system proposed Describe the functions performed by the various components of the proposed equipment. 3. Lists of hardware and software Itemise all hardware and software offered. 4. Answers to Section 4 of the OR (general requirements) Respond to each of the detailed requirements in sections 4, stating how the proposed solution meets the requirement; it is important that each requirement is answered individually: generalised descriptions that do not answer all the questions are not acceptable. 5. Answers to Section 5 of the OR (communications requirements) Respond to each of the requirements in Section 5 in the same way as for Section 4. 6. Answers to Section 6 of the OR Respond to each of the points raised in this section. 7. Timetable Summarise how the timetable for procurement and implementation will be met. It should be made clear if any of the components of the proposed system will not be available within the specified timescale. 8. Costs Itemise capital and recurrent costs of the proposed system, for both hardware and software. Costs for third-party hardware or software to meet the minimum requirements must be included. 9. Further information Any other information which the supplier sees fit to include. It is vital that suppliers realise the importance of answering fully all the questions in the OR. Proposals that do not meet all the minimum requirements or incomplete proposals will not be shortlisted. The return of proposals will be followed by a period of initial evaluation following which a shortlist of proposals approved by the Computer Board will be produced. More detailed discussions and evaluation will then take place with the shortlisted suppliers to fully investigate the capabilities of the proposed system and its ability to meet the university's requirements. If these discussions are satisfactory, a Memorandum of Agreement will be signed and suppliers will then receive a formal Invitation to Tender. Prospective suppliers may be required to demonstrate items of hardware or software for which a proposal has been made. Such demonstrations should, as far as is practicable, simulate use of the item in the user environment. Suppliers will be invited to include actual user programs and data in the demonstration. Suppliers will be expected to bear their own costs for such demonstrations. Timetable The provisional timetable is as follows: Week 0 Issue of Operational Requirement 6 Return of proposals 10 Announcement of shortlist 20 Completion of discussions, signing of MoA 22 Invitation to tender 26 Return of tenders 29 Tender evaluation completed, referral to Computer Board 34 Award of contract ?? Delivery of equipment ?? Start of user service While the University will endeavour to adhere to this timetable, it should be noted that changes may become inevitable as the procurement proceeds. Any changes will be notified to suppliers as necessary. Conditions of contract The supply of the the equipment will be subject to the Universities' Standard Conditions of Contract for the Supply of Data Processing Equipment (Edition 1980 including amendment sheet dated 5 August 1980, revised 21 May 1984), except that Appendix C to Schedule 1 (Form of Contract for Class A Maintenance) will be replaced by Appendix I to Schedule 1 of CON 84. It should also be noted that the aforementioned Appendix I will itself be amended for university contracts so that Condition 1 a(i) reads "...contractual period to provide 4 times 13 weekly reporting figures per reporting year." Definitions relating to serviceability assessments are given in Appendix B. Suppliers should note that CON 84 requires a commitment to maintain the system for 10 years. Acceptance Acceptance will be conditional on the satisfactory completion of equipment trials conducted by or on behalf of CCTA in accordance with the Universities Standard Conditions of Contract for Trials of Data Processing Equipment (CS16, Universities Edition 1973). While every endeavour has been made to give suppliers an accurate description of the University's requirements, suppliers should form their own conclusions about the methods and resources needed to meet these requirements; The University cannot accept responsibility for the supplier's assessment of any system proposed. Appendix A Evaluation criteria Evaluation will be based on the total capital and recurrent costs of purchasing the system and running it for 7 years. In addition to tendered costs, many other costs may be taken into account if they are significantly different between suppliers. An assessment will also be made of subjective factors where real costs cannot sensibly be derived but where there are real differences between tendered systems that will influence their worth to the university and its work. Table 1 lists many of the areas which may be evaluated if appropriate. The table includes both quantifiable and unquantifiable (subjective) items. Appendix A Table 1 Evaluation guidelines for suppliers Area Factor to be evaluated System life Total costs costs Third party Extra costs support Performance CPU power, number of terminals supported, batch throughput, memory size, I/O throughput, response times, other performance factors Arithmetic range and accuracy Performance (throughput) of major utilities and packages Operating Range of peripherals supported system functionality Range of software available Operating User interface system quality Quality of utilities and applications Systems Effort required programming Software maintenance effort Data storage Quantity Back-up Archival Communications Performance Functionality JNT standards OSI standards Operation User management and control, performance tuning and monitoring Remote and unattended operation, automatic restart after power failure Operator interface Maintenance Cost and reliability Availability Facilities required on site Reporting and call-out procedures Documentation Cost Quality Machine-readability Training Costs Conversion Costs Compatibility Upgradeability Ease and cost-effectiveness Environment Machine-room costs Electricity costs Delivery Schedule Appendix B Definitions relating to Serviceability Assessments Serviceability assessment will be over discrete 13-week periods (ie there will be 4 such periods per year), and the criteria for serviceability must be achieved in each of these 13-week periods. Serviceability is defined as: % serviceability = 100 x (SP-SM-MT-OSD-SRT) / (SP-SM-MT) MTBSI (Mean Time Between System Incidents) is defined as: MTBSI = (SP-SM-OSD-SRT) / N where N is the number of system incidents for which the contractor is responsible; OS (operational system) is the configuration of hardware and software facilities which the University requires at any time during maintenance cover time; SP (scheduled period) is the scheduled operational period; SM (scheduled maintenance) is the agreed time during SP when the OS is unavailable to the University because of scheduled preventive maintenance; MT (modification time) is the time during SP when the OS is unavailable because the contractor has requested and the University has permitted the inclusion of system modifications; OSD (operational system downtime) is the unscheduled downtime which results when the OS is unavailable to the University during SP because of a contractual incident attributed to the maintained system; SRT (system recovery time) is a fixed time allowance for each incident or system failure that results in system recovery action. Appendix C Checklist of Minimum, Highly Desirable, Desirable and Information Requirements This checklist is provided purely for convenience. In the event of there being any discrepancy between this appendix and Section 4 of the OR, Section 4 will be regarded as being definitive. Minimum requirements MR1 ??? concurrent active users MR2 Batch workload of ??% concurrent with timesharing load MR3 Minimum CPU sizes for multi-CPU solutions MR4 Response times MR5 Virtual memory of 16MB MR6 Floating-point accuracy MR7 ??GB of filestore MR8 Sufficient hardware to backup the filestore MR9 Tapedecks for tape interchange MR10 Print capacity of ???? lpm MR11 Quality of user interface MR12 Screen editor MR13 Same interface for screen- and scroll-mode terminals MR14 Help facility MR15 Batch system MR16 Same interface for batch and interactive users MR17 Single filestore image MR18 ??? thousand files and ???? usernames MR19 Filestore privacy mechanisms MR20 Ease of allocation of filestore MR21 Recording of all resources used MR22 Allocation of budgets MR23 Recovery from filestore corruption MR24 Retrieval of individual files from backup tapes MR25 High level of security MR26 Unattended running for up to ??? hours MR27 Job-scheduling during unattended running MR28 Enhanceability by ??% MR29 High level of reliability MR30 Level of preventive maintenance MR31 Comprehensive documentation MR32 One copy of documentation within the budget Highly desirable requirements HDR1 Support of more than the specified workload HDR2 Balanced configuration HDR3 Same floating-point format on all processors HDR4 More user filestore HDR5 Support of type-ahead HDR6 Redirection of input and output HDR7 Scheduling algorithms for interactive use HDR8 Command language features HDR9 Structured, interactive help system HDR10 Ability to enhance the help system locally HDR11 Efficient batch scheduling of small jobs HDR12 Filenames of at least ?? characters HDR13 Automatic load balancing on multi-machine systems HDR14 Restore of files from a single corrupted track HDR15 Ability to upgrade system incrementally HDR16 Easy conversion to the new system HDR17 Aids to assist migration of the existing database Desirable requirements DR1 Better response times DR2 Larger virtual address space DR3 Automatic archiving and retrieval of files DR4 Hardware-assisted database searching DR5 800 bpi tapedeck DR6 Utilities to read tapes from other systems DR7 More than 1 printer, for resilience DR8 Printer with graphics capability DR9 Ability to log terminal sessions DR10 Software to interwork with workstations DR11 Availability of operating system source DR12 Ability to test a new version of software DR13 Efficient use of frequently used software DR14 Screen editor with SSMP DR15 Special editor features DR16 Log of all help request DR17 Facility for users to monitor batch jobs DR18 File access permissions DR19 Transparent filestore structure DR20 System monitoring tools DR21 Record of usage of facilities DR22 Hierarchical allocation of budgets DR23 Ability to restrict access to particular facilities DR24 Common-user interface for languages DR25 Flagging of language extensions by compilers DR26 Provision of a Unix subsystem DR27 Automatic restart after a power-failure DR28 Commitment to future level of discount DR29 Degree of resilience to failure DR30 Documentation in machine-readable form DR31 Permission to reproduce copyright documentation DR32 On-line self-teaching aids Information requirements IR1 Processing speed for floating-point IR2 Floating-point accuracy IR3 Time taken to backup the on-line filestore IR4 Future plans for easier backup IR5 Facilities for special purpose routines IR6 Special licence conditions for software IR7 Effort required to maintain and update the system IR8 Expansion paths IR9 Figures for contractual commitments on reliability IR10 Details of hardware and software maintenance arrangements IR11 Environmental requirements IR12 Policy for bulk purchase of documentation IR13 Training courses IR14 Support offered Subject: Re: Procurement Procedures From: CCTA Date: Fri, 19 Jun 87 12:13 BST To: P.E.Williams@UK.AC.EDINBURGH In reply to: <17 Jun 87 15:17:33 bst 110889@BUSH> Via: UK.AC.EDINBURGH.EMAS-A ; (to uk.ac.edinburgh.bush) 19 Jun 87 12:40:03 bst Via: UK.AC.RUTHERFORD.GEC-B ; (to uk.ac.edinburgh.emas-a) 19 Jun 87 12:11:36 bst Msg ID: <19 JUN 1987 12:13:20 NJIN03@UK.AC.RUTHERFORD.GEC-B> Two separate messages following this one contain the text of the documents on Computer Board procedures. I am not sure if they will answer all your questions on things such as timescales - please give me a call or send me a message if you have any more questions. Regards Rob Whetnall --- End of forwarded message ________________________________________________________________________ (Message 99) Subject: Re: Procurement Procedures From: P.E.Williams @ uk.ac.edinburgh Date: 06 Jul 87 11:34:59 bst Reply to: CCTA To: j.livingstone Via: UK.AC.EDINBURGH.BUSH ; (to uk.ac.edinburgh.emas-b) 06 Jul 87 11:33:36 bst Msg ID: <06 Jul 87 11:34:59 bst 110313@BUSH> --- Forwarded message: Subject: Re: Procurement Procedures From: CCTA Date: Fri, 19 Jun 87 12:17 BST To: P.E.Williams@UK.AC.EDINBURGH In reply to: <17 Jun 87 15:17:33 bst 110889@BUSH> Via: UK.AC.EDINBURGH.EMAS-A ; (to uk.ac.edinburgh.bush) 19 Jun 87 12:40:21 bst Via: UK.AC.RUTHERFORD.GEC-B ; (to uk.ac.edinburgh.emas-a) 19 Jun 87 12:19:08 bst Msg ID: <19 JUN 1987 12:17:57 NJIN03@UK.AC.RUTHERFORD.GEC-B> Guide to the Procurement process for University systems funded by the Computer Board for Universities and Research Councils Part II Evaluation, award of contract, final acceptance 1. Introduction This guide discusses the procurement procedures for university projects after the Operational Requirement has been issued. (Part I gives advice on the preparation and issue of the OR.) The normal phases after issue of the OR to prospective suppliers are as follows: shortlisting: detailed system proposals from a number of suppliers are evaluated and a shortlist agreed for more detailed investigation; detailed discussions: proposals from shortlisted suppliers are investigated to verify their ability to meet the requirement, and assess their overall desirability; agreement of MoA: a Memorandum of Agreement is prepared for each shortlisted supplier that has been proved acceptable during the detailed discussions; invitation to tender: ITTs are issued to all suppliers with whom an MoA was agreed; final evaluation: the tender offering the best overall system is chosen; award of contract: following any necessary post-tender negotiation and CBU approval, award of contract is made; acceptance: the installed system is subjected to acceptance tests before payment is made. These are the standard procedures, but they may be varied in special circumstances. Each of these subjects is considered in more detail in the following sections. 2. Shortlisting Objectives The objective of shortlisting is to select from the proposals received those solutions which appear to best meet the requirement and are therefore worthy of more detailed investigation. The decisions at this stage can be fairly subjective, but should be based on rational factors. Shortlisting meeting The university should arrange a shortlisting meeting to discuss the proposals submitted and decide on a shortlist. Those represented at the meeting will include the Godfather, the Computer Board Secretariat, the JNT, and CCTA (London and Norwich). The shortlist will normally require the approval of the Board, although in some circumstances it is possible to agree in advance that, if the decision is not controversial, it will be delegated to the Godfather. Length of shortlist There is no fixed number of companies that should appear on the shortlist, but experience has shown that 3 is the best number in most cases. If only 2 companies are shortlisted and it subsequently proves impossible to an agree an MoA with one of them, the project will end up with only a single supplier being invited to tender. Neither this, nor the shortlisting of only one company is acceptable to the Computer Board except in exceptional circumstances. A shortlist of 4 or more can often be attractive and initially avoid difficult decisions, but given a finite resource and a restricted timescale for evaluation of the products and agreeing MoAs, it will generally mean that the companies cannot be evaluated in sufficient detail. However, it may sometimes be appropriate to shortlist more than 3 companies who appear to meet the requirement if one or more of them are already well known to the university as existing suppliers. Companies that have no chance of winning the procurement should not be shortlisted just to make up the numbers as this only wastes a great deal of time for all concerned, and does not in reality make the procurement more competitive. If the OR is divided into a number of separate functional areas, a shortlist of up to 3 for each area is normally appropriate. Choice of companies The first objective in shortlisting will always be to eliminate those proposals that do not meet the minimum requirements of the OR. The remaining proposals should then be compared on the basis of how many of the highly desirable and desirable requirements have been met, along with external factors such as how well the company has been performing on other similar contracts, particularly in the university sector. Other subjective considerations may be brought in with the aim of producing a balanced shortlist with a variety of different companies all of which are potential winners. For instance there are sometimes two or more suppliers proposing systems that run the same operating system (eg VM/CMS or Unix); unless such systems are seen as ideal solutions to the requirements, it is unwise to shortlist two of them at the expense of a good alternative. Due regard should be given to proposals from British suppliers; the Computer Board expect to see such proposals kept in at shortlisting stage if they meet, or can commit to meet within an acceptable timescale, the minimum requirement. While proposals from relatively new start-up companies can often be very attractive, it is generally best to avoid a shortlist consisting solely of this type of company as there is a much higher degree of risk that the product will not meet its specification or the company will run into trouble. If there is a solution of this type proposed that is particularly attractive it is permissible to shortlist it, but it is advisable to balance it with two well-established companies with which it should be easy to agree an MoA. In the unlikely event that after taking all the above factors into account the shortlist is still too long, it is sensible to look ahead to the more detailed financial evaluation that will be done on the final tenders; this should give enough information to show which companies are unlikely to win and should therefore be excluded. Inadequate shortlist If it is impossible to form an adequate shortlist using the above guidelines, the condition that all shortlisted proposals must clearly meet the minimum requirements may exceptionally have to be relaxed in the interest of avoiding undue delay. As a guideline, suppliers who have not provided full evidence on one or more points may be shortlisted if it is judged that an acceptable agreement can be reached in subsequent negotiations. If it is still not possible to form an adequate shortlist, review action will be necessary on the following lines: either: major changes are made to the OR. The whole procedure must then be restarted from the beginning. (With good preparation, this step is likely to be necessary only in rare cases.) or: minor changes (in the view of the university and the Computer Board) are made to the OR. The list of changes must be sent to all suppliers who received a copy of the original OR (including those who did not respond) seeking their comments. Normally, those suppliers who have previously responded will make minor changes to their proposal and the others will again decline to make an offer. If any supplier who had previously declined to offer now changes his mind and provides satisfactory evidence of his ability to meet the revised OR (without at this stage a full proposal), the whole procedure must be restarted from the beginning. Otherwise, the university should proceed to the subsequent steps using the modified OR and any revised proposals it has received. 3. Detailed discussions Objectives The objectives of detailed discussions are two-fold: firstly, to fully investigate the suppliers proposals to confirm that they can meet the minimum requirement; and secondly, to assess how well the desirable requirements are met in order to compare the value of the different offerings to the university. Discussions also help suppliers to understand the requirement more fully and hence to ensure that they are offering their best solution. It is quite common for minor changes to be made to the configuration at this stage, but the university is free to reject major changes if the necessary evaluation would cause significant delay. The university should aim to keep to the timetable in the OR, but is free to take longer for the evaluation if it is thought to be necessary. However, the university is not obliged to extend the deadline because a supplier is being unhelpful or slow in responding to queries. In these circumstances, the university should make it clear to the supplier that the original timetable is being adhered to, and if discussions have not been satisfactorily concluded by the announced date, they will not be further involved in the project. Initial meeting As soon as possible after shortlisting, meetings should be arranged with each of the suppliers to inform them of which areas the university wish to investigate in more depth. It is particularly important that companies whose presence on the shortlist is somewhat marginal are informed of all areas of weakness as soon as possible; this helps to minimise problems later on if they are asked to withdraw. The methods used to investigate proposals will vary considerably depending on the nature of the system, and the resources available within the university. Meetings with technical specialists from suppliers will provide basic information about how a system works, and most suppliers will provide large quantities of manuals. Other important aspects of evaluation are site visits, benchmarking and demonstrations. User site visits Visits to other sites (preferably universities) running the same or similar systems are one of the best sources of information on the problems of a particular system. Suppliers should be asked to give a list of relevant sites from which one or two can be chosen. Ideally, visits to both long-standing and relatively new sites should be arranged. Well-established sites can give a lot of information about what can be got out of a system when it is pushed to its limits, but will probably have forgotten the problems encountered when setting-up the system for the first time. A visit to a new site will not give the same depth of information on the capabilities of the system, but should give an excellent indication of what initial problems there are likely to be. Relevant user site visits can be one of the best sources of information, so consideration should be given to visits abroad if there are no suitable sites in the UK. Visits to the headquarters of companies (often in the US) are sometimes suggested, but it is wise to be clear on the objectives of such visits. Most of the larger companies can supply almost as much technical information in the UK as the US. However, such visits may give valuable information on future product plans (depending on the policy of the company) and can also provide information on the long-term strategy of the company. It has been found that visits to smaller new companies are generally the most valuable because they are much less restrictive about discussing plans, and it is often possible to talk to the people who are responsible for designing and implementing the system. Demonstrations and benchmarking It is generally sensible to plan a specific program of work to try out on each system. This may range from compilation and execution of a few Fortran programs to running a full interactive benchmark. (The OR will have informed suppliers of the universities plans in this area.) Most suppliers are able to arrange some form of dial-in access to a system either at their own offices or at a user site. This should be used to give hands-on experience of what a particular system is like to use in practice. Full interactive benchmarks are rarely undertaken because they are extremely expensive for both the supplier and the university, and no matter how much effort goes in to preparing them, the results they give often have little relevance to how systems will actually perform in practice. It is unwise to assume that because a system will support 100 terminals in a simulated interactive environment it will support the same number in practice. The best use of such benchmarks is for comparisons, rather than to obtain absolute figures: given a representative benchmark, if System A supports 100 users, and System B 110, then it is likely that System B will support more users than System A with the real workload (but it is most unlikely that the number will be 110). Simple performance demonstrations take far less effort than full benchmarks, and can give useful results. A good approach is to select a number of typical programs, and to measure compilation speeds and run times. Compilation listings from the first attempt at compiling the program can also be very useful at estimating how much effort will be needed for converting to the new system. The standard CCTA Whetstone programs can be used for approximate comparisons of floating-point performance. They are currently being updated to Fortran-77 versions; the source will be available via JANET on request to "Whetnall@RL.GB". CCTA are currently investigating the possibility of developing some standard programs to use as building blocks for benchmarks. 4. Memorandum of agreement Purpose of MoA The MoA is the formal document which marks the end of the discussion stage between shortlisted suppliers and the university. An MoA provides the basis upon which an Invitation to Tender (ITT) is drafted but it is not itself a contractual document. It is important that the MoA is both comprehensive and unambiguous. It must set out precisely what the company has to provide to satisfy the requirement; this will generally consist of hardware, software and supporting services jointly agreed as being the company's best offer for meeting the OR. It is permissible for the MoA to contain as options alternative configurations from the same company. It is vitally important for the university to realise that signature of an MoA means that the technical and operational viability of a supplier's offering is accepted. Consequently, the university will have no reason to reject a valid tender based on an agreed MoA. A skeleton MoA is included at Annex A. 5. Invitation to tender Shortly after receiving the signed MoAs, CCTA CT5 will issue formal Invitations to Tender. These will be based on the MoAs, and will ask the supplier to make contractual any commitments made within the MoA. Once ITTs have been issued, there should be no further contact between the university and suppliers. This is because the MoAs represent an agreed acceptable position, and further negotiations are unnecessary. If any substantive discussions were to take place which led to a change in a supplier's tender, competing suppliers would have a sustainable complaint about unfair treatment; this could cause considerable delay to the project, and final award of contract. Direct contact between the university and suppliers must not be resumed until after award of contract. Suppliers are normally allowed three weeks for return of tenders. Tenders always go in the first instance to CCTA in Norwich who, having independently opened and verified them, forward them to the university as soon as possible. If there are any queries about the content of the tenders they should be sent to CCTA CT5, who will telex them to the supplier. 6. Final evaluation Objectives The objective of final evaluation is to consider the tenders submitted by a number of suppliers and reach a recommendation on which provides the best solution for the university based on rational factors. The main criteria to be considered are how much will the system cost to purchase and run (or lease/hire) over the expected 7-year life, and how well the requirements (essentially as specified in the OR) have been met. This objective is realised by preparation of an evaluation model which will consist of an analysis of all costable items (both tendered and non-tendered) along with an assessment of all areas where there are significant differences between proposals, but for which no sensible cost can be derived. If the total cost for one supplier is the lowest by a significant margin then the Computer Board will expect that supplier to be chosen; however, if the total costs are similar then the weighting factors assigned to the uncostable items may be brought into consideration, and used to provide a justification for selecting a more expensive system. The greater the difference in total costs between the cheapest and the desired system, the stronger the justification will have to be. Suppliers must be advised of how their tenders will be evaluated and which areas are particulary important, but specific costs and weightings must not be revealed. Appendix A to the Draft OR for University Procurements shows the level of information that should be released. Evaluation model Before release of tenders by CCTA, the university should prepare an evaluation model, which must be submitted to the CBU Secretariat and CCTA for approval before the university may receive the tenders. The evaluation model consists of an analysis of all capital and recurrent costs over the 7-year life of the system discounted at 5% along with weighting factors assigned to any other areas where there is a quantifiable difference, but where no sensible cost can be assigned to that difference. Tendered costs obviously cannot be included at this stage, but all other costs which are to be taken into account must be declared at this stage. A very simple example of a discounted cash flow model is given at Annex B. The costs of buying and running the system will cover more than just the tendered costs, and can be divided into the following categories: a) capital costs paid by the CBU (mainly basic hardware costs) b) recurrent costs paid by the CBU (software and maintenance costs, to an annual limit of 10% of (a)) c) capital costs met by the university (eg preparing a suitable environment) d) recurrent costs met by the university (eg staff costs, costs from (b) above the 10% level funded by the CBU) Annex C lists a range of areas which may be considered in the evaluation model. The table is not necessarily exhaustive. Any special requirements from the OR not mentioned in the table should be included if they are likely to have a significant effect on the evaluation. The various factors have been classified in the table as capital costs, recurrent costs or ranked. These classifications are intended only as general guidelines and not absolute rules. For instance, if the user control facilities in a system are inadequate it might be possible to remedy this by paying a one-off charge for an additional piece of software, which would be counted as a capital cost. Alternatively, it could be necessary to employ an extra person for the life of the system, which would be a recurrent cost. The final possibility is that the deficiencies are such that nothing can be done about them, and the system would have to be used for 7 years with inadequate user control. Generally, it would be impracticable to put a financial cost on this situation, so it would be counted as unquantifiable, and assigned a weighting factor. For many of the points listed there may be no significant difference between the suppliers, so they can safely be ignored in any comparison. Evaluation report Once tenders have been received, the actual costs can be put into the evaluation model, and the total cost of each system calculated. This should be a straightforward process. The university should prepare a Final Evaluation Report for submission to the Computer Board. This will contain the costings from the evaluation model. If the university wish to choose a system which does not have the lowest overall cost, full justification must be provided. Evaluation meeting When the draft Final Evaluation Report has been completed and circulated to the external advisors, the university should arrange a final evaluation meeting to discuss the report and its conclusions. Those represented at the meeting will include the Godfather, the Computer Board Secretariat, the JNT, and CCTA (London and Norwich). The report should be revised to take account of any changes agreed at the meeting, and then submitted to the Computer Board for approval. 7. Award of contract As soon as possible after the Computer Board approve the recommendation, CCTA CT5 will award the contract. The contract consists of the ITT, the tender submitted, any subsequent telexes and/or letters of clarification, and the acceptance of tender telex and letter. The university must not give any advance indication of the decision until CCTA CT5 has formally notified the successful and unsuccessful tenderers. 8. Acceptance When the system has been installed, it is necessary to perform some testing to ensure (as far as is practicable) that it functions correctly and is in accordance with the contract. CCTA's recommended procedure for acceptance of most systems is a CS16 Parts A & F trial. This consists of a 20-working-day period of use of the system where anything may be run on the machine (including, for example, a re-run of a benchmark). The criteria for success are that the system meets certain serviceability standards, and performs in accordance with the contract. If the criteria have not been met after the first 20 days, the trial is extended on a day-by-day basis until the criteria have been met. If after 60 days the system has not completed a successful period of 20 consecutive days, the supplier is in breach of contract. Further details are given in Annex D. Once acceptance has been completed, the university should notify CCTA CT5, who will formally notify the supplier. The supplier will then submit an invoice direct to the university. 9. Further information Further advice on the preparation of a Discounted Cash Flow model and any other contractual aspects of procurement is available from CCTA CT5 Division (Norwich). Advice on benchmarking, the technical content of the evaluation model, acceptance procedures, and any other technical aspect of procurement is available from CCTA CT1 Division (Riverwalk House, London). Annex A (Draft) Memorandum of Agreement between and The University of in respect of computer equipment, software and services for the University's mainframe replacement project. This document presents the conclusions reached between the University of (hereinafter called "the University") and (hereinafter called "the Supplier") in respect of the University's Operational Requirement, issued on the . Discussions were based on this Operational Requirement and the Supplier's response dated . This memorandum, together with any subsequent written amendments or additions, will form the basis for a formal Invitation to Tender from the Central Computer and Telecommunications Agency. Nothing herein is contractually binding upon the Supplier or the University and any features which the parties require to be so binding shall be incorporated into an official invitation to tender and the Supplier's response thereto. Signed on behalf of the University Signed on behalf of the Supplier Signature Signature Name Name Position Position Date Date Index of contents Section Description 1 Introduction 2 Goods and services to be provided by the supplier 2.1 Hardware 2.2 Software 2.3 Functions and facilities 2.4 Communications functions and facilites 2.5 Project and product support 2.6 Maintenance and serviceability 2.7 Training 2.8 Delivery and installation 2.9 Environment 2.10 Documentation 2.11 Warranties 2.12 Performance 3 Goods and services to be provided by the University 3.1 Hardware 3.2 Software 3.3 Communications 3.4 Accommodation 3.5 Project support 4 General 4.1 Acceptance and trialling 4.2 Special conditions Annex A Hardware to be supplied Annex B Software to be supplied Annex C Functions and facilities Annex D Communications functions and facilities Annex E Serviceability Annex F Documentation Annex G Timetable Annex H Training 1. Introduction The supplier confirms that the hardware, software and services detailed in this memorandum will meet the requirements specified in the Operational Requirement for the University of , issued on , and clarified during subsequent discussions between the project staff and the supplier. 2. Goods and services to be provided by the supplier 2.1 Hardware The hardware to be supplied is listed at Annex A. 2.2 Software The software to be supplied is listed at Annex B. 2.3 Functions and facilities The system will perform or provide the functions and facilities listed in Annex C. 2.4 Communications functions and facilities The system will support the communications facilites defined in Annex D. 2.5 Project and product support 2.5.1 Project support [Define any services to be offered by the supplier in areas such as planning and installation of software; record any provisions for machine time for development prior to delivery.] 2.5.2 Product support The supplier agrees to support all hardware and software items supplied for 10 years from the date of acceptance. [Record any agreed provisions for support of historic versions of hardware or software and the position regarding updates introduced by the supplier.] 2.6 Maintenance and serviceability 2.6.1 [Define when, where and how preventive maintenance and maintenance cover will be provided in order to achieve the required serviceability levels, the details of which will be contained in the appropriate Appendices to Schedule 1.] 2.6.2 [Define any agreed spares holding or acquisition necessary in order that the serviceability levels can be met.] 2.7 Training The training to be offered by the supplier is identified in Annex H. 2.8 Delivery and installation The supplier will be responsible for delivery, installation of the equipment, and presentation of the integrated system for trialling. 2.9 Environment The supplier has inspected the proposed accommodation and confirms that the equipment to be supplied will operate in the environment to be provided. [If there are any particular constraints imposed by the accommodation or the supplier's equipment, list them here, or refer to them in a separate annex.] 2.10 Documentation A list of the documentation to be supplied is at Annex F. [If appropriate, record permission to reproduce copyright material.] 2.11 Warranties [If any special warranty has been negotiated with the supplier then the scope and extent should be stated, together with any impact on maintenance charges.] 2.12 Performance [If the results of performance demonstrations or benchmarks are to be used as part of the acceptance procedures then the assessments and results should be detailed in an Annex.] 3. Goods and services to be provided by the University 3.1 Hardware [If any third-party hardware will form part of the total solution, and is not subject to a separate MoA, record it in a separate appendix. It must be made clear if the cost of such hardware is to be met from the budget, and the method of determining the costs.] 3.2 Software [If any third-party software will form part of the total solution, and is not subject to a separate MoA, record it in a separate appendix. It must be made clear if the cost of such software is to be met from the budget, and the method of determining the costs.] 3.3 Communications [Define the communications interfaces (eg lines, modems, cables) to be supplied by the university.] 3.4 Accommodation [Define the accommodation and environment to be provided by the university for computer hardware and support staff, including maintenance engineers. Accommodation plans should be included in an Annex. Define any works services to be carried out by the university.] 3.5 Project support [Define any tasks to be performed and personnel to be provided by the university to assist the supplier in the performance of his task.] 4. General 4.1 Acceptance and trialling Acceptance will be subject to successful completion of CS16 trials, the content of which will be agreed between CCTA CT1, the university, and the supplier. [Define any special conditions which must be met before acceptance of the system. In particular, if there is more than one major supplier of equipment but no prime contractor, conditions should be included to ensure that the no part of the system should be accepted until it has been shown that it works with any alien equipment to which it is interfaced.] 4.2 Special conditions [If the requirements of the system are such that the standard conditions of CON 84 are considered inadequate, inappropriate or in need of supplement then special conditions should be included. These conditions will be included in the Invitation to Tender.] Annexes Annex A: Hardware to be supplied [For each item of hardware to be supplied (including any cabling, consumables etc.), list the quantity, supplier's reference and a brief supplier's description.] Annex B: Software to be supplied [For each item of software to be supplied, list the quantity, supplier's reference and a brief supplier's description. Specially written software should be listed separately from standard software; specifications for any software to be written by the supplier should be included.] Annex C: Functions and facilities [List the functions and facilities to be performed or provided. These will refer to particular aspects not necessarily apparent from the hardware and software descriptions but which are essential and should be made contractual.] Annex D: Communications functions and facilities The system to be supplied will support the following communications facilities as defined in the Operational Requirement. It is assumed that all mandatory, highly desirable and desirable facilities are available unless explicitly excluded in the following text. OR reference 1. Network service [At least one of the following three network services to be included. The number and type of physical interfaces and speeds may need to be confirmed, particularly for X.25.] 5.2.1 PSS-compatible X.25 5.2.2 Cambridge slotted ring 5.2.3 10 Mbps CSMA/CD baseband communications (Ethernet) 2. Network Management 5.3.1 Name-to-address mapping 5.3.2 Call management 5.4.1 3. Terminal access [At least one of the following three access methods to be included.] 5.4.2 Incoming terminal access over X.25 5.4.3 Incoming terminal access over Cambridge ring 5.4.4 Incoming terminal access over CSMA/CD 5.4.5 Outgoing terminal access 5.4.6 Screen-oriented functions 5.5 4. File transfer 5.6 5. Job transfer [Specify type, eg Full RJE station and Simple Serving Site.] 5.7 6. Electronic mail 5.4.7 7. Performance 5.9.3 [Any quantitative communications requirements in the OR which it is felt should be reiterated in the MoA should be stated here.] [In general it is also useful to reproduce any specific text from the OR which, as a result of discussions with manufacturers, requires clarification and explicit confirmation, eg specific features of a protocol not currently available.] Annex E: Serviceability [A completed copy of the appropriate appendix to Schedule 1 of CON 84 should be enclosed.] Annex F: Documentation [List quantity, supplier's reference and subject for each manual to be supplied. List quantities of free copies separately.] Annex G: Timetable [Record the installation and acceptance timetable in weeks relative to award of contract.] Annex H: Training [A schedule of courses showing location, duration, number of attendees and subject should be included. Identify any training credits offered.] Annex B Example of a DCF calculation Supplier 1 (cost in #000s) Year 1 Year 2 Year 3 Year 4 Year 5 Year 6 Year 7 Capital cost 250 75 Annual maintenance 25 25 25 25 25 25 25 Conversion costs 75 15 Environment 30 Total 380 40 25 100 25 25 25 DCF @ 5% 1.00 0.95 0.90 0.86 0.81 0.77 0.74 Discounted total 380.0 38.0 22.6 85.7 20.4 19.3 18.4 Total NPV Supplier 1 584.38 Supplier 2 (cost in #000s) Year 1 Year 2 Year 3 Year 4 Year 5 Year 6 Year 7 Capital cost 270 90 Annual maintenance 22 22 22 22 22 22 22 Conversion costs 15 Environment 40 Total 332 37 22 112 22 22 22 DCF @ 5% 1.00 0.95 0.90 0.86 0.81 0.77 0.74 Discounted total 332.0 35.2 19.9 96.0 17.9 17.0 16.2 Total NPV Supplier 2 534.15 Annex C Evaluation Criteria Area Factor to be evaluated Typical questions Evaluation System life Total costs What is the DCF at a discount rate of 5% Capital costs over 7 years? How does a lease/hire option compare? Third party Extra costs What third party support is needed? Recurrent support Has the manufacturer reduced his cost to allow it to be met within the cash limit? Is the manufacturer prepared to accept prime contractor status for equipment other suppliers offered as part of the tender? If not, are there any significant university costs associated with management of a mixed vendor system? Performance CPU power, number of terminals How do benchmark results and evidence from Ranked supported, batch throughput, other sites suggest that the system meets memory size, I/O throughput, or exceeds the mandatory requirements of response times, other the OR? performance factors Arithmetic range and accuracy How accurate are the floating point formats Ranked Performance (throughput) of major Is speed of Fortran compilation particularly Ranked utilities and packages good or bad? Operating Range of peripherals supported Is there a wide range of peripherals Ranked system available from both the supplier and 3rd functionality parties? Range of software available Is there a wide range of software Ranked applications available from the supplier and 3rd parties? Is a Unix subsystem available? Ranked Operating User interface How good is the user interface in terms of Ranked system the prospective users? It is easy for quality novice users to get started, for occasional users to remember? Is it powerful enough for frequent sophisticated users? Quality of utilities and Are utilities and applications good quality Ranked applications products with good documentation or are they unreliable and idiosyncratic? Systems Effort required Are a lot of systems programmers required? Recurrent programming Need they be very high quality? Software maintenance effort Is the system stable or are there frequent Recurrent updates? Are a lot of patches required? Are these supplied in machine-readable form, and can they be supplied over a communic- ations link? Data storage Quantity How much is provided in excess of mandatory Ranked requirements? How much is wasted by inefficient usage? Back-up Will extra staff be needed? Is there Recurrent sufficient hardware? How good are the utilities? Archival Are automatic archival facilities provided? Ranked Communications Performance How much extra capacity is provided in Ranked excess of the mandatory requirements? Functionality Are all the desirable features implemented? Ranked JNT standards How well does the solution conform to Ranked recommended JNT practice? OSI standards How good is the suppliers commitment to Ranked future international standards? Operation User management and control, Are the facilities provided by the supplier Capital performance tuning and adequate? Are special utilities needed to monitoring improve them? Will extra staff be required to manage Recurrent the system? Remote and unattended operation, Will the system require fewer operators Recurrent automatic restart after power because of these features? failure Operation Will more computing time be provided Ranked (continued) (eg because system restarts after power interruption at weekend, providing time that would otherwise have been lost)? Operator interface How good is the operator interface? Is a Recurrent a lot of training necessary? Are a lot of operators needed? Maintenance Cost What is the cost? Recurrent and reliability Availability What is the expected overall availability Ranked of the system? Is this just predicted, or based on experience at other sites? Will the supplier guarantee this level? How much downtime is needed for preventive maintenance? Are remote diagnostics available? Facilities required on site Does the supplier require significant Recurrent space on site for engineers? Reporting and call-out How complex are the procedures? Do they Recurrent procedures require a lot of operator time? Documentation Cost How much is provided free, and what is the Capital cost of extra copies? Are updates free? and Do updates come as replacement pages or as recurrent new manuals? Will the supplier allow adequate reproduction rights? Quality Is the documentation easy to read and Ranked comprehensive? Machine-readability Can documentation be provided in machine- Ranked readable format? Training Costs How expensive are courses? Is self- Recurrent instruction material available? Conversion Costs How compatible is the system with the Capital equipment it is replacing? How much effort will be required to convert to the new system? Are there are any utilities available for program and data conversion? How much of the work will need to be transferred? Compatibility How compatible is the equipment with other systems on campus? Will there be much transfer of work to and from other systems? Will many users need to learn both systems? Upgradeability Ease and cost-effectiveness How cost-effectively can the system be Ranked upgraded? Is such transition easy, or will it have a major impact on the users? Supplier Initial support How much support will the supplier provide Capital relationship to help make the system operational? Ongoing support How much support will the supplier provide Recurrent throughout the life of the system? Track record What is the suppliers record of meeting Ranked commitments at other sites, particularly universities? What is the supplier's record like on Ranked delivery? Understanding of requirement How good is the supplier's understanding Ranked of the requirement, and university requirements in general? Future commitment What is the level of future commitment Ranked to the university market? Future products Has the supplier demonstrated commitment Ranked by discussing plans for relevant future products? Other universities Are there other universities with similar Ranked systems? Is there an active user group? Risk Are all aspects of the system proven, or Ranked are there areas of uncertainty? Environment Machine-room costs Is any building work required, new air- Capital conditioning, power supplies, water-cooling? Electricity costs Recurrent Delivery Schedule Can all equipment be delivered within the Ranked required timescale, or is a phased delivery necessary? Annex D Acceptance procedures CCTA's recommended procedure for acceptance of most systems is a CS16 Parts A & F trial. This consists of a 20-working-day period of normal use of the system, optionally preceded by a demonstration phase. Before the start of the trial, CCTA CT5 Division (Norwich) will send an Acceptance Trial Certificate (CCTA Form 155) to the university which lists the equipment to be tested, and the key contractual dates. Upon completion of normal installation procedures the supplier's representative signs this form to indicate that the system has been commissioned and is ready for a user service. The system then becomes the responsibility of the university. If previously agreed in the MoA, specific demonstrations can be run before the start of the 20-day period to ensure that the system is suitable for a user service. These demonstrations may take a variety of forms. For instance, it may be desirable to test satisfactory operation of the interfaces to existing third-party communications equipment. If peripherals or special interfaces are being retained from a previous system, the connections can be tested at this stage. It may also be appropriate to re-run a benchmark. However, when considering any special demonstration, it should be kept in mind that many of them can be performed just as easily during the 20-day period. The purpose of the 20-day user test period is two-fold: to test the reliability of the hardware (insofar as this is possible in such a short period with modern high-reliability hardware); and to test that the system functions as committed to by the supplier. The 20-day period is intended to be representative of normal use of the system when it goes into service; any work may be run on the system, although it would be unreasonable to use the whole period running programs that stressed electro-mechanical devices. While it is appreciated that new systems are typically much more powerful than the ones they are replacing and users will take some time to learn to use them, it is usually possible to achieve a reasonable level of loading on the system, if necessary by introducing artificial background tasks. The normal process of installing and testing software is usually sufficient to provide a comprehensive test of the functionality of the system. If any faults are found during the 20 days which indicate that the system is not in accordance with the contract, the trial is extended on a day-to-day basis until a successful 20-day period has been achieved, or 60 days have elapsed, when the supplier is in breach of contract. Once the university is satisfied that the system is acceptable, the Form 155 should be signed and returned to CCTA CT5. The supplier will then be formally notified of the successful outcome of the trial, and asked to submit invoices. In practice, it is exceptional for a complete system to be accepted without some reservations. The most common problem is that some components of the system are not delivered on time. If sufficient equipment is present then the trial may start on the reduced configuration, with a separate 20-day testing period applied to items that are delivered later. However, the university is not obliged to accept such a situation, and may decline to start the trial until the full configuration is present. It is important to realise that if this option is exercised, the system remains the property of the supplier until the start of the trial, and the university have no right to use it whatsoever. Any use of the system before the start of the 20-day period could constitute acceptance of the system because of the implication that the system was suitable for processing some user work. The other common reason for partial acceptance is that some part of the system fails to function correctly. If this happens, the remainder of the equipment can be accepted, and a separate 20-day trial re-run once the deficiencies have been rectified. With either case of partial acceptance it is vitally important that the acceptance certificate makes it clear exactly what has and has not been accepted. Appropriate sums of money will be withheld until deficiencies have been rectified. The university must keep full records on all incidents during the trial. CCTA (CT1) do not need to attend during a Part F trial, but would expect to be involved if there are any problems or disputes. Further advice on acceptance trials is available from CCTA CT1 Division, Riverwalk House. --- End of forwarded message