@b(8 THE EXPERIMENTER TOOL)

The experimenter tool supports the design of experiments and their execution
using objects produced by modelling tools and producing outputs acceptable
to the reporter. In general terms it allows series of connected runs and 
analysis of results.

Modelling tools which have been implemented or modified in accordance with the
Simmer standards for shared objects, such as output specifications, will be
capable of inclusion in the experimenter's facilities.

Alien models and modelling tools may be used in limited ways, probably using
the provision for entering results from real world systems or something
similar.

Conceptual provision is also made for analysis of observations of real 
systems and of hybrid systems. This will not be fully implemented in the pilot,
although a limited formatting facility may be provided for real world data,
but will form a subject for an advanced study.

@b(8.1 Concepts)

Experiments will involve one or more systems or models of systems. Where more 
than one such is to be studied, it is the responsibility of the user to define 
the meaning of any comparative results.

Experiments involve the definition of @i(factors) whose values may be set to 
desired @i(levels). A factor is either a selection amongst comparable 
models and systems, a free variable in a model's input or control
specification or a variable quantity in a real system.
For a particular experiment a factor may be fixed for all runs, allowing the 
effects of others to be isolated and appropriate analysis to demonstrate the 
particular relationships under study.

Analyses will use @i(response variables) defined by the user for that experiment. 
Such variables must have been specified in the output specification for the 
model or system, when the corresponding SIMMER model or system object was
generated by the appropriate tool.

Simulation studies can only estimate the value of their response variables to 
some degree of statistical precision. Unless the method of independent
replications is to be used this requires a method of estimation to be built
into the model by the appropriate modelling tool. However, the 
estimation method may be capable of parameterisation via a free variable in 
the control specification. Thus the experimenter will control replications 
completely and other estimations to the extent that they are parameterised.

Experimental factors which cannot readily be incorporated as free variables 
will require variant models to be run. This can be expensive and error prone.

In general, support will be intended for pre-planned experiments involving 
simple binary comparison, regression analysis on a single variable factor or 
factorial analysis on multiple factors. No automated support for more elaborate 
strategies, such as heuristic searches, is deemed feasible currently.

@b(8.2 Facilities)

@b(8.2.1 Creation and editing) of an experimental plan is the core of the
experimenter tool. This is a SIMMER object and amongst its attributes will
be:

~the model or models~ to be used in the experiment

~the factors to be varied~ and the mapping of these onto free variables in
each model used

~the fixed values~ to be mapped onto remaining free variables in each model

~a run matrix~ which will define the values used for the variable factor or 
factors in each run, including the model

@b(8.2.2 A @i(real system data processor)), allowing response data from real world systems
or alien modelling tools to be processed as if they were outputs from a model and including a means of
defining a dummy output specification to match them may be provided. This
would allow such data to be "edited in" along with model outputs, for analysis
and/or comparison.

@b(8.2.3 Replication of runs) of simulations will be supported.

@b(8.2.4 Automatic performance of experiments) from experimental plans will be 
provided. This may be performed in @i(batch) or @i(interactive feedback) may be 
provided.

@b(8.2.5 Automatic analysis and presentation of results) in user requested forms, 
such as graphs and tables will be performed. Analysis techniques will include 
test of means, regression, curve fitting and analysis of variance. Output 
objects will be compatible with the reporter tool.

@b(8.2.6) Where @b(further or alternative analysis) of a set of results is 
appropriate, the user will be able to invoke a suitable analyser for a set of 
results independently of a run or set of runs. This will be important in using 
imported real world observations.

@b(8.3 The user interface)

At the outer level of SIMMER the user will be able to choose to manipulate
experiments in three ways.

    *  Icons representing those experiments currently under construction,
       being modified or ready for use will be presented - work in hand.

    *  A tool icon which invokes generation of a new experiment will be
       presented.

    * A database of archived models may be acessed to provide an
          experiment for review or modification.


@b(8.3.1 Inside an experiment object)

The experimenter will accept input from forms, which will essentially construct
and/or complete sets of data required to allow experiments to be run.
Some forms will already exist as a result of model generation, e.g. free
variable lists and output specifications.  These will need to be completed.
Other forms may be constructed from scratch, e.g. run matrices.

It may be sensible to allow free variable lists to be constructed by the
experimenter before models are constructed or selected.  Models used 
would then have to match the requirements of the experiment (or be 
modified to do so).

At all levels users will be encouraged to supply notes and comments,
which could be used by the reporter.
@n()