Quality Domains
###############

This page describes the different domains that identify aspects of methods. It describes each domain and then illustrates the elements of
Business Object Model that are used to describe the domain.
   
.. figure:: /objects/images/xin_model_h_yout.png
	:scale: 100%
										
	Simple method 'h' with inputs 'x' and outputs 'y'
   
========
Overview
========

There are several important |quality domains| that describe a method, these are illustrated below.
   
.. figure:: /objects/images/ql_domains_graph.png
	:scale: 100%
										
	Quality Domains - Single Input, Single Output illustration

For a method that has a single input and single output this is a fairly simple diagram. Adding an second input changes the
diagram into a surface plot. Adding more inputs is not possible to illustrate in a 3D diagram. The below diagram shows a delta
error from the real output to the simulated output with two input value. For a perfect simulation the values would all be zero, and the
graph would be a horizontal planar surface. The colours represent the deviation from the ideal, with green as acceptable, yellow and orange
as use at risk, and red as unacceptable.
   
.. figure:: /objects/images/ql_domain_2x_input.png
	:scale: 100%
										
	Two inputs, single output, showing the delta error
   
.. figure:: /objects/images/ql_domain_1x_input.png
	:scale: 100%
										
	Single input, single output showing the delta error
   
================================================
Enterprise Architect Concept Model Documentation
================================================

For more information see EA - VVA concepts for MS
   
Domain of Verification
**********************
   
The conditions under which verification was conducted. [1][2]
Practically, the Domain of Verification of a simulation model can be characterised by the extent of the requirements on which the model
is demonstrated to be correct.

The verification effort shall be intended to define this Domain of Verification. Reviews and tests of the models and simulations are
classical means to perform the Verification.  

..note::
               
  * The Domain of Verification of simulation model is often characterised by M&S requirements coverage matrix.
  * Bugs identification and simulation unexpected stop are also tracked to identify whether simulation model can contain
    defects or show anomalies (like any software based application.           
               
.. note::
   References:
   
   * NASA-STD-7009, Standard for models and simulations, National Aeronautics and Space Administration Washington, DC 20546-0001,
   * November 7th 2008, available online. 
   * CRESCENDO FP7-234344 EC project, BDA Quality Laboratory Specifications, deliverable D5.4.2, version R2.1 
   
This is the limits of the over which the method was verified. Using graph shown on the Overview image, it is shown as
the maximum and minimum of the curve.
   
.. figure:: /objects/images/domain_of_verification.png
	:scale: 100%
										
	Domain of Verification

If there are not explicit points at the maximum and minimum then the limits are dependent on the curve that is fitted
through the data points, i.e. on the curve fitting algorithm.

Domain of Validity
******************

Complete set of all experimental frames in which the model is valid. 

Practically, the Domain of Validity of a simulation model can be characterised by the limits of the possible values
for input variables and parameters on which the simulation of the model will give valid simulation outputs or results.
               
The validation effort shall be intended to delimit this Domain of Validity. Note as the resources (time and budget) can
increase to an unaffordable level, most of the time the Domain of Validation is the only available knowledge on the domain
of validity restricted to a limited number of input vectors (i.e. sample).
               
.. note:: 
   In analogy with a real scalar which can take values between lower and upper bounds, one can alternatively refer to range
   of validity.
               
.. note::
   References:
   
   * Simulation Interoperability Standards Organisation (SISO) General Methodology for Verification and Validation (GM-VV)
     and Acceptance of Models, Simulations and Data Handbook, Reference Manual and Recommended Practice Guide [Draft documents
     for comment round] [available online].
               
This is the limits over which the method is valid. This is usually a smaller range than the Domain of Verification. 
   
.. figure:: /objects/images/domain_of_validity_1.png
	:scale: 100%
										
	Domain of Validity - simple case
   
An allowable deviation between the real and the simulated outputs is set. This gives the range of outputs that the method is valid.

From this range it should be possible to identify the range of inputs that are valid. 
e.g. InputA>200 and InputB>0.

If there is more than one input, then it may be there are some cases where the limits can be extended, but only when limits are
imposed on other inputs. This means that there need allow links between the limitations on the inputs.

E.g. if 100<InputA<200 then 5<InputB<10, otherwise InputA>200 and InputB>0. 

Sometimes there may be more than one range where the method is valid. This is illustrated below. If this is the case, then their needs to be
more than one set of limitations.

E.g. 100<InputA<500 and 600<InputA<1000 
   
.. figure:: /objects/images/domain_of_validity_2.png
	:scale: 100%
										
	Domain of Validity - complex case
   
Domain of Validation
********************
   
Subset of the Domain of Validity that has been effectively covered by the validation methods. This is the good enough knowledge
that can be obtained on the domain of validity according to the planned Validation effort documented in V&V plan.
   
These are the discrete points from the original validation of the method that are within the limits of the Domain of Validity.
Each point will have a specific set of inputs values (or models) and a specific output value (or model).
   
.. figure:: /objects/images/domain_of_validation.png
	:scale: 100%
										
	Domain of Validation
   
Domain of Intended Use
**********************
            
It describes the problem to be addressed by the M&S, including the system or process being represented and the role it plays
in the overall program.

What the model is intended to cover, from the requester view point. This information is part of the specification, and in
particular is part of the rational for M&S requirements.

Intended use is defined at the planning stage and typically give the context on which the simulation study in terms of goals
and purpose.
               
=======
Example
=======

In S4P2 Fireproof gearbox scenario, the intended use of a virtual test bench with simulation models is aimed at provide
confidence that the gearbox design will be structurally resistant to flame exposure and provide virtual means of compliance
towards associated certification requirements.
                              
.. note::
   References:
   
   * MIL-STD-3022, Department of Defense Standard Practice Modeling and Simulation (M&S) Verification, Validation and
     Accreditation (VV&A) Documentation Templates, Approved on 28 January 2008
                                
Acceptable Domain of Use
************************
   
It refers to the actual domain of application of the simulation related to the experimental frame defined with limits
that constraint the execution of simulation to an bounded set of conditions and hypothesis. Typically acceptance
(or accreditation in US Defense terminology) is concerned in comparing the applicable domain of use and intended domain of use
and deciding to approve or authorise the usage of this accredited model limited to this acceptable domain of use
                              
=========================
Analogy for Understanding
=========================

Let consider a similar situation to the computation of the cosine function over radian angle. The cosine function
is defined mathematically for all real numbers but implementation by computer is based on floating point (IEEE Std 745 compliant)
numbers. Note that IEEE std 745 floating point format will consequently limit the range to less than 3.40282347e+38 for single
precision 32-bits float. Depending on the implementation of this cosine the accuracy of the result is limited to a given range
of input values and rounding will degrade numeric precision for large number. Even if largely valid for input larger than pi it
can be decided to limit the domain of use of this function to computation of the cosine of input in range [-pi, pi]. When a
given application needs to manipulate larger angles than pi like 2pi, the simulation user should pre-process the
input angle in order to normalise it, for instance changing 2*pi in 0.
                              
=======
Example
=======

Assuming we want to compute the maximum temperature over a aircraft structure submitted to heat flow, the simulation model
shall work as intended use. The validation of this model is performed on a subset of the intended use domain and gives accurate
results with positive comparison with experimental measurements. Nonetheless a the border of this Domain of Validation, the
accuracy degrades rapidly, so acceptance decision limits the domain of use to where the simulation model results validity is
better.
               
               
.. note::
   References:
   
   * Simulation Interoperability Standards Organization (SISO) General Methodology for Verification and Validation (GM-VV)
     and Acceptance of Models, Simulations and Data Handbook, Reference Manual and Recommended Practice Guide [Draft documents
     for comment round] [available online].
               
M&S Fitness (for purpose)
*****************************
   
Providing the capabilities, correctness, accuracy, and usability needed for the intended use or current application
               
.. note::
   References:

   * DMSO VV&A RPG Glossary
               
.. sectionauthor:: |Judith Crockford|

.. include:: /objects/documents/main/Keywords.rst