Quality Gates and Associated Capabilites

This page describes the Quality Gate concept and describes the elements of Business Object Model that are used in relation with them

Overview

Quality gates are predefined steps in a simulation process where quality-related elements associated to the process must be assessed and qualified prior to any action. The process might terminate or choose among different paths according to the results of the Quality Gate.

Quality gates mark programme decision points where assessment of models is conducted based on collected V&V evidence. It aims at reconciling the different viewpoints from the supplier and the customer of M&S results into consistent objects capture. It provides to the user or to the architect simple and unambiguous indicators that can be shared in the context of Extended Enterprise

Elements associated to Quality Gates

The tests and activities performed at a Quality Gate depend on the context of the study. However, some elements are common to most gates. Here is an example of two quality gates implemented in the thermal process S4T1

S4T1 process with Quality Gates

Quality Gate #1: Check simulation and test data

• Traceability
• Consistency
• Completeness (data and attached information)
• Version concordance
• Expert assessment

Quality Gate #2: Check the comparison

• Relevance of the prepared input data
• Traceability of output data and link to input data
• Reusability of output data
• Concordance between output data and specification
• Completeness of output data,
• Expert assessment (report) of the identification tool
• Expert assessment (report) of the error quantification tool
• Definition of the error threshold
• Existence of a comparison report

Quality Gate representation using Business Object Model classes

The following describes the information that can be captured concerning a Quality Gate

Note

A QualityGateInstance is represented by a gate.

Quality Gate Instance icon

A QualityGateType is represented by a gate with a mortar board - the common symbol to denote a methodology object.

Quality Gate Type icon

A QualityReportInstance is represented by a document with a rosette.

Quality Report Instance icon

A QualityReportType is represented by a document with a rosette and a mortar board - the common symbol to denote a methodology object.

Quality Report Type icon

Quality Gate Type

In some cases a Quality Gate is a formal process that follows a rigorous procedure. The example shows the Quality Gate Type which has formalDocumentation pointing to a procedure document for the gate. It also shows the typesOfPersons that are to be approvers and authorisers of the gate. The gate type has input types (IsGateTypeInput) and shows the type(s) that it is to control (ModelTypeIsControlledByGate). These are all in the context of the CollaborativeModelTemplate. This means that the same QualityGateType can have different inputs and control different ModelTypes in another context.

This is illustrated below, which shows the QualityGateType in the bottom right of the diagram.

View of classes associated with a Quality Gate Type

For more information see: SysML - Quality Gate Template Management.

Quality Gate Instance

A Quality Gate does not create any model data itself; it is just a Go/NoGo decision for something. Any updates to models should be part of the normal ModelInstance derivedFrom order; any checking that needs to be recorded would be stored as Quality Reports or meta-data for the item being checked.

The diagram below shows the information surrounding a Quality Gate Instance.

View of classes associated with a Quality Gate Instance

When an instance of a QualityGate is created, it belongs to an AssociativeModelNetwork. It is an instance of a QualityGateType from which it gets the types it should connect to and the typeOfPerson that should sign off the Quality Gate.

The Gate instance is then connected to the inputs (IsGateInstanceInput) and to the ModelInstance(s) that is controls (ModelInstanceIsControlledByGate). Though not shown, both of these can be connected to ModelInstanceDataFlows for when the gate has an associated method with declared inputs (see Methods DataFlow for more details).

The lifecycle of the QualityGateInstance in this example would be as follows:

• Creation according to the QualityGateType definition, with status “Planned”

• Selection of Person(s) to be the plannedApprover, and these also have status “Planned”

• A meeting is planned, and happens (This part of the example is further elaborated in the sub-chapters below.)

• On conclusion of the meeting:

  • The status of each approval is updated to “Passed” (or “Failed”)
  • The “actualApprover” relationships are added; in this example “sign off mesh” has a different actual approver to the planned approver.
  • Rationale documents can be added to the approvals or justifications added that point to other objects as support. This is illustrated for “sign off mesh”, where justification has been added that it is OK to sign because of a good Quality Report. The justification connects the approval of the Quality Report to the approval for the Quality Gate.
  • The status of the gate is updated to “Passed” (or “Failed”)

For more information see: SysML - Quality Gate Management and SysML - Justifications and Assumptions.

Planned Meeting for Quality Gate Instance

Icons for Business Object Model Classes

A PlannedMeeting is represented by a two people talking in front of a calendar.

Planned Meeting icon

An ActualMeeting is represented by a two people talking.

Actual Meeting icon

The diagram below shows a planned meeting for a Quality Gate Instance.

View of classes for a planned meeting for a Quality Gate

The features of the planned meeting are:

• The QualityGateInstance is the subject of the PlannedMeeting.

  • For illustration, the planned meeting has a minutes template which points to the formal documentation from the QualityGateType.

  • It has three meeting roles:

    • “Chair/QualityGateManager”
    • “Mass Expert”
    • “Mesh Expert”. There are two of these roles, the first one (red border) has a declined acceptanceStatus which is why there is a second one with a different person as the attendee.

The QualityGateInstance shows the status of the approvals as planned, and at this stage in the gate lifecycle there are only plannedApprover relationships.

Actual Meeting for Quality Gate Instance

The diagram below shows an actual meeting for a Quality Gate Instance.

View of classes for a actual meeting for a Quality Gate

The features of the AcualMeeting are:

• The QualityGateInstance is the subject of the ActualMeeting which points to the PlannedMeeting as its plan. The details of the planned meeting are hidden in the diagram.
• The meeting points to a document as the minutes of the meeting.
• The Person “Jennifer Bernard” is assignee in the role “Chair/QualityGateManager” for the meeting.
• She is also the plannedApprover and actualApprover for the QualityGate authorisation “authorise to proceed” and the status of this has been set to “Passed”.
• The minutes from the meeting have been added as rationaleDocument for the “authorise to proceed” of the Quality Gate
• The Person “John Lee” is assignee in the role “Mass Expert” for the meeting.
• He is also the plannedApprover and actualApprover for the QualityGate approval “sign off mass” and the status of this has been set to “Passed”.
• The minutes from the meeting have been added as rationaleDocument for the “sign off mass” approval of the Quality Gate
• The Person “Clever Chap” is assignee in the role “Mesh Expert” for the meeting.
• He is also the actualApprover for the QualityGate approval “sign off mesh”, and the status of this has been set to “Passed”.
• He is not the plannedApprover for “sign off mesh” this was the person “Pascal Martin” who declined the PlannedMeeting so was replaced (see sub chapter above)”
• The minutes from the meeting have been added as rationaleDocument for the “sign off mesh” approval of the Quality Gate
• A justification for approving “sign off mesh” has been provided. This points to the ActualMeeting as support, and also has support as the approval for a Quality Report. It has a description “OK cos good QR”

For more information see: SysML - Meetings and SysML - Justifications and Assumptions.

Decision capture mechanisms - Argumentation

This section describes the collaborative design support framework to track group decisions.

Note

Note this capability is for collaborative design but could be used for any collaborative decision e.g: process termination, trade-off, validation by a group of experts, etc.

Overview

Collaborative Design Support is a process to support an experts’ discussion concerning a specific decision, like collaborative modelling, and a formal framework to understand and analyse the experts’ choice. We propose a methodology in four steps, supported by a graphical modelling of the debate, and a formal framework to record debate allowing analysis of the decision.

Benefits:

• produce a more useful capture of the meeting that can be reviewed and analysed later on,
• support the decision – the decision captured is part of the Quality Gate report
• help future projects to reach a quicker and better substantiated decision

Benefits for engineers:

• rational engineers thinking during a meeting
• make links between everyones ideas in meetings
• clarify the design in an alternative domain
• support certification by tracking the design process

Benefits for decision makers:

• provide metrics to help understand the decision
• support decision maker in the light of experts’ arguments
• record motivations of the decision, in a corporate memory in a useful form

Motivation

Keeping a record of meetings that captures both the arguments and their relationships can help in the understanding of design rational and design choices. Keeping a record of meetings is also very interesting for the constitution of a corporate memory.

Very often, when a project is over, no one remembers why certain choices have been made, mainly because people have left the company or have forgotten, and the reasons for decisions have not been recorded. Keeping records of the arguments would avoid having the same discussion all over again, with the same arguments. Recording the debate would allow the discussions to be resumed and experts could introduce new arguments if they feel the need to. In addition, having a visual representation of a debate allows its better understanding.

Four step Process

Example of an Argumentation network

Worked Example using Business Object Model Classes

Icons for Business Object Model Classes

An Argumentation Network is represented by a three people in a network with the connections glowing red and green.

Argumentation Network icon

An Argument is represented by crossed swords.

Argument icon

A Claim is reprensented by crossed swords with a hand palm upwards.

Claim icon

A Decision is represented by crossed swords, and shaking hands.

Decision icon

A Justification is represented by a pointing hand.

Justification icon

An Approval is represented by a thumbs-up hand.

Approval icon

The image below shows a very simple argumentation network for the Quality Gate. It shows a series of arguments made by different people concerning the quality of the input to the gate. The arguments are then linked to (support) the final decision (orange) to pass the gate.

The details are as follows:

• The ArgumentationNetwork is shown at the bottom of the diagram

  • All the seven arguments and the decision “belong” to the argumentation network
  • It was recorded as part of a meeting; this is shown by the “argumentation” link from the ActualMeeting (middle of image).

• The first argument is “Mesh is good according to the Quality Report“

  • This argument is made by “Clever Chap”.

    • It has two pieces of supporting evidence for the argument, the QualityReportInstance and its approval.
    • The argument is “about” (justifies - green) signing off the mesh for the Quality Gate.
    • It is a supporting argument (blue) for the decision to pass the gate.

• The second argument is “Mass model is good”

  • This argument is made by John Lee.

    • The argument is “about” (justifies - green) the mass model.
    • It is challenged by the next argument “no evidence to show mass model is good”

• The third argument is “no evidence to show mass model is good”

  • This challenges the “Mass model is good” argument because there is no evidence for the argument
  • It is made by “Jennifer Bernard” the authoriser of the Quality Gate.
  • The argument is “about” (justifies - green) the mass model.
  • This is then countered again by John Lee with the argument that the “mass model created by reliable person”

• The fourth argument is “mass model created by reliable person”

  • This challenges the lack of evidence argument and supports the original argument that the “Mass model is good”

    • This argument is made by John Lee.
    • The argument is “about” (justifies - green) the mass model.

  • It has supporting evidence (blue) pointing to the TypeOfPerson that created the mass model.

    • It is a supporting argument for the decision to pass the gate.

• There are two final arguments to agree that the mesh and mass are acceptable.

• The decision to pass the gate is shown to be supported by several of the arguments.

  • It is a decision “about” (justifies - green) the Quality Gate.
  • It could also point to all the support evidence for the supporting arguments, but this derivable so does not need to be explicitly defined.

For more information see: SysML - Decisions and Argumentation and SysML - Justifications and Assumptions.

Quality Dashboards

A Quality Dashboard provides a simple set of visual indicators associated with the level of quality reached at the gate

Quality Dashboard

Section author: Judith Crockford