Verification

image of military personnel performing test of equipment

About

The Verification process provides the evidence that the system or system element performs its intended functions and meets all performance requirements listed in the system performance specification and functional and allocated baselines. Verification answers the question, "Did you build the system correctly?" Verification is a key risk-reduction activity in the implementation and integration of a system and enables the program to catch defects in system elements before integration at the next level, thereby preventing costly troubleshooting and rework.

Role of the PM and SE

The Program Manager (PM) and Systems Engineer, in coordination with the Chief Developmental Tester, manage verification activities and methods as defined in the functional and allocated baselines and review the results of verification. Guidance for managing and coordinating integrated testing activities can be found in Test & Evaluation Enterprise Guidebook and in DoDI 5000.89, Test and Evaluation.

Activities

Verification begins during Requirements Analysis, when top-level stakeholder performance requirements are decomposed and eventually allocated to system elements in the initial system performance specification and interface control specifications. During this process, the program determines how and when each requirement should be verified and the tasks required to do so, as well as the necessary resources (i.e., test equipment, range time, personnel, etc.). The resulting verification matrix and supporting documentation become part of the functional and allocated baselines.

Verification may be accomplished by any combination of the following methods:

Demonstration. Demonstration is the performance of operations at the system or system element level where visual observations are the primary means of verification. Demonstration is used when quantitative assurance is not required for the verification of the requirements.
Examination. Visual inspection of equipment and evaluation of drawings and other pertinent design data and processes should be used to verify conformance with characteristics such as physical, material, part and product marking and workmanship.
Analysis. Analysis is the use of recognized analytic techniques (including computer models) to interpret or explain the behavior/performance of the system element. Analysis of test data or review and analysis of design data should be used as appropriate to verify requirements.
Test. Test is an activity designed to provide data on functional features and equipment operation under fully controlled and traceable conditions. The data are subsequently used to evaluate quantitative characteristics.

Designs are verified at all levels of the physical architecture through a cost-effective combination of these methods, all of which can be aided by modeling and simulation.

Verification activities and results are documented among the artifacts for Functional Configuration Audits (FCA) and the System Verification Review (SVR) (see SE Guidebook Section 3.6). When possible, verification should stress the system, or system elements, under realistic conditions representative of its intended use.

The individual system elements provided by the Implementation process are verified through developmental test and evaluation (DT&E), acceptance testing or qualification testing. During the Integration process, the successively higher-level system elements may be verified before they move on to the next level of integration. Verification of the system as a whole occurs when integration is complete. As design changes occur, each change should be assessed for potential impact to the qualified baseline. This may include a need to repeat portions of verification in order to mitigate risk of performance degradation.

Outputs

The output of the Verification process is a verified production-representative article with documentation to support Initial Operational Test and Evaluation (IOT&E). The SVR provides a determination of the extent to which the system meets the system performance specification.

Products and Tasks

Product	Tasks
AWQI 7-1-1: Design and implement a testing process to compare actual performance with required system/item performance	Identify system technical and operational requirements. Identify modeling and simulation system verification documents as applicable. Identify developmental test and evaluation documents as applicable. Identify operational test and evaluation documents as applicable. Review existing test and modeling and simulations data, analysis and reports for program applicability. Design and implement a testing process to compare actual performance with required system / item performance.
AWQI 7-2-1: Verify system compliance with defined physical architecture	1. Identify system build-to specifications. 2. Examine production representative article and compare with build-to specifications. 3. Assess system compliance with defined physical architecture. 4. Identify any deficiencies between physical architecture and build-to specifications. 5. Correct deficiencies and document compliance with defined physical architecture.

Product

Tasks

AWQI 7-1-1: Design and implement a testing process to compare actual performance with required system/item performance

Identify system technical and operational requirements.
Identify modeling and simulation system verification documents as applicable.
Identify developmental test and evaluation documents as applicable.
Identify operational test and evaluation documents as applicable.
Review existing test and modeling and simulations data, analysis and reports for program applicability.
Design and implement a testing process to compare actual performance with required system / item performance.

AWQI 7-2-1: Verify system compliance with defined physical architecture

1. Identify system build-to specifications.
2. Examine production representative article and compare with build-to specifications.
3. Assess system compliance with defined physical architecture.
4. Identify any deficiencies between physical architecture and build-to specifications.
5. Correct deficiencies and document compliance with defined physical architecture.

Source: AWQI eWorkbook

Resources

Key Terms

Developmental Test Adequacy Evaluation
Developmental Test & Evaluation (DT&E)
Developmental Test Adequacy Evaluation (aka Developmental Test Assessment)
Developmental Test and Evaluation (DT&E)
Functional Configuration Audit
Inspection
Prototype Models
System Verification Review
Test & Evaluation Master Plan (TEMP)
Test Readiness Review (TRR)
Testing
Verification
Verification, Validation, and Accreditation

Source:
DAU ACQuipedia
DAU Glossary

Policy and Guidance

SE Guidebook Section 4.2.6. Verification Process
DoDI 5000.89, Test and Evaluation, Section 5, DT&E
DoD Cybersecurity Test and Evaluation Guidebook
DoD Cybersecurity Test and Evaluation Guidebook, Addendum
IEEE 15288.1-2014 Section 6.4.9 Verification process
SEBoK ??? System Verification
Systems Engineering Plan (SEP) Outline Section 2.1 Requirements Development
Test and Evaluation Enterprise Guidebook

DAU Training Courses

TST 102: Fundamentals of Test and Evaluation
TST 2040V: Test and Evaluation for Practitioners
LOG 0030: Supportability Test & Evaluation
TST 1100 :Intro to Systems Engineering for Testers
TST 2100 :Applied Systems Engineering for Testers
CLE 023: Modeling and Simulation in Test and Evaluation
CLE 084: Models, Simulations, and Digital Engineering
WSE 006 :Engineering Management Workshop (EMW)
WSE 026 :Test & Evaluation Across the Acquisition Life Cycle
WSS 013 :Cyber Training Range ??? Aviation Systems