Take a sneak peek at the new NIST.gov and let us know what you think!
(Please note: some content may not be complete on the beta site.).

View the beta site
NIST logo

MBE PMI Validation and Conformance Testing

Overview

NIST has created a Test System to measure conformance of Computer-Aided Design (CAD) software to American Society of Mechanical Engineers (ASME) standards for Product and Manufacturing Information (PMI), specifically geometric dimensioning and tolerancing (GD&T) information.

PMI includes geometric dimensioning and tolerancing, 3D annotations, surface texture specifications, finish requirements, process notes, material specifications, welding symbols, and other information. One of the objectives of PMI standards is to define the semantics of the words and symbols used to communicate manufacturing information in 3D computer models. The use of PMI Representation (aka semantic PMI) will allow software developers to automate various design (CAD), manufacturing (CAM), and inspection (CMM) functions because the engineering application software associated with these functions can process the PMI directly. PMI presentation (aka graphical PMI) consists of geometric elements preserving the exact appearance of the PMI annotations and is meant to be human-readable.

Test System Components

The test system has five main components:

  1. Test Case definition and expert review
  2. Test CAD Model creation based on the test case definition
  3. Verification of the CAD models against the test case definitions
  4. Generation of derivative STEP, JT, and 3D PDF files by the Implementor Forums
  5. Validation of the derivative files against the CAD models and test case definitions

MBE-PMI-validation-data-process-flow

Test Cases

A representative collection of PMI concepts commonly used in practice were selected from ASME Y14.5-1994 (Dimensioning and Tolerancing) and ASME Y14.41-2003 (Digital Product Definition Data Practices) from which a set of test cases were developed for determining whether CAD software correctly implements the PMI concepts. ASME Y14.5 defines the symbology used for specifying PMI.  ASME 14.41 specifies how to present PMI in 2D and in 3D. The most current versions of ASME Y14.5 (2009) and Y14.41 (2012) are not included as they have not yet been widely implemented in CAD software.

There are two types of Test Cases. Both types of test cases are 2D drawings of an annotated 3D model that is used as the basis for authoring the CAD test models.

Combined Test Case (CTC)

A Combined (or Complex) Test Case (CTC) is a combination of Atomic Test Cases (ATC). An Atomic Test Case highlights an individual PMI annotation to be tested, called the measurand. The ATC is not a complete specification of the part’s PMI, but rather contains only the PMI needed to specify enough context information to understand the measurand. Typically this means one or more examples of the measurand, along with any datum features referenced. ATCs, although useful for conformance testing, are not “realistic” in that they include only measurand-related PMI.

A Combined Test Case (CTC) is superset of the PMI of all ATCs for a particular part model. Each CTC specifies one of the five distinct part geometries developed for the project. Depending on the complexity of the part, its CTC may provide more than one view. A CTC is not intended to be a fully-toleranced test case, it is only a combination of PMI from a set of ATC.

Fully-Toleranced Test Case (FTC)

An FTC is a Fully-Toleranced Test Case where the intent is to have all of the geometric features fully-toleranced in the context of GD&T. Fully-toleranced means that each tolerancing feature on a model is adequately controlled and constrained by tolerances that comply with the applicable dimensioning, tolerancing, and modeling standards. Each FTC includes as many annotation types and constructs needed to fully control and constrain each feature relative to one or more datum reference frames.

Test Case Expert Review

The Test Cases were peer-reviewed by several GD&T experts to ensure that PMI concepts were correctly specified according to the ASME Y14 standards, that appropriate concepts were covered, and to minimize errors and misunderstandings. The Test Cases are not intended to represent best practice in how to apply GD&T to a part. Simpler GD&T strategies could have been used. The Test Cases are intended to exercise valid presentations of GD&T defined in the ASME Y14 standards.

Test CAD Models

Test Cases alone are insufficient for automated conformance testing to take place. For each Test Case, a Test Model generated in four CAD systems. Test models were generated from the geometrical definition and PMI concepts represented in the corresponding test case. As best as possible, the PMI concepts were modeled in the CAD system as semantic information rather than just graphical presentations. Issues in modeling PMI concepts in each CAD system were documented and shared with the CAD system software vendor to determine whether the issue was due to misuse of the CAD system to generate a particular PMI construct or truly resulted from a lack of capability of the CAD system.

The native CAD models have been made available to the CAx Implementor Forum (CAx-IF), JT Implementor Forum (JT-IF), and 3D PDF Implementor Forum (3DPDF-IF).

MBE PMI validation test case example

CAD Model Verification and Derivative File Validation

A key objective of the project is to verify and validate the semantic representation of PMI concepts in CAD models. For model verification and validation, each of the native CAD models from one CAD system are compared to the equivalent native CAD models from the other CAD systems. This round-robin verification process serves to ensure that all of the native CAD models are as equivalent as possible to each other from a geometry and PMI standpoint. Feedback from the verification process is used to refine the CAD system test models. Verification categories include many characteristics of geometry and GD&T annotations.

The validation process compared derivative STEP, JT, and 3D PDF files to the original native CAD models. Feedback from the validation process is used to improve CAD system STEP, JT, and 3D PDF translator software.

*
Bookmark and Share

Download

Click here to download: 

  • CAD Models for both Datasets (see below)
  • 2D drawings with annotations for all test cases
  • STEP AP203 files for all test cases (geometry only)
  • STEP AP203 and AP242 files with PMI, generated from the Dataset 1 CAD models

Download the STEP File Analyzer

Test Results

The verification and validation results measure PMI implementation capabilities in CAD software and derivative STEP, JT, and 3D PDF files.  The results are based on the test case specifications, CAD modeling methods, versions of CAD software, and comparison methodologies.  PMI representation and presentation are evaluated by many verification characteristics and validation issues.  The results may identify issues relevant to your use cases for downstream automated-consumption of PMI representation or human-interpretation of PMI presentation.

Other Testing

AFNeT STEP AP242 Benchmark Testing - This is a European project that is testing AP242 with a larger set of models, a different set of CAD software, and a different test methodology and criteria.


Dataset 1 (December 2014)

Dataset 1 includes five CTC (Combined/Complex Test Cases) and associated CAD Models.

Click on the images for 2D drawings of each test case.  CTC 02 and 04 fit together in an assembly.
       CTC-01             CTC-02              CTC-03
PMI Validation Complex Test Case 1 PMI Validation Complex Test Case 2 PMI Validation Complex Test Case 3
       CTC-04             CTC-05
PMI Validation Complex Test Case 4 PMI Validation Complex Test Case 5

The CTC Test Cases have been modeled in four CAD systems: Dassault Systemes CATIA V5 R21, PTC Creo 2.0, Siemens NX 8.0, Dassault Systemes SolidWorks 2012.

Dataset 2 (June 2015)

Dataset 2 includes six FTC (Fully-Toleranced Test Cases) and associated CAD Models for only the three FTC highlighted below.

Click on the images for 2D drawings of each test case.  FTC 07, 08, 09, and 10 fit together in an assembly.

       FTC-06               FTC-07              FTC-08
Screenshot of FTC-06 test case  Screenshot of FTC-07 test case  Screenshot of FTC-08 test case
       FTC-09               FTC-10              FTC-11
Screenshot of FTC-09 test case  Screenshot of FTC-10 test case  Screenshot of FTC-11 test case

The FTC Test Cases have been modeled in four CAD systems (more recent versions than Dataset 1): Dassault Systemes CATIA V5-6R2014, PTC Creo 3.0, Siemens NX 9.0, Dassault Systemes SolidWorks 2015

Other Data

CAD models and associated files from the Design to Manufacturing and Inspection project (D2MI) are included with the download of the two datasets.


Project Team

Robert Lipman, Josh Lubell, Tom Hedberg,
Allison Barnard Feeney, Simon Frechette - NIST

Doug Cheney, Asa Trainer, Tyler Smith - ITI TranscenData

Bryan Fischer - Advanced Dimensional Management
Curtis Brown - Dept. of Energy NNSA Kansas City Plant
Ted Anderson - Sigmetrix
Rich Eckenrode, Malkin Shaw - RECON Services
Sagar Patil - Neilsoft

The MBE PMI Validation and Conformance Testing Project is funded by the Digital Thread for Smart Manufacturing project.

Disclaimers

NIST Disclaimer

NIST License

The test cases are not intended to represent best practice in how to apply GD&T to a part.  Simpler GD&T strategies could have been used. The test cases are intended to exercise valid presentations of GD&T defined in the ASME Y14 standards.

The test cases, CAD models, and STEP files can be used without any restrictions.  Their use in other software or hardware products does not imply a recommendation or endorsement of those products by NIST.

We would appreciate acknowledgement if any of the test cases, CAD models, and STEP files are used, however, the use of the NIST logo is not allowed.

Resources

Contact

Robert Lipman, robert.lipman@nist.gov
Tom Hedberg, thomas.hedberg@nist.gov