Advancing Competitiveness

Advancing Competitiveness Homepage

Credit: Pixabay

A Conceptual Framework for Economic Decision Making in Advancing Manufacturing Industry Competitiveness

Proposed in NIST AMS 100-80 is an enterprise-wide system of continuous improvement for any organization trying to increase impact in advancing U.S. manufacturing competitiveness. It is proposed that to maximize the impact of each dollar invested in research and development (R&D), change agents (i.e., those who invest in advancing manufacturing competitiveness such as governments or trade organizations) will likely need to transition from intuition-based selection to rigorous measurement science. By treating each R&D project as an experiment with a clear hypothesis (ex ante prediction) and measurable outcome (ex post estimate), organizations create a double feedback loop generating actionable information that can be used to systematically improve future investment accuracy. The potential benefits include:

• Near real-time aggregatable impact estimates
• Growing layered body of evidence for impact that supports drilling down to underlying data
• Interoperable impact accounting 
• Enabling systematic impact growth
• Compounding enterprise-wide learning (de-siloing) 
• Actionable impact information

The foundation of this framework—hypothesis testing and impact tracking—is essential to realizing a change agent’s full impact potential. Many change agents use well-established metrics such as publications, standards development, technology transfer outputs, stakeholder usage, and other indicators of activity and adoption. An issue arises in that these metrics are typically not integrated into a unified framework that systematically links intermediate outputs to downstream economic outcomes or benefit-cost impacts in a way that supports predictive decision-making and project configuration optimization. Without these characteristics, planning does not strategically map to impact. This framework uses a system of continuous improvement to generate an engine of growth in impact. It is a means for collaboratively applying scientific rigor across an organization to achieve growth in impact. The framework can be broken conceptually in to three parts, as described below. An abridged discussion of the components, logic, and implementation infrastructure are discussed in the pages linked near the bottom of this page. For a more complete discussion, please see NIST AMS 100-80.

Part One: Opportunity Mapping and Hypotheses Generation

The first part of the framework is to develop an opportunity map, which consists of data characterizing potential impacts. Cost data on the manufacturing industry can be thought of as a cube of cubes (see illustration below). All the little cubes together represent the total of all costs. Each little cube represents an individual cost with particular characteristics such as the industry, firm size, and other factors (not shown). As illustrated in the figure below, the data granularity can be increased over time providing increasing ability to accurately predict the costs that manufacturers face related to a particular project proposal. Additionally, the data is extensible where future data collections can be implemented into the data cube.

In addition to predicting cost, change agent organizations will want to aim for accurate predictions of percent reduction in cost resulting from R&D. Predicted percent cost reduction is estimated using bottom-up, mechanism-based models informed by reference classes, early pilot evidence, and uncertainty bounds. Estimates are expressed probabilistically and updated over time, with evaluation emphasizing calibration and learning rather than point accuracy. The reduction in cost comes from manufacturing process efficiencies, input substitution, capital utilization, labor productivity, and quality/reliability. Estimates of reduction might map these categories where each one is associated with a range of potential reductions.

Part Two: Validation and Recalibration

After a project or selection of products are completed, the resulting impact needs to be evaluated for some portion of them. The impact that is measured can be used to justify budgets, evaluate prediction accuracy, and evaluate the effectiveness of different aspects of a project/product (e.g., diffusion efforts). The last two equate to a double feedback loop. To evaluate prediction accuracy, the estimate of realized (ex post) impact is compared to the predicted (ex ante) impact. Each round of projects will result in revealing an estimated level of error associated with the approach for predicting impact. As time goes on, the error can be systematically reduced through learning and eventually it might be minimized.

Part Three: Enterprise-Wide Utilization

The last part of the framework is to build and utilize the opportunity map, the prediction and tracking of impact, and hypothesis testing organization wide. This can generally be seen as harnessing economies of scale and standardizing metrics. As more entities collaborate, there is more data being shared moving away from episodic learning that happens in silos to enterprise-wide learning that compounds. This results in a sharing of the costs for collecting information and sharing knowledge, resulting in more knowledge per dollar of expenditure. 

Framework Components, Logic, and Implementation

Credit: Pixabay The Framework	Credit: Pixabay Framework Logic	Credit: Pixabay Implementation Infrastructure
The Framework's Process	Background: Impact Forecasting	Notes and Cautions
Data Layers and Feed Back Loops	Supporting Logic and Evidence	Standards and Platforms
Metrics and Units of Observation	Economies of Scale	Modular Adoption

Collaboration is a key component to reducing change agent costs and enabling compound learning. If you are considering adopting this framework, consider reaching out to the project lead Douglas Thomas, Economist: douglas.thomas [at] nist.gov (douglas[dot]thomas[at]nist[dot]gov)