Many enterprises have become driven by digital data, including manufacturing, engineering, and scientific domains. With the ubiquity of Web browsers and Web document formats across a range of platforms and devices, many communities are using the Web as an application environment. As such, standard interfaces must be defined for the technologies related to developing applications on the Web, including programming interfaces for client-side development, network requests, timed events and platform interaction. As these standards are promulgated throughout the industry, the demand for metrics to measure conformance to these standards has increased. Effective use of these metrics dictates that they be developed in a short amount of time, and be done so in a way that allows changes to the specifications to be easily propagated through a given set of tests.
Measurement is difficult due to the complexity of the underlying technologies being addressed. In addition, measuring conformance in web applications requires large test sets covering a wide a range of standards.
Currently, W3C WEB Applications working group members are responsible for developing the necessary sets of standards for web applications. Until recently, no testing effort had begun.
The long term goal of this project is to establish a metrology infrastructure that automatically generates conformance tests directly from a specification, thus improving relevant standards and interoperability among implementations of the respective standard.
It has become well-known that software is not adequately tested and is rushed to market full of bugs. A recent NIST study indicated that the annual cost to the country for inadequate testing is as high as 59 billion dollars. NIST has vast experience in technology evaluation applicable to data exchange and is uniquely positioned to provide the metrology, test and measurement infrastructure required to advance the state of the art. Conformance test development requires significant resources -- beyond the means of most organizations. Common data and metrics permit direct comparisons of approaches and NIST is viewed as the neutral objective arbiter of evaluation design and implementation.