Linguistic and AI Support for Collective Standards Development

Summary

Increasing digitalization brings new opportunities and challenges to standards development. This project supports the future of standards development through research and development of new tools and methods for reaching consensus, navigating, building understanding, and performing computations in dynamic, interdisciplinary contexts. These tools and methods leverage advances in AI, natural language processing, linguistic theory, and applied mathematics to provide metrics of language use and understanding, useful formalizations, and increased interoperability between standards. This can improve the speed and validity of standards development, increase confidence in voluntary consensus standards, and produce knowledge and methods which can be transferred to other areas in the future.

Description

Credit: Neo4j

Standards are increasingly part of complex digital ecosystems which may span dozens of documents, several different domains of science and engineering, and hundreds of contributors from different countries and backgrounds. It can be challenging to ensure that standards are consistent, meet requirements, and communicate effectively, especially when standards cross domains. Checking conformance to standards is also an increasingly significant challenge given the dynamic nature of emerging technologies. This project aims to meet some of these challenges by providing tools to support the development of smart standards – standards which incorporate formal or logical specifications alongside or in place of natural language. There are two parts to this: representing standards and developing standards.

Representing standards with formal specifications allows them to be used in rigorous, predictable computations. This means that the content of standards can be verified for certain key properties, that standards from different sources can interact seamlessly, and that information from standards can be dynamically retrieved to meet the needs of users. This project studies the automatic construction of semi-formal standards using natural language processing tools. Though not fully specified, this semi-formal content can improve consistency in language use, identify conflicts in terminology, and bootstrap smart standards development.

Formalization in standards development allows for the integration of techniques from computer science and mathematics into the process. This project imagines an “integrated development environment” for standards development, which enables advanced information retrieval, hyperlinks, and dynamic analysis of content to highlight errors or missing values. These techniques can also improve version control, taking inspiration from version control systems in software.

Though this project is focused on standards development, the tools and techniques in development can be applied to many other use cases, including guidelines, scientific publications, patents, and many other collaborative efforts. In each of these areas, tools for formalization can improve consensus building, help trace information flow, identify conflicts, manage information, and measure linguistic and structural properties of complex systems. This reduces the time and effort needed to write, develop, and manage documents. It reduces ambiguity and enhances collective understanding, allowing for the management of variation, collective memory, and bias. This results in increased confidence in scientific research, standards, and policy.

Major Accomplishments

• Developed QuerCUS, a prototype application for automatically constructing linguistic databases for standards.
• Analyzed various standards to improve terminological consistency and identify circular definitions.
• Engaged with standards developers and standards development organizations to build a repository of key needs and challenges in standards development.