NCAL: Data Management

In the mechanical testing and materials science communities there is not a widely accepted solution for data management of test data.  As part of NIST efforts on data archiving, provenance, and availability, we have worked to improve our data practices.  We have extended our system beyond many out-of-the-box commercial systems or industrial data archiving methods, as our use case in test method development and unique test and characterization capabilities have additional requirements for data and metadata recording and retention. 

Some of the key features:

• Data management system is designed to track provenance in all steps of our work, from tracking the original material as delivered through reporting our results in publications.
• We track machine configuration as it evolves over time or is reconfigured for a new test campaign for purposes of archiving, provenance, troubleshooting, and restoration of a machine configuration.
• Data management system has a web based interface for searching and locating data in a single location.
• Over 10,000 individual data records, and nearly 20 Tb of data files.

While built upon a commercial data management system, we have developed a generic workflow that can be applied to a wide variety of laboratories.  We are open to sharing schemas, data dictionaries, and/or lessons learned from our work internally and externally.