June 2021 – NIST NSRL RDS 2.73 Publication Error

Summary

The NIST NSRL June 2021 Reference Data Set (RDS) version 2.73 contained several errors, stemming from one cause. The errors were not related to the cryptographic hash values. The errors cause non-attribution of file cryptographic hashes to source product titles, if the RDS data is used in such a fashion.

Scope

The errors are limited to one file, NSRLFile.txt but inter-file dependecies between the NSRLFile.txt file and the NSRLProd.txt file are affected.

RDS 2.73 NSRLFile.txt contains 6,319,360 new file descriptions, all of which are mis-attributed to source products. The hashes, filename, filesize and specialcode column values are correct.

RDS 2.73 NSRLProd.txt contains 6,295 new product descriptions. All column values are correct.

Source of Error

During NSRL publication production, a previous version of internal code was mistakenly used to build the NSRLFile.txt file. The quality control checks for publication which check for inter-file dependencies did not issue an alert for the lack of dependencies. There were no inter-version changes, so no QC alerts were issued for historic inconsistancies.

The build process has been changed to ensure the correct code shall be used in future, and the QC checks have been augmented to cover this case.

Remedy

NIST NSRL published RDS version 2.73.1 (July 2021) to be used in place of version 2.73.

NSRL staff are available to answer further questions and assist with remedy; please contact nsrl [at] nist.gov (nsrl[at]nist[dot]gov) or find contact information at https://www.nsrl.nist.gov

Examples of Erroneous Data

In the example data shown below, the NSRLProd.txt file contains the “ProductCode” column, shown in green. This file contains correct data.

The NSRLFile.txt file contains the “ProductCode” column, shown in red. Version 2.73 contains  6,319,360 new rows with erroneous values in the “ProductCode” column. Those codes do not match any code in the NSRLProd.txt file resulting in no attribution of a file to a source.

NSRLProd.txt

"ProductCode","ProductName","ProductVersion","OpSystemCode","MfgCode","Language","ApplicationType"
229120,"Sniper Ghost Warrior Contracts","6065035","189","89867","multilanguage","Game"
229121,"Grow Defense","6108399","189","89868","english","Game"
229126,"Ruff Night At The Gallery","5913284","189","89873","english","Game"
229127,"War Thunder","6001334","189","80762","multilanguage","Game"
229128,"Among Us","5946232","189","89874","multilanguage","Game"
229129,"Project Wingman","5965936","189","89875","multilanguage","Game"

NSRLFile.txt

"SHA-1","MD5","CRC32","FileName","FileSize","ProductCode","OpSystemCode","SpecialCode"

"00000052A9EEEC6C8348CFB2AEA77BC1FBF8D239","F46CA74CA3D89E9D3CF8D8E5CD77842D","2F9CC135","__DATA__mod_init_func",772,448056,"362",""

"0000183C8BA2EA8AA4452E82AF3B4F88D72F40E0","3C3E90D20FBF770135E486D494E05014","0B071930","HEADSET3421.PNG",2493,445011,"362",""

"00001CE03DE7DF8C24667EAFE30C2904F3817326","01C915E5EA2BC216D8FD3C4AD8DF30A8","C8F52A88",".modinfo",231,452137,"362",""

"000026E815D30E5DD7464881BB04A4C875F2FA43","69CAEB49A48F38E62340F791A23E76A3","2AE1510C","11.dbghelp.dll",58,452142,"362",""

"0000286E6E923FF47368E84F44B3432E254029DD","12AD01F526B3011DABC23A8DEA6B5BD0","3336B6B9",".rel.debug_frame",624,452137,"362",""

"00004932CF0D10A0D4A5BF0BFFC18FD87F6A9A72","B46D8107FC0C21CD6E29D32ADAF7476F","47334FB8","macros.conf.example",1678,446088,"362",""

"00004932CF0D10A0D4A5BF0BFFC18FD87F6A9A72","B46D8107FC0C21CD6E29D32ADAF7476F","47334FB8","macros.conf.example",1678,446089,"362",""

The NSRL tracks both “products” and “objects” in a database. A “product” may be comprised of several disks or downloadable objects, such as a main installation object, a fonts library object, video drivers object, etc. In this case, the code mistakenly used to create the publication files used the internal “objectcode” values to fill in the “productcode” column for the newly added 6,319,360 NSRLFile.txt rows.

Return to the NSRL Home