Before shipping, we make each release available to vendors of leading forensic tools for testing against their products. For completeness, we also test the data in house against some of the major forensic tools. We try to resolve some questions, but in our experience the vendor support staff will likely give you a better response.
Each release of the NSRL hash set is made available for download free of charge one month after the subscriber CD release. The download is available on this page. You should also verify that the hash set is correct by using the digital signature associated with the downloaded file.
As of the March 2018 RDS release (2.60) the NSRL is no longer accepting new subscriptions for physical DVD releases of our dataset. Our most recent release is available from the NSRL Download page, under Current RDS Hash Set.
Please direct any questions to the NSRL team at nsrl [at] nist.gov.
The format of the data is described in our paper Data Formats of the NSRL Reference Data Set (RDS) Distribution (PDF).
We purchase most of the software in the NSRL. We try to get everything on major retailers top selling lists. Some software we hear about by word of mouth, some by schedule (like tax programs each tax year, security, antivirus) and some by requests from law enforcement and other agencies. We accept donations from manufacturers and have paperwork to state we will not use the software license . All donations of new software should be COTS shrink-wrapped and exactly what a consumer would purchase. We accept donations of used software as long as it is in useable condition but there is no guarantee that it will make it into the NSRL RDS. We do keep a limited number of duplicate software for media degradation testing and in order to keep a back up of the most popular software, such as operating system packages.
To donate software to the NSRL, please mail packages to:
Department of Commerce - NIST
ATTN: NSRL Project
100 Bureau Drive, Stop 8970
Gaithersburg, MD 20899-8970
We apologize, but we cannot lend out copies of the software in our collection.
We make the hashes (MD5, SHA, etc.) available to everyone, but the software itself is (a) stored in an evidence locker, (b) is often donated by vendors with a non-use agreement, or (c) can't be redistributed due to copyright.
However, our experience suggests you might want to try hitting tech swap meets, used bookstores, bargain bins in non-chain stores - they've been a gold mine for us.
We will do what we can within the bounds of the licensing of contents of our collection.
Please contact our subscription department.
No. The NSRL is prevented by law from handling such files, and NSRL policy prevents us from including the hash of a file in the NSRL RDS unless we possess the original copy of that file.
The NDIC HashKeeper project is one source of illicit data hashes (see below).
NIST also has a catalog of digital forensics databases which you may find useful.
The NSRL RDS and the NDIC's Hashkeeper are collections of File Identification Information (FII) which are typically used to identify computer files during forensic investigations of computer systems. The principal differences between the two collections are as follows:
Yes, we will be collecting SHA-256, Whirlpool, and several other pieces of metadata that we don't gather now.
The additional metadata will be included in a separate product - the RDS will continue in its present format.
The members of our steering committee (federal, state and local law enforcement) consider the files in the NSRL database as "known" - NOT "known good" OR "known bad" - just "known application files."
NIST does not make a decision about "known bad' or "malicious" or "notable", because there are various case scenarios where that classification is not cut-and-dried.
Note, however, that the NSRL database does contain hashes of files from applications which are traditionally viewed as malicious (encryption, steganography, hacker tools).
You can partition the applications according to your specific needs using the "ApplicationType" field in the "NSRLProd.txt" file - if you consider steganography apps as bad, you can identify them as such using that data.
We have had reports from several investigators that a small number of files - on the order of 10 or 12 - will cause "alerts."
It is our opinion that someone unknown to us has designated all of the file hashes associated with some NSRL hacker applications as "notable" or "malicious" (probably inside a tool that imports the NSRL hash set). Unfortunately, a few of the files used by those hacker apps are very common files used by normally harmless software. If you have a small number of "alert" hits, it is very likely that those are false positives.
Yes, installed software results are included in the RDS. The file entries are marked with the SpecialCode "D".
You can look in a file called NSRLProd.txt and find a column called "ApplicationType". We have classified the programs, and you can look for the description of your interest - steganography, keylogger, office suite, etc.
No. However the NSRL has a research computing environment containing millions of unique original files, along with a database containing metadata about the files (filename, bytesize, etc).
The format for running an algorithm against the file collection is basically that you would submit a job - in the form of your code - to the NSRL. We would then run your job against the file collection, returning the results and your code to you upon completion.
There are various conditions of access to the research environment, including:
Please contact us for details.
Below is a high-level illustration of the NSRL:
A High-level Illustration of the NSRL