Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

The Generation of Software Security Scoring Systems Leveraging Human Expert Opinion



Peter Mell


While the existence of many security elements can be measured (e.g., vulnerabilities, security controls, or privacy controls), it is challenging to measure their relative security impact. In the physical world we can often measure the impact of individual elements to a system. However, in cyber security we often lack ground truth (i.e., the ability to directly measure significance). In this work we propose to solve this by leveraging human expert opinion to provide ground truth. Experts are iteratively asked to compare pairs of security elements to determine their relative significance. In the back end our knowledge encoding tool performs a form of binary insertion sort on a set of security elements using each expert as an oracle for the element comparisons. The tool not only sorts the elements (note that equality maybe permitted) but it records the strength or degree of each relationship. The output is a directed acyclic 'constraint' graph that provides a total ordering among the sets of equivalent elements. Multiple constraint graphs are then unified together to form a single graph that is used to generate a scoring or prioritization system. For our empirical study, we apply this domain agnostic measurement approach to generate scoring/prioritization systems in the areas of vulnerability scoring, privacy control prioritization, and cyber security control evaluation.
Conference Dates
October 3-6, 2022
Conference Location
Virtual, MD, US
Conference Title
2022 IEEE 29th Annual Software Technology Conference (STC)


software, measurement, security, scoring, prioritization


Mell, P. (2022), The Generation of Software Security Scoring Systems Leveraging Human Expert Opinion, 2022 IEEE 29th Annual Software Technology Conference (STC), Virtual, MD, US, [online],, (Accessed April 14, 2024)
Created November 18, 2022, Updated October 30, 2023