Event Details
Improving Vulnerability Detection Measurement: Test suites and Software Security Assurance
Presenter: Alexander M. Hoole
Supervisor:
Date: Fri, September 29, 2017
Time: 12:30:00 - 14:00:00
Place: EOW 430
ABSTRACT
The Software Assurance Metrics and Tool Evaluation (SAMATE) project at the National Institute of Standards and Technology (NIST) has created the Software Assurance Reference Dataset (SARD) to provide researchers and software security assurance tool developers with a set of known security flaws. Locating, or creating, datasets is especially challenging for use in empirical evaluation of flaw detection and verification tools. During an empirical evaluation of our runtime monitoring framework, deficiencies were discovered in two existing test suites which led to a collaboration with NIST to provide replacements. Test Suites 45 and 46 are analyzed, discussed, and updated to improve accuracy, consistency, preciseness, and automation. Empirical results show metrics such as recall, precision, and F-Measure are all impacted by invalid base assumptions regarding the test suites.