Benchmarking Technical Quality of Software Products
This paper describes SIG's approach to collecting measurement results into a central benchmark repository. Some open-source software products are used to exemplify how comparisons regarding technical quality can be performed.
by José Pedro Correia (SIG), Joost Visser (SIG)
Published in WCRE 2008, Proceedings of the 15th Working Conference on Reverse Engineering, October 15-18, 2008, Antwerp, Belgium.
Abstract: To enable systematic comparison of technical quality of (groups of) software products, we have collected measurement data of a wide range of systems into a benchmark repository. The measurements were taken over the course of several years of delivering software assessment services to corporations and public institutes. The granularity of the collected data follows the layered structure of a model for software product quality, based on the ISO/IEC 9126 international standard, which we developed previously. In this paper, we describe the design of our benchmark repository, and we explain how it can be used to perform comparisons of systems. To provide a concrete illustration of the concept without revealing confidential data, we use a selection of open source systems as example.