Introduction

Methodology
Results
Conclusions

Towards Automated Quality Models for Software

Development Communities
The QualOSS and FLOSSMetrics case

Daniel Izquierdo Cortázar, Jesús M. González-Barahona,

Santiago Dueñas, Gregorio Robles

{dizquierdo, jgb, sduenas, grex}@libresoft.es

GSyC/Libresoft, Universidad Rey Juan Carlos

Quatic 2010, Porto, Portugal

29th, September, 2010

Daniel Izquierdo Cortázar, Jesús M. González-Barahona, Santiago Dueñas,

Towards Gregorio
Automated
Robles
Quality Models for Software Development Co
 Introduction
Methodology
Results
Conclusions

(cc) 2010 Daniel Izquierdo Cortázar.

Some rights reserved. This document is distributed under the Creative
Commons Attribution-ShareAlike 2.5 licence, available in
http://creativecommons.org/licenses/by-sa/2.5/

D. Izquierdo, J. G. Barahona, S. Dueñas, G. Robles Towards Automated Quality Models

 Introduction
Methodology
Results
Conclusions

Table of contents

1 Introduction

2 Methodology

3 Results

4 Conclusions

D. Izquierdo, J. G. Barahona, S. Dueñas, G. Robles Towards Automated Quality Models

 Introduction
Methodology
Results
Conclusions

Index

1 Introduction

2 Methodology

3 Results

4 Conclusions

D. Izquierdo, J. G. Barahona, S. Dueñas, G. Robles Towards Automated Quality Models

 Introduction
Methodology
Results
Conclusions

Motivation

Quality models and FLOSS communities

There exist several quality models focused on FLOSS
In most of the cases, there are some new additions to the
traditional point of view
Being the community of developers supporting the product of
special interest

D. Izquierdo, J. G. Barahona, S. Dueñas, G. Robles Towards Automated Quality Models

 Introduction
Methodology
Results
Conclusions

Motivation

General process of execution:

Most of them have followed a Goal-Question-Metric approach
Thus, there is a development of several high-level quality
attributes
And go down to reach the final set of metrics
That final estimation of the metrics have been manually
calculated (in most of the cases)
Or based on experts estimations

D. Izquierdo, J. G. Barahona, S. Dueñas, G. Robles Towards Automated Quality Models

 Introduction
Methodology
Results
Conclusions

Motivation

Nowadays FLOSS offer a huge set of data mainly based on

next data sources:
Source Code Management System: CVS, Subversion, Git,
Mercurial, etc.
Mailing Lists or Forums
Bug Tracking Systems: Bugzilla, GNATs, Trac, Redmine, etc.
Source code: Releases
Others.

D. Izquierdo, J. G. Barahona, S. Dueñas, G. Robles Towards Automated Quality Models

 Introduction
Methodology
Results
Conclusions

Motivation

Even more: there exist meta-repositories of information

OSSMole, FLOSSMetrics, SRDA, Ohloh
Those meta-repositories store information from development
activities and from different perspectives

D. Izquierdo, J. G. Barahona, S. Dueñas, G. Robles Towards Automated Quality Models

 Introduction
Methodology
Results
Conclusions

Motivation

Thus, what if we start to use meta-repositories as a centralized

way of retrieving information for those quality models?

D. Izquierdo, J. G. Barahona, S. Dueñas, G. Robles Towards Automated Quality Models

 Introduction
Methodology
Results
Conclusions

Motivation

General problems found from people willing to use information

from software repositories:
Different systems with different APIs
Different representations of the same kind of information
Performance when analyzing a huge set of data
Lost data (e.g. when migrations take place)
Damage to FLOSS projects infrastructure

D. Izquierdo, J. G. Barahona, S. Dueñas, G. Robles Towards Automated Quality Models

 Introduction
Methodology
Results
Conclusions

Motivation

This paper discusses how this automation is done for the

QualOSS (quality model) project and using information from
the FLOSSMetrics (meta repository of information) database
And how this could be extended to other quality models and
meta-repositories

D. Izquierdo, J. G. Barahona, S. Dueñas, G. Robles Towards Automated Quality Models

 Introduction
Methodology
Results
Conclusions

Index

1 Introduction

2 Methodology

3 Results

4 Conclusions

D. Izquierdo, J. G. Barahona, S. Dueñas, G. Robles Towards Automated Quality Models

 Introduction
Methodology
Results
Conclusions

QualOSS Quality Model

D. Izquierdo, J. G. Barahona, S. Dueñas, G. Robles Towards Automated Quality Models

 Introduction
Methodology
Results
Conclusions

QualOSS Quality Model

Size and Regeneration Adequacy: The degree to which the

size evolution and regeneration of a FLOSS community
happens at an adequate rate to maintain a sustainable
community size.
Interactivity and Workload Adequacy: The degree to which
the community interacts adequately and partition the
workload among FLOSS community members adequately to
maintain a community cohesion and motivated.

D. Izquierdo, J. G. Barahona, S. Dueñas, G. Robles Towards Automated Quality Models

 Introduction
Methodology
Results
Conclusions

Methodology

To measure risky situations, thresholds were defined: green,

yellow, red and black
Those thresholds were derived from a statistical analysis
Those thresholds show how a given project is located among
others.
Having a black color does not mean that this project is “bad”,
but it means that the results were in the set of projects that
are most far from the average

D. Izquierdo, J. G. Barahona, S. Dueñas, G. Robles Towards Automated Quality Models

 Introduction
Methodology
Results
Conclusions

Metrics

sra2- New code contributors evolution.

sra3- New non-code contributors evolution.
sra4- New core contributors evolution.
sra5- Evolution of core members who stopped contributing.
sra6- Balance between new core contributors and those who left the
project.

D. Izquierdo, J. G. Barahona, S. Dueñas, G. Robles Towards Automated Quality Models

 Introduction
Methodology
Results
Conclusions

Index

1 Introduction

2 Methodology

3 Results

4 Conclusions

D. Izquierdo, J. G. Barahona, S. Dueñas, G. Robles Towards Automated Quality Models

 Introduction
Methodology
Results
Conclusions

QualOSS method

22 out of 29 metrics were fully automated

Limitations
Environment highly dependable of the data sources
If the data sources are not supported by the FLOSSMetrics
database or the tools, then the metrics have to be manually
retrieved.
From them:
SCM: 15
Mailing lists: 4
BTS: 3

D. Izquierdo, J. G. Barahona, S. Dueñas, G. Robles Towards Automated Quality Models

 Introduction
Methodology
Results
Conclusions

QualOSS method

Extending this analysis to other communities, we can find, in

the FLOSSMetrics database, hundreds of projects analyzed
using the QualOSS approach (the community side, so far).
Thus, why not an extension to other quality models?

D. Izquierdo, J. G. Barahona, S. Dueñas, G. Robles Towards Automated Quality Models

 Introduction
Methodology
Results
Conclusions

OpenBRR, QSoS, SQO-OSS and QualOSS

Quality Method Num. metrics Num. automat. metrics

QualOSS 29 22
SQO-OSS 9 7
OpenBRR 2 2
QSoS 6 2
Table: Quality methods and their automatized community metrics

D. Izquierdo, J. G. Barahona, S. Dueñas, G. Robles Towards Automated Quality Models

 Introduction
Methodology
Results
Conclusions

Index

1 Introduction

2 Methodology

3 Results

4 Conclusions

D. Izquierdo, J. G. Barahona, S. Dueñas, G. Robles Towards Automated Quality Models

 Introduction
Methodology
Results
Conclusions

Conclusions

Other quality models different from QualOSS could be

partially automated
Other quality attributes from the QualOSS model could be
partially automated
Some strong limitations found in the automation of the
quality models:
Not found data sources
Too complex metrics (e.g. there are not tools)
It is necessary to have a more objective way of defining metrics
and their ranges

D. Izquierdo, J. G. Barahona, S. Dueñas, G. Robles Towards Automated Quality Models

 Introduction
Methodology
Results
Conclusions

Questions

How friendly are quality models for an automation of their

metrics?
Is so important the human factor in the evolution of the
systems? if so, what can we do to improve their analysis?

D. Izquierdo, J. G. Barahona, S. Dueñas, G. Robles Towards Automated Quality Models

 Introduction
Methodology
Results
Conclusions

Questions?

Thanks for your attendance!

Questions?
–
Daniel Izquierdo Cortázar
dizquierdo@gsyc.es
GSyC/LibreSoft - Universidad Rey Juan Carlos

D. Izquierdo, J. G. Barahona, S. Dueñas, G. Robles Towards Automated Quality Models