Motivation for the seminar

Early design time prediction of software architectures to determine extra-functional quality attributes like performance or reliability is subject of intense research. The motivation for early quality predictions is based on the assumption that corrective measures are easy to realise during the design phase with low costs. Furthermore, such predictions allow an engineering approach to software development: software architects create models of the system under development, analyse their extra-functional properties, redesign them if necessary, and only if the requirements are met, the system is implemented. Since models assume a central role in this process, it is usually called model-based software design in literature. Model-based software design also helps if there are multiple design alternatives which fulfil the requirements. In this case, systematic trade-off analyses become available in which software architects can, for example, maximize the utility-to-cost ratio in collaboration with the software's stake-holders.

Recent research enhances the former model-based methods (as surveyed for performance by Balsamo et al.) into model-driven quality prediction techniques. Model-driven software analysis and prediction tries to automate the process of analysing the information available in the model, thus replacing the manual analysis dominant in model-based analysis. To put model-driven software development into practice, the software architect first creates a (semi-)formal model of the system's architecture, including aspects of the system's static structure, dynamic behaviour, and deployment on hardware nodes. Afterwards, the software architect refines the model with quality annotations relevant to the quality attributes of interest. For example, for performance evaluation, the architect annotates steps in an activity diagram with the amount of CPU, hard disk, or network resources needed to perform individual actions. The following steps then execute automatically.

• First, the created model (called the software design model) is transformed into a prediction model specific to the quality attribute under evaluation. For example, for performance the transformation can generate queueing network or layered queuing network models, stochastic timed Petri nets, or stochastic process algebras (SPAs).
• Next, the resulting prediction model is solved by standard solvers relying on analytical or simulation-based methods. The solution contains values for metrics relevant to the respective analysis method. These have to be transformed back into annotations on the original software design model.
• Finally, the software architect takes these values as foundation for design decisions.

While the whole process has demonstrated its applicability in several case studies and while the use of model-driven techniques made them much more efficient in contrast to model-based approaches, the process of creating a model still remains a time-consuming task. This is especially true where the annotated software design model is concerned - even if we assume that the design model itself has to be created in the common software development process anyway (as is the case for example in the Rational Unified Process), this model needs to be annotated with reliable estimates of the quality annotations. Examples of these annotations include resource demands, loop iteration counts, branching probabilities, or failure rates. Estimating values of even the simple quality annotations is a difficult task. The estimates become still more complex when the annotations support parametrisation as introduced by Koziolek. In these cases, the model explicitly address the fact that many annotations also depend on the data processed by the system (e.g. processing 1kB of data takes less time than processing 1MB). However, as the data is usually supplied by the system's users, it is unknown at the time when the specifiation of single components or subsystems is created, which would be desirable in order to build reusable models. Typically, estimates are also too difficulat to be made just by guessing using a rule of thumb strategy. In such cases, annotations are based on measurements taken on existing components of the system, such as pre-existing application components, middleware platforms, database systems, or operating system services. Taking these measurements usually involves deploying existing components in test environments and measuring them in prototypes, so that the software architect can take the collected measurements and convert them into the needed annotations for the software design model. Most parts of this process today need manual interaction.

As explained in the previous paragraph, measurements of existing components are hindered by the dependency of the measurements on the characteristic of the test environments. Obviously, a component that is measured in an isolated test environment performs differently from a component that is integrated in an application, where it can share resources with other components. Component benchmarking therefore aims to include the dependency of the annotations on the shared resources into a parametric expression of the measurement results. Coupled with a model of resource sharing, which can be built from the models of the application architecture and the application deployment, such annotations compensate for the dependency of the measurement results on the test environment. The complexity of creating the highly parameterised models obviously hinders a wide-spread application of model-driven prediction methods. Today, a lot of expertise and experience is required to design and conduct experiments. To lower the effort needed, we therefore also need model-driven methods that collect and interpret data and transform it into annotations to be added to the software design model.

Although several ideas exist in this area, there is still no approach that would be integrated into a model-driven tool chain. There are some approaches which tackle the issue of collecting performance model input from generated prototypes. For example, Woodside and Schramm use prototypes generated from Layered Queuing Networks (LQNs) to do so. Similar approaches based on different source models were presented by Zhou, Gorton, and Liu, Grundy, Cai et al., or Becker et al.. The question how to automatically interpret measurements to come to performance model inputs was recently tackled by Woodside using statical models or by Krogmann et al. using genetic algorithms. However, none of the mentioned approaches integrates explicitly in a model-driven performance prediction process. From the summarised state-of-the art, it can be concluded that there are two mostly disjunct communities working on predictable software development. One group is researching model-driven software analysis methods for quality attributes like performance and reliability. The other group focuses on benchmarking and measuring existing software systems or prototype implementations. In cases where the model creation should not be based on manual estimates, model-driven software analysis approaches try to use results from the measurement community. To foster research in both areas but especially in the intersection of these research topics, the proposed Dagstuhl seminar is expected to come up with a state-of-theart survey of both fields and their intersection. This survey should provide a solid foundation for further research in the area and tries to close the gap by identifying the missing elements.

Contact

The seminar is organized by

Steffen Becker (Forschungszentrum Informatik, Germany, steffen dot becker at fzi dot de)
Raffaela Mirandola (Politecnico di Milano, Italy, mirandola at elet dot polimi dot it)
Petr Tuma (Charles University, Czech Republic, petr dot tuma at dsrg dot mff dot cuni dot cz)

Please contact us in case of questions or comments.

Acknowledgments

The seminar is kindly supported by the German Computer Science Foundation (GI e.V.) and the FP7 EU Project Q-ImPrESS.