This book constitutes the thoroughly refereed post-proceedings of the International Dagstuhl-Seminar on Empirical Software Engineering, held in Dagstuhl Castle, Germany in June 2006. The purpose of this workshop was to identify the progress of empirical software engineering since 1992, to summarize the state-of-the-art in ESE, to summarize the state-of-the-practice in ESE in industry, and to develop an ESE roadmap for research, practice, education and training. The 54 revised full papers in this state-of-the-art survey were carefully reviewed and selected from the presentations during the workshop, that provided a forum for a small but representative group of leading experts in software engineering with an emphasis on empirical studies from both universities and industry to meet and reflect on past successes and failures, assess the current state of the practice and research, identify challenges, and define future directions. The papers are organized in topical sections on the empirical paradigm, measurement and model building, technology transfer and education, as well as roadmapping. Victor R. Basili, Dieter Rombach, and Kurt Schneider Introduction In 1992, a Dagstuhl seminar was held on ¿Experimental Software Engineering Issues¿ (seminar no. 9238). Its goal was to discuss the state of the art of empirical software engineering (ESE) by assessing past accomplishments, raising open questions, and proposing a future research agenda. Since 1992, the topic of ESE has been adopted more widely by academia as an interesting and promising research topic, and in industrial practice as a necessary infrastructure technology for goal-oriented, sustained process improvement. At the same time, the spectrum of methods applied in ESE has broadened. For example, in 1992, the empirical methods applied in software engineering were basically restricted to quantitative studies (mostly controlled experiments), whereas since then, a range of qualitative methods have been introduced, from observational to ethnographical studies. Thus, the field can be said to have moved from experimental to empirical software engineering. We believe that it is now time to again bring together practitioners and researchers to identify both the progress made since 1992 and the most important challenges for the next five to ten years. Session 1 The Empirical Paradigm The Empirical Paradigm Introduction 1 Dieter Rombach Approaches for Empirical Validation Techniques for Empirical Validation 4 Marvin V. Zelkowitz Status of Empirical Research in Software Engineering 10 Andreas H fer and Walter F. Tichy Position Papers Aggregation of Empirical Evidence 20 Marcus Ciolkowski Empirical Evaluation in Software Engineering: Role, Strategy, and Limitations 21 Lionel C. Briand New Opportunities for Empirical Research 22 Markku Oivo Empirical Paradigm: Position Paper 23 Carolyn B. Seaman The Value of Empirical Evidence for Practitioners and Researchers 24 Austen Rainer Exploration Versus Confirmation Empirical Paradigm The Role of Experiments 25 Barbara Kitchenham The Role of Controlled Experiments in Software Engineering Research 33 Victor R. Basili Position Papers Creating Real Value in Software Engineering Experiments 38 James Miller From Silver Bullets to Philosophers' Stones: Who Wants to Be Just an Empiricist? 39 Guilherme H. Travassos Social and Human Aspects of Software Engineering 40 Helen Sharp Longitudinal Studies in Evidence-Based Software Engineering 41 Tracy Hall The Use of Grounded Theory in Empirical Software Engineering 42 Jeffrey Carver Historical Review Exploration and Confirmation: An Historical Perspective 43 Michael S. Mahoney Working Group Results Combining Study Designs and Techniques Working Group Results 50 Carolyn B. Seaman Optimizing Return-On-Investment (ROI) for Empirical Software Engineering Studies Working Group Results 54 Lutz Prechelt The Role of Controlled Experiments Working Group Results 58 Andreas Jedlitschka and Lionel C. Briand Discussion and Summary The Empirical Paradigm Discussion and Summary 63 Marcus Ciolkowski, Barbara Kitchenhani, and Dieter Rombach Session 2 Measurement and Model Building Measurement and Model Building Introduction 68 Victor R. Basili Data Sharing Data Collection, Analysis, and Sharing Strategies for Enabling Software Measurement and Model Building 70 Richard W. Selby Knowledge Acquisition in Software Engineering Requires Sharing of Data and Artifacts 77 Dag I.K. Sj berg Effective Data Interpretation Effective Data Interpretation 83 J rgen M nch Software Support Tools and Experimental Work 91 Audris Mockus Position Papers Measurement and Interpretation of Productivity and Functional Correctness 100 Hakan Erdogmus Synthesising Research Results 101 Barbara Kitchenham On the Quality of Data 102 Thomas Ostrand Working Group Results Potential of Open Source Systems as Project Repositories for Empirical Studies Working Group Results 103 Nachiappan Nagappan Data Sharing Enabling Technologies Working Group Results 108 Marvin V. Zelkowitz Documenting Theories Working Group Results 111 Dag I.K. Sj berg Discussion and Summary Measurement and Model Building Discussion and Summary 115 Sira Vegas and Vic Basili Session 3 Technology Transfer and Education Technology Transfer arid Education Introduction 121 Kurt Schneider Technology Transfer Empirical Studies as a Basis for Technology Transfer 125 Elaine J. Weyuker Position Papers Relationships and Responsibilities of Software Experimentation 128 Giovanni Cantone The (Practical) Importance of SE Experiments 129 Tore Dyb TD> How to Improve the Use of Controlled Experiments as a Means for Early Technology Transfer 130 Andreas Jedlitschka Extending Empirical Studies to Cover More Realistic Industrial Development and Project Management Issues 131 Marek Leszak Empirical Case Studies in Industry: Some Thoughts 132 Nachiappan Nagappan Software Process Simulation Frameworks in Support of Packaging and Transferring Empirical Evidence 133 Dietmar Pfahl Structuring Families of Industrial Case Studies 134 Laurie Williams Education Empirical Software Engineering: Teaching Methods and Conducting Studies 135 Claes Wohlin Educational Objectives for Empirical Methods 143 Natalia Juristo Position Papers On "Landscaping" and Influence of Empirical Studies 151 Frank Houdek Involving Industry Professionals in Empirical Studies with Students 152 Letizia Jaccheri and Sandro Morasca Working Group Results Industry-Research Collaboration Working Group Results 153 Lutz Prechelt and Laurie Williams Teaching Empirical Methods to Undergraduate Students Working Group Results 158 Austen Rainer, Marcus Ciolkowski, Dietmar Pfahl, Barbara Kitchenham, Sandro Morasca, Matthias M. M ller, Guilherme H. Travassos, and Sira Vegas Discussion and Summary Technology Transfer and Education Discussion and Summary 163 Andreas Jedlitschka, Dietmar Pfahl, and Kurt Schneider Roadmapping Empirical Software Engineering Research Roadmap Introduction 168 Richard W. Selby Working Group Results Roadmapping Working Group 1 Results 172 Ross Jeffery Roadmapping Working Group 2 Results 175 Marcus Ciolkowski and Lionel Briand Roadmapping Working Group 3 Results 178 Frank Houdek Roadmapping Working Group 4 Results 181 Laurie Williams, Hakan Erdogmus, and Rick Selby Discussion and Summary Empirical Software Engineering Research Roadmap Discussion and Summary 184 Richard W. Selby Appendix List of Participants 188 Author Index 193