skip to main content
10.1145/3183440.3195001acmconferencesArticle/Chapter ViewAbstractPublication PagesicseConference Proceedingsconference-collections
poster

There is no random sampling in software engineering research

Published:27 May 2018Publication History

ABSTRACT

Representative sampling is considered crucial for predominately quantitative, positivist research. Researchers typically argue that a sample is representative when items are selected randomly from a population. However, random sampling is rare in empirical software engineering research because there are no credible sampling frames (population lists) for the units of analysis software engineering researchers study (e.g. software projects, code libraries, developers, projects). This means that most software engineering research does not support statistical generalization, but rejecting any particular study for lack of random sampling is capricious.

References

  1. Jorge Echeverría, Francisca Pérez, Andrés Abellanas, Jose Ignacio Panach, Carlos Cetina, and Óscar Pastor. 2016. Evaluating Bug-Fixing in Software Product Lines: an Industrial Case Study. In Proceedings of the 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement. ACM, 24. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Barbara Kitchenham and Stuart Charters. 2007. Guidelines for performing systematic literature reviews in software engineering. Technical report (2007).Google ScholarGoogle Scholar
  3. Kai Petersen, Robert Feldt, Shahid Mujtaba, and Michael Mattsson. 2008. Systematic Mapping Studies in Software Engineering.. In Proceedings of the Evaluation and Assessment in Software Engineering Conference (EASE) (Bari, Italy). ACM, 68--77. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Ewan Tempero, Craig Anslow, Jens Dietrich, Ted Han, Jing Li, Markus Lumpe, Hayden Melton, and James Noble. 2010. The Qualitas Corpus: A Curated Collection of Java Code for Empirical Studies. In Proceedings of the 17th Asia Pacific Software Engineering Conference. IEEE, Sydney, Australia, 336--345. D0I Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. There is no random sampling in software engineering research

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in
        • Published in

          cover image ACM Conferences
          ICSE '18: Proceedings of the 40th International Conference on Software Engineering: Companion Proceeedings
          May 2018
          231 pages
          ISBN:9781450356633
          DOI:10.1145/3183440
          • Conference Chair:
          • Michel Chaudron,
          • General Chair:
          • Ivica Crnkovic,
          • Program Chairs:
          • Marsha Chechik,
          • Mark Harman

          Copyright © 2018 Owner/Author

          Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 27 May 2018

          Check for updates

          Qualifiers

          • poster

          Acceptance Rates

          Overall Acceptance Rate276of1,856submissions,15%

          Upcoming Conference

          ICSE 2025

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader