How to effectively use topic models for software engineering tasks? An approach based on Genetic Algorithms

Annibale Panichella, Bogdan Dit, Rocco Oliveto, Massimilano Di Penta, Denys Poshynanyk, Andrea De Lucia

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

261 Scopus citations

Abstract

Information Retrieval (IR) methods, and in particular topic models, have recently been used to support essential software engineering (SE) tasks, by enabling software textual retrieval and analysis. In all these approaches, topic models have been used on software artifacts in a similar manner as they were used on natural language documents (e.g., using the same settings and parameters) because the underlying assumption was that source code and natural language documents are similar. However, applying topic models on software data using the same settings as for natural language text did not always produce the expected results. Recent research investigated this assumption and showed that source code is much more repetitive and predictable as compared to the natural language text. Our paper builds on this new fundamental finding and proposes a novel solution to adapt, configure and effectively use a topic modeling technique, namely Latent Dirichlet Allocation (LDA), to achieve better (acceptable) performance across various SE tasks. Our paper introduces a novel solution called LDA-GA, which uses Genetic Algorithms (GA) to determine a near-optimal configuration for LDA in the context of three different SE tasks: (1) traceability link recovery, (2) feature location, and (3) software artifact labeling. The results of our empirical studies demonstrate that LDA-GA is able to identify robust LDA configurations, which lead to a higher accuracy on all the datasets for these SE tasks as compared to previously published results, heuristics, and the results of a combinatorial search.

Original languageEnglish
Title of host publication2013 35th International Conference on Software Engineering, ICSE 2013 - Proceedings
Pages522-531
Number of pages10
DOIs
StatePublished - 2013
Event2013 35th International Conference on Software Engineering, ICSE 2013 - San Francisco, CA, United States
Duration: 18 May 201326 May 2013

Publication series

NameProceedings - International Conference on Software Engineering
ISSN (Print)0270-5257

Conference

Conference2013 35th International Conference on Software Engineering, ICSE 2013
Country/TerritoryUnited States
CitySan Francisco, CA
Period18/05/1326/05/13

Keywords

  • Genetic Algoritms
  • Latent Dirichlet Allocation
  • Textual Analysis in Software Engineering

Fingerprint

Dive into the research topics of 'How to effectively use topic models for software engineering tasks? An approach based on Genetic Algorithms'. Together they form a unique fingerprint.

Cite this