Understanding the Low Inter-Rater Agreement on
Aggressiveness on the Linux Kernel Mailing List

— Supplementary Website

Thomas Bock, Niklas Schneider, Angelika Schmid, Sven Apel, and Janet Siegmund


Here we provide two tables containing the papers that we collected by means of our systematic literature review: Back to Main Page

List of Papers that We Considered Relevant for Our Study (Included Papers)

In this table, we present all the papers that we found via any of the pillars of our systematic literature review and that we consider relevant for our study on sentiment analysis of developers' communications in the software-engineering domain, that is, that fulfilled our inclusion criteria.
In the third-last column, we provide the category we assigned the paper to (either Tool Development + Evaluation, or Tool Application + Usage).
In the second-last column, we denote whether a paper contains a manual data labeling (i.e., the authors have performed a human annotation study to assign a sentiment or something similar to their data items).
If a paper presents a new tool, we provide the name of the developed tool in the last column.

# Venue Year Authors Title Found by
Obaidi & Klünder
Found by
Extension of Obaidi & Klünder
Found by
Lin et al.
Considered
by Us
Category (Assigned by Us) Manual Data
Labeling
Developed Tool
1
2018 Lin et al.

Sentiment Analysis for Software Engineering: How
Far Can We Go?

Sentiment Analysis for Software Engineering: How
Far Can We Go?
✔️ ✔️ ✔️ ✔️ Tool Development + Evaluation ✔️ Stanford CoreNLP SO
2
2017 Islam and Zibran

Leveraging Automated Sentiment Analysis in Software
Engineering

Leveraging Automated Sentiment Analysis in Software
Engineering
✔️ ✔️ ✔️ Tool Development + Evaluation SentiStrength-SE
3
2018 Novielli et al.

A Benchmark Study on Sentiment Analysis
for Software Engineering Research

A Benchmark Study on Sentiment Analysis
for Software Engineering Research
✔️ ✔️ ✔️ ✔️ Tool Evaluation
4
2017 Jongeling et al.

On Negative Results When Using Sentiment
Analysis Tools for Software Engineering Research

On Negative Results When Using Sentiment
Analysis Tools for Software Engineering Research
✔️ ✔️ ✔️ ✔️ Tool Evaluation
5
2015 Jongeling et al.

Choosing Your Weapons: On Sentiment Analysis
Tools for Software Engineering Research

Choosing Your Weapons: On Sentiment Analysis
Tools for Software Engineering Research
✔️ ✔️ ✔️ Tool Evaluation
6
2018 Islam and Zibran

SentiStrength-SE: Exploiting Domain Specificity for Improved
Sentiment Analysis in Software Engineering Text

SentiStrength-SE: Exploiting Domain Specificity for Improved
Sentiment Analysis in Software Engineering Text
✔️ ✔️ ✔️ ✔️ Tool Development + Evaluation
7
2020 Zhang et al.

Sentiment Analysis for Software Engineering: How
Far Can Pre-trained Transformer Models Go?

Sentiment Analysis for Software Engineering: How
Far Can Pre-trained Transformer Models Go?
✔️ ✔️ ✔️ ✔️ Tool Development + Evaluation Fine-tune pre-trained Transformer-based models (BERT, RoBERTa, XLNet) for SE
8
2018 Ding et al.

Entity-Level Sentiment Analysis of Issue Comments

Entity-Level Sentiment Analysis of Issue Comments
✔️ ✔️ ✔️ ✔️ Tool Development + Evaluation ✔️ SentiSW
9
2017 Souza and Silva

Sentiment Analysis of Travis CI Builds

Sentiment Analysis of Travis CI Builds
✔️ ✔️ ✔️ ✔️ Tool Application + Usage
10
2017 Ahmed et al.

SentiCR: A Customized Sentiment Analysis Tool
for Code Review Interactions

SentiCR: A Customized Sentiment Analysis Tool
for Code Review Interactions
✔️ ✔️ ✔️ ✔️ Tool Development + Evaluation ✔️ SentiCR
11
2015 Jurado and Rodriguez

Sentiment Analysis in Monitoring Software Development
Processes: An Exploratory Case Study on
GitHub's Project Issues

Sentiment Analysis in Monitoring Software Development
Processes: An Exploratory Case Study on
GitHub's Project Issues
✔️ ✔️ ✔️ ✔️ Tool Application + Usage
12
2016 Blaz and Becker

Sentiment Analysis in Tickets for IT
Support

Sentiment Analysis in Tickets for IT
Support
✔️ ✔️ Tool Development + Evaluation ✔️ 3 dictionary-based models
13
2019 Shen et al.

Evaluating the Performance of Machine Learning
Sentiment Analysis Algorithms in Software Engineering

Evaluating the Performance of Machine Learning
Sentiment Analysis Algorithms in Software Engineering
✔️ ✔️ ✔️ ✔️ Tool Evaluation
14
2018 Calefato et al.

Sentiment Polarity Detection for Software Development

Sentiment Polarity Detection for Software Development
✔️ ✔️ ✔️ ✔️ Tool Development + Evaluation ✔️ Senti4SD
15
2014 Guzman et al.

Sentiment Analysis of Commit Comments in
GitHub: An Empirical Study

Sentiment Analysis of Commit Comments in
GitHub: An Empirical Study
✔️ ✔️ ✔️ ✔️ Tool Application + Usage
16
2018 Islam and Zibran

Sentiment Analysis of Software Bug Related
Commit Messages

Sentiment Analysis of Software Bug Related
Commit Messages
✔️ ✔️ ✔️ Tool Application + Usage
17
2015 Novielli et al.

The Challenges of Sentiment Detection in
the Social Programmer Ecosystem

The Challenges of Sentiment Detection in
the Social Programmer Ecosystem
✔️ ✔️ ✔️ ✔️ Tool Evaluation
18
2018 Islam and Zibran

DEVA: Sensing Emotions in the Valence
Arousal Space in Software Engineering Text

DEVA: Sensing Emotions in the Valence
Arousal Space in Software Engineering Text
✔️ ✔️ ✔️ ✔️ Tool Development + Evaluation ✔️ DEVA
19
2016 Sinha et al.

Analyzing Developer Sentiment in Commit Logs

Analyzing Developer Sentiment in Commit Logs
✔️ ✔️ ✔️ ✔️ Tool Application + Usage
20
2017 Mäntylä et al.

Bootstrapping a Lexicon for Emotional Arousal
in Software Engineering

Bootstrapping a Lexicon for Emotional Arousal
in Software Engineering
✔️ ✔️ Tool Development + Evaluation ✔️
21
2018 Imtiaz et al.

Sentiment and Politeness Analysis Tools on
Developer Discussions Are Unreliable, but So
Are People

Sentiment and Politeness Analysis Tools on
Developer Discussions Are Unreliable, but So
Are People
✔️ ✔️ ✔️ ✔️ Tool Evaluation ✔️
22
2013 Guzman and Bruegge

Towards Emotional Awareness in Software Development
Teams

Towards Emotional Awareness in Software Development
Teams
✔️ ✔️ ✔️ ✔️ Tool Application + Usage
23
2020 Huq et al.

Is Developer Sentiment Related to Software
Bugs: An Exploratory Study on GitHub
Commits

Is Developer Sentiment Related to Software
Bugs: An Exploratory Study on GitHub
Commits
✔️ ✔️ ✔️ Tool Application + Usage
24
2014 Novielli et al.

Towards Discovering the Role of Emotions
in StackOverflow

Towards Discovering the Role of Emotions
in StackOverflow
✔️ Tool Application + Usage
25
2018 Novielli et al.

A Gold Standard for Emotion Annotation
in Stack Overflow

A Gold Standard for Emotion Annotation
in Stack Overflow
✔️ ✔️ Tool Application + Usage ✔️
26
2018 Efstathiou et al.

Word Embeddings for the Software Engineering
Domain

Word Embeddings for the Software Engineering
Domain
✔️ Tool Development + Evaluation word2vec
27
2017 Gachechiladze et al.

Anger and Its Direction in Collaborative
Software Development

Anger and Its Direction in Collaborative
Software Development
✔️ ✔️ Tool Development + Evaluation ✔️
28
2019 Biswas et al.

Exploring Word Embedding Techniques to Improve
Sentiment Analysis of Software Engineering Texts

Exploring Word Embedding Techniques to Improve
Sentiment Analysis of Software Engineering Texts
✔️ ✔️ ✔️ Tool Development + Evaluation RNN4SentiSE
29
2015 Calefato et al.

Mining Successful Answers in Stack Overflow

Mining Successful Answers in Stack Overflow
✔️ ✔️ ✔️ Tool Application + Usage
30
2015 Ortu et al.

Are Bullies more Productive? Empirical Study
of Affectiveness vs. Issue Fixing Time

Are Bullies more Productive? Empirical Study
of Affectiveness vs. Issue Fixing Time
✔️ ✔️ ✔️ ✔️ Tool Application + Usage
31
2016 Ortu et al.

The Emotional Side of Software Developers
in JIRA

The Emotional Side of Software Developers
in JIRA
✔️ ✔️ Tool Application + Usage ✔️
32
2021 Sun et al.

Exploiting the Unique Expression for Improved
Sentiment Analysis in Software Engineering Text

Exploiting the Unique Expression for Improved
Sentiment Analysis in Software Engineering Text
✔️ ✔️ Tool Development + Evaluation SESSION
33
2014 Tourani et al.

Monitoring Sentiment in Open Source Mailing
Lists – Exploratory Study on the
Apache Ecosystem

Monitoring Sentiment in Open Source Mailing
Lists – Exploratory Study on the
Apache Ecosystem
✔️ ✔️ ✔️ ✔️ Tool Application + Usage ✔️
34
2019 Ferreira et al.

A Longitudinal Study on the Maintainers’
Sentiment of a Large Scale Open
Source Ecosystem

A Longitudinal Study on the Maintainers’
Sentiment of a Large Scale Open
Source Ecosystem
✔️ ✔️ Tool Application + Usage
35
2020 Novielli et al.

Can We Use SE-specific Sentiment Analysis
Tools in a Cross-Platform Setting?

Can We Use SE-specific Sentiment Analysis
Tools in a Cross-Platform Setting?
✔️ ✔️ ✔️ Tool Evaluation ✔️
36
2022 Ferreira et al.

How Heated Is It? Understanding GitHub
Locked Issues

How Heated Is It? Understanding GitHub
Locked Issues
✔️ Tool Application + Usage (human only) ✔️
37
2014 Murgia et al.

Do Developers Feel Emotions? An Exploratory
Analysis of Emotions in Software Artifacts

Do Developers Feel Emotions? An Exploratory
Analysis of Emotions in Software Artifacts
✔️ ✔️ ✔️ Tool Application + Usage (human only) ✔️
38
2019 Cheruvelil and daSilva

Developers' Sentiment and Issue Reopening

Developers' Sentiment and Issue Reopening
✔️ ✔️ ✔️ ✔️ Tool Application + Usage
39
2019 Robbes and Janes

Leveraging Small Software Engineering Data Sets
with Pre-trained Neural Networks

Leveraging Small Software Engineering Data Sets
with Pre-trained Neural Networks
✔️ ✔️ Tool Evaluation
40
2019 Chen et al.

SEntiMoji: An Emoji-Powered Learning Approach for
Sentiment Analysis in Software Engineering

SEntiMoji: An Emoji-Powered Learning Approach for
Sentiment Analysis in Software Engineering
✔️ ✔️ ✔️ ✔️ Tool Development + Evaluation SEntiMoji
41
2021 Venigalla and Chimalakonda

StackEmo: Towards Enhancing User Experience by
Augmenting Stack Overflow with Emojis

StackEmo: Towards Enhancing User Experience by
Augmenting Stack Overflow with Emojis
✔️ ✔️ Tool Development + Evaluation StackEmo
42
2020 Claes and Mäntylä

20-MAD - 20 Years of Issues
and Commits of Mozilla and Apache
Development

20-MAD - 20 Years of Issues
and Commits of Mozilla and Apache
Development
✔️ Tool Application + Usage
43
2020 Kuutila et al.

Chat Activity Is a Better Predictor
than Chat Sentiment on Software Developers
Productivity

Chat Activity Is a Better Predictor
than Chat Sentiment on Software Developers
Productivity
✔️ Tool Application + Usage
44
2016 Marshall et al.

Outcomes of Emotional Content from Agile
Team Forum Posts

Outcomes of Emotional Content from Agile
Team Forum Posts
✔️ Tool Application + Usage (human only) ✔️
45
2022 Miller et al.

“Did You Miss My Comment or
What?” Understanding Toxicity in Open Source
Discussions

“Did You Miss My Comment or
What?” Understanding Toxicity in Open Source
Discussions
✔️ Tool Application + Usage ✔️
46
2020 Sarker et al.

A Benchmark Study of the Contemporary
Toxicity Detectors on Software Engineering Interactions

A Benchmark Study of the Contemporary
Toxicity Detectors on Software Engineering Interactions
✔️ Tool Evaluation ✔️
47
2021 Cohen

Contextualizing Toxicity in Open Source: A
Qualitative Study

Contextualizing Toxicity in Open Source: A
Qualitative Study
✔️ Tool Application + Usage ✔️
48
2021 Cheriyan et al.

Towards Offensive Language Detection and Reduction
in Four Software Engineering Communities

Towards Offensive Language Detection and Reduction
in Four Software Engineering Communities
✔️ ✔️ ✔️ Tool Evaluation ✔️
49
2022 Sayago-Heredia et al.

Exploring the Impact of Toxic Comments
in Code Quality

Exploring the Impact of Toxic Comments
in Code Quality
✔️ Tool Application + Usage
50
2023 Sarker et al.

Automated Identification of Toxic Code Reviews
Using ToxiCR

Automated Identification of Toxic Code Reviews
Using ToxiCR
✔️ Tool Development + Evaluation ToxiCR
51
2022 Hata et al.

GitHub Discussions: An Exploratory Study of
Early Adoption

GitHub Discussions: An Exploratory Study of
Early Adoption
✔️ ✔️ Tool Application + Usage ✔️
52
2020 Raman et al.

Stress and Burnout in Open Source:
Toward Finding, Understanding, and Mitigating Unhealthy
Interactions

Stress and Burnout in Open Source:
Toward Finding, Understanding, and Mitigating Unhealthy
Interactions
✔️ Tool Development + Evaluation ✔️ STRUDEL toxicity detector
53
2022 Qiu et al.

Detecting Interpersonal Conflict in Issues and
Code Review: Cross Pollinating Open- and
Closed-Source Approaches

Detecting Interpersonal Conflict in Issues and
Code Review: Cross Pollinating Open- and
Closed-Source Approaches
✔️ Tool Evaluation ✔️
54
2021 Ferreira et al.

The “Shut the f**k up” Phenomenon:
Characterizing Incivility in Open Source Code
Review Discussions

The “Shut the f**k up” Phenomenon:
Characterizing Incivility in Open Source Code
Review Discussions
✔️ ✔️ Tool Evaluation ✔️
55
2017 Islam and Zibran

A Comparison of Dictionary Building Methods
for Sentiment Analysis in Software Engineering
Text

A Comparison of Dictionary Building Methods
for Sentiment Analysis in Software Engineering
Text
✔️ ✔️ ✔️ ✔️ Tool Evaluation
56
2018 Islam and Zibran

A Comparison of Software Engineering Domain
Specific Sentiment Analysis Tools

A Comparison of Software Engineering Domain
Specific Sentiment Analysis Tools
✔️ ✔️ ✔️ ✔️ Tool Evaluation
57
2017 Calefato et al.

EmoTxt: A Toolkit for Emotion Recognition
from Text

EmoTxt: A Toolkit for Emotion Recognition
from Text
✔️ ✔️ ✔️ ✔️ Tool Development + Evaluation ✔️ EmoTxt
58
2020 Kritikos et al.

An Empirical Investigation of Sentiment Analysis
of the Bug Tracking Process in
Libre Office Open Source Software

An Empirical Investigation of Sentiment Analysis
of the Bug Tracking Process in
Libre Office Open Source Software
✔️ ✔️ ✔️ Tool Application + Usage
59
2018 Murgia et al.

An Exploratory Qualitative and Quantitative Analysis
of Emotions in Issue Report Comments
of Open Source Systems

An Exploratory Qualitative and Quantitative Analysis
of Emotions in Issue Report Comments
of Open Source Systems
✔️ ✔️ ✔️ ✔️ Tool Development + Evaluation ✔️ ESEM-E
60
2017 Guzman et al.

An Exploratory Study of Twitter Messages
about Software Applications

An Exploratory Study of Twitter Messages
about Software Applications
✔️ ✔️ ✔️ ✔️ Tool Application + Usage ✔️
61
2018 Freira et al.

Analyzing The Impact Of Feedback In
GitHub On The Software Developer's Mood

Analyzing The Impact Of Feedback In
GitHub On The Software Developer's Mood
✔️ ✔️ ✔️ ✔️ Tool Application + Usage
62
2021 Novielli et al.

Assessment of Off-the-Shelf SE-Specific Sentiment Analysis
Tools: An Extended Replication Study

Assessment of Off-the-Shelf SE-Specific Sentiment Analysis
Tools: An Extended Replication Study
✔️ ✔️ ✔️ Tool Evaluation ✔️
63
2020 Ahasanuzzaman et al.

CAPS: A Supervised Technique for Classifying
Stack Overflow Posts Concerning API Issues

CAPS: A Supervised Technique for Classifying
Stack Overflow Posts Concerning API Issues
✔️ ✔️ ✔️ ✔️ Tool Application + Usage
64
2020 Umer et al.

CNN-Based Automatic Prioritization of Bug Reports

CNN-Based Automatic Prioritization of Bug Reports
✔️ ✔️ ✔️ Tool Application + Usage
65
2016 Robinson et al.

Developer Behavior and Sentiment from Data
Mining Open Source Repositories

Developer Behavior and Sentiment from Data
Mining Open Source Repositories
✔️ ✔️ ✔️ ✔️ Tool Application + Usage
66
2019 Neupane et al.

EmoD: An End-to-End Approach for Investigating
Emotion Dynamics in Software Development

EmoD: An End-to-End Approach for Investigating
Emotion Dynamics in Software Development
✔️ ✔️ ✔️ ✔️ Tool Development EmoD
67
2018 Kaur et al.

Emotion Mining and Sentiment Analysis in
Software Engineering Domain

Emotion Mining and Sentiment Analysis in
Software Engineering Domain
✔️ ✔️ ✔️ ✔️ Tool Evaluation ✔️
68
2020 Cagnoni et al.

Emotion-Based Analysis of Programming Languages on
Stack Overflow

Emotion-Based Analysis of Programming Languages on
Stack Overflow
✔️ ✔️ ✔️ Tool Development + Evaluation
69
2019 Wang

Emotions Extracted from Text vs. True
Emotions–An Empirical Evaluation in SE Context

Emotions Extracted from Text vs. True
Emotions–An Empirical Evaluation in SE Context
✔️ ✔️ ✔️ ✔️ Tool Evaluation ✔️
70
2019 Calefato et al.

EMTk – The Emotion Mining Toolkit

EMTk – The Emotion Mining Toolkit
✔️ ✔️ ✔️ Tool Development + Evaluation EMTk
71
2013 Zhang and Hou

Extracting Problematic API Features from Forum
Discussions

Extracting Problematic API Features from Forum
Discussions
✔️ ✔️ ✔️ ✔️ Tool Application + Usage
72
2018 Werner et al.

How Angry Are Your Customers? Sentiment
Analysis of Support Tickets that Escalate

How Angry Are Your Customers? Sentiment
Analysis of Support Tickets that Escalate
✔️ ✔️ ✔️ ✔️ Tool Application + Usage
73
2018 Mostafa and Abd Elghany

Investigating Game Developers' Guilt Emotions Using
Sentiment Analysis

Investigating Game Developers' Guilt Emotions Using
Sentiment Analysis
✔️ ✔️ ✔️ Tool Application + Usage
74
2019 Islam et al.

MarValous: Machine Learning Based Detection of
Emotions in the Valence-Arousal Space in
Software Engineering Text

MarValous: Machine Learning Based Detection of
Emotions in the Valence-Arousal Space in
Software Engineering Text
✔️ ✔️ ✔️ ✔️ Tool Development + Evaluation MarValous
75
2018 Werder and Brinkkemper

MEME: Toward a Method for Emotions
Extraction from GitHub

MEME: Toward a Method for Emotions
Extraction from GitHub
✔️ ✔️ ✔️ ✔️ Tool Development + Evaluation MEME
76
2018 Ortu et al.

Mining Communication Patterns in Software Development:
A GitHub Analysis

Mining Communication Patterns in Software Development:
A GitHub Analysis
✔️ ✔️ ✔️ ✔️ Tool Application + Usage
77
2017 Goyal and Sardana

NRFixer: Sentiment Based Model for Predicting
the Fixability of Non-Reproducible Bugs

NRFixer: Sentiment Based Model for Predicting
the Fixability of Non-Reproducible Bugs
✔️ ✔️ ✔️ Tool Application + Usage
78
2018 Claes et al.

On the Use of Emoticons in
Open Source Software Development

On the Use of Emoticons in
Open Source Software Development
✔️ ✔️ ✔️ ✔️ Tool Application + Usage
79
2019 Lin et al.

Pattern-Based Mining of Opinions in Q&A
Websites

Pattern-Based Mining of Opinions in Q&A
Websites
✔️ ✔️ ✔️ ✔️ Tool Development + Evaluation ✔️ POME
80
2014 Pletea et al.

Security and Emotion: Sentiment Analysis of
Security Discussions on GitHub

Security and Emotion: Sentiment Analysis of
Security Discussions on GitHub
✔️ ✔️ ✔️ ✔️ Tool Application + Usage ✔️
81
2014 Rousinopoulos et al.

Sentiment Analysis of Free/Open Source Developers:
Preliminary Findings from a Case Study

Sentiment Analysis of Free/Open Source Developers:
Preliminary Findings from a Case Study
✔️ ✔️ ✔️ Tool Application + Usage
82
2017 Patwardhan

Sentiment Identification for Collaborative, Geographically Dispersed,
Cross-Functional Software Development Teams

Sentiment Identification for Collaborative, Geographically Dispersed,
Cross-Functional Software Development Teams
✔️ ✔️ ✔️ ✔️ Tool Application + Usage ✔️
83 PeerJ Computer Science 2016 Destefanis et al.

Software Development: Do Good Manners Matter?

Software Development: Do Good Manners Matter?
✔️ ✔️ ✔️ Tool Application + Usage
84
2018 Werder

The Evolution of Emotional Displays in
Open Source Software Development Teams: An
Individual Growth Curve Analysis

The Evolution of Emotional Displays in
Open Source Software Development Teams: An
Individual Growth Curve Analysis
✔️ ✔️ ✔️ ✔️ Tool Application + Usage
85
2013 Garcia et al.

The Role of Emotions in Contributors
Activity: A Case Study on the
GENTOO Community

The Role of Emotions in Contributors
Activity: A Case Study on the
GENTOO Community
✔️ ✔️ ✔️ ✔️ Tool Application + Usage
86
2016 Islam and Zibran

Towards Understanding and Exploiting Developers' Emotional
Variations in Software Engineering

Towards Understanding and Exploiting Developers' Emotional
Variations in Software Engineering
✔️ ✔️ ✔️ ✔️ Tool Application + Usage
87
2013 Guzman

Visualizing Emotions in Software Development Projects

Visualizing Emotions in Software Development Projects
✔️ ✔️ ✔️ ✔️ Tool Application + Usage
88
2020 Brisson et al.

We Are Family: Analyzing Communication in
GitHub Software Repositories and Their Forks

We Are Family: Analyzing Communication in
GitHub Software Repositories and Their Forks
✔️ ✔️ ✔️ Tool Application + Usage
89
2020 Biswas et al.

Achieving Reliable Sentiment Analysis in the
Software Engineering Domain Using BERT

Achieving Reliable Sentiment Analysis in the
Software Engineering Domain Using BERT
✔️ ✔️ Tool Development + Evaluation ✔️ BERT4SentiSE
90
2021 Mahbub et al.

Analysis of Factors Influencing User Contribution
and Predicting Involvement of Users on
Stack Overflow

Analysis of Factors Influencing User Contribution
and Predicting Involvement of Users on
Stack Overflow
✔️ ✔️ Tool Application + Usage
91
2021 Park and Sharif

Assessing Perceived Sentiment in Pull Requests
with Emoji: Evidence from Tools and
Developer Eye Movements

Assessing Perceived Sentiment in Pull Requests
with Emoji: Evidence from Tools and
Developer Eye Movements
✔️ ✔️ Tool Evaluation ✔️
92
2021 Uddin and Khomh

Automatic Mining of Opinions Expressed About
APIs in Stack Overflow

Automatic Mining of Opinions Expressed About
APIs in Stack Overflow
✔️ ✔️ ✔️ Tool Development + Evaluation ✔️ OpinerDSO
93
2021 Wu et al.

BERT for Sentiment Classification in Software
Engineering

BERT for Sentiment Classification in Software
Engineering
✔️ ✔️ Tool Development + Evaluation BERT-FT
94
2020 Cabrera-Diego et al.

Classifying Emotions in Stack Overflow and
JIRA Using a Multi-Label Approach

Classifying Emotions in Stack Overflow and
JIRA Using a Multi-Label Approach
✔️ ✔️ Tool Evaluation
95
2021 Chen et al.

Emoji-Powered Sentiment and Emotion Detection from
Software Developers’ Communication Data

Emoji-Powered Sentiment and Emotion Detection from
Software Developers’ Communication Data
✔️ ✔️ Tool Evaluation
96
2021 Herrmann and Klünder

From Textual to Verbal Communication: Towards
Applying Sentiment Analysis to a Software
Project Meeting

From Textual to Verbal Communication: Towards
Applying Sentiment Analysis to a Software
Project Meeting
✔️ ✔️ Tool Application + Usage ✔️
97
2021 Mansoor et al.

How Developers and Tools Categorize Sentiment
in Stack Overflow Questions – A
Pilot Study

How Developers and Tools Categorize Sentiment
in Stack Overflow Questions – A
Pilot Study
✔️ ✔️ Tool Evaluation ✔️
98 PLOS ONE 2021 Sokolovsky et al.

Is It Feasible to Detect FLOSS
Version Release Events from Textual Messages?
A Case Study on Stack Overflow

Is It Feasible to Detect FLOSS
Version Release Events from Textual Messages?
A Case Study on Stack Overflow
✔️ ✔️ Tool Application + Usage
99
2016 Mäntylä et al.

Mining Valence, Arousal, and Dominance –
Possibilities for Detecting Burnout and Productivity?

Mining Valence, Arousal, and Dominance –
Possibilities for Detecting Burnout and Productivity?
✔️ ✔️ ✔️ Tool Application + Usage
100
2021 Zhang et al.

SentiLog: Anomaly Detecting on Parallel File
Systems via Log-Based Sentiment Analysis

SentiLog: Anomaly Detecting on Parallel File
Systems via Log-Based Sentiment Analysis
✔️ ✔️ Tool Development SentiLog
101
2020 Valdez et al.

Sentiment Analysis in Jira Software Repositories

Sentiment Analysis in Jira Software Repositories
✔️ ✔️ Tool Application + Usage
102 Mathematics 2021 Dao and Yang

Severity Prediction for Bug Reports Using
Multi-Aspect Features: A Deep Learning Approach

Severity Prediction for Bug Reports Using
Multi-Aspect Features: A Deep Learning Approach
✔️ ✔️ Tool Application + Usage
103
2021 Sanei et al.

The Impacts of Sentiments and Tones
in Community-Generated Issue Discussions

The Impacts of Sentiments and Tones
in Community-Generated Issue Discussions
✔️ ✔️ Tool Application + Usage ✔️
104
2020 Madampe et al.

Towards Understanding Emotional Response to Requirements
Changes in Agile Teams

Towards Understanding Emotional Response to Requirements
Changes in Agile Teams
✔️ ✔️ ✔️ Tool Application + Usage
105
2021 Venigalla and Chimalakonda

Understanding Emotions of Developer Community Towards
Software Documentation

Understanding Emotions of Developer Community Towards
Software Documentation
✔️ ✔️ Tool Application + Usage
106 PLOS ONE 2019 Sapkota et al.

A Network-Centric Approach for Estimating Trust
Between Open Source Software Developers

A Network-Centric Approach for Estimating Trust
Between Open Source Software Developers
✔️ ✔️ Tool Application + Usage ✔️
107
2015 Serva et al.

Automatically Mining Negative Code Examples from
Software Developer Q & A Forums

Automatically Mining Negative Code Examples from
Software Developer Q & A Forums
✔️ ✔️ Tool Development + Evaluation ✔️
108 IEEE Software 2019 Werner et al.

Can a Machine Learn Through Customer
Sentiment?: A Cost-Aware Approach to Predict
Support Ticket Escalations

Can a Machine Learn Through Customer
Sentiment?: A Cost-Aware Approach to Predict
Support Ticket Escalations
✔️ ✔️ Tool Application + Usage
109 IEEE Software 2019 Lanovaz and Adams

Comparing the Communication Tone and Responses
of Users and Developers in Two
R Mailing Lists: Measuring Positive and
Negative Emails

Comparing the Communication Tone and Responses
of Users and Developers in Two
R Mailing Lists: Measuring Positive and
Negative Emails
✔️ ✔️ Tool Application + Usage
110
2016 da Cruz et al.

Estimating Trust in Virtual Teams: A
Framework Based on Sentiment Analysis

Estimating Trust in Virtual Teams: A
Framework Based on Sentiment Analysis
✔️ ✔️ Tool Application + Usage
111
2018 Licorish and MacDonell

Exploring the Links Between Software Development
Task Type, Team Attitudes and Task
Completion Performance: Insights from the Jazz
Repository

Exploring the Links Between Software Development
Task Type, Team Attitudes and Task
Completion Performance: Insights from the Jazz
Repository
✔️ ✔️ Tool Application + Usage
112
2017 Munaiah et al.

Natural Language Insights from Code Reviews
that Missed a Vulnerability

Natural Language Insights from Code Reviews
that Missed a Vulnerability
✔️ ✔️ Tool Application + Usage
113
2015 Rahman et al.

Recommending Insightful Comments for Source Code
Using Crowdsourced Knowledge

Recommending Insightful Comments for Source Code
Using Crowdsourced Knowledge
✔️ ✔️ Tool Application + Usage
114
2014 Licorish and MacDonell

Relating IS Developers' Attitudes to Engagement

Relating IS Developers' Attitudes to Engagement
✔️ ✔️ Tool Application + Usage
115
2019 Ferreira et al.

Sentiment Analysis of Open Source Communities:
An Exploratory Study

Sentiment Analysis of Open Source Communities:
An Exploratory Study
✔️ ✔️ Tool Application + Usage
116
2019 Alesinloye et al.

Sentiment Analysis of Open Source Software
Community Mailing List: A Preliminary Analysis

Sentiment Analysis of Open Source Software
Community Mailing List: A Preliminary Analysis
✔️ ✔️ Tool Application + Usage
117 Information Systems 2019 Morales-Ramirez et al.

Speech-Acts Based Analysis for Requirements Discovery
from Online Discussions

Speech-Acts Based Analysis for Requirements Discovery
from Online Discussions
✔️ ✔️ Tool Application + Usage
118
2016 Tourani and Adams

The Impact of Human Discussions on
Just-in-Time Quality Assurance: An Empirical Study
on OpenStack and Eclipse

The Impact of Human Discussions on
Just-in-Time Quality Assurance: An Empirical Study
on OpenStack and Eclipse
✔️ ✔️ Tool Application + Usage
119
2018 Portugal and do Prado Leite

Usability Related Qualities Through Sentiment Analysis

Usability Related Qualities Through Sentiment Analysis
✔️ ✔️ Tool Application + Usage
120
2019 Ferreira et al.

Winning of Hearts and Minds: Integrating
Sentiment Analytics into the Analysis of
Contradictions

Winning of Hearts and Minds: Integrating
Sentiment Analytics into the Analysis of
Contradictions
✔️ ✔️ Tool Application + Usage
121 IEEE Access 2019 Ramay et al.

Deep Neural Network-Based Severity Prediction of
Bug Reports

Deep Neural Network-Based Severity Prediction of
Bug Reports
✔️ Tool Application + Usage
122
2017 Yang et al.

Analyzing Emotion Words to Predict Severity
of Software Bugs: A Case Study
of Open Source Projects

Analyzing Emotion Words to Predict Severity
of Software Bugs: A Case Study
of Open Source Projects
✔️ Tool Application + Usage
123
2018 Yang et al.

An Emotion Similarity Based Severity Prediction
of Software Bugs: A Case Study
of Open Source Projects

An Emotion Similarity Based Severity Prediction
of Software Bugs: A Case Study
of Open Source Projects
✔️ Tool Application + Usage
124
2018 Ahasanuzzaman et al.

Classifying Stack Overflow Posts on API
Issues

Classifying Stack Overflow Posts on API
Issues
✔️ Tool Application + Usage
125
2020 Klünder et al.

Identifying the Mood of a Software
Development Team by Analyzing Text-Based Communication
in Chats with Machine Learning

Identifying the Mood of a Software
Development Team by Analyzing Text-Based Communication
in Chats with Machine Learning
✔️ Tool Development + Evaluation ✔️
126
2022 Herrmann et al.

On the Subjectivity of Emotions in
Software Projects: How Reliable Are Pre-labled
Data Sets for Sentiment Analysis?

On the Subjectivity of Emotions in
Software Projects: How Reliable Are Pre-labled
Data Sets for Sentiment Analysis?
✔️ Tool Development + Evaluation ✔️
127
2022 Graßl and Fraser

Scratch as Social Network: Topic Modeling
and Sentiment Analysis in Scratch Projects

Scratch as Social Network: Topic Modeling
and Sentiment Analysis in Scratch Projects
✔️ Tool Application + Usage ✔️
128
2022 Uddin et al.

An Empirical Study of the Effectiveness
of an Ensemble of Stand-Alone Sentiment
Detection Tools for Software Engineering Datasets

An Empirical Study of the Effectiveness
of an Ensemble of Stand-Alone Sentiment
Detection Tools for Software Engineering Datasets
✔️ Tool Development + Evaluation Sentisead
129 Journal of Software: Evolution
and Process
2023 Almarimi et al.

Improving the Detection of Community Smells
Through Socio-Technical and Sentiment Analysis

Improving the Detection of Community Smells
Through Socio-Technical and Sentiment Analysis
✔️ Tool Application + Usage
130
2021 Huang et al.

Predicting Community Smells' Occurrence on Individual
Developers' Emotions and Progress

Predicting Community Smells' Occurrence on Individual
Developers' Emotions and Progress
✔️ Tool Application + Usage
131
2022 Patnaik and Padhy

Sentiment Analysis of Software Project Code
Commits

Sentiment Analysis of Software Project Code
Commits
✔️ Tool Application + Usage
132
2022 Obaidi et al.

On the Limitations of Combining Sentiment
Analysis Tools in a Cross-Platform Setting

On the Limitations of Combining Sentiment
Analysis Tools in a Cross-Platform Setting
✔️ Tool Development + Evaluation
133
2023 Swillus and Zaidman

Sentiment Overflow in the Testing Stack:
Analysing Software Testing Posts on Stack
Overflow

Sentiment Overflow in the Testing Stack:
Analysing Software Testing Posts on Stack
Overflow
✔️ Tool Application + Usage
134
2023 Sarker et al.

ToxiSpanSE: An Explainable Toxicity Detection in
Code Review Comments

ToxiSpanSE: An Explainable Toxicity Detection in
Code Review Comments
✔️ Tool Development + Evaluation ✔️ ToxiSpanSE
135
2020 Sengupta and Haythornthwaite

Learning with Comments: An Analysis of
Comments and Communityon Stack Overflow

Learning with Comments: An Analysis of
Comments and Communityon Stack Overflow
✔️ Tool Application + Usage (human only) ✔️
136
2022 Gao et al.

Understanding the Impact of Bots on
Developers Sentiment and Project Progress

Understanding the Impact of Bots on
Developers Sentiment and Project Progress
✔️ Tool Application + Usage
137
2022 Imran et al.

Data Augmentation for Improving Emotion Recognition
in Software Engineering Communication

Data Augmentation for Improving Emotion Recognition
in Software Engineering Communication
✔️ Tool Evaluation ✔️
138
2022 Prenner and Robbes

Making the Most of Small Software
Engineering Datasets With Modern Machine Learning

Making the Most of Small Software
Engineering Datasets With Modern Machine Learning
✔️ Tool Development + Evaluation StackOBERTflow
139
2019 Sarker et al.

Socio-Technical Work-Rate Increase Associates With Changes
in Work Patterns in Online Projects

Socio-Technical Work-Rate Increase Associates With Changes
in Work Patterns in Online Projects
✔️ Tool Application + Usage
140
2016 Ortu et al.

Arsonists or Firefighters? Affectiveness in Agile
Software Development

Arsonists or Firefighters? Affectiveness in Agile
Software Development
✔️ ✔️ Tool Application + Usage
141
2021 Fucci et al.

Waiting Around or Job Half-Done? Sentiment
in Self-Admitted Technical Debt

Waiting Around or Job Half-Done? Sentiment
in Self-Admitted Technical Debt
✔️ Tool Evaluation ✔️
142
2019 Imtiaz et al.

Investigating the Effects of Gender Bias
on GitHub

Investigating the Effects of Gender Bias
on GitHub
✔️ Tool Application + Usage
143
2022 Kadhar and Kumar

Deep-Learning Approach for Sentiment Analysis in
Software Engineering Domain

Deep-Learning Approach for Sentiment Analysis in
Software Engineering Domain
✔️ Tool Development + Evaluation
144
2022 Sun et al.

Incorporating Pre-trained Transformer Models into TextCNN
for Sentiment Analysis on Software Engineering
Texts

Incorporating Pre-trained Transformer Models into TextCNN
for Sentiment Analysis on Software Engineering
Texts
✔️ Tool Development + Evaluation EASTER
145
2022 Mula et al.

Software Sentiment Analysis Using Machine Learning
with Different Word-Embeddings

Software Sentiment Analysis Using Machine Learning
with Different Word-Embeddings
✔️ Tool Development + Evaluation
146
2022 Kumar et al.

Sentiment Analysis of Developers' Comments on
GitHub Repository: A Study

Sentiment Analysis of Developers' Comments on
GitHub Repository: A Study
✔️ Tool Application + Usage
147
2022 Rong et al.

An Empirical Study of Emoji Use
in Software Development Communication

An Empirical Study of Emoji Use
in Software Development Communication
✔️ Tool Application + Usage ✔️
148
2023 von der Mosel et al.

On the Validity of Pre-trained Transformers
for Natural Language Processing in the
Software Engineering Domain

On the Validity of Pre-trained Transformers
for Natural Language Processing in the
Software Engineering Domain
✔️ Tool Development + Evaluation seBERT
149
2023 Assavakamhaenghan et al.

Does the first Response Matter for
Future Contributions? A Study of First
Contributions

Does the first Response Matter for
Future Contributions? A Study of First
Contributions
✔️ Tool Application + Usage
150
2021 Uddin et al.

Automatic API Usage Scenario Documentation from
Technical Q&A Sites

Automatic API Usage Scenario Documentation from
Technical Q&A Sites
✔️ Tool Application + Usage
151
2020 Uddin et al.

Mining API Usage Scenarios from StackOverflow

Mining API Usage Scenarios from StackOverflow
✔️ ✔️ Tool Application + Usage
152
2022 Uddin et al.

A Qualitative Study of Developers' Discussions
of Their Problems and Joys During
Early COVID-19 Months

A Qualitative Study of Developers' Discussions
of Their Problems and Joys During
Early COVID-19 Months
✔️ Tool Application + Usage (human only) ✔️
153
2019 El Asri et al.

An Empirical Study of Sentiments in
Code Reviews

An Empirical Study of Sentiments in
Code Reviews
✔️ ✔️ Tool Application + Usage ✔️
154 IEEE Software 2019 Maipradit et al.

Sentiment Classification Using N-Gram Inverse Document
Frequency and Automated Machine Learning

Sentiment Classification Using N-Gram Inverse Document
Frequency and Automated Machine Learning
✔️ ✔️ Tool Development + Evaluation
155
2022 Schroth et al.

On the Potentials of Realtime Sentiment
Analysis on Text-Based Communication in Software
Projects

On the Potentials of Realtime Sentiment
Analysis on Text-Based Communication in Software
Projects
✔️ Tool Application + Usage
156
2022 Kaur et al.

Analysis of Factors Influencing Developers' Sentiments
in Commit Logs: Insights from Applying
Sentiment Analysis

Analysis of Factors Influencing Developers' Sentiments
in Commit Logs: Insights from Applying
Sentiment Analysis
✔️ Tool Application + Usage
157
2019 Paul et al.

Expressions of Sentiments During Code Reviews:
Male vs. Female

Expressions of Sentiments During Code Reviews:
Male vs. Female
✔️ Tool Application + Usage
158
2022 Bleyl and Buxton

Emotion Recognition on StackOverflow Posts Using
BERT

Emotion Recognition on StackOverflow Posts Using
BERT
✔️ Tool Development + Evaluation
159
2017 Singh and Singh

How Do Code Refactoring Activities Impact
Software Developers' Sentiments? – An Empirical
Investigation into GitHub Commits

How Do Code Refactoring Activities Impact
Software Developers' Sentiments? – An Empirical
Investigation into GitHub Commits
✔️ ✔️ Tool Application + Usage
160
2022 Sayago-Heredia et al.

A Datset for Analysis of Quality
Code and Toxic Comments

A Datset for Analysis of Quality
Code and Toxic Comments
✔️ Tool Application + Usage
161
2015 Ortu et al.

Would You Mind Fixing This Issue?
An Empirical Analysis of Politeness and
Attractiveness in Software Developed Using Agile
Boards

Would You Mind Fixing This Issue?
An Empirical Analysis of Politeness and
Attractiveness in Software Developed Using Agile
Boards
✔️ Tool Application + Usage
162
2019 Huq et al.

Understanding the Effect of Developer Sentiment
on Fix-Inducing Changes: An Exploratory Study
on GitHub Pull Requests

Understanding the Effect of Developer Sentiment
on Fix-Inducing Changes: An Exploratory Study
on GitHub Pull Requests
✔️ ✔️ Tool Application + Usage
163
2018 Destefanis et al.

On Measuring Affects of GitHub Issues'
Commenters

On Measuring Affects of GitHub Issues'
Commenters
✔️ ✔️ Tool Application + Usage
164
2019 Ortu et al.

Empirical Analysis of Affect of Merged
Issues on GitHub

Empirical Analysis of Affect of Merged
Issues on GitHub
✔️ ✔️ Tool Application + Usage
165
2017 Yang et al.

Sentiments Analysis in GitHub Repositories: An
Empirical Study

Sentiments Analysis in GitHub Repositories: An
Empirical Study
✔️ ✔️ Tool Application + Usage
166
2021 Chatterjee et al.

Automatic Extraction of Opinion-Based Q&A from
Online Developer Chats

Automatic Extraction of Opinion-Based Q&A from
Online Developer Chats
✔️ Tool Application + Usage
167
2022 Cassee et al.

Self-Admitted Technical Debt and Comments' Polarity:
An Empirical Study

Self-Admitted Technical Debt and Comments' Polarity:
An Empirical Study
✔️ Tool Evaluation ✔️
168
2019 Skriptsova et al.

Analysis of Newcomers Activity in Communicative
Posts on GitHub

Analysis of Newcomers Activity in Communicative
Posts on GitHub
✔️ ✔️ Tool Application + Usage
169
2021 Chouchen et al.

Anti-Patterns in Modern Code Review: Symptoms
and Prevalence

Anti-Patterns in Modern Code Review: Symptoms
and Prevalence
✔️ Tool Application + Usage (human only) ✔️
170
2022 Robe et al.

Pair Programming Conversations with Agents vs.
Developers: Challenges and Opportunities for SE
Community

Pair Programming Conversations with Agents vs.
Developers: Challenges and Opportunities for SE
Community
✔️ Tool Application + Usage (human only - but suggest tools) ✔️
171
2023 Batoun et al.

An Empirical Study on GitHub Pull
Requests' Reactions

An Empirical Study on GitHub Pull
Requests' Reactions
✔️ Tool Application + Usage (human only) ✔️
172
2020 Li et al.

Sentiment Analysis over Collaborative Relationships in
Open Source Software Projects

Sentiment Analysis over Collaborative Relationships in
Open Source Software Projects
✔️ Tool Application + Usage
173
2021 Batra et al.

BERT-Based Sentiment Analysis: A Software Engineering
Perspective

BERT-Based Sentiment Analysis: A Software Engineering
Perspective
✔️ Tool Evaluation
174
2021 Li et al.

Monitoring Negative Sentiment-Related Events in Open
Source Software Projects

Monitoring Negative Sentiment-Related Events in Open
Source Software Projects
✔️ Tool Application + Usage
175
2023 Yin et al.

On the Self-Governance and Episodic Changes
in Apache Incubator Projects: An Empirical
Study

On the Self-Governance and Episodic Changes
in Apache Incubator Projects: An Empirical
Study
✔️ Tool Application + Usage
176
2023 Wang et al.

More Than React: Investigating the Role
of Emoji Reaction in GitHub Pull
Requests

More Than React: Investigating the Role
of Emoji Reaction in GitHub Pull
Requests
✔️ Tool Application + Usage
177
2024 Ferreira et al.

Incivility Detection in Open Source Code
Review and Issue Discussions

Incivility Detection in Open Source Code
Review and Issue Discussions
✔️ Tool Evaluation
Jump to top of page

List of Papers that We Considered Not-Relevant for Our Study (Excluded Papers)

In this table, we present all the papers that we found via any of the pillars of our systematic literature review but that we consider not-relevant for our study on sentiment analysis of developers' communications in the software-engineering domain, that is, that did not fulfill our inclusion criteria. In the last column, we denote the reason why we consider the paper not-relevant for our study, that is, why we excluded the paper.

# Venue Year Authors Title Found by
Obaidi & Klünder
Found by
Extension of Obaidi & Klünder
Found by
Lin et al.
Considered
by Us
Reason Why Not Considered
1 Journal of King Saud
University
2020 Issa Atoum

A Novel Framework for Measuring Software
Quality-in-Use Based on Semantic Similarity and
Sentiment Analysis of Software Reviews

A Novel Framework for Measuring Software
Quality-in-Use Based on Semantic Similarity and
Sentiment Analysis of Software Reviews
✔️ ✔️ User Reviews
2 Rapport de Recherche RR-LIRIS 2014 Collomb et al.

A Study and Comparison of Sentiment
Analysis Methods for Reputation Evaluation

A Study and Comparison of Sentiment
Analysis Methods for Reputation Evaluation
✔️ ✔️ Not specific to the software engineering domain
3
2015 Panichella et al.

How Can I Improve My App?
Classifying User Reviews for Software Maintenance
and Evolution

How Can I Improve My App?
Classifying User Reviews for Software Maintenance
and Evolution
✔️ ✔️ ✔️ User Reviews
4
2012 Goul et al.

Managing the Enterprise Business Intelligence App
Store: Sentiment Analysis Supported Requirements Engineering

Managing the Enterprise Business Intelligence App
Store: Sentiment Analysis Supported Requirements Engineering
✔️ ✔️ User Reviews, before 2013
5
2017 Martens and Johann

On the Emotion of Users in
App Reviews

On the Emotion of Users in
App Reviews
✔️ ✔️ ✔️ User Reviews
6
2017 Kumar and Abraham

Opinion Mining to Assist User Acceptance
Testing for Open-Beta Versions

Opinion Mining to Assist User Acceptance
Testing for Open-Beta Versions
✔️ ✔️ User Reviews
7
2020 Whiting et al.

Parallel Worlds: Repeated Initializations of the
Same Team to Improve Team Viability

Parallel Worlds: Repeated Initializations of the
Same Team to Improve Team Viability
✔️ ✔️ Not specific to the software engineering domain
8
2015 Guzman et al.

Retrieving Diverse Opinions from App Reviews

Retrieving Diverse Opinions from App Reviews
✔️ ✔️ ✔️ User Reviews
9
2016 Qian et al.

SatiIndicator: Leveraging User Reviews to Evaluate
User Satisfaction of SourceForge Projects

SatiIndicator: Leveraging User Reviews to Evaluate
User Satisfaction of SourceForge Projects
✔️ ✔️ ✔️ User Reviews
10
2014 Haque and Rahman

Sentiment Analysis by Using Fuzzy Logic

Sentiment Analysis by Using Fuzzy Logic
✔️ ✔️ Not specific to the software engineering domain
11
2017 Aung and Myo

Sentiment Analysis of Students' Comment Using
Lexicon Based Approach

Sentiment Analysis of Students' Comment Using
Lexicon Based Approach
✔️ ✔️ Not specific to the software engineering domain
12
2017 Gkontzis et al.

Sentiment Analysis to Track Emotion and
Polarity in Student Fora

Sentiment Analysis to Track Emotion and
Polarity in Student Fora
✔️ ✔️ Not specific to the software engineering domain
13 Journal of Big Data 2015 Fang and Zhan

Sentiment Analysis Using Product Review Data

Sentiment Analysis Using Product Review Data
✔️ ✔️ Not specific to the software engineering domain
14
2014 El-Halees

Software Usability Evaluation Using Opinion Mining

Software Usability Evaluation Using Opinion Mining
✔️ ✔️ ✔️ User Reviews
15
2018 Hassan et al.

Studying the Dialogue Between Users and
Developers of Free Apps in the
Google Play Store

Studying the Dialogue Between Users and
Developers of Free Apps in the
Google Play Store
✔️ ✔️ User Reviews
16
2015 Kaewyong et al.

The Possibility of Students' Comments Automatic
Interpret Using Lexicon Based Sentiment Analysis
to Teacher Evaluation

The Possibility of Students' Comments Automatic
Interpret Using Lexicon Based Sentiment Analysis
to Teacher Evaluation
✔️ ✔️ Not specific to the software engineering domain
17
2021 Jeong and Kim

Does Sentiment Help Requirement Engineering: Exploring
Sentiments in User Comments to Discover
Informative Comments

Does Sentiment Help Requirement Engineering: Exploring
Sentiments in User Comments to Discover
Informative Comments
✔️ User Reviews
18
2014 Guzman and Maalej

How Do Users Like This Feature?
A Fine Grained Sentiment Analysis of
App Reviews

How Do Users Like This Feature?
A Fine Grained Sentiment Analysis of
App Reviews
✔️ ✔️ User Reviews
19
2021 Garousi et al.

Mining User Reviews of COVID Contact-Tracing
Apps: An Exploratory Analysis of Nine
European Apps

Mining User Reviews of COVID Contact-Tracing
Apps: An Exploratory Analysis of Nine
European Apps
✔️ User Reviews
20 Irish Journal of Medical
Science
2021 Rekanar et al.

Sentiment Analysis of User Feedback on
the HSE's Covid-19 Contact Tracing App

Sentiment Analysis of User Feedback on
the HSE's Covid-19 Contact Tracing App
✔️ User Reviews
21
2021 Yang et al.

TOUR: Dynamic Topic and Sentiment Analysis
of User Reviews for Assisting App
Release

TOUR: Dynamic Topic and Sentiment Analysis
of User Reviews for Assisting App
Release
✔️ User Reviews
22
2018 Muñoz et al.

A Cognitive Agent for Mining Bugs
Reports, Feature Suggestions and Sentiment in
a Mobile Application Store

A Cognitive Agent for Mining Bugs
Reports, Feature Suggestions and Sentiment in
a Mobile Application Store
✔️ User Reviews
23
2020 Zhou et al.

A Domain Knowledge Incorporated Text Mining
Approach for Capturing User Needs on
BIM Applications

A Domain Knowledge Incorporated Text Mining
Approach for Capturing User Needs on
BIM Applications
✔️ User Reviews, not specific to the software engineering domain
24
2018 Luiz et al.

A Feature-Oriented Sentiment Rating for Mobile
App Reviews

A Feature-Oriented Sentiment Rating for Mobile
App Reviews
✔️ User Reviews
25
2018 Kuriachan and Pervin

ALDA: An Aggregated LDA for Polarity
Enhanced Aspect Identification Technique in Mobile
App Domain

ALDA: An Aggregated LDA for Polarity
Enhanced Aspect Identification Technique in Mobile
App Domain
✔️ User Reviews
26
2017 Guzman et al.

A Little Bird Told Me: Mining
Tweets for Requirements and Software Evolution

A Little Bird Told Me: Mining
Tweets for Requirements and Software Evolution
✔️ User Reviews, Twitter
27
2017 Yin and Pfahl

A Method to Transform Automatically Extracted
Product Features into Inputs for Kano-Like
Models

A Method to Transform Automatically Extracted
Product Features into Inputs for Kano-Like
Models
✔️ Online sources (not specified), not specific to the software engineering domain
28
2019 Messaoud et al.

A Multi-label Active Learning Approach for
Mobile App User Review Classification

A Multi-label Active Learning Approach for
Mobile App User Review Classification
✔️ User Reviews
29
2019 Khan et al.

Analysis of Requirements-Related Arguments in User
Forums

Analysis of Requirements-Related Arguments in User
Forums
✔️ User Reviews, Reddit, not really related to sentiment analysis
30
2013 Carreno and Winbladh

Analysis of User Comments: An Approach
for Software Requirements Evolution

Analysis of User Comments: An Approach
for Software Requirements Evolution
✔️ User Reviews
31
2015 McIlroy et al.

Analyzing and Automatically Labelling the Types
of User Issues that Are Raised
in Mobile App Reviews

Analyzing and Automatically Labelling the Types
of User Issues that Are Raised
in Mobile App Reviews
✔️ User Reviews
32
2017 Williams and Mahmoud

Analyzing, Classifying, and Interpreting Emotions in
Software Users' Tweets

Analyzing, Classifying, and Interpreting Emotions in
Software Users' Tweets
✔️ User Reviews, Twitter
33
2017 Ciurumelea et al.

Analyzing Reviews and Code of Mobile
Apps for Better Release Planning

Analyzing Reviews and Code of Mobile
Apps for Better Release Planning
✔️ User Reviews
34 Journal of Software: Evolution
and Process
2018 Liu et al.

Analyzing Reviews Guided by App Descriptions
for the Software Development and Evolution

Analyzing Reviews Guided by App Descriptions
for the Software Development and Evolution
✔️ User Reviews
35
2013 Jamroonsilp and Prompoon

Analyzing Software Reviews for Software Quality-Based
Ranking

Analyzing Software Reviews for Software Quality-Based
Ranking
✔️ User Reviews
36
2016 Peng et al.

An Approach of Extracting Feature Requests
from App Reviews

An Approach of Extracting Feature Requests
from App Reviews
✔️ User Reviews
37
2018 Dhinakaran et al.

App Review Analysis via Active Learning

App Review Analysis via Active Learning
✔️ User Reviews
38
2015 Hoon et al.

App Reviews: Breaking the User and
Developer Language Barrier

App Reviews: Breaking the User and
Developer Language Barrier
✔️ User Reviews
39
2016 Keertipati et al.

Approaches for Prioritizing Feature Improvements Extracted
from App Reviews

Approaches for Prioritizing Feature Improvements Extracted
from App Reviews
✔️ User Reviews
40
2018 Nayebi et al.

App Store Mining Is Not Enough
for App Improvement

App Store Mining Is Not Enough
for App Improvement
✔️ User Reviews, Twitter
41
2016 Panichella et al.

ARdoc: App Reviews Development Oriented Classifier

ARdoc: App Reviews Development Oriented Classifier
✔️ User Reviews
42
2014 Chen et al.

AR-miner: Mining Informative Reviews for Developers
from Mobile App Marketplace

AR-miner: Mining Informative Reviews for Developers
from Mobile App Marketplace
✔️ User Reviews
43
2013 Zou et al.

Assessing Software Quality through Web Comment
Search and Analysis

Assessing Software Quality through Web Comment
Search and Analysis
✔️ User Reviews
44
2017 Licorish et al.

Attributes that Predict Which Features to
Fix: Lessons for App Store Mining

Attributes that Predict Which Features to
Fix: Lessons for App Store Mining
✔️ User Reviews
45 Innovations in Systems and
Software Engineering
2017 Pandey et al.

Automated Classification of Software Issue Reports
Using Machine Learning Techniques: An Empirical
Study

Automated Classification of Software Issue Reports
Using Machine Learning Techniques: An Empirical
Study
✔️ Not related to sentiment analysis
46
2017 Kurtanovic and Maalej

Automatically Classifying Functional and Non-Functional Requirements
Using Supervised Machine Learning

Automatically Classifying Functional and Non-Functional Requirements
Using Supervised Machine Learning
✔️ Requirements classification
47
2017 Deocadez et al.

Automatically Classifying Requirements from App Stores:
A Preliminary Study

Automatically Classifying Requirements from App Stores:
A Preliminary Study
✔️ Requirements classification, User Reviews
48
2018 Chuanyi et al.

Automatically Classifying User Requests in Crowdsourcing
Requirements Engineering

Automatically Classifying User Requests in Crowdsourcing
Requirements Engineering
✔️ Requirements classification
49
2013 Denkharghani and Yilmaz

Automatically Identifying a Software Product's Quality
Attributes Through Sentiment Analysis of Tweets

Automatically Identifying a Software Product's Quality
Attributes Through Sentiment Analysis of Tweets
✔️ User Reviews, Twitter
50
2019 Kilani et al.

Automatic Classification of Apps Reviews for
Requirement Engineering: Exploring the Customers Need
from Healthcare Applications

Automatic Classification of Apps Reviews for
Requirement Engineering: Exploring the Customers Need
from Healthcare Applications
✔️ User Reviews
51
2017 Lu and Liang

Automatic Classification of Non-Functional Requirements from
Augmented App User Reviews

Automatic Classification of Non-Functional Requirements from
Augmented App User Reviews
✔️ User Reviews
52
2019 Li et al.

Automatic Identification of Assumptions from the
Hibernate Developer Mailing List

Automatic Identification of Assumptions from the
Hibernate Developer Mailing List
✔️ Assumption mining: assumption and non-assumption
53
2019 Phetrungnapha and Senivongse

Classification of Mobile Application User Reviews
for Generating Tickets on Issue Tracking
System

Classification of Mobile Application User Reviews
for Generating Tickets on Issue Tracking
System
✔️ User Reviews
54
2019 Stanik et al.

Classifying Multilingual User Feedback Using Traditional
Machine Learning and Deep Learning

Classifying Multilingual User Feedback Using Traditional
Machine Learning and Deep Learning
✔️ User Reviews
55 IET Software 2020 Liu et al.

Combining Goal Model with Reviews for
Supporting the Evolution of Apps

Combining Goal Model with Reviews for
Supporting the Evolution of Apps
✔️ User Reviews
56
2017 Ebert et al.

Confusion Detection in Code Reviews

Confusion Detection in Code Reviews
✔️ Confusion detection, not related to sentiment analysis
57 Concurrency and Computation: Practice
and Experience
2019 Ben-Abdallah et al.

CROSA: Context-Aware Cloud Service Ranking Approach
Using Online Reviews Based on Sentiment
Analysis

CROSA: Context-Aware Cloud Service Ranking Approach
Using Online Reviews Based on Sentiment
Analysis
✔️ User Reviews
58
2017 Nayebi et al.

Crowdsourced Exploration of Mobile App Features

Crowdsourced Exploration of Mobile App Features
✔️ User Reviews
59
2015 Di Sorbo et al.

Development Emails Content Analyzer: Intention Mining
in Developer Discussions

Development Emails Content Analyzer: Intention Mining
in Developer Discussions
✔️ Content extraction, not related to sentiment analysis
60
2018 Abad et al.

ELICA: An Automated Tool for Dynamic
Extraction of Requirements Relevant Information

ELICA: An Automated Tool for Dynamic
Extraction of Requirements Relevant Information
✔️ Emotion recognition from oral conversations
61
2019 Gao et al.

Emerging App Issue Identification from User
Feedback: Experience on WeChat

Emerging App Issue Identification from User
Feedback: Experience on WeChat
✔️ User Reviews
62
2015 Guzman et al.

Ensemble Methods for App Review Classification:
An Approach for Software Evolution

Ensemble Methods for App Review Classification:
An Approach for Software Evolution
✔️ User Reviews
63
2016 Qian et al.

Evaluating Quality-in-Use of FLOSS Through Analyzing
User Reviews

Evaluating Quality-in-Use of FLOSS Through Analyzing
User Reviews
✔️ User Reviews
64
2013 Leopairote et al.

Evaluating Software Quality in Use Using
User Reviews Mining

Evaluating Software Quality in Use Using
User Reviews Mining
✔️ User Reviews
65
2019 Wang et al.

Extracting API Tips from Developer Question
and Answer Websites

Extracting API Tips from Developer Question
and Answer Websites
✔️ Content extraction, not related to sentiment analysis
66
2017 Wang et al.

Extracting User-Reported Mobile Application Defects from
Online Reviews

Extracting User-Reported Mobile Application Defects from
Online Reviews
✔️ User Reviews
67
2014 Guzman et al.

Fave: Visualizing User Feedback for Software
Evolution

Fave: Visualizing User Feedback for Software
Evolution
✔️ User Reviews
68
2016 Shah et al.

Feature-Based Evaluation of Competing Apps

Feature-Based Evaluation of Competing Apps
✔️ User Reviews
69
2016 Benjamin Matthies

Feature-Based Sentiment Analysis of Codified Project
Knowledge: A Dictionary Approach

Feature-Based Sentiment Analysis of Codified Project
Knowledge: A Dictionary Approach
✔️ Not specific to the software engineering domain
70
2014 Jiang et al.

For User-Driven Software Evolution: Requirements Elicitation
Derived from Mining Online Reviews

For User-Driven Software Evolution: Requirements Elicitation
Derived from Mining Online Reviews
✔️ User Reviews
71
2019 Nicolai et al.

Healthcare Android Apps: A Tale of
the Customers' Perspective

Healthcare Android Apps: A Tale of
the Customers' Perspective
✔️ User Reviews
72
2018 Gao et al.

INFAR: Insight Extraction from App Reviews

INFAR: Insight Extraction from App Reviews
✔️ User Reviews
73 IEEE Access 2019 Liu et al.

Information Recommendation Based on Domain Knowledge
in App Descriptions for Improving the
Quality of Requirements

Information Recommendation Based on Domain Knowledge
in App Descriptions for Improving the
Quality of Requirements
✔️ Requirements classification
74
2019 Scalabrino et al.

Listening to the Crowd for the
Release Planning of Mobile Apps

Listening to the Crowd for the
Release Planning of Mobile Apps
✔️ User Reviews
75
2016 Iacob et al.

MARAM: Tool Support for Mobile App
Review Management

MARAM: Tool Support for Mobile App
Review Management
✔️ User Reviews
76
2017 Jha and Mahmoud

MARC: A Mobile Application Review Classifier.

MARC: A Mobile Application Review Classifier.
✔️ User Reviews
77
2018 Liu et al.

Mining Android App Descriptions for Permission
Requirements Recommendation

Mining Android App Descriptions for Permission
Requirements Recommendation
✔️ Content extraction, not related to sentiment analysis
78
2019 Jha and Mahmoud

Mining Non-Functional Requirements from App Store
Reviews

Mining Non-Functional Requirements from App Store
Reviews
✔️ User Reviews
79
2017 Williams and Mahmoud

Mining Twitter Feeds for Software User
Requirements

Mining Twitter Feeds for Software User
Requirements
✔️ User Reviews, Twitter
80
2017 Kurtanovic and Maalej

Mining User Rationale from Software Reviews

Mining User Rationale from Software Reviews
✔️ User Reviews
81
2016 Maalej et al.

On the Automatic Classification of App
Reviews

On the Automatic Classification of App
Reviews
✔️ User Reviews
82
2019 Fucci et al.

On Using Machine Learning to Identify
Knowledge in API Reference Documentation

On Using Machine Learning to Identify
Knowledge in API Reference Documentation
✔️ Not related to sentiment analysis
83 Turkish Journal of Electrical
Engineering and Computer Science
2016 Ikram et al.

Open Source Software Adoption Evaluation Through
Feature Level Sentiment Analysis Using Twitter
Data

Open Source Software Adoption Evaluation Through
Feature Level Sentiment Analysis Using Twitter
Data
✔️ User Reviews, Twitter
84
2018 Durelli et al.

Please Please Me: Does the Presence
of Test Cases Influence Mobile App
Users' Satisfaction?

Please Please Me: Does the Presence
of Test Cases Influence Mobile App
Users' Satisfaction?
✔️ User Reviews
85
2017 Arianto et al.

Quality Measurement of Android Messaging Application
Based on User Experience in Microblog

Quality Measurement of Android Messaging Application
Based on User Experience in Microblog
✔️ User Reviews, Twitter
86
2017 Alkadhi et al.

Rationale in Development Chat Messages: An
Exploratory Study

Rationale in Development Chat Messages: An
Exploratory Study
✔️ Content extraction, Not related to sentiment analysis
87
2019 Jiang et al.

Recommending New Features from Mobile App
Descriptions

Recommending New Features from Mobile App
Descriptions
✔️ Content extraction, Not related to sentiment analysis
88
2019 Dalpiaz and Parente

RE-SWOT: From User Feedback to Requirements
via Competitor Analysis

RE-SWOT: From User Feedback to Requirements
via Competitor Analysis
✔️ Requirements classification
89 Computers & Security 2019 Hatamian et al.

Revealing the Unrevealed: Mining Smartphone Users
Privacy Perception on App Markets

Revealing the Unrevealed: Mining Smartphone Users
Privacy Perception on App Markets
✔️ User Reviews
90
2017 Ali et al.

Same App, Different App Stores: A
Comparative Study

Same App, Different App Stores: A
Comparative Study
✔️ User Reviews
91
2018 Buchan et al.

Semi-Automated Extraction of New Requirements from
Online Reviews for Software Product Evolution

Semi-Automated Extraction of New Requirements from
Online Reviews for Software Product Evolution
✔️ User Reviews
92
2016 Alkalbani et al.

Sentiment Analysis and Classification for Software
as a Service Reviews

Sentiment Analysis and Classification for Software
as a Service Reviews
✔️ User Reviews
93 Future Internet 2019 Zhao and Zhao

Sentiment Analysis Based Requirement Evolution Prediction

Sentiment Analysis Based Requirement Evolution Prediction
✔️ User Reviews
94
2016 Liu et al.

Stratify Mobile App Reviews: E-LDA Model
Based on Hot "Entity" Discovery

Stratify Mobile App Reviews: E-LDA Model
Based on Hot "Entity" Discovery
✔️ User Reviews
95
2018 Hu et al.

Studying the Consistency of Star Ratings
and Reviews of Popular Free Hybrid
Android and iOS Apps

Studying the Consistency of Star Ratings
and Reviews of Popular Free Hybrid
Android and iOS Apps
✔️ User Reviews
96
2019 Abad et al.

Supporting Analysts by Dynamic Extraction and
Classification of Requirements-Related Knowledge

Supporting Analysts by Dynamic Extraction and
Classification of Requirements-Related Knowledge
✔️ Requirements classification, not related to sentiment analysis
97
2018 Huang et al.

Tell Them Apart: Distilling Technology Differences
from Crowd-Scale Comparison Discussions

Tell Them Apart: Distilling Technology Differences
from Crowd-Scale Comparison Discussions
✔️ Not related to sentiment analysis
98
2016 Mercado et al.

The Impact of Cross-Platform Development Approaches
for Mobile Applications from the User's
Perspective

The Impact of Cross-Platform Development Approaches
for Mobile Applications from the User's
Perspective
✔️ User Reviews
99
2018 Yin and Pfahl

The OIRE Method-Overview and Initial Validation

The OIRE Method-Overview and Initial Validation
✔️ User Reviews, Requirements classification
100
2019 Kallis et al.

Ticket Tagger: Machine Learning Driven Issue
Classification

Ticket Tagger: Machine Learning Driven Issue
Classification
✔️ Content extraction, not related to sentiment analysis
101 Journal of the Association
for Information Science and
Technology
2018 Li et al.

To Do or Not To Do:
Distill Crowdsourced Negative Caveats to Augment
API Documentation

To Do or Not To Do:
Distill Crowdsourced Negative Caveats to Augment
API Documentation
✔️ Content extraction, not related to sentiment analysis
102
2015 Vu et al.

Tool Support for Analyzing Mobile App
Reviews

Tool Support for Analyzing Mobile App
Reviews
✔️ User Reviews
103
2021 Noei et al.

Too Many User-Reviews, What Should App
Developers Look at First?

Too Many User-Reviews, What Should App
Developers Look at First?
✔️ User Reviews
104
2019 Truelove et al.

Topics of Concern: Identifying User Issues
in Reviews of IoT Apps and
Devices

Topics of Concern: Identifying User Issues
in Reviews of IoT Apps and
Devices
✔️ User Reviews
105 Procedia Computer Science 2019 2019 Fazyeli et al.

Towards Auto-labelling Issue Reports for Pull-Based
Software Development using Text Mining Approach

Towards Auto-labelling Issue Reports for Pull-Based
Software Development using Text Mining Approach
✔️ Content extraction, not related to sentiment analysis
106
2017 Shi et al.

Understanding Feature Requests by Leveraging Fuzzy
Method and Linguistic Analysis

Understanding Feature Requests by Leveraging Fuzzy
Method and Linguistic Analysis
✔️ Content extraction, not related to sentiment analysis
107
2019 Shah et al.

Using App Reviews for Competitive Analysis:
Tool Support

Using App Reviews for Competitive Analysis:
Tool Support
✔️ User Reviews
108
2018 Jha and Mahmoud

Using Frame Semantics for Classifying and
Summarizing Application Store Reviews

Using Frame Semantics for Classifying and
Summarizing Application Store Reviews
✔️ User Reviews
109
2012 Brooks and Swigger

Using Sentiment Analysis to Measure the
Effects of Leaders in Global Software
Development

Using Sentiment Analysis to Measure the
Effects of Leaders in Global Software
Development
✔️ Not really sentiment analysis, before 2013
110 Applied Sciences 2019 Ali and Hong

Value-Oriented Requirements: Eliciting Domain Requirements from
Social Network Services to Evolve Software
Product Lines

Value-Oriented Requirements: Eliciting Domain Requirements from
Social Network Services to Evolve Software
Product Lines
✔️ User Reviews, Twitter
111
2019 Werner et al.

What Can the Sentiment of a
Software Requirements Specification Document Tell Us?

What Can the Sentiment of a
Software Requirements Specification Document Tell Us?
✔️ Requirements Specification Documents
112
2015 Gu and Kim

"What Parts of Your Apps Are
Loved by Users?"

"What Parts of Your Apps Are
Loved by Users?"
✔️ User Reviews
113
2018 Huebner et al.

What People Like in Mobile Finance
Apps: An Analysis of User Reviews

What People Like in Mobile Finance
Apps: An Analysis of User Reviews
✔️ User Reviews
114
2016 DiSorbo et al.

What Would Users Change in My
App? Summarizing App Reviews for Recommending
Software Changes

What Would Users Change in My
App? Summarizing App Reviews for Recommending
Software Changes
✔️ User Reviews
115
2017 Bakiu and Guzman

Which Feature Is Unusable? Detecting Usability
and User Experience Issues from User
Reviews

Which Feature Is Unusable? Detecting Usability
and User Experience Issues from User
Reviews
✔️ User Reviews
116
2013 Fu et al.

Why People Hate Your App: Making
Sense of User Feedback in a
Mobile App Store

Why People Hate Your App: Making
Sense of User Feedback in a
Mobile App Store
✔️ User Reviews
117
2019 Ali et al.

Your Opinions Let Us Know: Mining
Social Network Sites to Evolve Software
Product Lines

Your Opinions Let Us Know: Mining
Social Network Sites to Evolve Software
Product Lines
✔️ User Reviews, Twitter
118
2022 Malgaonkar et al.

Prioritizing User Concerns in App Reviews
– A Study of Requests for
New Features, Enhancements and Bug Fixes

Prioritizing User Concerns in App Reviews
– A Study of Requests for
New Features, Enhancements and Bug Fixes
User Reviews
119
2019 Ronchieri et al.

Sentiment Assessment for Software Code Assessment

Sentiment Assessment for Software Code Assessment
Only idea, no concrete method (2 pages paper)
120
2016 Guzman et al.

A Needle in the Haystack: What
Do Twitter Users Say about Software?

A Needle in the Haystack: What
Do Twitter Users Say about Software?
User Reviews, Twitter Only manual labeling
121
2017 Uddin and Khomh

Automatic Summarization of API Reviews

Automatic Summarization of API Reviews
API Reviews = User Reviews
122
2022 Dabrowski et al.

Analysing App Reviews for Software Engineering:
A Systematic Literature Review

Analysing App Reviews for Software Engineering:
A Systematic Literature Review
Literature Review on User Reviews
123
2018 Lin et al.

Two Datasets for Sentiment Analysis in
Software Engineering

Two Datasets for Sentiment Analysis in
Software Engineering
Not really a paper (only 1 page), manually labeled data sets of user reviews and StackOverflow data
124
2015 Ortu et al.

The JIRA Repository Dataset: Understanding Social
Aspects of Software Development

The JIRA Repository Dataset: Understanding Social
Aspects of Software Development
Only provide a JIRA dataset, but no idea and no analysis
125
2017 Fountaine and Sharif

Emotional Awareness in Software Development: Theory
and Measurement

Emotional Awareness in Software Development: Theory
and Measurement
Provide a study with EEG, GSR, EyeTracker and ask study participants to perform tasks and label their own emotions
126 IEEE Software 2019 Novielli and Serebrenik

Sentiment and Emotion in Software Engineering

Sentiment and Emotion in Software Engineering
Not a research paper, just an introducation to a special issue by the IEEE Software Guest Editors
127
2019 Alami et al.

Why Does Code Review Work for
Open Source Software Communities?

Why Does Code Review Work for
Open Source Software Communities?
Not related to sentiment analysis (though mentioning negative mood in Linux code reviews)
128
2016 Schneider et al.

Differentiating Communication Styles of Leaders on
the Linux Kernel Mailing List

Differentiating Communication Styles of Leaders on
the Linux Kernel Mailing List
Not related to sentiment analysis, but analyzing expletive content
129
2018 Mäntylä et al.

Natural Language or Not (NLoN) –
A Package for Software

Natural Language or Not (NLoN) –
A Package for Software
Not related to sentiment analysis, but provide an approach to separate natural language from technical text (e.g., source code)
130
2020 Cheriyan et al.

Norm Violation in Online Communities –
A Study of Stack Overflow Comments

Norm Violation in Online Communities –
A Study of Stack Overflow Comments
Not related to sentiment analysis, but identify reasons of norm violations when comments on StackOverflow are deleted (such as swearing)
131
2022 Gunawardena et al.

Destructive Critisism in Software Code Review
Impact Inclusion

Destructive Critisism in Software Code Review
Impact Inclusion
Not related to sentiment analysis,
132
2015 Squire and Gazda

FLOSS as a Source for Profanity
and Insults: Collecting the Data

FLOSS as a Source for Profanity
and Insults: Collecting the Data
Search for profane/insulting words in LKML and provide a dataset of examples.
133
2016 Carillo et al.

Towards Developing a Theory of Toxicity
in the Context of Free/Open Source
Software & Peer Production Communities

Towards Developing a Theory of Toxicity
in the Context of Free/Open Source
Software & Peer Production Communities
Develop ideas towards a theory on toxicity in OSS, only ideas, only theory, no practical parts
134
2016 Carillo and Marsan

“The Dose Makes the Poison” -
Exploring the Toxicity Phenomenon in Online
Communities

“The Dose Makes the Poison” -
Exploring the Toxicity Phenomenon in Online
Communities
Develop a theory on toxicity in OSS and conduct an interview study to empirically validate their theory
135 CoRR abs/1812.04863 2018 Lu et al.

A First Look at Emoji Usage
on GitHub: An Empirical study

A First Look at Emoji Usage
on GitHub: An Empirical study
not peer-reviewed (only published on arXiv)
136
2022 Calefato and Lanubile

Using Personality Detection Tools for Software
Engineering Research: How Far Can We
Go?

Using Personality Detection Tools for Software
Engineering Research: How Far Can We
Go?
Personality Detection is not related to Sentiment
137
2020 Egelman et al.

Predicting Developers’ Negative Feelings about Code
Review

Predicting Developers’ Negative Feelings about Code
Review
Interviews + regression analysis of identified characteristics of pushback
138
2023 Uddin et al.

An Empirical Study of Deep Learning
Sentiment Detection Tools for Software Engineering
in Cross-Plattform Settings

An Empirical Study of Deep Learning
Sentiment Detection Tools for Software Engineering
in Cross-Plattform Settings
Arxiv pre-print of submission, not peer-reviewed yet
Jump to top of page