Joyce & Kraut, JCMC 2006
Citation
Joyce, E., and Kraut, R. E. (2006). "Predicting continued participation in newsgroups." Journal of Computer-Mediated Communication, 11(3).
Online Version
Summary
In this paper, Joyce & Kraut look at the factors that make newcomers to an online group more likely to continue participation in the group, with a particular focus on Usenet newsgroups. Turnover in online communities is very high, and the authors hope to shed light on what influences people to commit to participating in an online community. Their analysis tests six hypotheses:
| Hypothesis | Claim | Support |
|---|---|---|
| 1 | Receiving a response to an initial post will increase the likelihood that the poster will post again. | Supported |
| 2 | Receiving a response that provides information rather than asks a question will increase the likelihood that the poster will post again. | Disconfirmed |
| 3 | Receiving a response will be more likely to encourage someone to post again if the response is a direct answer to an initial question. | Not supported |
| 4 | Receiving a response that has a positive emotional tone rather than a negative one will increase the likelihood that a newcomer will post again. | Not supported |
| 5 | Receiving a response that affirms what the poster said rather than disagreeing with it will increase the likelihood that a newcomer will post again. | Not supported |
| 6 | Receiving a longer reply to an initial post rather than a shorter one will increase the likelihood that a newcomer will post again | Not supported |
The authors looked at posts in six Usenet newsgroups: netscape.public.mozilla.ui (task-based, primarily male), alt.support.diet (support-based, primarily female), alt.support.cancer.breast (support-based, primarily female), alt.politics.usa.constitution.gun—rights (interest-based), alt.sports.hockey.nhl.ny-rangers (interest-based), and alt.baldspot (support-based, primarily male). Six months of data was collected for each group, and new users were identified for the first three months of collection for each group. While this short data collection period might mean that people who made second posts more than 3-6 months after their first post would be missed, the authors claim that this risk is low based on previous research showing that only 4.6% of users posted for a period longer than three months. However, this finding was based on data from one newsgroup - the Mozilla newsgroup - and may not apply to other types of newsgroups.
Using this collection methodology, they identified 2,777 first posts (1,020 from the Mozilla group, 426 from the baldness group, 541 from the diet group, 164 from the breast cancer group, 224 from the gun rights group, and 302 in the hockey group). 44% of these users did go on to make a second post. On average, the newcomers posted to the groups 7.5 time, with a median of 1 post - a highly skewed distribution. The survival time between first and last post was also highly skewed. On average, the mean survival time was 45 days, but the median was 0. Most people did not make a second post, and many of those who did make a second post did so within 24 hours.
Joyce & Kraut also looked at the replies the newcomers received. 61% of newcomers received a reply to their first post, and several factors appeared to be correlated with the likelihood of receiving a reply. First, posts to certain groups were more likely to receive replies than others. The participants in the Mozilla group in particular were significantly less likely to respond to a first post than the participants in other groups - only 55% of first posts to the Mozilla group received replies, compared to an average of 70% in the non-Mozilla groups. The length and form of the first post were also significantly correlated with replies. Increasing the word count of a first post also increased the probability of a reply by 14.4%, and asking a question increased the likelihood of a reply by 16.4%. There was no association between the emotional tone or agreeableness of a first post and the likelihood of a reply.
The authors also attempted to predict whether the newcomers would make second posts, testing four possible models. The first model included only the post characteristics (e.g. emotional tone, question, etc) and the group the message was posted to. The groups were found to have different base reply rates. The breast cancer group and the gun rights group in particular had low second post rates (38.8% and 37.8% respectively). Longer first posts were positively correlated with making second posts, and posters who indicated higher levels of agreement in their first posts were slightly more likely to post again.
The second model added replies to the first post as a predictive variable, improving the fit of the model. Receiving a reply to the first post was correlated with a 12.4% increase in the likelihood of a second post, even with differences between post characteristics and groups held constant.
Model 3 introduced the characteristics of the replies as predictive variables, also improving the model fit among the subset of newcomers who received replies. Replies that asked a question were correlated with a 5% likelihood of posting again on top of the increase already associated with receiving a reply at all. The emotional tone of the reply was not correlated with the likelihood of a second post.
Model 4 looked at the message sequence; specifically, whether an initial question followed by a reply offering a statement, opinion, or advice was associated with a higher likelihood of making a second post. Adding these variables to Model 3 did not improve the model fit, and the sequence was not a significant predictor of a second post.
Finally, Joyce & Kraut attempted to predict characteristics of the replies. Controlling for the group, they found that replies to longer posts were themselves longer. This is not an artifact of a tendency to quote the previous post in a reply, as the data had been cleaned to remove these quotes prior to analysis. While most replies to first posts were statements (89%), first posts that were questions were 6% more likely to receive statements as replies than first posts that were statements. The authors were surprised to note that there seemed to be a decline in positive emotional expression in replies associated with greater emotional expression in the initial post.
While the authors note that this research is preliminary, they say that it does provide evidence that interaction increases newcomers' willingness to return to the group to make a second post, a step toward becoming committed to the group as a whole.
