Lampe & Johnston, SIGGROUP 2005
Citation
Lampe, C. and Johnston, E., "Follow the (Slash) dot: Effects of Feedback on New Members in an Online Community." In International Conference on Supporting Group Work, GROUP '05.
Online Version
Summary
In this paper, Lampe and Johnston look at how new users learn to participate in an online community, specifically with regard to new users who join discussions on the news/discussion site Slashdot. Successful online communities need to attract new members on an ongoing basis, and in order to successfully participate in the community these new users need to learn the relevant guidelines and standards of behavior. Failure to do so can have consequences; the new member may inadvertently violate norms, may be more vulnerable to deception, may be ignored by other users, and may end up increasing information overload. The authors present three possible mechanisms by which new users can learn how to participate in the community: previous experience from education or participation on similar forums, observing current users and their interactions, and by receiving feedback from other members when the new member contributes.
Using server logs obtained from Slashdot, Lampe and Johnston analyzed the activity of 11,079 new users who created accounts on Slashdot between November 1, 2004 and December 6, 2004. Additionally, they surveyed 233 users who had created accounts since November 1, 2004. The survey respondents were recruited via Slashdot's homepage, and were assigned a unique identifier that allowed for matching with the server log data. Out of the 11,079 new users in the dataset, 1763 (16%) made a total of 6467 comments. 55.1% of these users only made one comment; the mean number of comments made was 3.7, and the median was 1.
Users who received no moderation - that is, no one voted their contribution up or down - on their first comment were less likely to make a second comment than were users who did receive moderation, regardless of whether the moderation was positive or negative. However, receiving feedback on the first comment alone isn't necessarily enough to keep a new user posting. 30% of users who made a second comment after receiving feedback on their first comment stopped commenting if the second comment failed to receive feedback. Though the authors caution that they do not have enough data to support a strong causal argument, there does seem to be an association between the the direction of feedback and the ratings future comments will receive.
Survey respondents indicated that they felt they knew how to recognize and write good comments on Slashdot, but this belief wasn't correlated with the actual scores their comments received. Further, measures of previous experience and observation were not predictive of the scores a user's first comment would receive on the site. With respect to posting second comments, receiving a positive score on the first comment was correlated with a shorter time gap between the posting of the first and second comments. Additionally, users who spent more time observing the site before commenting and who also made more frequent page views (as indicated by the server logs) were more likely to make multiple comments.
The authors note that the nature of the survey log data makes it difficult to tell who is truly a "new user" - a user who has observed the site anonymously for a year before signing up isn't a "new user" in the strict sense of the term, but with only two months of log data (as well as the potential for a user to be using multiple IP addresses over time) there's no way to tell how long a user has really been reading the site. They speculate that more qualitative work may be able to address this problem. They also point out that the results may not be entirely generalizable to other online communities, but the broad finding that users who receive feedback from more seasoned community members participate differently than users who do not receive this feedback may be more generalizable.
