Dawn Felagund (![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png)
dawn_felagund) wrote in ![[community profile]](https://www.dreamwidth.org/img/silk/identity/community.png)
b2mem2017-03-23 09:17 pm
dawn_felagund) wrote in b2mem2017-03-23 09:17 pm
Entry tags:
"Review Plz? Feedback Behavior in the Tolkien Fanfic Community" by Dawn Felagund
B2MeM Prompt and Path: Meta on Fandom and Multimedia (orange/nonfiction path)
Format: essay
Genre: nonfiction/meta
Rating: General
Warnings: n/a
Characters: n/a
Pairings: n/a
Summary: A look at Tolkien Fan Fiction Survey data on who leaves feedback and how often, and how many readers wish they did more.
A little over a year ago, I ran an online survey about Tolkien fan fiction as part of my ongoing research on the history and culture of the Tolkien fan fiction community. (Read more about the Tolkien Fan Fiction survey here.) I have been slowly posting the results of the survey over the last year or so. For the orange/nonfiction path prompt "Meta on Fandom," I decided to dig into a topic from the survey based on what other B2MeM participants would like to know more about. People who answer my poll wanted to know more about, "How many readers comment or give feedback on stories? Why do they do this?" I will eventually investigate the other topics as well, most likely over the summer while I'm on break from school.
This essay seeks to answer some basic questions on feedback behavior in the Tolkien fanfic community. Who leaves feedback? How often? I will also begin to look at why people leave feedback, specifically at social pressure to do so. This will be the first post in a series looking at feedback behavior; the series will in all likelihood extent beyond B2MeM; follow![[livejournal.com profile]](https://www.dreamwidth.org/img/external/lj-syndicated.gif)
heretic_lore, my Twitter, or the Tumblr tag #tolkien fan fiction survey for updates related to the survey.
Probably the first question to answer when thinking about commenting is: How often do people comment? I asked the question, "Do you leave comments or other feedback on Tolkien-based fan fiction stories?" Of the 1040 people who answered that question, 75.9% of them said YES.
Now it's important to note the "or other feedback" in the statement. This didn't ask just about comments or reviews; it could have included one-click feedback like kudos or likes as well. If I could go back and do this survey over, I'd likely change this question to distinguish between the two. For now, though, it's what I have to work with.
When we break down this question by the participant's role, the results become more interesting. I looked at the responses of writers versus readers only to this question. Writers were far, far more likely to leave feedback on what they read: 86.5% of writers (n = 635) answered YES compared to 59.3% of readers-only (n = 393). My initial reaction to this information is, "Well, of course, writers would best understand how much feedback matters to other writers." I think that's part of it, but there are probably other factors involved as well.
Interestingly, 13.5% of writers responded that they did not leave feedback on stories that they read. I find this group intensely interesting, and a future post will look specifically at this group of participants.
Of course, a participant could have left a single comment or liked one story posted on Tumblr and answered YES to the above. Any author can tell you that three-fourths of their readers do not leave feedback on a specific story; many of my stories, based on click counts, would have hundreds of comments, and it is rare for me to exceed ten comments, and I receive more comments than most authors. (The highest percentage of kudos-per-click on my AO3 stories is about 19%.) So what percentage of stories do readers leave comments on?
I asked participants to "Estimate the percentage of Tolkien-based fan fiction stories that you leave comments or other feedback on." Those who responded with a number greater than zero left comments on a median average of 30% of stories.
Breaking down the data a little further also shows that readers willing to leave feedback tend to leave it relatively infrequently. More than half of participants (54.7%) left feedback on one out of three stories, or less. The graph below shows the number of participants who left different amounts of feedback. The numbers drop steadily until spiking briefly around 50%--likely because someone is more likely to respond with 50% rather than dithering slightly to either side of that number; after the 50% mark, the numbers hang rather steadily. There is a small resurgence among the participants that, in my mind as I worked on these numbers, I termed "unicorns": those who left feedback on almost everything they read.
It's important to note that these numbers are likely slightly inflated. Even in anonymous surveys, like this one, there is a tendency to overstate positive behaviors, like one's habit of leaving feedback on the fiction one reads for free. To support this point in this particular survey, several participants left brief comments on their answers, suggesting that they'd recently increased their feedback due to growing awareness of its value to authors or that they felt they needed to do more; some participants offered excuses (such as English as a second language) or responded to a perceived low number with self-effacing humor (like a :P emoticon). In addition, numbers were potentially inflated because one is more likely to remember the stories one takes time to leave feedback on, especially comments. A ficlet skimmed quickly on Tumblr, for instance, is more likely to be forgotten than the same ficlet on AO3 where the reader leaves a one-sentence comment or even clicks a kudos; especially the comment requires more careful reading.
Looking at actual feedback numbers supports that 30% is likely inflated. I chose ten stories on AO3 from the section "The Silmarillion and other histories of Middle-earth -- J.R.R. Tolkien." The stories had been posted just over a week ago and were on the sixth page of results, so they had likely received the first heavy wave of readership. Since most AO3 readers who leave comments, in my experience, also leave a kudos (and since comment counts on AO3 also include author replies and further conversation on a story), then I looked just at kudos. For those ten stories, the kudos-per-click percentage was a median average of 9.2%, spanning a range of 1.7% to 26.2%: nowhere near the self-reported 30% rate from the survey.
As implied above, there is a degree of social pressure to leave feedback on stories. I was curious if readers felt they needed to do this more, or if they were happy with the current amount of feedback they left, so I looked at responses to the statement "I want to leave comments and other feedback more often on the stories I read." Participants had five options to choose from: Strongly Agree, Agree, Disagree, Strongly Disagree, and No Opinion/Not Sure.
Overwhelmingly, participants wanted to leave more feedback: 77.6% agreed or strongly agreed with the statement. In other words, three out of four readers think they need to leave more feedback. Fewer than one in ten (8.89%) disagreed or strongly disagreed. Of the latter category, a mere eight participants chose this option.
I looked closer at that group: participants who strongly disagreed that they wanted to leave feedback more often. Of those eight participants, three were part of the unicorn group that left feedback on almost everything they read; it's understandable why they felt they didn't need to do more! One participant left feedback a reported 70% of the time--still a relatively high number--so about half responded with Strongly Disagree because they really can't do much more than they already are. One person did not provide a response for the amount of feedback they left but answered NO to the question "Do you leave comments or other feedback on Tolkien-based fan fiction stories?" Three entered "zero" for the amount of feedback they left; these four responses felt somewhat defiant to me given how contrary to correct fandom etiquette it was. (I would say that this etiquette demands that one either leave feedback or feel badly for not doing so.)
I was also curious about the unicorn group: those who left feedback a reported 90 to 100% of the time. Despite leaving feedback on just about everything they read, 65% still agreed or strongly agreed that they wanted to do more. (Including those who reported that they left feedback 100% of the time: 67% of these participants still wanted to do more, including five who strongly agreed with the statement.)
Of the unicorn group, 21.7% chose No Opinion/Not Sure, a percentage much higher than the 13.5% of all participants who chose this option for this statement. I generally avoid making inferences about the No Opinion/Not Sure participants--there are a lot of reasons why people might choose this option, including that they truly do not understand what the statement is asking--but this discrepancy is too interesting to pass up hypothesizing about a little. I suspect that these respondents know that they are going above and beyond the majority of fandom but still feel uncomfortable stating directly that they don't think they need to do more. Choosing No Opinion/Not Sure is quite possibly the more socially acceptable option: a way to circumspectly admit that one really can't do much more.
The unicorns are an interesting group. Why do so many of them--about two out of three--feel that they need to do more? It is possible that the feedback they are leaving is mostly or entirely kudos or other one-click feedback, and they feel they should be writing more comments. (Readers who leave kudos on everything they read are a well-reported phenomenon on AO3; one participant even commented that they "kudos" everything they read.) It is also possible that social norms in fandom dictate that one should always be striving to improve on how much feedback one leaves on stories, and these readers feel that guilty gnawing even though they already are leaving feedback on almost everything they pick up. Here, I can turn to personal experience: I am in the unicorn group myself, leaving feedback on everything I read (in the form of comments) except when I regularly have to skim stories as part of my mod duties on the sites I run. (Sometimes even then I get sucked into a story and comment.) Despite the number of comments I leave, despite the hours of work I do in the fandom each week, I still feel guilty over not commenting on more of those stories skimmed in the course of daily site business. (I also feel guilty for not reading more, period.)
From this data, it is possible to draw a few conclusions:
If you have a question you'd like to see data on, please do share! Next time, I will likely look at why readers decide to leave feedback on a story, but if there's a topic or question you're interested in seeing analyzed and discussed in greater depth, let me know!
Format: essay
Genre: nonfiction/meta
Rating: General
Warnings: n/a
Characters: n/a
Pairings: n/a
Summary: A look at Tolkien Fan Fiction Survey data on who leaves feedback and how often, and how many readers wish they did more.
Review Plz? Feedback Behavior in the Tolkien Fanfic Community
A little over a year ago, I ran an online survey about Tolkien fan fiction as part of my ongoing research on the history and culture of the Tolkien fan fiction community. (Read more about the Tolkien Fan Fiction survey here.) I have been slowly posting the results of the survey over the last year or so. For the orange/nonfiction path prompt "Meta on Fandom," I decided to dig into a topic from the survey based on what other B2MeM participants would like to know more about. People who answer my poll wanted to know more about, "How many readers comment or give feedback on stories? Why do they do this?" I will eventually investigate the other topics as well, most likely over the summer while I'm on break from school.
This essay seeks to answer some basic questions on feedback behavior in the Tolkien fanfic community. Who leaves feedback? How often? I will also begin to look at why people leave feedback, specifically at social pressure to do so. This will be the first post in a series looking at feedback behavior; the series will in all likelihood extent beyond B2MeM; follow
heretic_lore, my Twitter, or the Tumblr tag #tolkien fan fiction survey for updates related to the survey.Probably the first question to answer when thinking about commenting is: How often do people comment? I asked the question, "Do you leave comments or other feedback on Tolkien-based fan fiction stories?" Of the 1040 people who answered that question, 75.9% of them said YES.
Now it's important to note the "or other feedback" in the statement. This didn't ask just about comments or reviews; it could have included one-click feedback like kudos or likes as well. If I could go back and do this survey over, I'd likely change this question to distinguish between the two. For now, though, it's what I have to work with.
Who Leaves Feedback?
When we break down this question by the participant's role, the results become more interesting. I looked at the responses of writers versus readers only to this question. Writers were far, far more likely to leave feedback on what they read: 86.5% of writers (n = 635) answered YES compared to 59.3% of readers-only (n = 393). My initial reaction to this information is, "Well, of course, writers would best understand how much feedback matters to other writers." I think that's part of it, but there are probably other factors involved as well.
- Writers are more likely to belong to the sites where they read. Many sites (SWG and MPTT, for example) do not allow comments from anonymous users.
- Writers are more likely to be comfortable enough with English (or the language the story is written in) to be able to write a comment. I can read tolerably in Spanish, for example, but would never dare attempt to comment on something written in Spanish.
- Writers are more likely to simply know what to say in a comment. They know what they like to hear on their own stories. They know what goes into crafting a story and are possibly more accustomed to noticing a characterization detail or a particularly good turn of phrase: the kind of thing you'd mention in a comment.
Interestingly, 13.5% of writers responded that they did not leave feedback on stories that they read. I find this group intensely interesting, and a future post will look specifically at this group of participants.
How Often Do Readers Leave Feedback?
Of course, a participant could have left a single comment or liked one story posted on Tumblr and answered YES to the above. Any author can tell you that three-fourths of their readers do not leave feedback on a specific story; many of my stories, based on click counts, would have hundreds of comments, and it is rare for me to exceed ten comments, and I receive more comments than most authors. (The highest percentage of kudos-per-click on my AO3 stories is about 19%.) So what percentage of stories do readers leave comments on?
I asked participants to "Estimate the percentage of Tolkien-based fan fiction stories that you leave comments or other feedback on." Those who responded with a number greater than zero left comments on a median average of 30% of stories.
Breaking down the data a little further also shows that readers willing to leave feedback tend to leave it relatively infrequently. More than half of participants (54.7%) left feedback on one out of three stories, or less. The graph below shows the number of participants who left different amounts of feedback. The numbers drop steadily until spiking briefly around 50%--likely because someone is more likely to respond with 50% rather than dithering slightly to either side of that number; after the 50% mark, the numbers hang rather steadily. There is a small resurgence among the participants that, in my mind as I worked on these numbers, I termed "unicorns": those who left feedback on almost everything they read.
It's important to note that these numbers are likely slightly inflated. Even in anonymous surveys, like this one, there is a tendency to overstate positive behaviors, like one's habit of leaving feedback on the fiction one reads for free. To support this point in this particular survey, several participants left brief comments on their answers, suggesting that they'd recently increased their feedback due to growing awareness of its value to authors or that they felt they needed to do more; some participants offered excuses (such as English as a second language) or responded to a perceived low number with self-effacing humor (like a :P emoticon). In addition, numbers were potentially inflated because one is more likely to remember the stories one takes time to leave feedback on, especially comments. A ficlet skimmed quickly on Tumblr, for instance, is more likely to be forgotten than the same ficlet on AO3 where the reader leaves a one-sentence comment or even clicks a kudos; especially the comment requires more careful reading.
Looking at actual feedback numbers supports that 30% is likely inflated. I chose ten stories on AO3 from the section "The Silmarillion and other histories of Middle-earth -- J.R.R. Tolkien." The stories had been posted just over a week ago and were on the sixth page of results, so they had likely received the first heavy wave of readership. Since most AO3 readers who leave comments, in my experience, also leave a kudos (and since comment counts on AO3 also include author replies and further conversation on a story), then I looked just at kudos. For those ten stories, the kudos-per-click percentage was a median average of 9.2%, spanning a range of 1.7% to 26.2%: nowhere near the self-reported 30% rate from the survey.
Do Readers Want to Leave More Feedback?
As implied above, there is a degree of social pressure to leave feedback on stories. I was curious if readers felt they needed to do this more, or if they were happy with the current amount of feedback they left, so I looked at responses to the statement "I want to leave comments and other feedback more often on the stories I read." Participants had five options to choose from: Strongly Agree, Agree, Disagree, Strongly Disagree, and No Opinion/Not Sure.
Overwhelmingly, participants wanted to leave more feedback: 77.6% agreed or strongly agreed with the statement. In other words, three out of four readers think they need to leave more feedback. Fewer than one in ten (8.89%) disagreed or strongly disagreed. Of the latter category, a mere eight participants chose this option.
I looked closer at that group: participants who strongly disagreed that they wanted to leave feedback more often. Of those eight participants, three were part of the unicorn group that left feedback on almost everything they read; it's understandable why they felt they didn't need to do more! One participant left feedback a reported 70% of the time--still a relatively high number--so about half responded with Strongly Disagree because they really can't do much more than they already are. One person did not provide a response for the amount of feedback they left but answered NO to the question "Do you leave comments or other feedback on Tolkien-based fan fiction stories?" Three entered "zero" for the amount of feedback they left; these four responses felt somewhat defiant to me given how contrary to correct fandom etiquette it was. (I would say that this etiquette demands that one either leave feedback or feel badly for not doing so.)
I was also curious about the unicorn group: those who left feedback a reported 90 to 100% of the time. Despite leaving feedback on just about everything they read, 65% still agreed or strongly agreed that they wanted to do more. (Including those who reported that they left feedback 100% of the time: 67% of these participants still wanted to do more, including five who strongly agreed with the statement.)
Of the unicorn group, 21.7% chose No Opinion/Not Sure, a percentage much higher than the 13.5% of all participants who chose this option for this statement. I generally avoid making inferences about the No Opinion/Not Sure participants--there are a lot of reasons why people might choose this option, including that they truly do not understand what the statement is asking--but this discrepancy is too interesting to pass up hypothesizing about a little. I suspect that these respondents know that they are going above and beyond the majority of fandom but still feel uncomfortable stating directly that they don't think they need to do more. Choosing No Opinion/Not Sure is quite possibly the more socially acceptable option: a way to circumspectly admit that one really can't do much more.
The unicorns are an interesting group. Why do so many of them--about two out of three--feel that they need to do more? It is possible that the feedback they are leaving is mostly or entirely kudos or other one-click feedback, and they feel they should be writing more comments. (Readers who leave kudos on everything they read are a well-reported phenomenon on AO3; one participant even commented that they "kudos" everything they read.) It is also possible that social norms in fandom dictate that one should always be striving to improve on how much feedback one leaves on stories, and these readers feel that guilty gnawing even though they already are leaving feedback on almost everything they pick up. Here, I can turn to personal experience: I am in the unicorn group myself, leaving feedback on everything I read (in the form of comments) except when I regularly have to skim stories as part of my mod duties on the sites I run. (Sometimes even then I get sucked into a story and comment.) Despite the number of comments I leave, despite the hours of work I do in the fandom each week, I still feel guilty over not commenting on more of those stories skimmed in the course of daily site business. (I also feel guilty for not reading more, period.)
Conclusion
From this data, it is possible to draw a few conclusions:
- Most readers of Tolkien fan fiction leave feedback, but most readers leave feedback on a relatively low number of the stories they read.
- Self reports of the number of stories a reader feedback on appear to be significantly inflated. This doesn't have to mean that participants wanted to deliberately mislead in their responses--there are a number of reasons why self reports might be inaccurate, discussed above--but it is worth keeping in mind for other items on the survey where self-reported and actual behavior are more difficult to compare.
- Authors are significantly more likely than readers-only to leave feedback on a story.
- The vast majority of readers express that they want to leave feedback more often on stories they read. This includes the so-called "unicorns": readers who leave feedback on almost everything they read. This suggests, to me, that there is enough social pressure to leave feedback that participants may have felt uncomfortable stating that they felt they were doing enough. If you have an alternate explanation, please share in the comments!
If you have a question you'd like to see data on, please do share! Next time, I will likely look at why readers decide to leave feedback on a story, but if there's a topic or question you're interested in seeing analyzed and discussed in greater depth, let me know!