dawn_felagund: Skeleton embracing young girl (Default)
[personal profile] dawn_felagund posting in [community profile] b2mem
B2MeM Prompt and Path: Meta on Fandom and Multimedia (orange/nonfiction path)
Format: essay
Genre: nonfiction/meta
Rating: General
Warnings: n/a
Characters: n/a
Pairings: n/a
Summary: A look at Tolkien Fan Fiction Survey data on who leaves feedback and how often, and how many readers wish they did more.

Review Plz? Feedback Behavior in the Tolkien Fanfic Community



A little over a year ago, I ran an online survey about Tolkien fan fiction as part of my ongoing research on the history and culture of the Tolkien fan fiction community. (Read more about the Tolkien Fan Fiction survey here.) I have been slowly posting the results of the survey over the last year or so. For the orange/nonfiction path prompt "Meta on Fandom," I decided to dig into a topic from the survey based on what other B2MeM participants would like to know more about. People who answer my poll wanted to know more about, "How many readers comment or give feedback on stories? Why do they do this?" I will eventually investigate the other topics as well, most likely over the summer while I'm on break from school.

This essay seeks to answer some basic questions on feedback behavior in the Tolkien fanfic community. Who leaves feedback? How often? I will also begin to look at why people leave feedback, specifically at social pressure to do so. This will be the first post in a series looking at feedback behavior; the series will in all likelihood extent beyond B2MeM; follow [livejournal.com profile] heretic_lore, my Twitter, or the Tumblr tag #tolkien fan fiction survey for updates related to the survey.

Probably the first question to answer when thinking about commenting is: How often do people comment? I asked the question, "Do you leave comments or other feedback on Tolkien-based fan fiction stories?" Of the 1040 people who answered that question, 75.9% of them said YES.

Now it's important to note the "or other feedback" in the statement. This didn't ask just about comments or reviews; it could have included one-click feedback like kudos or likes as well. If I could go back and do this survey over, I'd likely change this question to distinguish between the two. For now, though, it's what I have to work with.

Who Leaves Feedback?


How often authors and readers-only leave feedback
When we break down this question by the participant's role, the results become more interesting. I looked at the responses of writers versus readers only to this question. Writers were far, far more likely to leave feedback on what they read: 86.5% of writers (n = 635) answered YES compared to 59.3% of readers-only (n = 393). My initial reaction to this information is, "Well, of course, writers would best understand how much feedback matters to other writers." I think that's part of it, but there are probably other factors involved as well.

  • Writers are more likely to belong to the sites where they read. Many sites (SWG and MPTT, for example) do not allow comments from anonymous users.

  • Writers are more likely to be comfortable enough with English (or the language the story is written in) to be able to write a comment. I can read tolerably in Spanish, for example, but would never dare attempt to comment on something written in Spanish.

  • Writers are more likely to simply know what to say in a comment. They know what they like to hear on their own stories. They know what goes into crafting a story and are possibly more accustomed to noticing a characterization detail or a particularly good turn of phrase: the kind of thing you'd mention in a comment.


Interestingly, 13.5% of writers responded that they did not leave feedback on stories that they read. I find this group intensely interesting, and a future post will look specifically at this group of participants.

How Often Do Readers Leave Feedback?


Of course, a participant could have left a single comment or liked one story posted on Tumblr and answered YES to the above. Any author can tell you that three-fourths of their readers do not leave feedback on a specific story; many of my stories, based on click counts, would have hundreds of comments, and it is rare for me to exceed ten comments, and I receive more comments than most authors. (The highest percentage of kudos-per-click on my AO3 stories is about 19%.) So what percentage of stories do readers leave comments on?

I asked participants to "Estimate the percentage of Tolkien-based fan fiction stories that you leave comments or other feedback on." Those who responded with a number greater than zero left comments on a median average of 30% of stories.

Breaking down the data a little further also shows that readers willing to leave feedback tend to leave it relatively infrequently. More than half of participants (54.7%) left feedback on one out of three stories, or less. The graph below shows the number of participants who left different amounts of feedback. The numbers drop steadily until spiking briefly around 50%--likely because someone is more likely to respond with 50% rather than dithering slightly to either side of that number; after the 50% mark, the numbers hang rather steadily. There is a small resurgence among the participants that, in my mind as I worked on these numbers, I termed "unicorns": those who left feedback on almost everything they read.

Graph of how often readers leave feedback on stories

It's important to note that these numbers are likely slightly inflated. Even in anonymous surveys, like this one, there is a tendency to overstate positive behaviors, like one's habit of leaving feedback on the fiction one reads for free. To support this point in this particular survey, several participants left brief comments on their answers, suggesting that they'd recently increased their feedback due to growing awareness of its value to authors or that they felt they needed to do more; some participants offered excuses (such as English as a second language) or responded to a perceived low number with self-effacing humor (like a :P emoticon). In addition, numbers were potentially inflated because one is more likely to remember the stories one takes time to leave feedback on, especially comments. A ficlet skimmed quickly on Tumblr, for instance, is more likely to be forgotten than the same ficlet on AO3 where the reader leaves a one-sentence comment or even clicks a kudos; especially the comment requires more careful reading.

Looking at actual feedback numbers supports that 30% is likely inflated. I chose ten stories on AO3 from the section "The Silmarillion and other histories of Middle-earth -- J.R.R. Tolkien." The stories had been posted just over a week ago and were on the sixth page of results, so they had likely received the first heavy wave of readership. Since most AO3 readers who leave comments, in my experience, also leave a kudos (and since comment counts on AO3 also include author replies and further conversation on a story), then I looked just at kudos. For those ten stories, the kudos-per-click percentage was a median average of 9.2%, spanning a range of 1.7% to 26.2%: nowhere near the self-reported 30% rate from the survey.

Do Readers Want to Leave More Feedback?


As implied above, there is a degree of social pressure to leave feedback on stories. I was curious if readers felt they needed to do this more, or if they were happy with the current amount of feedback they left, so I looked at responses to the statement "I want to leave comments and other feedback more often on the stories I read." Participants had five options to choose from: Strongly Agree, Agree, Disagree, Strongly Disagree, and No Opinion/Not Sure.

Overwhelmingly, participants wanted to leave more feedback: 77.6% agreed or strongly agreed with the statement. In other words, three out of four readers think they need to leave more feedback. Fewer than one in ten (8.89%) disagreed or strongly disagreed. Of the latter category, a mere eight participants chose this option.

Graph of responses to the statement I want to leave comments and other feedback more often on the stories I read

I looked closer at that group: participants who strongly disagreed that they wanted to leave feedback more often. Of those eight participants, three were part of the unicorn group that left feedback on almost everything they read; it's understandable why they felt they didn't need to do more! One participant left feedback a reported 70% of the time--still a relatively high number--so about half responded with Strongly Disagree because they really can't do much more than they already are. One person did not provide a response for the amount of feedback they left but answered NO to the question "Do you leave comments or other feedback on Tolkien-based fan fiction stories?" Three entered "zero" for the amount of feedback they left; these four responses felt somewhat defiant to me given how contrary to correct fandom etiquette it was. (I would say that this etiquette demands that one either leave feedback or feel badly for not doing so.)

I was also curious about the unicorn group: those who left feedback a reported 90 to 100% of the time. Despite leaving feedback on just about everything they read, 65% still agreed or strongly agreed that they wanted to do more. (Including those who reported that they left feedback 100% of the time: 67% of these participants still wanted to do more, including five who strongly agreed with the statement.)

Of the unicorn group, 21.7% chose No Opinion/Not Sure, a percentage much higher than the 13.5% of all participants who chose this option for this statement. I generally avoid making inferences about the No Opinion/Not Sure participants--there are a lot of reasons why people might choose this option, including that they truly do not understand what the statement is asking--but this discrepancy is too interesting to pass up hypothesizing about a little. I suspect that these respondents know that they are going above and beyond the majority of fandom but still feel uncomfortable stating directly that they don't think they need to do more. Choosing No Opinion/Not Sure is quite possibly the more socially acceptable option: a way to circumspectly admit that one really can't do much more.

The unicorns are an interesting group. Why do so many of them--about two out of three--feel that they need to do more? It is possible that the feedback they are leaving is mostly or entirely kudos or other one-click feedback, and they feel they should be writing more comments. (Readers who leave kudos on everything they read are a well-reported phenomenon on AO3; one participant even commented that they "kudos" everything they read.) It is also possible that social norms in fandom dictate that one should always be striving to improve on how much feedback one leaves on stories, and these readers feel that guilty gnawing even though they already are leaving feedback on almost everything they pick up. Here, I can turn to personal experience: I am in the unicorn group myself, leaving feedback on everything I read (in the form of comments) except when I regularly have to skim stories as part of my mod duties on the sites I run. (Sometimes even then I get sucked into a story and comment.) Despite the number of comments I leave, despite the hours of work I do in the fandom each week, I still feel guilty over not commenting on more of those stories skimmed in the course of daily site business. (I also feel guilty for not reading more, period.)

Conclusion


From this data, it is possible to draw a few conclusions:

  • Most readers of Tolkien fan fiction leave feedback, but most readers leave feedback on a relatively low number of the stories they read.

  • Self reports of the number of stories a reader feedback on appear to be significantly inflated. This doesn't have to mean that participants wanted to deliberately mislead in their responses--there are a number of reasons why self reports might be inaccurate, discussed above--but it is worth keeping in mind for other items on the survey where self-reported and actual behavior are more difficult to compare.

  • Authors are significantly more likely than readers-only to leave feedback on a story.

  • The vast majority of readers express that they want to leave feedback more often on stories they read. This includes the so-called "unicorns": readers who leave feedback on almost everything they read. This suggests, to me, that there is enough social pressure to leave feedback that participants may have felt uncomfortable stating that they felt they were doing enough. If you have an alternate explanation, please share in the comments!


If you have a question you'd like to see data on, please do share! Next time, I will likely look at why readers decide to leave feedback on a story, but if there's a topic or question you're interested in seeing analyzed and discussed in greater depth, let me know!

Date: 2017-03-24 08:26 am (UTC)
hhimring: Tolkien's monogram (Tolkien)
From: [personal profile] hhimring
Very interesting.

Self-reporting may well be inflated for the reasons you say.
But some of the click ratio on which you base your calculation for the inflation could be re-reads or people briefly checking out a fic and then coming back to finish reading or read more thoroughly later.

(And also, of course, more negatively, people opening up stories to have a look and then deciding this is not the story they wanted to read just now after all)
Edited Date: 2017-03-24 09:18 am (UTC)

Date: 2017-03-24 10:13 am (UTC)
ext_45018: (tolkien - Tengwatrix Reloaded)
From: [identity profile] oloriel.livejournal.com
This!
Also, multi-chapter stories may require several sittings, yet in the end a reader leaves a comment only on the final chapter.
I, both as a reader and as a writer, would consider this a fully fair comment/review (getting comments on every chapter is awesome but far more likely on a WiP than on a finished story, I suspect). However, in the statistics, it means that the story may have been clicked, say, 10 times (by me alone) but only 1 comment was left.

Or people read the story on a different platform (like LJ or Tumblr) and comment there, and while they may re-read it on AO3 or SWG, they won't leave the comment again. (I am not likely to copy my comments from this comm into the comment field on SWG, for instance, though perhaps I should? I know some people did it for their MEFA reviews but I haven't. Yet, the story got read AND commented on, just not in the same place.)
For instance, I know I've commented on a couple of chapters of AMC, but only on LJ, never (?) on SWG (although I've re-read AMC there a couple of times), so in the statistics, that'll register as lots of reads (probably one read per three or four chapters) and not a single comment.

So while the different places in which people post fic - private sites, social media, event communities, forums and archives - are awesome and all need to exist, they also make it impossible to easily interpret the click data.

In conclusion, I'm perfectly happy to believe that the self-reporting is inflated (personally, I try to err on the side of caution, but I guess others may roll differently). But I don't think it's possible to conclude that quite so easily from the click:comment ratio. So 9,2 % is quite likely just as far from the truth as 30 % is.
Edited Date: 2017-03-24 10:14 am (UTC)

Date: 2017-03-25 09:37 am (UTC)
ext_45018: (tolkien - Tengwatrix Reloaded)
From: [identity profile] oloriel.livejournal.com
Yeah, "just as far" was likely badly exaggerated.

But hey! Glad that the numbers for complete single-chapter fics support (part of ;)) my theory. And there seems to be a unicorn story there for which the 30% are really appropriate!

If I were to like it enough to reread it on a different site, I don't see why I wouldn't give it a kudos, even if I had commented on it or liked it elsewhere.

Hm, I know I have to stop assuming that everybody would use these things the way I do, but I certainly know that until today, I wouldn't have left a kudos after reading a story on AO3 if I had already read the story on LJ, or a friend's tumblr, or was waiting for it to appear on SWG. Heck, I wouldn't leave a kudo if I commented on that story on AO3. That's because kudos don't really count for me (when my Russian translator reported me the stats on TTS on the Russian archive where she'd posted it, I ignored the Facebook-style "likes" completely, only the comments mattered.). I guess it would be different if there were 0 kudos/likes, though, because that would hurt. And I obviously have entitlement issues here, so I've resolved to leave kudos in the future when I read something on AO3 - whether I've already commented on it elsewhere or not. It might matter to someone else.
But, anyway, I was assuming that I might not be the only one who thought kudos unnecessary after leaving a comment.

So yes, kudos are probably the best choice of measure for the reasons you've listed, but they're still more or less eavily skewed by however every single reader uses them. But you've got to use something, I know!

Date: 2017-03-24 09:55 pm (UTC)
hhimring: Tolkien's monogram (Tolkien)
From: [personal profile] hhimring
I have on occasion opened up a story once, had a quick look, decided, yes, I will want to read this at some point, come back later, when the time was right, opened up the story a second time, read it and left a kudos, went away again, come up with a wording for a comment, opened the story up a third time and left the comment (and that might even be on a different site). I don't mean to say I do this all the time, but it's hardly vanishingly rare, either.

And I do quite often have other reasons not to read a story right now--it doesn't necessarily mean the author has failed to hook me...

Date: 2017-03-25 09:49 am (UTC)
ext_45018: (lotr - Fëanor invented the internets.)
From: [identity profile] oloriel.livejournal.com
I have to admit that I do this too, even if I know I only have 15 minutes and the story has 9K words: I can certainly use these 15 minutes to determine whether the characterisation and writing style make me want to invest the longer time span needed to actually read the whole 9K. That's because I'm not a statistician and didn't even think about mucking up the click counts until today. I just want to know that I'll have something good to read (or not!) when I come back. ;)

Or I may think I can read something within a certain time, but then the phone rings or the kids want something or the story is more complex than expected so I'll have to stop and return later. Maybe I just fail at time management. ^^

Or I may read a 1K story in my 15 minutes, be full of incoherent ~feelings~ afterwards so I just have to re-read the story in the evening, and then return for a third time the next day to finally leave a comment. Or not, if I still don't know what to say. (I should be a better person and just voice that initial incoherence, I guess: "I enjoyed this so much that I can't properly put it into words, but I'll try to come back later to talk about it some more!" Always room for improvement. >_>)

Date: 2017-03-25 10:12 am (UTC)
From: [identity profile] heartofoshun.livejournal.com
I guess: "I enjoyed this so much that I can't properly put it into words, but I'll try to come back later to talk about it some more!"

I do this a fair amount, especially if a story has very few or no comments and I think it is true worthy of appreciation. I also admit to on the rare occasion not commenting if a story is way over appreciated in my judgment. That's infrequent and usually in mega-fandoms.

Date: 2017-03-26 06:11 pm (UTC)
ext_45018: (tolkien - Tengwatrix Reloaded)
From: [identity profile] oloriel.livejournal.com
Collecting IPs for clicks might be the most (if not absolutely) precise measure, but perhaps one could achieve an approximation by asking a representative crowd of readers about their behaviour, and apply the corresponding ratio to the overall click number?

... yeah, I wouldn't want to do that, either.

Even in a controlled environment (as would have been provided by the MEFAs, back in the day, but I don't suppose the stats are still available), the results would likely be skewed - in the MEFA scenario, people basically read in order to review (and only refrained from commenting on something they'd read altogether if they really, really disliked that story), and in any study, people would be more aware that they're being watched and thus probably more prolific in their comment- writing...

So interviewing readers on their reading practices would probably be unavoidable to find out what's the most common behaviour, I suppose.

Date: 2017-03-26 06:38 pm (UTC)
From: [identity profile] heartofoshun.livejournal.com
in the MEFA scenario, people basically read in order to review (and only refrained from commenting on something they'd read altogether if they really, really disliked that story)

I approached the MEFA experience totally differently!!! First, I read a lot year 'round. One might call it an addiction. I never read FOR the MEFAs, I first looked through the MEFA nominations to write reviews of my favorite stories of the year. Only if I got through all of those did I write reviews of ones I read that I thought were pretty good, if not spectacular, or the author showed promise and needed encouragement and I wanted to see more of them in the future. Last of all did I ever read anything new, unless everyone was talking about it and I felt the odd one out.

I might use others' MEFA reviews in the future to find reading material. I never wrote an MEFA review to bolster my review count either, although I did try harder at times to get through my list because someone was counting (and churning out garbage reviews just to count them). I do admit to reviewing stories that were less than stellar IMHO out of friendship, but even then I gave an honest and thoughtful review and did not review anything without merit.

Date: 2017-03-26 07:01 pm (UTC)
ext_45018: (tolkien - mefa nominee)
From: [identity profile] oloriel.livejournal.com
OK, so one would have to interview participants about their reviewing behaviour under controlled circumstances too. ;) Wow, what a systematic approach! I did check for already-known favourites first and made a secondary list of "this sounds promising but I'll only tackle it if I have the time", though, now that you mention it. (If the summary didn't catch me, unless it was about favourite characters, I usually didn't read it at all unless something was explicitly recommended by someone whose tastes I knew were similar to mine...) I never found the time to go through other people's reviews in order to find out what everybody else loved!

I didn't write reviews in order to bolster my review count, but knowing that all reviews were counted did sometimes get me to write at least a short "I liked this" type review because I felt the story deserved more than zero points, and definitely to agonise a bit more about coming up with 4- or 5-point reviews when the story merited it. My normal reviewing behaviour is definitely different!

Date: 2017-03-26 07:58 pm (UTC)
From: [identity profile] heartofoshun.livejournal.com
Some people tried to review every story! Or get little icons with a specific number of reviews made on them. I found that practice annoying. Just me! Other found it fun! But I did get a review on a long story once from someone who I would have thought an unlikely audience for that story; it said: "What!!??"

Date: 2017-03-27 06:41 am (UTC)
ext_45018: (tolkien - mefa nominee)
From: [identity profile] oloriel.livejournal.com
Urgh. I'd normally have said that there's no harm in someone reviewing every story, if that's the kind of challenge they like, but a review like that... sort of undermines the reason why there are rewards for reviewing...
(On the other hand, I can use this as a handy explanation for my less-than-excited response to "this was interesting!" type replies, because I now suspect these may have come from the same people...)

(Why am I even keeping this icon around after 6 years? I don't rightly now! Hard to let go of the shiny.)

Date: 2017-03-27 06:45 am (UTC)
From: [identity profile] heartofoshun.livejournal.com
That is one of my favorite MEFA icons.

Profile

b2mem: (Default)
Back to Middle-earth Month

August 2020

S M T W T F S
      1
2345678
9101112131415
16171819202122
23242526272829
3031     

Most Popular Tags

Page Summary

Style Credit

Expand Cut Tags

No cut tags
Page generated Jul. 21st, 2025 06:31 am
Powered by Dreamwidth Studios