Tuesday, March 17, 2009

Review Scoring, Formatting and MAL

About a month ago, MyAnimeList revised its review system (this thread seems to have been damaged in the recent server problems; curiously, Xinil has chosen not to repost or fix it as he had done several other affected threads. I had to dig out this link from my browser history. Clicking the "last" post link on the page doesn't work, though you can still manually navigate the thread up to page 8, after which it redirects you to page 1). This involved three major changes:

1) The hiding of the score breakdown for every review. Previously, every review on MAL showed not only the overall score, but the individual scores for story, character, animation, sound, and enjoyment. Reviews can still fill in the subscores... there's just no way for readers to see them. (Later, Xinil conceeded and re-added the subscores, though users have to go through an extra click to get to them.)

2) The removal of BBCode support for all reviews. Reviews can no longer contain any kind of BBCode, including basic things like bold, italics, underline, and strikethrough.

3) The relocation of reviews on profile pages. Reviews are now to be accessed at the top of each user's profile instead of from the side. Users can choose to favorite reviewers so that their newest reviews appear under a tab in their own profile.

A handful of people expressed their disapproval of change #3, but most of the dissenters were more concerned with changes #1 and #2, and I was among those ranks.

Xinil did not really provide an argument in favor of #1, but eventually conceded to user complaint and re-added the subscores. Honestly, in retrospect, I think the main reason I was upset at the removal of the subscores was just that they had always been there before. Updates to a site should not take away features that were there previously, especially if no one had ever had anything bad to say about them. Users didn't have to use the subscores, but they were there. Despite that Xinil had little to say on the matter though, there was a lot of debate over the feature between both reviewers and readers alike.

A lot of people seemed to believe that numbers were too impartial and that the less numbers a review had, the better. No numbers would encourage readers to actually read through the entire review to get the writer's opinion rather than only taking a quick glance at the provided numbers (and rating a review "helpful" or "not helpful" based solely on those numbers). Some readers said that they liked the numbers because it allowed them to quickly assess a writer's opinion -- writer's said that this just made them lazy. Other writers claimed that the numbers encouraged readers to read through the actual review if the numbers intrigued them -- for example, if the reviewer scored all aspects of a series high except for animation.

I write reviews. I write a lot of reviews, and I only occasionally read others' reviews. Still, I argued in favor of the numbers and the subscores. My reasoning was that there are some things that are just easier to emphasize with numbers. For example, I loved the animation and sound in Final Fantasy VII Advent Children, but I abhored the character development and story. I expressed this in words, but it's probably just as effective for a reader to take a quick glance at my subscores and garner that information for himself: Story -- 5, Character -- 5, Animation -- 10, Sound -- 10. I will be the first to admit that I tend to write excruciatingly long reviews. Sure, it's nice if you actually read the whole thing, but I won't be offended if you're just looking for a quick, numerical summary.

A few readers also pointed out that they were afraid that reviews might contain spoilers, and thus it was safer to just glance over the scoring. With my own flaming hatred of spoilers, I make a point to never write spoilers in my reviews for MAL (except in the cases of sequel series, wherein I'll occasionally have spoilers for the parent series). I think most other reviews do the same, or at the very least, carefully mark where a spoiler might be. Still, it's not hard to sympathize with the cautious reader; it's true after all, numbers can't spoil the story for you, even if they might turn you off from it or get you overexcited.

So yes. I like my numbers, and I'm glad the subscores did eventually make it back. Now that the fight's over though, I come to realize this: I don't really use numbers for the reviews I write outside of MAL. For the live action movie and comic reviews that I write for this blog (because MAL is anime and manga only), I provide no numbers, only words. Lots and lots of words. I almost had numbers. I distinctly remember scoring the first NANA movie when I was reviewing it, but I ended up not posting them. I can't remember why, but because I'm obsessive compulsive and need to keep everything in the same format, every subsequent movie review was missing numbers.

So I guess I don't think numbers are so essential after all. I only ever assigned those numbers because the option was there. If it had never been there to begin with, it would have never bothered me. Reading a review, you get a pretty decent idea of someone's opinion, regardless of whether or not numbers are there to accompany the words. Numbers aren't essential, though they might be a cheating, kind of time saver for some. Besides, everyone's numbers seem to be different. It isn't uncommon for reviewers to have a page dedicated to explaining just what their numbers mean, especially in relation to one another. One person's 10 is not the same as another person's 10 because one person might give out 10's more frequently than others, lessening its "value." Of my reviews on MAL thus far, I've only given an overall score of 10 to two series (the Death Note manga and the Gurren Lagann anime). Most other people are less reserved with their praises, but that doesn't mean they're any more or less superior as a reviewer. We're just different people, and we review things differently.

And with subscores, the difference is even greater. Some reviewers average their subscores to create the overall score. Other reviewers, like me, assign an overall score independently of the other scores. For some series, I just feel as if some categories carry more weight than others. Sometimes, the enjoyment and charisma of a series just overrides the fact that it had shoddy animation and a cliche storyline. So I guess for readers who read reviews by a wide range of reviewers, numbers can mean very little. If you're unfamiliar with the particular reviewer, it can be hard to compare their scores with someone else's. Everyone's using a different scale, and that can't be very helpful at all.

In retrospect, I guess #1 wasn't really that big of a deal. I still like that we were able to compromise, and I still like that I can assign both an overall score and multiple subscores, but it wouldn't have been all that hard to just add those in myself at the end of every review if I really felt like it. It would have been annoying, but much more workable than #2...

Unfortunately, Xinil was much more adamant about keeping BBCode out of reviews. His position on the issue was:

I'm still set against bringing bbcode back. Newspapers, magazines, news websites...none of them use bold/italics. We don't need it.
I kind of find this to be bullshit. No one uses bold and italics? Are you serious? Font creation standards all but dictate the inclusion of a bold, italicized, and bold-italicized version of standard fonts. That is an obvious indication of their widespread use. In addition to that, newspapers and magazines have plenty of formatting via headers and margins, as well as images, to help break up the text. The tl;dr syndrome has only gotten worse with the age of the Internet, and thus, there is a need to minimalize giant walls of plaintext. If you want someone to read what you've written, you need to make it easy for them. This is why paragraphs were invented.

I've never seen anyone on MAL really abuse the use of BBCode in their reviews. Some people get a bit fancy with their tiered, multi-colored, and bullet point reviews, but all of those shenanigans are in an effort to make their opinions easier to understand -- organization isn't a crime. Myself, I like my bold and italics. They do their job: they emphasize things without having to result to CAPSLOCK, which is still "yelling" on the Internet. I'm also fond of the [url] tag because it allows me to link related reviews. It is beneficial to both me and the reader for me to link my review of the Death Note manga from my review of the Death Note anime. Similarly, I link reviews for spin-off series like SEED Supernova to their parent series because some aspects of the series, such as animation and sound, are shared between them. All right, so the linking isn't really kosher or necessary, but it's useful. At the very least, basic font formating like bold and italics should be left useable.

In the arguments for BBCode, there arose a discussion about the need to format reviews by blocking off sections for each subcategory. For this, citing professional news and review sources is actually applicable because it's true -- most reviews are just solid essays without any kind of partitions. Generally, for good reviews, all the subcategories would be covered at some point in the text anyway. Still, writing a review in sections appeals greatly to my OCD. If all the information is there regardless, does it really matter? Is there really such a huge difference between transitional phrases like "As for the characters..." and a header that declares "Characters"? The latter is easier to identify and read. Sectioning things off might make it easier for those that only want to know about a certain aspect of something. It also helps keep me, the writer, organized as well. Without those sections, it'd be much, much easier for me to ramble off in an incomprehensible mess... as if my writing wasn't rambly enough.

So here's to still wanting that BBCode back, at least in part. Unfortunately, now that the thread seems to be both hidden from the site and unusable anyway, there's no good platform on which to argue with Xinil. I'm not sure how much good logic would do him anyway; despite having asked for suggestions, he didn't seem very receptive to them. (Consider also that a majority of the legitimate threads in the Suggestions forum seem to never get a reply from him.) I dunno. Maybe it's not worth the drama.

For a little while after all the changes were made, I considered no longer using MAL as my review platform if compromises weren't made. That seemed needlessly dramatic though. I like MAL. I don't like these changes, but I still like MAL. It would be a lot of trouble for me to mirror all of my existing reviews on this blog too. I started and finished Antique Bakery on a random whim a few weeks ago, and it's sitting around on my backlog waiting to be reviewed. It'll be hard trying to find a way to emphasize things in my writing without italics, and I hate that my section headers no longer stand out for lack of bold, but... what can ya do? Oftentimes, I think I fret over this nonsense way, way too much.

5 comments:

Joe said...

The New York Times, and many other book reviewers don't use scores at all. I appreciate this, but at the same time It's sometimes difficult to determine if they liked or disliked a book if you don't have encyclopedic knowledge of the literary world; you're just left with a general feeling. They also very rarely summarize their opinion. I hate having to stuff a novel or something into a simplistic grading system but it seems necessary to get the idea across.

Iknight said...

It feels pretty weird not being able to italicise series titles. But hey, it's Xinil's website. I guess I'll leave it if he ever makes changes which are too irritating (these don't fall into that category).

You can actually get a review removed if it contains major spoilers. I suspect they have to be obvious, serious ones that the mods will recognise, though: the case I'm thinking of is a review of Magikano which said (essentially) 'Don't watch this because at the end [etc]'

Kiriska said...

@Joe: Yeah, I find that a lot of NYT's book and movie reviews are pretty unclear as to what position the author's taking, perhaps to keep offending the reader to a minimal. That's all well and good, but if I can't take away some kind of impression, then that's missing the point. I seem to recall the book review for Harry Potter 7 being vague like that, though the review for The Dark Knight was definitely positive. Maybe a number is oversimplifying a very complex opinion, but you can't argue with its ease of understanding.

@Iknight: Certainly its Xinil's site, but I always like to think that administrators should keep nice with their members. I don't see the point in needlessly initiating antagonisms by implementing changes without polling the public. The changes made to the review system seemed more like a random whim of his because there was no public call for it, and his refusal to compromise (at least with the BBCode matter) with the resulting backlash just leaves a bad taste in my mouth. It reminds me a bit of Facebook. Sure, MAL isn't anywhere near Facebook's caliber and its members aren't as vocal, but it's the same kind of idea. Like I said,

I like Mal a lot -- it's an incredibly useful tool, and for the most part, I enjoy the community. I'm with you in that I'll leave if something utterly offensive happens (and it hasn't yet), but I don't think it's too much to ask for Xinil to consult the masses a bit before implementing large changes.

Ryan A said...

I'm still set against bringing bbcode back. Newspapers, magazines, news websites...none of them use bold/italics. We don't need it.

It's really not about that... it's a security thing. It's also not about needing it, because it is available and why not allow it... personally, on the site I've been developing (melative.com) the objective is to not limit possibilities or confinements.

Unfortunately, I don't do on-site reviews, but link to reviews via associated bookmarking... also, there are no point-ratings... can't recommend it :( I'm sure there are ways to make relative ratings work with review-breakdowns, but I'll leave that for when it is needed.

Cheers

Kiriska said...

Well, I'd be curious to know how it's a security issue, but if that were the case, I don't see why Xinil couldn't have just mentioned it. From his words, it seems more like a personal vendetta against BBCode, which really just makes him out to be a selfish administrator who makes adjustments without considering the other users on the site. I don't know whether or not that's actually true, but that's the impression I'm getting from his explanations.