As of October 10th, 2016, Solereview’s content has evolved into a crowd-sourced review based format.
We’ve had a great ride so far. Since April 2014, solereview has published over 150 detailed reviews, shot more than 20,000 footwear images and replied to over 10,000 reader questions. Along the way, solereview has grown to become a destination for detailed, unbiased reviews of running shoes.
And solereview certainly wouldn’t have come so far if it hadn’t been for the support of readers who have unconditionally helped solereview – not only with their suggestions but also with content sharing and financial assistance. You have our heartfelt thanks.
Everything seems to be going well. So why fix something which isn’t broken?
While it is very enjoyable to publish 5,000-word+ reviews accompanied by dozens of studio-shot photographs, this approach isn’t without its limitations.
1) It is expensive and time-consuming to produce these reviews
Solereview buys most of its shoes for testing. Though this approach maintains editorial integrity in the form of completely unbiased reviews (and hence zero-reciprocity), it comes at a cost.
Also, the existing review process takes a LOT of time. Shoes bought in February gets reviewed in June; by the time half of the shoe’s annual lifecycle is over. The process involves running in multiple pairs of shoes for weeks at an end, followed by the photo-shoot, editing and writing the actual review.
2) The current format has proved hard to scale.
Back in February, solereview attempted to hire freelance writers to help with reviews, and we were prepared to pay a decent sum of money. After interviewing/meeting a lot of people, not a single hire was made.
We were very fortunate to have met with extremely talented people – it is incredible how many people reached out to us. Experienced journalists, prolific bloggers, Hollywood screenwriters – in many cases, the writers interviewed solereview instead of the other way around!
We would have hired many of the applicants in a heartbeat if solereview was in a position to offer full-time employment (instead of freelance) and training.
At the same time, it was hard to find independent contractors who had the following set of readymade expertise: passionate runner+deep footwear domain expertise+studio photography and Photoshop skills+the writing to bring everything together in a long form review. The best we got was deep domain knowledge+experienced footwear tester+writer+runner.
We’re hoping that we can tap into all that amazing freelance talent once we complete the transition to a more scalable format.
3) Solereview’s ultra long-form style is at odds with how people consume content today.
Back in May, solereview published an article on the subject of running shoe costs. While it resembled one of our long-form reviews, it had a couple of differentiating factors.
One, it used catchy infographics instead of studio photography, and two, the article went somewhat viral. It was first shared within the running community, followed by prominent sneaker-blogs, and finally discovered by an unlikely target audience – the Silicon Valley.
It got so much traffic that most of solereview’s data tables instantly crashed. That’s a good problem to have, and we also realized the limits of our current server hosting plan. Solereview moved to new upgraded servers last month, by the way.
But the most important lesson here was not a 101 on how to repair a crashed website, but understanding the manner in which most people consume content nowadays.
The shoe cost article was a few thousand words long, but nobody seemed to have read it. Most of the reactions/comments we read on the internet were just based off the hero infographic, and most of their questions would have been answered if they had read the entire article.
On the other hand, why should they?
Solereview’s reviews had become increasingly detailed over the last couple of years. The level of detail has increased from 1000 words and just over a dozen photographs to 5,500+ words and 50 photographs. And yet, when we compared the average view time between the two, there was no difference at all between the 1,000 and 5,000-word reviews.
This disparity demonstrates that there is a clear cut-off time in attention span; longer reviews did not equate to a corresponding increase in time on page. We feel this has a lot to do with the shrinking screen real-estate (mobile), combined with hundreds of other content sources and apps vying for everyday attention.
One also needs to consider the fact that solereview has a sizable reader base not from native English speaking countries. Given that machine translation is still not perfect, a 5,000-word review in English was a formidable barrier to consumption.
The new crowdsourced review format seeks to combine two best practices – medium depth content and infographics – to make the reviews easier to digest. We are certainly not going to publish thin reviews; the average review will be between 2,000-3,000 words. That is decidedly long-form writing by any standard.
4) It was hard to find relevant information in the current review format
Let’s say that your intention of visiting solereview was to find out the ride quality of a particular shoe model. So within the 5,000-word review, where do you look? Is it buried somewhere in the first 1,000 words or somewhere in the middle?
Without clear content navigation, the amount of detail proved to be counter-intuitive. We intend to address that limitation through the new content design.
5) Our reviews were not comprehensive
This statement sounds like a complete paradox given the amount of detail in each review. But it’s true. One of our readers summed it perfectly well a while ago:
A single person point of view on a running shoe does not necessarily reflect the common sentiment. Often you might have found yourself in a position where you appreciated solereview’s unbiased review approach, but disagreed with our take on the shoe. And that’s perfectly reasonable given varied personal tastes and requirements.
But what if we could broaden each review’s relevance with a crowd-sourced data approach? By combining multiple perspectives, the reviews could provide insights previously not captured.
So what’s new?
1) Reader review contributions
Some of you have noticed the ‘write a review’ section on this website. This questionnaire includes different areas of product experience from a user’s perspective. This is a conditional form, meaning that the fields will change depending on the category you select – road running, trail, XC, etc.
We will rely heavily on reader contributions to write our reviews. Thank you, everyone, who took the time out to fill these – it’s been only a month since the form went up, and we already have a large count of insightful reader reviews.
Till recently, our reviews were the most commented on the internet. The website is already populated with thousands of reader questions and reviews; so in a way, the new ‘write a review’ section just serves to give more structure to feedback.
2) Highly standardized template
One of our goals is to make the information much easier to find. A standard set of infographics will help achieve that – each review will have an instantly recognizable set of visual data tools adapted to a particular shoe model.
These infographics will be grouped by sub-categories such as ‘upper fit and feel,’ ‘ride quality and behavior,’ ‘pros and cons,’ and so on. You’ll be able to locate the required information with ease every time, regardless of the shoe model in question. There is a clear beginning, middle and end.
We’ll briefly cover these infographics:
You’re familiar with the sensory profile infographic. This won’t change, and will continue to accompany all reviews.
Estimated lifespan infographic: Many runners want to know how long a shoe will last. Based on crowdsourced data, we calculate the average lifespan of a shoe. This number will change over time based on the density of future user reviews.
Upper fit and feel infographic: In the past, we’ve relied on a large number of photographs and accompanying text to explain the upper fit quality. This visual method makes it easier to imagine your foot inside the shoe.
Ride quality infographic: Most of the ride information (cushioning, responsiveness, stability) is currently captured in the sensory profile section, but these visual tools just break down the different characteristics with greater clarity.
Pros and Cons: Very simple. Just sums up the shoe’s strengths and weaknesses in one infographic.
After answering over 10,000 reader questions, solereview has unique insights into how people buy and use their shoes. These infographics take all that experience into account and packages comprehensive information in digestible morsels.
3) Ever evolving, constantly improving
The beauty of the crowdsourced data approach is that our reviews will get better with time. Unlike the existing format which is written only once, the crowdsourced reviews will change based on future reader contributions.
Let’s say a particular shoe review is published today based on six reader reviews. After a month, it is likely that we’ll get a few more contributions for the same shoe. It is also likely that these new reader perspectives will contain unique insights previously not available in the first six. Then we’ll go back to the review, and make updates based on the new information received.
In short, each shoe review gets better over time as the density of reader contribution increases. A shoe review will be much improved on day 365 than it is on day 1.
Needless to say, a review based on reader contributions also makes the content more objective while being completely unbiased.
4) Crowdsourced data, manual curation
One thing which isn’t going to change is solereview’s tone of voice. We will rely on crowdsourced reviews, but all reviews will still be written by solereview. This will help maintain a consistent writing style across the website.
And would we make the reader reviews public? No, and there are a couple of reasons why.
For starters, public reviews and ratings can be manipulated. Manual curation ensures data quality; this is where solereview’s domain expertise and experience with pattern recognition comes in.
Secondly, even with genuine reviews, it is a daunting task to sift through all the data to find what you’re looking for. Want a live example? Head over to our Pegasus 31 review, where there are nearly 800 comments. While all those comments are packed with running shoe goodness, it is hard to quickly find what you’re looking for.
5) Increased frequency, diverse brands
If initial reader response is anything to go by, we’ll soon have a lot of crowdsourced data. This translates into a higher number of shoe reviews each month, not to mention closer to the initial launch date. Contrast this with the current 3-4 reviews a month and on an average five months after the shoe is released.
We’ll also expand our scope beyond the usual suspects (adidas, Asics, Brooks, Nike etc) to other brands. We’ve already received crowd-sourced reviews on specialty brands such as Salming, Topo Athletic, Innov8 and Salomon.
Further more, solereview can extend its coverage to other categories. For example, trail running is a sub-category we want to cover. And there are opportunities beyond running as well. Solereview was never meant to be a running-only review website.
What’s beyond crowdsourced reviews?
The immediate future (phase 1) – say next 18 months – will be focused on publishing 100% crowdsourced reviews. After that, phase 2 involves blending the best of both worlds – which is melding crowdsourced reviews with in-house wear-testing.
As for phase 3, solereview has some ambitious goals. Let’s say that the success of phase 3 depends on taking advantage of computer vision and machine learning. Phase 3 also requires massive amounts of data, so crowdsourced reviews ties in with that conveniently.
Here’s to the next two years.