Expert Reviews, Demystified

Harmony Central has been publishing Expert Reviews for some time, and occasionally we’re asked various questions about the process. We thought that it was time for an article that describes exactly what goes into them and how they’re done. We know that some of you have questions about the process, and that’s what we hope to answer here. 

How do you decide on what gear to review? 

This is where the process starts. Like you, our reviewers are all musicians, recording engineers, live sound technicians and/or otherwise deeply involved in the world of music and gear; usually with many years of professional-level experience. And like most musicians, we have our ears to the ground about exciting new announcements in the musical gear world. In fact, we have a better overview of what’s coming out than most musicians. Not only do we attend trade shows like NAMM and AES where new gear is often revealed for the first time, we also get new product announcements sent to us on a regular basis, which we pass along to you (our readers) via our site’s front page and Harmony Central News, which is updated every non-holiday weekday. We even occasionally get press releases in advance of their publication date. Such “embargoed” press releases are held in confidence until the product’s official announcement date, but the point is that we often hear about cool new things that are coming out before the general public does. Anything that catches our eye or sparks our interest is a likely review candidate. We then reach out to the product manufacturer or their marketing reps and request a review unit loaner. Sometimes the company will contact us directly and ask if we’d be interested in reviewing a new product. As with any other item, we consider its potential coolness factor - if it’s not something we’re interested in, or more importantly, if it’s not something that we think our readers will be interested in, we pass on it - after all, there’s a lot of gear released every year, and we can’t possibly cover everything, so we try to stick with things that are potentially cool and that people will want to learn more about. Why waste time and editorial space reviewing uninteresting junk, right? We figure you’re more interested in reading about the cool stuff, and the word will quickly get around about the really dreadful stuff anyway. 

The review begins with research

Research? Yes. We consider not only the gear under review, but also what else is available in the same basic product category. While a lot of this relies on our past individual experiences, we can’t count on that for everything, so we check to see what other products are out there that may be similar or competing products. We also check into what musicians are currently saying and the questions they’re asking - are they expressing a need for something like the new product that we’re going to be covering? If the product has already been announced to the public, how’s the street buzz? What questions are people asking about it? What are they most interested in finding out about the new product? That’s all information that we’re going to want to make sure we cover in the course of our review. We also consider what the manufacturer has to say about the product in their marketing materials. Claims that they make are something we take note of, because whenever possible, those claims are something we need to check and verify for ourselves in the course of the review. 

So the gear arrives - then what? 

Once the gear is shipped to us for evaluation, we get to work. A good review will need images to illustrate various aspects of the product that we want to point out, and features that it has. Sometimes we can just use professionally-shot product photos that are provided by the manufacturer, but in many cases we’ll take photos ourselves. Measurements are also taken. No, we don’t just trust the manufacturer’s specs - whenever possible, we verify them, and occasionally we even find errors in them. If we do, we point them out to the manufacturer, which occasionally results in them updating their website and advertising materials, even before the review is published. We look for flaws - in appearance, in performance, in unmet expectations. We compare the product not only to similar products, but also to the claims that the manufacturer made about it in the release announcement and in their marketing materials.  Anything that doesn’t quite live up to the manufacturer’s claims is something we want to make note of and mention in the review. While we normally won’t compare a $500 guitar to one that sells for $5,000, if the manufacturer is making claims that it’s just as good as the more expensive model, that’s fair game - we’re going to look into it, and if it doesn’t measure up, we’ll tell you. 

We heavily test the product - often in real-world situations, such as during live performances, on recording sessions, and also by just playing around with it at home. We don’t just play it for five minutes and make an evaluation - we work with products that are under review for a considerable amount of time in order to really get to know the product - its strengths and its limitations - and in the process, we develop our opinions. And let’s face it - reviews involve a considerable amount of personal opinion, but we strive to make those educated opinions that are based on the facts. We also try to clearly delineate between facts and the reviewer’s personal preferences and opinions. For example, I prefer guitars with slimmer necks, but I understand that many players prefer a beefier neck profile. I won't criticize a product for not matching my personal preferences, but I will note the neck profile details and let you decide whether or not it might be of interest to you. 

Do all of your reviews follow the same format?

No, not all of them, although many do. The format of most of our reviews starts out with an (untitled) Introduction section. We’ll often give a bit of background here, whether it be historical information about similar products, the people behind the product under review, or information about how the product idea came about and what the manufacturer wanted to achieve by developing and releasing it. The next section is often titled “What You Need To Know.” This is where we try to provide all of the important information about the product. We not only cover the specific features and specifications, but you’ll also often find relevant comments about specific features and capabilities in this section too. We also include a “Limitations” section. This is where any limitations on the product’s usefulness for its intended purpose will be mentioned, as well as any areas of concern that might pose a problem for you, the potential purchaser. In the Conclusions section, we wrap everything up, and discuss our overall impressions of the item under review. While reading the complete review will give you the fullest picture of what the product is all about, if you’re in a hurry, it’s okay to skip down to the conclusions for the quick synopsis. We also include a Resources section. Here’s where you’ll find the price for the item, as well as links to the manufacturer’s web page for the product. We also include links to various retailers who carry the product in case you’re interested in purchasing it. You’ll also often find video demonstrations of the product that either describe its features, demo its sound, or both. And there will often be a link to a forum discussion thread, where you can go to read opinions from others who own the product, as well as serving as a place where you can post your own thoughts, or even ask questions you may have about the product under review. 

Do you get to keep the review units? 

In most cases, the answer is no. We request a return shipping label from the manufacturer, pack the item back in the shipping box it arrived in, and off it goes, back to where it came from. In some cases, we are allowed to keep the review unit - that’s usually for “disposable” items like strings, or personal-use items such as ear plugs. Software is usually provided with a NFR (not for resale) license. Occasionally a manufacturer will decide that it’s more trouble than its worth to restock the item, and despite multiple requests for return tags, they aren’t provided. In those cases, we’ll hang on to the review unit, but keep it ready to return as soon as we do receive those tags - in rare cases, that can sometimes take a year, or even two. Storing review units can sometimes become an issue for reviewers in those cases. 

What if you really like the review unit? Can you purchase it?

In many cases, a reviewer will really bond with an instrument or other product under review and decide that they want to buy it. Many manufacturers will accommodate purchase requests and allow the reviewer to purchase the review unit (often at a discounted price, since it can no longer be sold as a new unit after being extensively used in the course of a review), although in some cases they may not be willing to sell the review unit. This happens most often with brand new products that are in limited supply that the manufacturer might need to have returned as soon as possible so they can pass it along to the next product reviewer at another publication. 

Does Harmony Central tie reviews to ad purchases?

No. A company doesn’t have to advertise with Harmony Central in order for their product to be reviewed. While this isn't a universal policy in the music gear reviewing world, it's the only policy that we feel we can support. 

What about Harmony Central’s ownership? Is it true that HC is owner by Gibson?

While it’s true that Harmony Central is a Gibson brand, and is wholly owned by Gibson, our instructions are to run the website in an editorially-neutral manner. We take that responsibility extremely seriously - from the forums to our reviews, we don’t take sides, and we don’t promote any company over any other, or at the expense of other companies. Our first and foremost responsibility is to you, our readers, and we do our best to make all reviews fair and impartial, regardless of who made the product. When reviewing products from another Gibson brand, we make sure to state that HC is also a Gibson brand in the interest of full disclosure.  

Where can I go to check out the Harmony Central Expert Reviews? 

The latest ones are posted on the front page of the site, and you can find the Expert Reviews section, with the entire collection of HC Expert Reviews, right here.

What if I have a product I’d like to have reviewed, or have questions that were not covered in this article? 

We welcome review requests, and are always happy to answer questions from our readers. For submitting new product announcements and review requests, please email news@harmonycentral.com. For questions or comments about this article and our review process, please feel free to post them in this thread in the Sound, Stage and Studio forum. -HC- 

_________________________________________________________________

Phil O'Keefe is a multi-instrumentalist, recording engineer / producer and the Senior Editor of Harmony Central. He has engineered, produced and performed on countless recording sessions in a diverse range of styles, with artists such as Alien Ant Farm, Jules Day, Voodoo Glow Skulls, John McGill, Michael Knott and Alexa's Wish. He is a former featured monthly columnist for EQ magazine, and his articles and product reviews have also appeared in Keyboard, Electronic Musician and Guitar Player magazines.