My Take On Product Reviews

Just last night, Al Dean of DEVELOP3D and I were having an exchange on Twitter about software product reviews. I was making the point that you simply cannot do what people love to call an “in-depth review” of a CAD system in about 2000 words. Al was of the opinion that things were relative and depended on how deep the writer needed to go and also on the audience.

Personally, I prefer reviews where the writer shares his expereinces with the product that he is reviewing. If I want to know what’s new in a product, I can simply read the “What’s New” document, which will have far more detail. Even a company press release would suffice at times. This applies to all kinds of products, not just software. For example, take the review of a car. I am really not interested in knowing by how much the diameter of the disk brakes were increased. The brochure will definitely highlight that. I am more interested in knowing how the brakes held on curves, loose gravel and wet roads. I have yet come across a car review on TV where the reviewer only sits in the driver’s seat and points to the new stuff in the car, without actually taking the car for a spin and telling viewers what he feels about the car.

To explain my point, let me bring to your attention two reviews I recently read. The first one is a review in a UK magazine  (not Al’s) of Pro/ENGINEER WildFire 5.0 that I read last night. Basically the review was a “What’s New?” document in better prose. I am sure there are people who will find this information useful and that is probably why the review found its place in the magazine. The review was full of “there is this” and “there is that” and sprinkles of “you can do this” and “you can do that”. I don’t believe I ever saw the word “I” anywhere.

The second review is of KeyCreator 9 on the December issue of Cadalyst that I read this morning. I got a completely different feeling of this review just by noticing how many times the word “I” was used. The screenshots used in the review also gave me the impression that the writer actually used the product. Incidentally, the writer is a casting engineer and has been using KeyCreator every day for five years since V3. The writer explained in some detail how and where he used the new features, not just what the new features were. The following sentences from the review will give you a good idea of what I am trying to get at here:

“On a customer file with approximately 6,400 faces, I saw a 55.5% increase in speed!”

“This tool, which began as a SolidWorks file, was imported and healed in KeyCreator in two hours’ time. Tooling design was completed in another three hours.”

“I can personally attest to this improvement on real-world files.”

“KeyCreator and its approach to direct modeling stand out among all the modeling tools I’ve worked with over the years.”

Now this is the kind of stuff I prefer to see in product reviews.

Another thing. In their attempt to get magazines to review their products, CAD vendors often send a CD to the magazine and sometimes arrange for someone from the magazine to get trained for a couple of days at a local reseller. That’s crazy. Its equivalent to having someone who has just passed his driver’s license test write a review of a new car. In a few hours, you basically learn how to navigate around the software and are probably able to design something as simple as a Lego block, like how I recently did in my series on PowerSHAPE. There is no way in hell someone like me could actually review something as complex as PowerSHAPE. And by review, I mean do something useful and compare it with what I did earlier in an older version of the same product or a totally different product. If I absolutely had to review PowerSHAPE 2010 I would probably take the “What’s New?” section of the product documentation and simple reword it.

I use AutoCAD, SolidWorks and Rhino on a daily basis. AutoCAD and SolidWorks because they are the largest selling platforms for my plug-ins and I need to constantly work with customer’s data. I use Rhino to prototype almost all my products, since all of them use OpenNURBS, the geometry library of Rhino. But even then, I do not believe I could do justice if I had to review the next version or AutoCAD, SolidWorks or Rhino. If there is one product that I can review, it is Microsoft Visual Studio, the development platform that I use to build all my products. Basically, my soul has become one with that piece of software.

Maybe I am understanding this whole concept of product reviews all wrong. But if the point of a product review is to only let users know what’s is new in a product then what are advertisements for?

As Al so eloquently put it, “And the ultimate answer is, if you don’t like the review, don’t read it or the publication anymore – pretty simple right?“. That is precisely what I find myself doing.