|
Around October 1, 2006 new rating critera were
developed. The rating criteria had to change due to something
known in the reviewing community as "quality creep".
A common problem experienced by reviewers is
that over time the general overall quality of the products
they are reviewing improves. Therefore, a product formerly
rated high is not as good relative to the newer and better
overall general quality in the product market. Many lights
that used to be above average are not as spectacular when
the technology and innovations that made them so good become
more commonly experienced.
It is precisely this type of adjustment to product ratings
which I feel will make my ratings more valid and adaptable
to future changes in the market. To keep the ratings static
without any changes, while general overall product quality
in the market improves, would result in most lights reviewed
eventually rating "5.0 Stars". This would have made
the rating system useless.
I decided to go through the laborious process of re-evaluating
almost all the lights reviewed thus far and decide if the
rating should change as a result of the new criteria. Many
did.
As a result of the changes, the upper end of the ratings
now allows for greater distinction between the higher quality
lights.
The general critera for each category is listed below. 4.5
is now going to be the category for the best quality/performance
lights, and 5.0 is for lights with extra refinement or superior
performance. All of this is, of course, in my opinion.
The Star ratings have a SUBJECTIVE component which may bump
a light up or down about 1/2 Spot (usually) or more (occasionally)
if I feel it deserves it.
If anyone sees what they feel is big discrepency in the new
rating for a light, please let me know via e-mail and I'll
consider your remarks. Please do not expect a response, as
I will probably hear from quite a few folks.
 |
1
Star = Poor / Not
Recommended at All
|
 |
1½
Stars = Poor+ / Not Recommended
/ Well Below Average
|
 |
2
Stars = Fair / Low
end equipment / Below Average
|
 |
2½
Stars = Fair+ / Low end equipment / Aproaching
Average
|
 |
3
Stars = Good / Average
/ Minimum Serious Use (think of "Maglite 3-D"
as a good example)
|
 |
3½
Stars = Good+ / Better / Above Average /
Better Quality
lights
|
 |
4
Stars = Very Good
/ Well Above Average / Highly recommended
|
 |
4½
Stars = Very Good+ / Way Above Average /
Some of the Best / Highly Recommended
|
 |
5
Stars = Excellent
/ Best of the Best / Extremely Well Designed
|
Remember, each Star rating encompases a RANGE of quality.
Not all lights rated with the same Star rating are the same
quality. Some are better, some worse, but they fit in the
same general range in my opinion.
I ran some quick calculations based on the new rating system
to show the approximate number of lights (percentage) in each
Star rating category. The number of lights in the "above
average" categories is greater than the number "below
average", but this is mostly due to somthing in statistics
called "sample bias" - manufacturers with high quality
lights send them for review - manufacturers with junky lights
do not. The end result of the new ratings criteria is that
only about 10% of the lights on the site rate 5.0 and another
10% rate 4.5. These are the absolutely best lights you can
get (in my opinion). Lights rating 4.0, 3.5 and 3.0 Stars
make up 65% of the lights reviewed. These are average to well
above average lights available on the market and some are
very economical in price but still display good to very good
quality. 2.5, 2.0, 1.5 and 1.0 Stars are marginal lights with
some usefulness, but not necessarily recommended for serious
use. These four category ratings make up the bottom 15% of
lights reveiwed on this site.
| 10% |
5
Stars |
| 10% |
4½ Stars |
| 20% |
4
Stars |
| 27% |
3½ Stars |
| 18% |
3
Stars |
| 5% |
2½ Stars |
| 5% |
2
Stars |
| 2.5% |
1½ Stars |
| 2.5% |
1
Star |
|
|
|