Tiny, sparse data points. Or, why even the most technically adroit sounding camera reviewers can be full of crap.

So, there's this lens I've been interested in but it gets a range of "expert" reviews. It's the Panasonic/Leica 15mm f1.7. Most people will write some drivel about how it's blah, blah wide open but upon stopping down a stop and a half it becomes legit and usable by the professional class of photography bloggers.

I'm calling their bluff on this kind of generic lens test writing. The reviewers are writing based on what is generically traditional knowledge about the way lenses have always worked. Almost every lens on the planet is probably sharper one and half or two stops down from the maximum aperture so pretty much the reviewer isn't telling you squat that's useful about a particular lens.

I think most reviewers get a lens for a few days, take a few images of their cat and their interesting lunch, and maybe a few coffee cups, and then make a pronouncement about the quality. It might all hinge on whether or not their cat/model is having a good hairball day or not. Or if the lunch in question had hard edges or was nicely rounded and lacked a certain actinic certainty (is that related to micro or nano contrast?).

The fact is that I have to use a lens (at least one that doesn't arrive out of the box defective) for months before I really, really become conversant enough with it to know it as well as I should. And you might pity the poor lens company whose lenses routinely get tested on someone's back focusing or front focusing camera....

As the folks at LensRental.com will tell you, there are plenty of variations between different samples of the exact same lens model. That's why they run tests of ten or so units and then average the results to really understand just how well a theoretically average version of a lens will perform. But I rarely read that XXXX reviewer has scrubbed through five or six, or eight or ten, different units before making some sort of declaration of quality.

I recently read a review by Lloyd Chambers in which he described the lens I'm discussing, in so many words, as 'underwhelming'. He was evaluating it in the context of the Panasonic Leica 42.5 f1.2 experiences he'd had. But in the next paragraph he let us know that (like most Leica lenses) there is field curvature to consider (not a fault or a feature) and that if one could focus where one wanted to focus then the lens could delivery very sharp and convincing results. He more or less ended his assessment by declaring his inability to correctly focus said lens. Now we must all avoid the lens. The Oracle has spoken...

I'm hard headed so I decide to ignore Mr. Chamber's assessment and make my own. I had some extra change rattling around in the center console of the car so I tossed it all in a Baggie and headed to Precision Camera to play with the lens and see if it was worth trading money for.

I brought the lens home and opened the box. The first thing I'll say is that the lens is a little jewel and so beautifully made, designed and finished that anyone sporting the m4:3 systems around town should buy one just because it looks so good mounted on a camera. The second thing is the really cool external aperture ring, calibrated in 1/3 stops, which in use feels like one is back to using a real and authentic camera.

There is a regular lens cap and then there is a rubber "plug" cap that fits into the front end of the lens's hood. I think it's a nice touch while others think all lens hoods "must" be reversible.

But, most people don't care about aesthetics or lens hoods ( I HATE IT WHEN I SEE SOMEONE WALKING AROUND SHOOTING WITH THEIR LENS HOOD STILL REVERSED ON THEIR LENS. EITHER USE THE HOOD OR LEAVE IT AT HOME. THERE IS NO DEFENSIBLE MIDDLE GROUND. ANYTHING ELSE IS LIKE WEARING YOUR BASEBALL CAP SIDEWAYS. OR SPITTING IN PUBLIC.) What would HCB do?

So, given that people don't care about anything but mathematical performance metrics and the price what can I say about the 15mm Panasonic lens? It makes very pretty photographs. They are sharp. Even wide open they are sharp. Not "Otus" sharp but very much Nikon-Canon-Fuji-Olympus sharp.

a 3:2 crop from the full frame. 15mm f1.7 Summilux. ISO 200

G9 Raw file. 

An unchanged detail from the bottom right of the top photograph.

How sharp is the word, "left"?

I'm happy I bought it and I ended up using it on several environmental portraits last week, shooting at f2.0 (which is almost wide open) and being very satisfied with the results. Even when viewed on the 27 inch iMac. I think it's pretty much the case that all of these lenses and many, many others are very, very good. Much better than most people's command and practice of techniques lead them to believe.

I'm sure Mr. Chambers is knowledgeable and his ability to harness technique is prodigious. I can only guess two things concerning his dismissive review of this lens. Either he got a bad copy (the problem with limited data samples) or he got the idea in his mind from a glancing usage that he was unsatisfied with the lens on a day when his cat was having a bad hairball day and the idea turned into a existential reality for him that he was not able to objectively overcome.

Bottom line? My singular sample point is a very good lens with very good imaging potential and easy to use. Go test your own. Or ignore this altogether and just continuing enjoying whatever exemplary lenses you bought to go with your brand of camera.

But when you read a reviewer's description of a lens always keep in mind that they are busy trying to sell you stuff and get your fingers to click through to a merchant, or whatever, and they are generally only working with one sample, and for a very short amount of time. You can generally do better by taking your camera to a reputable dealer, putting the lens you are interested in on the front of your camera and then shooting and shooting and shooting. Camera stores sell tripods. Put your camera and the test lens on one of their tripods and shoot, shoot, shoot. Bring a model with you. Bring your kid. Bring a test chart. And then you'll really know whether or not you like the lens, want to buy the lens, and if you have  any future interest in reading lens reviews that are "subjective", and "real world." as opposed to smart guy testing such as is done on Lenstip.com and a few other sites.

Me? I read the blogs by purported experts and then, if I'm interested enough, I go and try the stuff myself. My way. In my style. And if it works for me I buy it.