The Best Way To Test Your New Camera or Lens (or Both).

shot for CTRMA Annual Report.

The last few weeks show us that reviewers of cameras and lenses can: be subjective, make mistakes, mis-focus, mis-understand how to use new cameras and.... use the new cameras in a manner that may be antithetical to your actual, unique way of working. You may need a lens with smooth bokeh while one reviewer only values sharpness. You may need resolution while another reviewer may only value the bokeh of the lenses under review.

You might need a camera with fast frame rates while your friend who shoots landscapes and human portraits wouldn't care if the camera in question only had single frame shooting capabilities. I prefer EVFs while you might prefer optical viewfinders. When I review a camera (a rare occurrence these days) I mark a good electronic viewfinder as a big plus. If you love looking through glass and mirrors you'll probably mark the same camera down. The same goes for relative size and weight.

After reading about some tests failures, and then skimming through nearly a thousand comments about the vagaries of lens testing (and especially testing one brand of lens on a different brand of camera --- and then bitching about the AF performance!) I've come to the same conclusion that I came to many years ago. If you want to know what a camera and lens can do you'll need to load up and shoot the gear the way you like to shoot the gear and then see if it works for you. After all, if the way a camera feels in your hand is terrible it will hardly matter if its low light performance is 93 and its closest competitor is a 92.5. If you're smart, and plan to have the camera around for a while, I'm guessing you'll forgo the point five and get the camera that feels best. You might be disappointed once you start pixel peeping at 300% but the rest of the photographers (the ones who are not pathologically obsessed) will probably come to the understanding that it's a combination of handling, features and final image quality that matter and not one single parameter in isolation.

I've read reviews that ding cameras for not having just the right touch screen (as if that matters in the real world). Even more absurd, I've seen reviews that list as "cons" not having in camera raw processing. That, of course, is just insane. To me. People swooned and fell to the ground in agony when it was revealed that Fuji ISO settings might be less sensitive than comparable values in different brands.
And then there are the hordes who think the camera should focus autonomously and everywhere while thousands of objects move through the field of view. They call this focus tracking. Fourteen people in the world really need focus tracking.

The image above was shot for an annual report. By the time I made this shot I'd already used the camera for over 10,000 previous images. Same with the zoom lens I was using. I'm certain that I could have used a Sony or Nikon or a Canon (or Olympus, Fuji or Pentax) to make the image and my choice would have been transparent to the final viewer; the person looking at the image on coated paper stock that had been screened at 300 dpi and printed with four colored inks on moving sheets of paper. I was equally certain that any standard, slightly wide to slightly tele, zoom lens would have been more than up to the task to resolve all the detail I needed to do the job.

But it made me feel confident that I, personally, had tested the camera over and over again up until this point. I knew that the model I used, along with the lens I used, would be "good enough" to do the job.

This image may not at all reflect the way you use a camera. For me almost everything is done with intention and a modicum of control. Even when photographing a meet and greet with a former president and dozens of VIPs I like to take my time and get all the parameters just right. In the above example the image required lighting. We had to float a scrim over the subject to kill the direct sunlight. I set up an Elinchrom Ranger RX AS flash system with a softbox to get the lighting I wanted. None of this is random. None of this requires 20, or 12, or 10, or 8 or 5 or even 2 frames per second. 1200 watt second battery flash units, used at full power, couldn't recycle that fast anyway. None of this required focus tracking as the subject was confined to the pool of shade created by the diffusion.

Most pros shoot on a tripod in order to lock in composition. Then they can be sure the framing and subject relationships don't change while they are shooting, bracketing or just goofing around.

We've gotten to a point where camera reviewing is lost in the woods of generality. Every feature becomes a big deal; or a deal-killer. Every camera must be equally good at high operating speeds, resolution, handling, compactness, button intelligence, and the endless ability to be endlessly customized.  The reality is that in photography's most pure form the only thing that matters is: "Are you able to take the photograph you want with this machine?" Does it do what you want? Is the image sharp enough? Is the shutter fast enough? Is the sensor "quiet" enough at the ISO settings you need  to use? Are you able to focus the camera well enough? Once you've satisfied these basic requirements every other caveat, argument, pinhead dancing angel recital or bemoaning about the lack of 30 discrete steps of exposure bracketing is just showing off how weak you are as a photographer. And if you don't really use the camera for making art; if you just use it to make reviews, you are revealed as nothing more than a person who picks at nits on the edges of a craft with no real value (other than providing the entertainment of reading) to offer. Well, no value to the consumer but appreciable value to the manufacturer.

Touch screen? Might be as fun as a video game but not a photographic necessity; a working shutter is a necessity. In cars it's nice to have cupholders but they are not a necessity; brakes are a necessity (if you actually drive). In-camera raw processing might be convenient in the way heated toilet seats may seem convenient; but neither is a necessity. Having an idea about something to photograph is pretty important.  And on and on. Among their reasons to exist I think reviewers are here to make new photographers want the new features that no one asked for but which come (like glued on rhinestones) on the latest cameras. How else to make more differentiations between products? How else to provide continual gear fodder from which to distill valuable clicks? Wouldn't it be nice if reviewers just came back after three months of continually using the same camera and then told you what they liked and disliked about it; and then showed you several hundred interesting photographs that showed what that particular camera could do? Modern miracle, I'd say.

So, if you have an interest in a new camera because you need a new camera here's how to proceed. Grab a memory card and head to your local camera store. Don't have a local camera store? Get on a plane, go to New York City and go to B&H Photo. Or fly into Austin for nachos, margaritas and a trip to Precision Camera. Have the sales clerk put your favorite lens on the camera you are interested in, or bring along your favorite lens to put on the camera. Play with the camera for half an hour or an hour. Go through the menus. Practice the focusing. Shoot a bunch of test shots. Bring your hot girlfriend or studly boyfriend and have them pose for shot after shot after shot. Shoot raw. Shoot Jpeg. Use the buttons you would normally use. Handhold the camera. Put the camera on a tripod. Then take your memory card home with you and look at the images in your usual app. Blow em up. Print your stuff on your regular printer. Sit back and think about the camera.

Once you've decided on a camera and have tested it as above, buy it from a place with a good, fair return policy. Take the camera home on a Friday night, charge the batteries and read the manual forward and backward. Take the next week off from work. Shoot all day. Process all night long. All week long. Wear the camera to breakfast, and lunch and dinner. On the day before you would have to send the camera back (presuming you don't like it; didn't warm up to it, etc.) sit down and make a list of the pros and cons as you see them. Make a dispassionate decision. Is this the camera or lens for you? Are you still happy when you use it? Do the images look great to you? Can you afford it? If you decide to keep it that's great but the testing doesn't stop until you never need to glance at the manual again and you know with a fair degree of certainty what YOU will get out of the camera as you are shooting it.

The hell with the reviewer's expectations, or list of pros and cons. He's not paying for your camera. He doesn't shoot like you. He's got a job. His job is to create ever-fresh content about cameras to drive you to his employer's website. The website gets paid by bringing potential buyers in close proximity to paying advertiser's products. They hope the supposed objectivity of the site's content will confer subliminal value to their advertiser's products and that you will buy one of the products. The reviews really have nothing to do with how real photographers use real cameras in their jobs or in their hobbies. If configurability is the leading new feature of a camera you are probably already looking at the wrong camera...

Added 6-27: an interesting article about manufacturing and testing tolerances: http://www.imx.nl/photo/optics/optics/page62.html