The Best Way To Test Your New Camera or Lens (or Both).

shot for CTRMA Annual Report.

The last few weeks show us that reviewers of cameras and lenses can: be subjective, make mistakes, mis-focus, mis-understand how to use new cameras and.... use the new cameras in a manner that may be antithetical to your actual, unique way of working. You may need a lens with smooth bokeh while one reviewer only values sharpness. You may need resolution while another reviewer may only value the bokeh of the lenses under review.

You might need a camera with fast frame rates while your friend who shoots landscapes and human portraits wouldn't care if the camera in question only had single frame shooting capabilities. I prefer EVFs while you might prefer optical viewfinders. When I review a camera (a rare occurrence these days) I mark a good electronic viewfinder as a big plus. If you love looking through glass and mirrors you'll probably mark the same camera down. The same goes for relative size and weight.

After reading about some tests failures, and then skimming through nearly a thousand comments about the vagaries of lens testing (and especially testing one brand of lens on a different brand of camera --- and then bitching about the AF performance!) I've come to the same conclusion that I came to many years ago. If you want to know what a camera and lens can do you'll need to load up and shoot the gear the way you like to shoot the gear and then see if it works for you. After all, if the way a camera feels in your hand is terrible it will hardly matter if its low light performance is 93 and its closest competitor is a 92.5. If you're smart, and plan to have the camera around for a while, I'm guessing you'll forgo the point five and get the camera that feels best. You might be disappointed once you start pixel peeping at 300% but the rest of the photographers (the ones who are not pathologically obsessed) will probably come to the understanding that it's a combination of handling, features and final image quality that matter and not one single parameter in isolation.

I've read reviews that ding cameras for not having just the right touch screen (as if that matters in the real world). Even more absurd, I've seen reviews that list as "cons" not having in camera raw processing. That, of course, is just insane. To me. People swooned and fell to the ground in agony when it was revealed that Fuji ISO settings might be less sensitive than comparable values in different brands.
And then there are the hordes who think the camera should focus autonomously and everywhere while thousands of objects move through the field of view. They call this focus tracking. Fourteen people in the world really need focus tracking.

The image above was shot for an annual report. By the time I made this shot I'd already used the camera for over 10,000 previous images. Same with the zoom lens I was using. I'm certain that I could have used a Sony or Nikon or a Canon (or Olympus, Fuji or Pentax) to make the image and my choice would have been transparent to the final viewer; the person looking at the image on coated paper stock that had been screened at 300 dpi and printed with four colored inks on moving sheets of paper. I was equally certain that any standard, slightly wide to slightly tele, zoom lens would have been more than up to the task to resolve all the detail I needed to do the job.

But it made me feel confident that I, personally, had tested the camera over and over again up until this point. I knew that the model I used, along with the lens I used, would be "good enough" to do the job.

This image may not at all reflect the way you use a camera. For me almost everything is done with intention and a modicum of control. Even when photographing a meet and greet with a former president and dozens of VIPs I like to take my time and get all the parameters just right. In the above example the image required lighting. We had to float a scrim over the subject to kill the direct sunlight. I set up an Elinchrom Ranger RX AS flash system with a softbox to get the lighting I wanted. None of this is random. None of this requires 20, or 12, or 10, or 8 or 5 or even 2 frames per second. 1200 watt second battery flash units, used at full power, couldn't recycle that fast anyway. None of this required focus tracking as the subject was confined to the pool of shade created by the diffusion.

Most pros shoot on a tripod in order to lock in composition. Then they can be sure the framing and subject relationships don't change while they are shooting, bracketing or just goofing around.

We've gotten to a point where camera reviewing is lost in the woods of generality. Every feature becomes a big deal; or a deal-killer. Every camera must be equally good at high operating speeds, resolution, handling, compactness, button intelligence, and the endless ability to be endlessly customized.  The reality is that in photography's most pure form the only thing that matters is: "Are you able to take the photograph you want with this machine?" Does it do what you want? Is the image sharp enough? Is the shutter fast enough? Is the sensor "quiet" enough at the ISO settings you need  to use? Are you able to focus the camera well enough? Once you've satisfied these basic requirements every other caveat, argument, pinhead dancing angel recital or bemoaning about the lack of 30 discrete steps of exposure bracketing is just showing off how weak you are as a photographer. And if you don't really use the camera for making art; if you just use it to make reviews, you are revealed as nothing more than a person who picks at nits on the edges of a craft with no real value (other than providing the entertainment of reading) to offer. Well, no value to the consumer but appreciable value to the manufacturer.

Touch screen? Might be as fun as a video game but not a photographic necessity; a working shutter is a necessity. In cars it's nice to have cupholders but they are not a necessity; brakes are a necessity (if you actually drive). In-camera raw processing might be convenient in the way heated toilet seats may seem convenient; but neither is a necessity. Having an idea about something to photograph is pretty important.  And on and on. Among their reasons to exist I think reviewers are here to make new photographers want the new features that no one asked for but which come (like glued on rhinestones) on the latest cameras. How else to make more differentiations between products? How else to provide continual gear fodder from which to distill valuable clicks? Wouldn't it be nice if reviewers just came back after three months of continually using the same camera and then told you what they liked and disliked about it; and then showed you several hundred interesting photographs that showed what that particular camera could do? Modern miracle, I'd say.

So, if you have an interest in a new camera because you need a new camera here's how to proceed. Grab a memory card and head to your local camera store. Don't have a local camera store? Get on a plane, go to New York City and go to B&H Photo. Or fly into Austin for nachos, margaritas and a trip to Precision Camera. Have the sales clerk put your favorite lens on the camera you are interested in, or bring along your favorite lens to put on the camera. Play with the camera for half an hour or an hour. Go through the menus. Practice the focusing. Shoot a bunch of test shots. Bring your hot girlfriend or studly boyfriend and have them pose for shot after shot after shot. Shoot raw. Shoot Jpeg. Use the buttons you would normally use. Handhold the camera. Put the camera on a tripod. Then take your memory card home with you and look at the images in your usual app. Blow em up. Print your stuff on your regular printer. Sit back and think about the camera.

Once you've decided on a camera and have tested it as above, buy it from a place with a good, fair return policy. Take the camera home on a Friday night, charge the batteries and read the manual forward and backward. Take the next week off from work. Shoot all day. Process all night long. All week long. Wear the camera to breakfast, and lunch and dinner. On the day before you would have to send the camera back (presuming you don't like it; didn't warm up to it, etc.) sit down and make a list of the pros and cons as you see them. Make a dispassionate decision. Is this the camera or lens for you? Are you still happy when you use it? Do the images look great to you? Can you afford it? If you decide to keep it that's great but the testing doesn't stop until you never need to glance at the manual again and you know with a fair degree of certainty what YOU will get out of the camera as you are shooting it.

The hell with the reviewer's expectations, or list of pros and cons. He's not paying for your camera. He doesn't shoot like you. He's got a job. His job is to create ever-fresh content about cameras to drive you to his employer's website. The website gets paid by bringing potential buyers in close proximity to paying advertiser's products. They hope the supposed objectivity of the site's content will confer subliminal value to their advertiser's products and that you will buy one of the products. The reviews really have nothing to do with how real photographers use real cameras in their jobs or in their hobbies. If configurability is the leading new feature of a camera you are probably already looking at the wrong camera...

Added 6-27: an interesting article about manufacturing and testing tolerances: http://www.imx.nl/photo/optics/optics/page62.html


Wally said...

See. You don't understand...Bokeh See you don't understand... View Cameras See you don't understand, in fact, you really don't understand....Heavyweight tripod, See you don't understand. OVF EVF; See you don't understand. Sony See you don't understand, Nikon See you don't understand, Canon See you don't understand. See you don't understand, See you don't understand, See you don't understand.

Sigma Foveon Sensor What...... See you don't understand....

Kirk Tuck, Photographer/Writer said...

Wally... I don't understand......your comment. Can you flesh this out? I'm very curious.

Art in LA said...

For movies, I try to find a reviewer with similar tastes to mine ... minimizes my disappointment risk. I like reading camera reviews, but for my entire life, I have pretty much stuck with the Minolta, Konica-Minolta and Sony options. That early branding works! I like Coca Cola, Levi's, adidas and Honda products for similar reasons. I know, weird.

Anonymous said...

Hi Tuck, Boy did you hit the nail on the head. Glad to hear that your son is back safe n sound, trips and going to school overseas can be adventurous for sure.


Anonymous said...


You might find these comments by Erwin Puts on the topic of lens reviews interesting:



Ps. A similar approach to car reviews by certain consumer protection groups is probably why most of us drive SUVs...

crsantin said...

You are making too much sense here Kirk, cut it out.

Joe Dasbach said...

Well put, Kirk!
I like your perspectives
and your isolating what's meaningful
amidst the abundance of camera chit-chat.
The signals to noise ratio.


Andrew said...

While overall I thoroughly agree with the point you're making, I'm a bit surprised at the specific calling out of a touchscreen as a bone of contention.

A touchscreen is literally a usability feature. Nothing more, nothing less. Class it in exact the same category as an AF joystick, or multiple command dials for exposure control, or customizable function buttons, or an automatic EVF eye sensor, or multiple autofocus points. Or even a large, comfortable grip. Literally none of those are requirements for most photographic applications. But I imagine that you, like most photographers, would still prefer to have them.

Once you've experienced the joy of a touchscreen, particularly one that lets you instantly and accurately move the AF point even with your eye to the EVF, why would you ever want to go back? Or maybe we should all just use a center AF point and focus / recompose. That works for almost everyone, too. Most people probably wouldn't call it ideal, though.

I am still thrilled to make pictures with meterless fully manual film cameras, and even something as rudimentary as that rarely feels like a burden in use. But good user convenience features like touchscreens, or automatic exposure modes, or the others I listed above simply make the camera more transparent to use. They let you focus more on the concept, the composition, the lighting, the picture you actually want to make. That seems to me like something you would be fighting for.

Kirk Tuck, Photographer/Writer said...

Hi Andrew, I have two current cameras with touch screens and both add complexity to operation, often are actuated inconveniently by my nose or face and offer very little usability for me personally. I shoot with my eye to the EVF not by holding the camera away from my body and scrapping across the rear finder with my fingers. I can use the 4 way touch pad on the camera to position focusing points anywhere on the screen and they don't move on their own as they sometimes do with touch screens. It's mostly a feature looking for a problem to solve. Hate them. I don't think others should be deprived of them if they feel they need them, I just question if it's a positive or net negative.

Wally said...

Re: See you don't understand. Refers to anything you don't necessarily agree with and someone else thinks is stupid despite you having a good reason.

What's the best single way to improve image quality? In my opinion and experience using, a tripod and I would argue a heavy tripod. Does this interfere with being a hipster and doing run and gun shooting letting Auto everything take over making decisions on your behalf....See you don't understand....Sony vs Canon? See you don't understand.

It goes on and on. Today Donald Trump asked a female Irish reporter over in front of the press and its sexist while Ronald Regan did the same thing with a female reporter from South America and no one made any comments... See you don't understand...

Michael Meissner said...

While I don't use the touch screen all that often, preferring to use the arrow keys to set the focus point, but there are a few times when I use the back LCD and touchscreen. These are all somewhat specialized uses.

Mostly I use it when I have the camera mounted on a tripod setup to capture a stage production, and it is much faster to use the touch screen to set the focus point. If I used the arrow keys, the actors would likely have moved by the time I select the focus point (normally I select the focus point ahead of time and just move the camera, but not always).

Similarly, when I'm in the back row of a production and I have the camera on a monopod to get it over the audience heads, I can use the touch pad to set the focus point with the LCD tilted down to just over my head, and shoot with a wired shutter release. In this position, it is awkward to use the arrow keys. Note, I only use the monopod if nobody is behind me, because I don't want to block other peoples views.

The most specialized use is where I have my E-m5 mark I and 12-50mm lens in a wooden box made to resemble an old 4x6 bellows camera (with a lot of extra stuff). I have cut out the back of the box where the LCD is and use that to shoot images. Since I have one hand holding the box, I find it much simpler to set the camera into the touchscreen focuses and takes the shot mode, and when I ask to take peoples pictures, it is simple to just touch the screen on the person's image and take the picture. In previous generations of the box, I used cameras without a touch screen, and I would have to move the camera until the subject I wanted was in the focus spot. Sure, these are snapshots, and not serious photos, but hey, I mostly take pictures to capture memories.