I decided to do a quick test between my newly acquired, but used, Canon 17-40 L and my beloved, but thoroughly abused, Sigma 17-70. The results, well, I'll see what you guys think.

Test Info: The camera was put on the tripod, focus was locked on the front of that post thing in the crops, then switched to M. All shots were at 17mm with the same exposure values between lenses. No processing was done. A tripod, remote release, and MLU were all used as well. Also, these were at ISO 100.

Also, I should admit that I don't really know what a "100% crop" means, but I did them by viewing the shot at 100%, cropping what I wanted, and saving it with no change of resolution at any point. I hope thats right.

SEE BELOW FOR SHOT-BY-SHOT COMPARISON.

What do you guys think? Any glaring errors in my test procedure that might have caused an issue?

- Scott

Looks to me like the Canon might have simply been out of focus. I'd set up something more controlled indoors with a subject with more contrast to ensure correct focus. Not really an easy image to compare, also from the f4 result I'm fairly certain something is suspect with how you did it.

I'd agree the focus looks like it could be off on the canon, but it's twice the price because it covers twice the sensor... Put them both on a FF and see what wins. IMHO the 17-40mm is only worth while on FF because of the cheaper/better speced 1.6 specific lenses.

Originally Posted by jacobsen1

I'd agree the focus looks like it could be off on the canon, but it's twice the price because it covers twice the sensor... Put them both on a FF and see what wins. IMHO the 17-40mm is only worth while on FF because of the cheaper/better speced 1.6 specific lenses.

Hmm... good point, I didn't think of that.

And yeah, I thought focus might be slightly different... but, in a real world situation, if I lock the focus on something, I'd expect it to be right the first time. I mean, I don't want that kind of result in the field...

- Scott

It does appear to be a focus issue, meaning that the 17-70 appears to have more in focus than the 17-40. This led me to look at the 17-70 f/4 and f/9 / f/16 shots you posted - they all look nearly identical, so are you absolutely sure that the f/4 shot is really at f/4? The 17-40 seems to exhibit the correct amount of DOF/bokeh for that aperture, so why doesn't the Sigma do the same?

If the focus was to be on the post, then the Canon blew it, and by a wide margin, so could you re-run the test to make sure that this wasn't a one-off mishap? Most comparos do a best-out-of series, which not only checks sharpness, but also focusing consistency.

First off, I do plan on re-doing the tests a few times. The focus could have been off a bit, but I think there just might be a pretty good difference wide-open. The Sigma is stopped down 1-stop (from f/2.8 ) which should help it where the Canon is wide open. Either way, I don't really care below f/9 or f/10 what it looks like, I NEVER shoot at those apertures when its more than a snapshot.

Second, the Sigma 17-70 isn't compatible with the 5D, it only uses EF lenses, not EF-S so thats the first reason why all the big-time landscape photogs use the 17-40. I'd keep my 17-70 in a heartbeat if I could, I love that little thing. Its been to hell and back and just won't quit. Even the glass is still flawless (I cleaned it today before the test).

Next, as Ben said, the 17-40 might also look better on a full-frame sensor (or non-crop sensor) like it was designed for.

Who knows, I'm a little disappointed but I'm not writing the 17-40 off just yet.

- Scott

As much as the possibility of a focus issue is real, my point was the focus issue could have been you, not the lens. It's possible the 17-70 is better at f4, however I think your results are not indicative of the performance I've seen from the 17-40 so either as mentioned the lens itself has an issue, or you made a simple mistake.

That's why it's so important to repeat the test a couple times, reacquiring focus and re-evaluating. I personally think though that a better focus target will make this easier. I like office buildings with good contrast between the windows & walls for this, however in the home plenty of home made focus targets are just as good and often easier to work with.

Okay... here's ROUND 2, the results look a little better (it looks like the 17-40 definitely renders colors/contrast better):

17-40, f/4, 100%:


17-70, f/4, 100%:


17-40, f/16, 100%:


17-70, f/16, 100%:


17-40, f/4, full:


17-70, f/4, full:


What do you guys think now? I like this concept of "keep testing it til you get the results you want". But seriously, I think I did a better job making sure the focus was right on these. Focus was on the "0" in 17-40 by the way.

- Scott

Originally Posted by emorphien

As much as the possibility of a focus issue is real, my point was the focus issue could have been you, not the lens.

Definitely. See the new try (and I did this one 3 times with very similar results).

Originally Posted by Stime187

What do you guys think now? I like this concept of "keep testing it til you get the results you want". But seriously, I think I did a better job making sure the focus was right on these. Focus was on the "0" in 17-40 by the way.

It looks like a better comparison with more control going on now. And the 17-40 looks more like how I'd expect it to. Of course there could be differences across the focus range, an infinity check and somewhere in-between isn't a bad idea since this looks like a closer focus setup.

It's not trying until you get the results you want, you just needed the help of those familiar with the lens to know bad data when they see it. Which is more valid: testing until you get results that seem more reasonable, or immediately accepting possibly faulty data?

Plus, now you can sleep better at night without worrying that your 17-40 might be a dud.

Thaaat's more like it, and still shows that the Sigma is a damn good lens for the money.

How is the 100% crop a different size for the same image on the 2nd test? I dont understand.

It just means it's a portion of the image zoomed at 100%, he can take as much or as little as he wants, depending on how annoying he wants to be to us when we open the thread.

I understand that, but it looks like the same portion ends up being a different size.

I think its because the 17-70 seems to be slightly wider @ 17mm so by cutting the same portion, its slightly different. Notice that both the 17-70 100% crops are near the same size as are both of the 17-40s. But, I'm not sure that actually explains it, more of a theory.

- Scott

Originally Posted by anotherD80

all in all, this just adds to my confidence in buying Sigma lenses. i'm amazed they are this similar.

As it should, they are much better than many give them credit for. Like I said, I will have distinct reasons to get rid of mine... I wouldn't otherwise.

Interestingly enough, I also have the Sigma 70-200 f/2.8 and the Canon 70-200 f/4 L right now, I might do a comparison between those two one of these days when I get some time.

- Scott

Hmm you're right, I missed that. Maybe the 17-70 is actually a bit wider than the 17-40.

Originally Posted by Stime187

Next, as Ben said, the 17-40 might also look better on a full-frame sensor (or non-crop sensor) like it was designed for.
- Scott

My point wasn't really that the canon will be better, just that it will work on FF where the Sigma won't. So it's technically a million % better, but by default. Testing on a cropped body will show you which is optically better in the center. As we've both been saying forever, the 17-40mm L should only be considered if you're getting a FF body. Otherwise you'd be an L collector to consider it as the sigma/tamron/tokina options make more sense on a cropped body.

Originally Posted by Stime187

Okay... here's ROUND 2, the results look a little better (it looks like the 17-40 definitely renders colors/contrast better):

What do you guys think now? I like this concept of "keep testing it til you get the results you want". But seriously, I think I did a better job making sure the focus was right on these. Focus was on the "0" in 17-40 by the way.

- Scott

Yeah, people do tend to defend lenses until the results they want come out... These look much better. One weird thing I'm seeing is that the L seems to have better bokeh at F4 (look at the Sam Adams bottle)? Anyone else notice that? I wonder if the sigma is slightly off in what it call f/4 to what canon calls f/4...

BTW I've had lens test where one lens (sigma) BLEW the L out of the water only to rerun the test again and get different result... not sure what does it, my technique, lens issue or what, but it happens.

Originally Posted by anotherD80

all in all, this just adds to my confidence in buying Sigma lenses. i'm amazed they are this similar.

Yes. I have a few issues with sigma, but none are really IQ related. Their build is great on their EX DG/DC line, but their paint sucks. Similar to subaru?! Also their focus isn't as fast as canon's (I'd assume this is similar with other mounts too) but is still good/great... Optically I was and still am very happy with my sigmas. The 24-70mm f/2.8 went because it didn't have HSM so it focused too slow on the 5D for me. Weather sealing is a nice plus on the L on my 1D when it's absolute crap outside. But it was a very hard choice, especially considering the $$$. The UWA issue for me was tough. The 17-40mm L is sharper than the 12-24mm but they are very different lenses. The 12-24 is a great lens and the best UWA on a 1.3 IMHO. The 17-40mm is better for FF though. For the 70-200mm I love my sigma. Always have. It's cheaper than the canon and very good. If you don't need IS definately consider it.

Originally Posted by Stime187

I also have the Sigma 70-200 f/2.8 and the Canon 70-200 f/4 L right now.

For obvious reasons I'd love to see that... Although for me I've been considering selling mine for the 100-400... I'd miss the f/2.8 for sure, but sometimes I really want the longer range. They are close in size/weight so it would be nice to double the long end. Price is a lot different though.

I don't generally refer to that as bokeh since bokeh as i know it is the quality or shape of OOF highlights. It just plain looks like the Canon might have slightly less DOF. However the sigma appears wider, so it's a bit tough to tell. It could also be a very slight difference in focus, but who knows?

Originally Posted by emorphien

I don't generally refer to that as bokeh since bokeh as i know it is the quality or shape of OOF highlights. It just plain looks like the Canon might have slightly less DOF. However the sigma appears wider, so it's a bit tough to tell. It could also be a very slight difference in focus, but who knows?

Yeah, good point. It is more the OOF from the DOF, but you see what I'm saying right? The different FLs could be to blame for that though as wider lenses at the same aperture will have deeper DOF...

I noticed it too, but attributed it to a focus issue also. Just seems like slightly different specs at various focal lengths. My Nikon 17-55 doesn't actually make it all the way out to 17mm, according to Thom Hogan and other reviewers (have not tested it myself). The Sigma vs. Canon may be a case of the same thing, where both are close enough to 17mm to call them 17mm lenses, yet there is enough variance to cause a difference in the images.

Then when you consider lens to lens sample variation among each manufacturer, and I don't think you're ever going to get exact matching images.

M

I agree on the Sigma bokeh (this refers to all OOF stuff, not just highlights - http://en.wikipedia.org/wiki/Bokeh ) looks odd. I experienced this with the 24-70, the 50-500, and even the 300-800. It's nowhere near as nice as the big Canon glass bokeh. Oh well, small price to pay for otherwise excellent lenses.

Bokeh applies to all OOF, it's most noticeable on highlights, and it has nothing to do with magnitude of the blur b/c that depends on your aperture. That's why I suggested this isn't a bokeh issue but simply either a change in focus or the amount of DOF.