DxOMark results are...

[Lance B]

I agree it is a rather unscientific test, but think about this; they rate the 5D3 and the D5100 only 1 point different, are you suggesting they are capable of similar results? I can't imagine getting an image straight out of the camera from a 5D3 like the first one above. If that were the case why bother getting better bodies, just invest in glass, which reinforces my thoughts that such ratings are a waste of time.

No I am not suggesting that the 5D3 is only 1 point better than the D5100 overall. The 5D3 is far the better camera, what DxOMark measures is sensor performance, not camera performance, which is two different things. The 5D3 has much better AF and faster frame rates, better built, more comprehensive menus and customisability. This is whay the 5D3 is by far the better camera over the D5100.

It's like saying a WRX is better than a Porsche simply because it has more HP or whatever. It just ain't so.

[zollo]

It's well known that Nikon don't apply as much 'polish' to their in camera jpegs as canon do. Probably takes a bit of getting used to after the over saturated jpegs canon puts out.

Also I can't see/tell where the focus is in your example, its certainly not on the person, camera error too... Maybe.

If you really want to see bad jpegs from a camera use ISO 400 or above on the 7d ;-)

[I @ M]

Mic, I really think that you need to sit back, have a cuppa and look into the DXO methodology a little harder.

I'm not either leaping to the defence of or decrying the DXO results because I simply don't understand them and Lance ^ above is on the mark with his summation of the DXO tests.

DXO, simply put, don't produce any images in their testing. That means that lens IQ differences are a moot point. The actual image presented on a screen is also a moot point because the image output from the camera especially in jpeg form is heavily dependant on the processor in the camera and with their testing consisting of electrical input and output and mathematical formulae being applied only to the sensor and before the processor stage DXO are merely presenting numerical results.
Those results may or may not be able to be proven correct or otherwise but until someone agrees on a standard the results from DXO are representative of their and their alone, testing.

I think with your "testing" and then understanding of the published DXO results comparing the bodies that you have overlooked a few things.

With the DXO results, you have applied the wording "massively higher" and I don't see it that way at all. What does the difference between 80 and 66 represent? Is it a percentage, number of oranges in a kilogram or tonnes of wasted whale meat from the Japanese fleet between 2009 and 2012?
Looking beyond the final "score" and into the graphs and things that I can understand a little more like ev and iso I don't see massive differences.

As for your presented image being preventative of what a particular camera can produce and used as an example I'm afraid that it simply does not work.
Too many questions are needed to be asked about that shot before any credence can be paid to the performance of the sensor ( which after all is what the DXO tests are about ) and you need to answer them thoroughly to be able to substantiate any observations about either the end image quality or the performance of the sensor.

From the very start.

#1 What picture control was used in the camera at the time of the shot?
#2 What alterations were made to that picture control, either on purpose or accidentally by the previous user of the camera or by yourself?
#3 What white balance settings were used and what alterations to the tint were applied?
#4 If the D5100 has the facility within the menus, as other Nikon models do, to set an overall exposure compensation adjustment without displaying that compensation adjustment, was that enabled?

Looking at that image and applying some very minor adjustments in the software that is supplied with that camera suggest to me that it was taken in either standard picture control with the contrast reduced ( even maybe no lens hood in place? ) or in the neutral picture control.

Either way, I don't see that image as representative of the mathematical testing of a sensor or as an indication of IQ from a particular camera.

[farmer_rob]

It seems to me that the DXO test results are technical evaluations of sensor performance, not of the secret sauce that Canon and Nikon use for converting raw sensor data to jpegs. (IIRC, DXO do tests on RAW files only.)

I think you have picked a difficult image - at least for NIkon cameras. It took me quite a lot of work to figure out the correct camera settings for black cattle and black dogs - the default nikon exposure calculations give you pictures like your first one, or (if you are not careful) pictures with black featureless masks and nicely exposed surrounds.

It may well be that the Canon exposure/processing calculations assess the scene "better" given the need to nicely expose the animals without washing out the rest of the scene.

These exposure and processing settings are not assessed by the DXO tests. Instead the tests just provide plenty of "canon" fodder for nikon fanboys :-)

Regards,
Rob

[ricktas]

I would ask. When you borrowed this D5100 did you do a complete reset of the camera? I see you initial photo as being SOOC, so what settings were used (not the ISO etc) but shooting in JPG lets you choose a huge range of things in the camera menu, like saturation, sharpening, etc. Did you reset these to default and then adjust them to how your 7D is setup for you?

Without doing a thorough scientific test where all settings are set to as similar as possible, and similar lenses used across cameras, any testing like this in the real world is going to have a negative effect on the results from one of the cameras compared to the other. We need to know ALL and EVERY setting in both cameras are the same, before we could even begin to discuss the variations. DXO do this, they setup the cameras under as similar environments as possible and then test them.

[ricktas]

You were also using a superzoom (55.0-300.0 mm f/4.5-5.6), basically a kit lens, not known for producing L quality results. There are way to many variables here to determine anything, either with your photos, or the DXO comparison.

[etherial]

Thanks guys for contributing to the debate, makes for interesting reading.

You are right, I didn't go through menus to check every setting, but knowing the owner of the camera, they use it on full auto all the time and hadn't had it very long so I can only assume that those settings were left alone. As I said from the top, this wasn't a scientific test, I didn't even set out to do a test, just borrowed the only camera I could get my hands on to take some pics of my boy working at the trial. All my points are anecdotal, but one thing it is, is real world experience.

In any case all the above from all of you supports my thoughts that there are way too many variables to consider to make these ratings useful to apply to real world situations. Sure if you want to drill down and analyse and understand every test in detail you might get something out of it, but from camera sensor to computer screen or print there are many more important factors that contribute to final image quality than the sensor and its DxOMark score. So my advice for the punters out there looking for a camera, don't take any notice of the scores, there are more important things to worry about when choosing your next camera (one being the lens!!).

[farmer_rob]

More importantly, did your boy work well?


Regards,
Rob

[etherial]

Yes Rob he did pretty well. He's almost 10 and only been trialling for a couple of years. The cows were a bit of a challenge for him, they are a bit more stubborn and he failed on those runs but he tried hard. He did some intermediate sheep runs in the afternoon and passed one to get another title so it was a good day. More photos here if you're interested.

[ricktas]

You are right, I didn't go through menus to check every setting, but knowing the owner of the camera, they use it on full auto all the time and hadn't had it very long so I can only assume that those settings were left alone.

HEHE. When you get a new camera, most people take a few shots then start 'playing' with the menu's. It is so easy to adjust something (even without realising) and then forget you have done it.

[arthurking83]

I think Rick is starting to hit on the point here Mic.

You have to remember something here too, Nikon's D7000, Pentax's K5 all have very similar DxO scores for a reason .. they all use this same sensor!!

Where the D7000 and D5100 only score 1 point less than the 5DmkIII, the K5 scores 1 point higher than the MkIII.
The pattern repeats itself due to the fact that Pentax and Nikon have eeked out everything they can from the Sony based sensor.
If you also place Sony into the scoring mix, again their camera's that also use this same 16Mp sensor also score well too!

I think the major point here is not that DxO is bollocks .. but that it's bollocks for your situation!!

You don't use DxO's software, so their results are useless for your purposes.

If Adobe produced some testing procedure that was similar to DxO's, using their software, then you would be singing their praises being so perfectly spot on.

You're comparing apples to cheese here, due to your workflow method.
You're reading on how DxO software evaluates the raw files, but using Adobe software to try to achieve the same results!

I can tell 'ya straight up, I can't the same look from my raw files using LightRoom, as I get using Nikon software(and this is the free Nikon software too boot!).
The free Nikon ViewNX2 always gives me nicer looking images instantly if I've got the capture right on.


Now the other issue I can see that may have tainted your unscientific results, is that by default, the D5100 is set to use Active D-Lighting, and it may be set to auto(this is how I found my brother's D5100! .... can't really remember, but ADL is not an ideal feature to use all the time. It's handy for specific conditions, but when conditions are such that it's not required, ADL will set your images to look flat and washed out.
I reckon this is what may have happened. Go to the Camera Menu item(Shooting Menu) and scroll down till you find Active D-Lighting. Any setting other than Off will produce slightly flat looking results where the histogram is all bunched up in the middle!!

On a raw file, Lightroom won't recognise the ADL setting you set, but the jpg will be affected so.

Also, Lightroom doesn't render the Nikon raw files as well as Nikon's software does. An exposure adjustment is ALWAYS required in LR, where it's not in Nikon's software, and this can be either up or down in Ev. I's very rare(for me) to have found a Nikon raw file that is exposed the same way when comparing Nikon's software with LR.
Again, as it's a raw file, it's simply a difference of opinion(between Nikon-Adobe-DxO-and others) in the way the file should be rendered.

While I'm not a fanboi of DxO's, I still respect their results, as it's most probably very pertinent when using their software for evaluating or comparing various cameras.
But if you're only using other software, then the recorded results may not coincide with the theory.


As a quick test: If you have access to FSViewer you can use this to compare how the images look between software.
Open any raw image in both FSViewer and Lightroom, and notice the difference in rendering between the two software.
It'll be blindingly obvious if Active D-Lighting is enabled.

[etherial]

Thanks Arthur, all very valid points... which reinforce my view that high DxO scores will not lead to happiness.

[arthurking83]

..... which reinforce my view that high DxO scores will not lead to happiness.

Possibly not, if you're preferred raw converter is not DxO's own!!

This is why I mentioned FSViewer.

With FSViewer, there are options to render the embedded preview file(the jpg in the raw file), and also to view the raw file via the raw file data(slower image rendering).
This clearly shows the difference between raw file viewers/converters more aptly than most other software.

eg, in Nikon raw files, the embedded preview file is almost exactly as Nikon software will render it, too, which is as the camera captures it .. so Nikon software will render the raw file as you saw it in the camera, whereas third party software is always going to render the file differently.
So it goes that DxO renders raw files differently to how Adobe's raw converter(ACR) will render the file.
This is also complicated by the different manner in which each manufacturer's raw files are handled by the various software too.

maybe ACR has a better understanding of how to interpret Canon raw files than it does for Nikon raw files.

Basically it's as simple as this: DxO's rankings are meaningless as a single source of information without the use of their software.
Unless we find ourselves in a situation where all the various manufacturers use DxO's raw conversion engine to display and edit images, DxO's reviews are only one guide amongst many others on the topic of relative camera performances.

So, with all things information related, you use DxO's reviews as part of a whole swag of reference reference material for determining how a camera performs against another.
For Adobe users, DPR(who only seem to use ACR as their main raw file conversion program) would be a better source of info.

FWIW: if you took FSViewer's interpretation of Nikon's D800 raw files as the only source of information as to how the raw files look, and hence how the camera performs ... you'd be seriously disappointed!
Great software and I think it's the best at what it does, but I never use it for raw file conversion.

I can't remember DxO all that well other than it never impressed me much(long time ago), Bibble(now After Shot Pro) was fast but awkward, and I ended up with LR for any of my non Nikon raw file handling.

[patrickv]

Damn, I own a 7D and I don't even know what its JPEG files look like ...

[arthurking83]

Damn, I own a 7D and I don't even know what its JPEG files look like ...

They either look like whatever you want them to look like, or they look like what's displayed on the review screen!

You're not seriously going to tell us you've never reviewed an image on the review screen, are you?

[agb]

It's not only the sensor scores that give a bit of heartache. Try having a look at the scores for your lenses.
Especially Canon lenses.
According to DXO a score of 10 is required for a lens to be rated as capable of producing an excellent 8"x12" print on a home printer.
The 70-300mm f 4-5.6 L lens on the 7D gets a rating of ...............you guessed it 10.
How does that relate to the MTF charts for the lens that Canon produce which shows that this lens is a really good lens.
http://www.usa.canon.com/cusa/consum..._4_5_6l_is_usm
Surely it can be good enough to produce better than a 8x12 print?
Users of the 70-300 have rated it as good as the 70-200 f4L IS which is also supposed to be a good lens, but it also only gets a 10. No telephoto lens on the 7D gets a rating good enough to print a 16x24, and most of the telephoto lenses would not give a print bigger than a postcard.
http://dancarrphotography.com/blog/2...0-200-f4-l-is/

[Tannin]

Your comment that they rate the D200 higher than the 5D MkI in terms of low light performance intrigued me however as it seems ridiculous that they would do so, so I looked up the scores. Seems to me that they rate the 5D MkI significantly higher (as they should) than the D200 in terms of low light performance.

Yeah. they do now, 'cause that was one of their stupidest mistakes and they had no choice but to fix it. But each new version of DXO seems to make a habit of throwing in a different howler. Etherial started this thread with yet another one. (I should mention that I'm going on memory for my model numbers, it may have been some other equally ridiculous finding that first drew my attention to DXO all those years ago. But it doesn't matter in any case, there are plenty of other howlers. The sensible thing to do with DXO is simply ignore it in favour of other methodologies that produce more rational results.)

[Eberbachl]

Remembering that they're rating sensors and not cameras, most of the results I've read since stumbling across this thread have seemed pretty credible thus far. If you have any current examples of howlers I'd love to read them.

[Tannin]

^ Etherial started the thread with one. Given that we already know the program makes huge mistakes, what would be the point ion looking for more examples? I've got 1001 better things to do with my time, but you could always go looking for yourself if you are bored enough. Not me. Life is too short!

[JM Tran]

^ Etherial started the thread with one. Given that we already know the program makes huge mistakes, what would be the point ion looking for more examples? I've got 1001 better things to do with my time, but you could always go looking for yourself if you are bored enough. Not me. Life is too short!

Im with you there, DxO is rubbish as established. Less debating numbers and decimal points, and more photo taking needed!