storage and organization

[poider]

G'day, just wondering what are the better options for storage and organization of photos?
I have over 40,000 photos from my wife and I and our travels, usually I put the photos in folders with place names etc, but find it excrutiating when I want to look at said photos, any folder with 200 or more photos takes ages to load in and settle, as I am scrolling it continually updates and changes the pictures order etc.
Is there a way that I can put in a search in my main (parent folder) that will search for and tell me how many photos I have taken at 300mm or how many with centre weighted or whichever perimeter I Choose, I know if I put all my photos in one folder I can group them by many different perimeters but having 40,000 photos in one folder would probably crash??
Peter

[ameerat42]

This will do all or most of that.
Faststone Image Viewer. (http://www.faststone.org/FSViewerDetail.htm)

It also does some basic editing. Lots of people use it on AP.

[paulheath]

Lightroom

[Tannin]

Ameerat42 is spot on.

What you need is an image viewer with decent performance. There are dozens of image viewers, many of them very good. Rule One is don't even consider anything from Adobe - Adobe products are a byword for bloated, slug-like performance which would put a sloth to sleep and bring a fast computer to its knees.

Faststone is fine. XNView is excellent. I still love the ancient PMView because of its outstanding ergonomics. Then there is ACDC, IrfanView, and various others I can't remember. Any one those mentioned will be around 1000% faster than any Adobe product this side of Pluto. None of them have the slightest trouble loading large folders.

(To be fair to Adobe, none of their products are actually designed-for-purpose in the way that the programs mentioned above are. Image viewers are not the same thing as image editors. Adobe doesn't make an image viewer (unless you count Bridge, which would be stretching a point), they only make image editors which can, at a pinch, be pressed into service as a second-rate viewer. But if they did make one, there is no reason to suspect that it would be any less bloated, glacial, and awkward in use than the likes of Lightroom, Acrobat, or Photoshop. Go with one of those I mentioned, or an equivalent of your choice.)

[poider]

Thank you all I will look at faststone, I have photoshop elements 14 for editing and have never really liked light room.
Thank you
Peter

[ricktas]

Thank you all I will look at faststone, I have photoshop elements 14 for editing and have never really liked light room.
Thank you
Peter

Photoshop (even elements) is an editing package. Lightroom is an asset manager with editing capabilities. Lightroom does have wonderful cataloguing, when used correctly.

But as with everything from camera brand to lens selection, it is personal choice.

[arthurking83]

For file viewing I have a preference for Fastone Viewer too.

BUT!! .. (big but here). If I need to view a lot of files in one go, nothing beats XNViewerMP.
What I mean by that is that as you store files in specific folders, most viewer software forces you to view the images from within each folder.
So if you have 20K image files, in 1000 folders with XNViewMP you can leave it to display only the files within each folder, OR you can set it to display all the files within the most senior folder recursively(it's your choice).

It does take a while to build up it's cache when viewing a lot of files in one go(ie. recursively) but it still displays a very high number as you view them whilst it build it's cache.
On my last usage it took 10mins to build the thumbnail cache for about 1000 80+ Mb NEF images.
If you only have jpg files saved, it's super duper much faster. Raw files take a bit of time to decode/read.

What I didn't like about Lightroom is that it forces you to make a catalog .. ie. that you can view all files at once once they're catalogged.
it's OK in what it does, but I much prefer the option to do so or not.
And it's catalog is not just a simple catalog .. it's a monstrous behemoth of a database.
Mine once ended up at over 6Gb with barely half my 200K images forcefully catalogged. I used to regularly delete it(only as I never wanted/needed another catalog of images).

On my primary and secondary archive locations I have about 200K images all up(jpgs, tifs and mostly NEFs).
XNViewMP will take about 20-30mins to rebuild the thumbnail cache(noting that i always clear it out, or delete it). So every time I use XNViewMP I basically start over from scratch.
For me this is important for one specific reason .. and why I use XNViewMP.
I once had a data corruption whilst moving images from one drive to another. 100+ K images to view in their entirety is too big to go through individually via each separate folder, so unknown to me I'd lost about 20 NEF images due to that data corruption.
Literally .. too many folders too many images to sift through each one.
By the time I'd found this out, I'd already updated my archives a few times over, but in that update I saved corrupted images where before the corruption they weren't.
So I was backing up corrupted images.

For me, XNViewer was the only practical way to locate every corrupted image I may have had.
Problem with Lightroom is that it builds it's own database, and relies on it's thumbnail which is stored separately as it's own file.
People may think this is good, but if the original file is corrupted, displaying the perfectly fine thumbnail doesn't show you that the actual raw file is corrupted.
That's where XNViewer comes into it.
Having to build the thumnail cache from scratch it showed a non image thumbnail. That is, for those corrupted raw files, it didn't display a thumnail of the raw file, it displayed a generic icon.
From that, I know there's an issue with the raw file data.
So that's my primary use for XNViewer. I set the thumbnail size to the smallest size, and open my primary and secondary photo archives recursively. I can sift through all 200K images in about 10 mins on that page.
Once I feel secure that they aren't corrupted I resave one of those those archives again to another location as a backup and then update those archives again once a year.

Hope that made sense.

Basically:
1a. For viewing and very basic edits of NEF files ; Nikons' ViewnNX2
1b. For general viewing of many image formats and device types ; FSViewer. it's the smallest fastest viewer that you don't even have to install(you can ruin it as a portable install).
3.. For the more complex viewing tasks ; XNViewerMP.


Question. Do you shoot in raw(NEF) mode or jpg?
I have to make dinner now, but a very important point to note for you is Keywording/Tagging, and IPTC.
I've found over the years, that catalogs the way that programs like Lr and other do them are pretty much useless unless you plan to lock yourself into those programs systems for ever.
Keyword raw files with embedded data, not thirdparty external data!
On a raw file this is not so easy to do.

[John King]

Keywords in Bridge are industry standards compliant. That's a big plus.

Keywording is a PITA!!

However, once done, retrieving any subset of files is very fast. Bridge can search using multiple keywords and logical associations between them.

Far be it from me to defend Adobe in any way, but this works.

Bridge CC is a free download, and can use any editor linked to it.

[Tannin]

It wouldn't work for everyone, but I store everything - absolutely everything - by date, with folders further labelled by location. I practically never have to spend more than a few moments searching, because all I have to do is remember the place, which leads in turn to the journey, which provides me with the approximate date - and I never, ever have to enter a keyword. Once in a long while I trip up over a place I've been to many times and have to look harder. When (to take an example that occurred just yesterday) did I see that vast flock of Flock Bronzewings near Cunnamullah? I've visited the district perhaps six times, and passed through on the way to somewhere else another dozen times, often camping there overnight. Took me about ten minutes to track the pictures down last night, which is very unusual. Are they good enough to publish? Hmmmm .... maybe - which is exactly the same conclusion I failed to come to the last five times I looked at them :(

Come to think of it, it's not quite as primitive a system as that. If I'm uncertain, provided that there was at least one picture I thought worth posting on that day or close to it, my website is database-driven and readily searchable by things like species, lens, time of day, habitat type, geographical location, and so on, so it's a simple task to find that picture, and thus the date, which leads directly to the other one I'm after. So having said "I don't use or need tagging", I actually do use a form of it (albeit for a very small subset of all pictures) and it is very useful. The point here is that if the task of tagging a gazillion pictures seems way, way too difficult, provided that your basic storage system is robust, you usually only need to tag a very few representative examples. 98% of the benefit for 1% of the effort.

[arthurking83]

Keywords in Bridge are industry standards compliant. That's a big plus.

....

When they set the standards, then yes. But interoperability with other software is next to zero(unless the other software has an import feature) .. even then, sometimes the imported data becomes garbled in some ways.

For true(er) 'industry standard' stick with ITPC data tagging into the raw file and you're safer for the future.

My history was similar to Tannin, been to many places multiple times, and it then becomes hard to differentiate some tiny insignificant detail difference between dates.
But I do the same.
Uppermost folder structure starts with dates, and then locations.

[John King]

Arthur, that's exactly what Bridge does.

Almost no other software supports IPTC keywords. Picmeta does, but it's glacially slow at searching compared with Bridge, and does not support multi-level searching or logical operators.

Credit where and when it's due ...

[Tannin]

^ All of that granted, Bridge remains bloated, slow, and horrible to use. Why would anyone want fat, ugly, waddling Bridge when there are so many slim, fit, attractive beauties like XNVView and Faststone willing and eager to dance with you?

[John King]

Tony, I've replied twice now, and both times the site has lost my post/s. I won't try again.

Bridge handles IPTC keywords and searches better/faster than any other program I have come across. It also does this with complex searches on multiple keywords/image attributes.

[ameerat42]

JK. I can see 3 of your posts in this thread. Is that what you mean?
Do you use the autosave feature? You can restore that content before
making a new post. At worst, you have to switch into Go Advanced mode.

[John King]

JK. I can see 3 of your posts in this thread. Is that what you mean?
Do you use the autosave feature? You can restore that content before
making a new post. At worst, you have to switch into Go Advanced mode.

Thanks, Am. The posts disappeared, and auto-save hadn't saved them :(.

[Tannin]

I can't comment on keywords, John, IPTC or otherwise. I've never used them or even seriously considered them. Possibly this is a mistake, but I have always made the assumption that the time overhead for entering them for thousands upon thousands of images is prohibitive.

Now if my pictures were, say, a collection of news photographs, that might be different. It would take a hell of a lot of work, but I can imagine some politician (e.g.) dying in a car crash and wanting to see every picture I had of that person, and probably of his close associates, and see all of those before a given deadline. So I'm not saying it's useless, I just cannot imagine a realistic non-commercial reason compelling enough to justify such a massive time investment.

(And don't you hate losing posts?) :( Somewhere, far, far away in another part of the multiverse, there is a planet where all those millions upon millions of lost posts go; a planet where they wait patiently for some dimension-skipping space-time traveller to come along and appreciate them in all their delicately faded glory. As you know, all the very best posts are lost posts, and all those laboriously typed replacements which make it to the web are just B-grade copies of the lost magnificent originals.)

[John King]

I can't comment on keywords, John, IPTC or otherwise. I've never used them or even seriously considered them. Possibly this is a mistake, but I have always made the assumption that the time overhead for entering them for thousands upon thousands of images is prohibitive.

Tony, my brother has used your system for many years, since the early 2000s. He now has such a mess and huge duplication of images that he is undertaking basic keywording of all his images. This will make the identification of duplicates far easier and faster. It has taken him about 1-2 weeks so far, and now largely completed.


Now if my pictures were, say, a collection of news photographs, that might be different. It would take a hell of a lot of work, but I can imagine some politician (e.g.) dying in a car crash and wanting to see every picture I had of that person, and probably of his close associates, and see all of those before a given deadline. So I'm not saying it's useless, I just cannot imagine a realistic non-commercial reason compelling enough to justify such a massive time investment.

I incrementally upload files from a card to its own folder. Each card is labelled both physically and digitally so that I can easily identify possible card failures. With today's huge cards, I initially put the files into dated folders under the main card's folder. When the card becomes full, I move all the images into the main card folder, then audit them against the card to ensure that none have been missed. Once that process is done, I then put a text file into that folder with the filename "Audited against card". Moving files on the same NTFS volume doesn't actually move the file (as I'm sure you know), it just rewrites the MFT entry to refer to a new head entry in the volume's database. That is to say, there is no possibility of data change/loss doing this process, other than by catastrophic HDD failure.

I do not erase the card files until I have at least one verified backup of the files onto an offline HDD.

With my filing system, once uploaded, files never get moved again unless for some kind of major system change. All variations of the original file/s (RAW+JPEG) always contain the original camera filename. I am currently 'between computers'. I am building a "new" PC, which is nearly finished. Once complete, I will use Windows Easy Transfer, to make a checklist of all programs and settings on my old main w/s, then remove its enterprise level HDDs into the new PC, while preserving the drive mappings.

Files are backed up multiple times to external HDDs. I have a 2TB Toshiba portable HDD that lives in the front pocket of my main camera bag ... I use SYNCBACK for this. If you are particularly neurotic about the backup integrity (like me), SYNCBACK allows you to use checksums and the like to verify backups. Just takes (far) longer.

Keywording is actually fairly fast. Because you can select all the images that you want to apply a particular keyword to, then just tick that keyword. Bridge does not use a catalog system, it (now) uses a MySQL database for searching and manipulation, but also embeds the keywords directly into the file where possible (e.g. JPG, TIFF, DNG) and into sidecar (.XMP) files where impossible to embed into the RAW file. I do know of at least one program that will embed keywords directly into RAW files, but it is klutzy (and I consider this to be an undesirable practice ... ). Bridge automatically keeps RAW and XMP file pairs together if you use Bridge to move, copy, delete the RAW file. Otherwise, it's hardly difficult to keep them together using any normal file manager, like Windows Explorer.

How I do keywording is to avoid the IPTC classification system like the plague!! It doesn't even make much sense to me ... Over the years, I have built up a couple of thousand keywords and phrases that are organized under major categories of things, people, creatures, activities and the like. I backup all these keywords to a text file pretty regularly.

After a shoot, I will (theoretically ... ) upload the images to the computer, then apply a general descriptor keyword - e.g. "Beaumaris Concourse car show 2017" to each class of shots. This allows me to isolate these a bit later (either by using the search facility, or by ticking the keyword in the L/H pane, Keywords in Bridge), then apply other keywords such as "Aston Martin", "Elfin", etc to individual shots.

Ticking the "No keywords" selection in the L/H pane, Keywords section of Bridge makes these entries hide themselves after application of the first keyword. This tells you how much pain there is still to go!


(And don't you hate losing posts?) :(

YES :nod:!! Usually some carefully worded response that addresses specific issues in a specific order. Clear, concise, never to be repeated ... Usually done painfully on a tablet ... :eek: :(.



Somewhere, far, far away in another part of the multiverse, there is a planet where all those millions upon millions of lost posts go; a planet where they wait patiently for some dimension-skipping space-time traveller to come along and appreciate them in all their delicately faded glory. As you know, all the very best posts are lost posts, and all those laboriously typed replacements which make it to the web are just B-grade copies of the lost magnificent originals.)

Quite ... :D :lol:

- - - Updated - - -

Tony, as an example of the power of keywording, on my new PC (where all my E-M1 MkII images are stored/worked on), there are 10,694 image files. Bridge took 35 seconds to extract the 858 files with "cats" in their keywords. "Cats" images are sprinkled randomly throughout all my images.

I could just as easily have made this search to find all images with the keyword "cat", while excluding those that contained the keyword "cats". I use the keyword "cat" for cats other than our own pair, and "cats" exclusively for our own two villains. Bridge is very, very powerful in this way.

These files are currently located on a very slow hard disk, with a minimal cache (if any at all!). I would expect this process to be much faster after I have moved the enterprise class HDDs out of my current old w/s.

[Tannin]

^ Not bad for a B-grade copy. :)

[John King]

Beats the heck out of the eyeball search algorithm. .. :nod: ;).

[John King]

Interestingly, I just did the same search on my old w/s. It took just under 60 seconds to extract 4305 files from about 92,693.

What's interesting about this is the apparent difference HDD speed makes here, along with general differences between the two PCs.

New PC:

Intel Core 2 Duo clocking at 2.93 GHz, SATA3 SSD 3Gb/s (running at SATA2 speed due to M/B limitations), separate old office/commercial PC 500GB HDD for image storage currently (with basic cache??), 2GB DDR5 RAM graphics card, 16 GB DDR3 main RAM, Win7Pro 64 bit.

Old PC:

Intel Core 2 Duo clocking at 2.13 GHz, SATA2 HDD for OS, separate Seagate Enterprise 3TB HDD for image storage (64MB cache), 512MB DDR2 RAM graphics card, 4 GB DDR2 main RAM, WinXPPro 32 bit.

Bridge is appallingly slow to initially cache images, but very fast to do anything else. FSV (for example) is pretty fast to do the initial caching process, but just cannot do most of the rest of what Bridge does, or can do. I never took to Lightroom, worst of all possible worlds!

The Olympus Viewer 3 program is much faster since upgrading the 512MB DDR2 graphics card to the 2GB DDR5 one in the "new" PC yesterday. Went from 50+ seconds to update changes to a RAW file to 7 seconds. Still tragic, but tolerable. I do not use OV3 for image editing or management. Apart from anything else, it can only use an aRGB colour space (16 bit), and I almost always use a ProPhotoRGB 16 colour space for editing. PS6 and Bridge are noticeably faster, but not by that sort of order.

[arthurking83]

Arthur, that's exactly what Bridge does.

Almost no other software supports IPTC keywords. ..

maybe they changed :confused013
In my experience Bridge never added IPTC data into NEF, CRW nor PEF files that I've tried .. but in saying that, I'm talking at least 3years ago now.
Could do jpgs and tiffs .. but who needs jpgs and tiffs for archiving(well OK, tiffs maybe, but camera files .. not tiff .. always only only ever raw)

Maybe they have better interoperability with Olympus raw files?

Only software that I've ever found to allow IPTC data entry into raw files has been:
PhotoSupreme(now) .. but back in the day it used to be IDImager(which I still have a license for) PhotoSupreme wasn't a good upgrade path tho, may have to eventually reconsider it as an option.
I haven't yet tried Pentax's own software, but Canon and Nikon's raw file software allows IPTC data to be embedded into their respective raw files.
The only other software that allows embedding IPTC data into the raw file is M$'s Photo Gallery(free), with the stipulation that you have your manufacturers raw codec installed on Windows.
(no idea if this is possible in a Mac environment).
They are the only few software I've tried(and I reckon I tried them all! :p) that will add keyword data into a raw file. (as well as Exiftool based software).

So my primary and secondary keyword/tag info is entered via Nikon's software and Windows Photo Gallery(but Gallery is about to be completely run out of town with the next incarnation of Windows .. so I'm on borrowed time using Gallery)

I have a sneaking suspicion that you may be confusing embedded IPTC data into a raw file(if that's what you were referring too) with Bridge's usage of 'linked data'
So in the instance(of linked data) Bridge knows of the file that you seemingly have entered IPTC data into .. but the reality is it's linked to that file, not embedded. Break the link and you see no IPTC data in that raw file.

What I'm referring too is:
Keyword with either Nikon software or Tag with Window Photo Gallery(makes no difference which).
Once done in one, the other sees it too. (that's really all I use).
But! .. open the folder of the raw files, click on a raw file in Windows Explorer and the data entered via Galley of Nikon's software is also usable in Win Explorer.
Almost all other software will also see the embedded data too. I've yet to come across software(of any use) that doesn't use that embedded data.
This is the major difference!
Add the data via Bridge, and Windows can't see it .. matter of fact no other software that I know of can either(other than Adobe software).
I know that soem software can be made to use the database of Adobe metadata, but it's an arduous process and basically regurgitates all that data into a new database. IDImager had a tool to do this. But it couldn't see Adobe 'linked' metadata.

Anyhow, this is the way it works on Nikon/Canon/Pentax(PEF!!) files
If it's different in ORW land .. could be a good reason to seriously look into Olympus!

Note tho .. I have limited experience with DNG, have converted some files, downloaded others(I don't have a DNG capable camera) .. but some data gets garbled in the conversion from CRW and NEF to DNG, but IPTC data is seen on some other DNG capable software.
Actually, my real hope is that one day camera makers will just suck it up and adopt DNG as a proper alternative to silly TIFF files in camera!
I can't ever remeber anyone in the last 15 years having shot in TIFF format in camera!

It'd be interesting to hear back if your use of IPTC data into Olympus raw files is seen across the board by other software(and which does, and which don't).

[John King]

Arthur, Among others, I have used both Nikon .NEF and Olympus .ORF raw files.

I have used both of these since 2005 ...

With vanishingly small exceptions, almost all IPTC keywording/tagging is either embedded in writable files (JPG, TIFF, DNG, etc), or written as sidecar files (XMP).

Almost all properly written s/w reads the original RAW file and the XMP file as if they were one file.

I have come across only one imaging program that writes data into a RAW file. I cannot recall its name. That process strikes me as being ill advised, to say the least!

Adobe Bridge (and Photoshop, and all other Adobe programs that support metadata keywords or tags) either write the metadata into writable image files, or into sidecar XMP files with non-writable RAW files. One can use a Hex editor to change the RAW files if you are feeling brave and have a thing for pain ...

In Windows 7, Windows Explorer can finally read IPTC standard metadata tags, or keywords.

For those wishing to look further into this arcane subject, you could start here:
https://iptc.org/standards/photo-metadata/iptc-standard/

Olympus image editing s/w does not write IPTC standard tags/keywords. Neither did my old NikonView s/w. Adobe s/w does, and always has.
I have been using PS since around 2005. It has always done what it does today, and does it no differently for any type/brand of RAW file that it is used with.

[arthurking83]

....

Olympus image editing s/w does not write IPTC standard tags/keywords. Neither did my old NikonView s/w. Adobe s/w does, and always has.
....

Like I said, IPTC was a standard long before Adobe got their grubby little hands on it and made it a domain of their choosing .. which even they seem to break with some features they choose to use.
Personally I don't like the idea of clutter up storage with as many small sidecar files as there are images .. but that's just me.
But, what I don't like about this convoluted debacle is that whilst it's supposed to be an 'industry standard, no-one actually does it as 'industry standard' .. if they did, you'd have Adobe sidecars(xml files) usable in Nikon software, also usable in .. <insert any software here.

So the reality end up more like:
You have sidecar files for Adobe's instance of a raw file, and you may have a sidecar file for Nikon's software version of that same file,.. add DxO's version, Affinity's incarnation .. and well the list goes on so far .. it's easy to see that you don't have as many sidecar files as you do images, , you're literally swamped with a number that turns out to be a factorial number of the number of the images you have. Now they don't take up a lot of room, but they clutter things so madly on your storage space it beggars belief.

So you end up with a situation where you have raw files that you may one day want to transfer, but you have to also transfer all those 'sidecar files' as well which only serves to clutter up more storage space over and again.

In 12 years, I've never experienced any issues with writing to an NEF file, with software that does it well... and that's the only way I do it!
Nikon software used to do it(did it badly for a short while in a single specific instance, but they rectified it and came good on fixing corrupted images.
All my corrupted images have only ever been a few times directly out of camera, and then this one time when I transferred massive amounts of data(most likely those obnoxious sidecar files! :D) and the transfer stalled, and began again.
Didn't think it corrupted the data, and the data seemed to be intact, but it ended up not so.

In terms of writing tagging data to raw file .. I'm totally comfy with my processes, based on historical perspective.

Your camera can be set to write IPTC data into the raw file directly as you shoot. It's not an ideal workflow, but can be handy in some instances.
I've tried it .. just clunky to manage on an image by image basis.
But the data it writes is in the raw file .. just as I do in my process above.

if I cant' locate my raw files(as Tanning described) .. many instances of a specific location over the course of many years, complicated with brief visits and prolonged visits.
Trying to sift through them to find one specific image with a specific bit of detail for me has become an ordeal sometimes.

BTW: for me Bendigo has become a 'cross to bear' I've been that way so many times fleeting and prolonged, I know an image has some specific detail, but I have so many <Date>_Bendigo/ folders that it makes me shudder at the thought of locating an image with <insert detail here>.

I delete jpg files, so they are of no use to me, and if I do end up locating that image with <specific detail inserted here> .. my first reaction is to tag it with that detail so as to never lose it ever again.

ps. I've now written close to 150K NEF files using Photo Gallery, I still have about 40+K remaining. I do it when I can both remember AND can be bothered. When I use locate an image I had been arduously seaching for for as described above, I almost always do it in Nikon's software.
Simply out of habit, rather than any quality issue tho.
It should be noted that the program itself doesn't 'write' the tagged data into the raw file.
It can't even manage to do much at all as a program without the use of manufacturers codec. So in effect, it uses the manufacturers codec to do the writing!
So to mistrust other software to write data to raw files is not as ill advised as stated. The writing process, when maintained using a correct workflow is really no different than trusting your camera to write the raw file in the first place.

pps. Nikon View could write IPTC data, but not directly via the main interface. Had to be used via a separate tool within the software, which opened a separate window for tag data to be entered. (as I remember it).

IPTC was a standard .. proper industry standard! .. long before Adobe was even an idea.
Look back into the significant ITPC history pre Adobe, and the only organisational name that maintains a presence up to the mid 2K's is IPTC. '79 and 90's it was ITPC.
In the mid 2000's Adobe entered the fray and well Adobe'd it. :p

actually: I do understand what it was they tried to do with it all, as video and other multimedia and accessorised devices entered the media world. But! it should have only ever been implemented by a totally non partisan committee of "proper geeks" with no affiliations or market clout/presence.
They would have received submissions by al interested parties involved, and the implementation would have been truly industry standard.
Like jpg .. 'is'(or maybe it isn't .. or is) but you get the idea.

There's really only one way to write a jpg, and it's coding has to conform to a set standard, that any jpg capable software can read .. and camera can create. . and it works.
Adobe 'gave the world' DNG and stuffed it, to the point where the majority of manufacturers still avoid using it even tho they're free to use it!(And even tho it's advantages are multivarious!)
My hunch is that it's a mistrust of all things Adobe on the part of the manufacturers(the large manufacturers) .. not simply a product of their inherent mistrust of openness.
Why bother to stick with TIFF which is an open format(even tho Adobe also own that too), when a better format exists?

Sorry for the arduous reply.

Basically: Tagged raw files rule! :th3:
Anything else is a compromise. :lol2:

[Tannin]

No-one has yet satisfactorily addressed the massively time-consuming task of tagging many thousands of files. Specific commercial applications aside, I'll lay you Sydney to a brick that I can find files using my simple, logical, time-efficient method much faster than you can with a tedious tag-everything approach provided we are fair and count the time spent tagging as well as the time spent searching - in my case the former is zero, in the other case more hours than I care to contemplate. I could take on average ten times longer to find a given file (which I don't) and still be a mile in front.

[John King]

Tony, my old w/s took less than a couple of seconds to display the 4 images tagged "Tarrant" out of 96,000+. One only has to keyword files once ...

Arthur, sorry, I don't mean to be rude, but I can't be bothered pursuing this with you. It works for me, regardless of whether I use Adobe products or a dedicated DAMS program. I do not use, and would never recommend any catalog based system (Lightroom, etc).

[Tannin]

I note that you are carefully avoiding any mention of the time required to tag 96,000 images. Which is my point.

It's like my great aunt, who once asked my father to drive her all the way across town to go to a particular grocery shop. Being a polite and dutiful lad (and wanting to put his best foot forward insofar as he was hoping to marry her niece) he cheerfully did so, which cost him half an hour each way and perhaps a sixpence or a shilling for petrol and other running expenses. (Cars were dear back then, and even dearer to run.)

My great aunt (well, she wasn't my great aunt then because my birth was still ten years in the future, but you know what I mean) was happy. As she explained to my father, thanks to his help she'd saved a ha'penny on the pound of butter she went there for. True story.

He, very wisely, kept his mouth shut, married the niece and the result, in the fullness of time, was me. (Which may of course not be regarded as an entire success.) Nevertheless, the tale is as good an example of spending a shilling to save a ha'penny as we are likely to see this side of tagging 96,000 files to find four of them once in a while.



PS: I'll let you know my address so you can send that brick you owe me.

[John King]

Keywording doesn't have to be painful.

I can tag all the images on a 32 GB card in minutes, depending on content.

Keywording everything in minute detail can be, and is almost never necessary IMO.

The benefit is that one never loses a file again.

Your system of adding information to a date folder is almost as time consuming, but nowhere near as powerful for retrieval. Trust me, I have lots of experience with this via my brother when using remote access software ...

Brick? What brick?

[ameerat42]

My great aunt... saved a ha'penny on the pound of butter...

Whence arose the expression: in for a ha'penny in for a pound, which held currency up till the
time of decimalisation in 1966. So that makes Tannin's age... predecimal.

[Tannin]

Your system of adding information to a date folder is almost as time consuming

Huh? Downloader Pro asks me for a folder name once per photographic day, which requires that I type in something like "Laratinga" or "Birdsville Track". That's it. Entire task completed (other than popping in the other flash cards and telling it to upload them.) Downloader Pro takes care of renaming all the files (in the form "180514-103214" for a picture taken at this moment) and placing them in the appropriate folder (which might be /2018/05-May/14-Laratinga). That provides all the information necessary to find whatever it is I'm after.

[John King]

Tony, I simply cannot comprehend why you are fighting so hard against something that, on your own admission, you have zero experience with!

I take lots and lots of photos of cars. These are spread out over six different cameras over 15+ years. If I want to find every photo of every Ferrari (or Bolwell, Tarrant, P76, Mustang, whatever ... ) that I have ever taken, it takes seconds to find them. Regardless of which camera I used, or where/when I took them.

You take lots of bird photos. If you want to find all the Australasian Pipit photos you have ever taken, how do you go about this? Or say you want to find all the pelican shots, or rainbow lorikeets, or ...

The more common the subject, the harder it becomes to find THAT particular shot that you have a vague memory about taking. Was it at Lake Eyre? Maybe it was at Lake Wendouree? Or maybe it was at Port Macquarie, or Macquarie Harbour ... Which year was it? What date/trip?

It doesn't matter whether you use an Adobe product, there are third party products that will also do this - e.g. https://www.fastpictureviewer.com/help/#fileutilities

As I get older, my memory is not what it used to be. The anaesthetic for my open heart surgery didn't help. Keywords/tags help enormously.

[ameerat42]

Tomatoes/tomatoes.... Potatoes/potatoes:D:D

[poider]

I had a quick look at all the programs suggested. Last night I opened bridge and typed in focal lengths under 300mm, and low and behold it listed every focal length. (Which is what I wanted originally, but it wouldn't give it to me), I took many hours to do and when I woke up the list was there waiting for me, I took a screenshot of the list and will check it out at my leisure, this afternoon I will reopen bridge and check if the list is still available.
Thank you all for you suggestions and enjoy your debate.
Peter

[arthurking83]

....

short version:
Arthur, sorry, I don't mean to be rude, but I can't be bothered pursuing this with you. It works for me, regardless of whether I use Adobe products or a dedicated DAMS program. I do not use, and would never recommend any catalog based system (Lightroom, etc).


No need to apologise.
My only concern was to highlight the strengths and weaknesses(limitations) of certain systems. My preferred system is to keep it as lean as possible and raw(file type! Not slang for whatever meaning the hipsters use the term for now :p)

I'm unsure of your use of the term 'catalog based system' tho.
I never delved into the guts of Bridge's innards, but my understanding is that it is a catalog based system.
If it wasn't, then every search would require that it re-indexed every image in the archive over and over again.. making it painfully slow to do a search(as the archive grew in size).


....

Warning! .. lonnngggg version!

On the point of keywording, John and I agree that tagging/keywording isn't time consuming or difficult unless you want it to be.

In Photo Gallery, I start to add a keyword, and as it's already in the system, it automagically auto pops up on a side panel as an option to choose from, along with multiple others with a similar alphabetical structure.
In ViewNX2 this doesn't auto pop up, but there is a click link with a long list of all the recently entered data. handy, but far less useful than Gallery's method.

So9in Gallery) typing "B" will initiate all the data currently starting with B, entering an 'e' then brings up all the relevant keywords with the name "Be....." one of which is "Bendigo".

Hit the down arrow to scroll to "Bendigo" and it's done.

For adding keywords to multiple images, Cntrl-A those images(within the folder) and do as above.
Once they're all tagged, search those images with(as my example) "Bendigo".

I then see hundreds of thousands of image in and around Bendigo.(as a search should provide).
Choose your 'file view' setup to assist with the ease of which you then add further tag data.
If you set the file list view to date created order, then all the images shot in succession are easier to choose collectively and you then add more tag data.

This is quite easy in M$'s Photo Gallery, and you only add the folder(s) that you wish to have catalogued. Easy, but time consuming if you need to play 'catch up'. But far easier than any other software that allows embedding into the raw file(where John and I diverge).

But even easier than using Gallery, is on initial browsing or downloading of the files to the computer device.
** If your computer device is a disposable/ultra portable/tablet type thing .. then I have no further advice than to bin it and get a rear device! :p

in the Nikon world, the two easy-ish options are:
ViewNX2: once the images are loaded up, scroll the RHS pane down to Metadata look for the Keywords input box add a keyword hit [add], hit save and it's done.
Alternatively: you can use Nikon Transfer(which I do, for file renaming!) and use the keywording feature in that program to keyword all the images in one hit. I used to do this and have many keyword data sets saved now, but it's just as easy to do the same in ViewNX2 anyhow. So I haven't used keywording in Transfer for a long time now.
The issue is what keywords to use? That's the hard part.

For me it's obviously the primary theme or subject in the scene! That could be a bird, a bridge, a bicycle, or a Brian. But sometimes I'm lazy and won't add that keyword into the image.
I may have travelled to Bendigo and shot 4 images of a Brian, a bird, a bicycle and a bridge.
Entering all the individual keywords IS tedious as Tanning presumes it to be, but what I do instead is add the common keyword .. ie. Bendigo, instead!
The point is to get the images into the catalog(or database) so that further tagging is then made easier.

**if there is one thing that is for sure, it's that I'm inherently lazy. Why do something tedious that you can always put off till another time. :p
Sometimes I can't be bothered, other times I look forward to doing the tedious stuff!!

So, once the main tag data is decided on, select all images(make sure they're active!) and add the common keyword hit add hit save.
This takes all of about 2 seconds. What I like about ViewNX2 is that this is done on the NEF files. Saving over 400 images once with the keyword "2015/Lake Eyre" had the 400 NEF files saved in about 5 seconds. So time consuming is not a point of contention.
I had no interest in keeping that keyword as it was in any of those files, it was simply a place keeper(or notation) to simply get those newly downloaded files into the catalog system.
Had I used 'Lake Eyre' without the 2015, I'd have confused those images with the lot I already have of Lake Eyre.
I wanted those images separated till I could be bothered to tag them properly with more detailed data.
As I remember I got back to those images at a much later date .. maybe a year or so later .. ie. when I could be bothered!

Tony: from experience I'd say I could tag those 96K images in about 4-5hrs or so.
Basically what I had to do when I first learned of my error in not tagging my images for the first 5 or so years. Can't remember exactly when I realised I got in way too deep, and changed my ways.
I wouldn't sit there for hours doing them all, I'd do some(many!) for about an hour, then come back to them again another day.
Note tho that this was from scratch, which is much harder, AND! ... with not much tag data already in the system!
Now with more data, tagging is easier. I think I have about 2500 individual words saved in my system. Why this is important is that it saves your soul/sanity from typo errors(hugely important) and RSI/tedium.

Only time I take longer than I'd like too is when trying to determine what species of bird/insect/flower/etc a subject is.

And as already noted earlier. I know that I've been to Bendigo far more often than anyone with zero links to that town ought to have visited. I think 10 trips in one year where I actually had the time to take photos(far more when it was just work and straight back).
So to locate the time when I took that photo of a bird on Brian's shoulder whilst he was riding that bicycle over a bridge in Bendigo .. for me is a monumental task!

XNView makes that process a bit easier as the thumbnail would be distinctive, but you still need to sift through hundred of thousands of images.
In the years to come I'll forget Brian's name, so I'll most likely just use the search term 'Bendigo'.
Once that's done and the 100K images of Bendigo have loaded in the preview pane, across to the RH pane is the list of all the other keyword data that relate to all those 'Bendigo' images.
That additional tagged data may be 'Blue Banded Bees', 'Brown Brick Building', 'Big Bad Bozo', 'Bridge'(yay! bingo) ... that's the new keyword I click on.
Now the new list of images(maybe 1000) is only of bridges in bendigo, and the new tag list on the RH pane is Bird, Balloon, Badminton, Berries, Bicycle(yay! .. click that!)
And there's Brian on his bike on the bridge, along with Benny driving his Bentley, and Billy walking his Bullock!

:D

In all seriousness tho those click to search endeavours take a mere few seconds to enact.

[John King]

Sorry to be short with you, Arthur. I have pain from many causes almost all the time. It can make me intolerant and downright bad tempered at times.

Bridge both embeds all applied metadata in images (or sidecar XMP files) and keeps them in a central MySQL database.

Unlike the cataloguing system in Lightroom and Photoshop Elements, the Bridge central database is not essential for Bridge to function. It can be moved or deleted completely without having any effect on a single image.

What Bridge does keep in its database is the thumbnails along with all associated keywords. Both of these are also embedded in the image files, so nothing is lost irrevocably if the link to the database is broken.

If I understand correctly, loss of the link in Lightroom leads to loss of all applied keywords and all image edits unless the image has been exported from Lightroom. Quite a few people have had this happen for various reasons, and there appears to be no recovery system that works. Hence my comments about catalog based editors/DAMS methods.

Agree with you about keywording not being particularly onerous. The benefits far outweigh the time spent. Like any classification system, whether in accounting, cladistics or photography, there are significant benefits>costs or no one would bother!

[Geoff79]

A lot of posts here and I didn’t get to read them all. But just wanted to add my 2 cents on a couple of things.

Firstly, I get the impression that everyone here uses a PC and there are no Mac users? The single hardest thing to accept when I switched from PC to Mac was that Faststone is not compatible with Mac. That was, and obviously still is, a great program and I mourned it’s loss heavily.

However, if there are any Mac users out there, I tried a few alternatives on the Mac and the one I personally settled on was Photoscape X. These days, I only use this type of program strictly as a browsing/sorting agent, and all edits are done in ACR/Photoshop. But as far as I can tell, it’s a pretty worthy comparison to Faststone, for Mac users.

And just quickly on Bridge, I also find it to be a quick search tool. I don’t label individual photos, but tag every dated folder with whatever I know will show up in whatever broad searches I might do to locate particular shots. It still requires decent amounts of searching once the folders are found, but at least helps to dramatically narrow the field very quickly. [emoji4]


Sent from my iPhone using Tapatalk

[Tannin]

keywording: The benefits far outweigh the time spent. Like any classification system, whether in accounting, cladistics or photography, there are significant benefits>costs or no one would bother!

On the contrary, if there were significant benefits, everybody would do it.

Nowhere in this thread has there been any sensible attempt to quantify the cost of keywording. Having tried it for myself, I know for a fact that in my case (and probably in 90+ percent of other cases), on a time = money basis keywording an entire collection is the equivalent of paying $1000 a year to insure your car for $100.

[arthurking83]

John, no need to apologise. no offence taken on my part. My only concern is info and accuracy of info:
ps. didn't know about the MySQL database either. Is it truly open format? That is, can you search it with other software.


....
Bridge both embeds all applied metadata in images (or sidecar XMP files) and keeps them in a central MySQL database.

....

This is what I don't understand, because my experience is that it really doesn't!
That is it doesn't embed metadata into raw files .. which is really the only important aspect of this conversation for me.
Yes, it embeds metadata into TIFF and JPG files, but then again almost all the software I use for imaging purposes does that anyhow, and once that's done you don't need any other software other than your computer's file browser to see it and search it!

The issue(or more accurately MY issue) with almost all metadata handling software is that they don't help with keeping track of raw files.
Except in the specific condition that you're happy to be tied to the current software into perpetuity.
I'm not(happy to be stuck with only a single software for the rest of my capable existence(capability of existence debate, notwithstanding! :p)

So the litmus test is:(assuming you're a Windows user, using Bridge), and only for your raw files:
can you see any metadata you've entered via Bridge, in the Tags section in Windows Explorer's file list(or Finder for Mac users)?

as an example of my preferred system, see my old thread about tagging/keywording/searching (http://www.ausphotography.net.au/forum/showthread.php?141274-Tagging-cataloging-searching-hints-and-tips&highlight=keywording)
(scroll down a bit to see my screenshots of Windows Explorer).

So my experience is that using Bridge(as well as Lr) I couldn't get the tagged data into the NEF files. I searched the topic and found that it doesn't do that. it does for commonly used raster images(ie. jpgs/tiffs) and some video formats.

For me, jpg and tiff files simply waste space and resources that can be better utilised by other stuff.
Previously I said I had about 200K images, but that was everything! I'd forgotten that I'd deleted about 40-50K of jpgs and tiff files, as I just checked my archive and I have 90K tagged images, and still 47K files still not tagged(that I'll get around too one day).
The only jpg/tiff files I keep are jpgs made by jpg only camera devices(point and shoots/smartphones/etc) and the tiffs I've scanned of old photos/slides.
So I have to be careful when shift-deleting those image file types, that I don't accidentally delete the jpgs/tiff that I want to keep.

Once the system is setup for this, you don't need any other software to enter tag data into the raw file .. Windows Explorer itself can do it. Clunky! .. but can do it.
It's search function is faster than Bridge, and can be used on any system with little effort.

Note!! I'm not questioning your reasons for doing how you do what you do. Important to note this point, as I'm not arguing any point with you here.
As this is a forum, and the forum is here to serve varied user tastes, I feel it's important to highlight advantages and disadvantages of each method. That really is the only reason I post about this stuff.

I know of people that program their own software and maintain their self made SQL databases for their archives and all sorts of variable workflows to similar effects .. so we each have a preference.

My issue(not that it's just my issue) is, that I've gone through the process not fully understanding the limitations of the various methods.
Once I did discover the disadvantange of using Lr .. actually Adobe's software, I was mightily pissed .. thinking that my files(raw .. only raw) were tagged, where they weren't.

Like Tannin said, keywording can be arduous, painful, annoying, etc .. and to a degree I agree with that sentiment, but only in that I was bitten(by Adobe) without realising it, and in the end all that work was for nothing!

never again!
side note: I was hoping to teach myself a bit of programming(of which I know completely zilch, even tho I persevered for a short time) to make my own software that could do what I wanted above.
Part of that process was to delve into various tagging software systems as well, for the ability to transfer their data into various other systems too.
(end result = 0! .. not even 01, or 10. Just a single solitary zero, as I couldn't find the time and ability to program consistently! :p)

In summary:

1/. If you use Adobe software, you need to keep using Adobe software to maintain that searchable tagged data, if your only concern is for your raw files.
If you transfer image rights/usage to any other system you need to be sure that system can also use Adobe software(and it's associated installation/costs) on that system too.

2/. If you use the manufacturer's codec system, you don't need any other software to tag or search your raw files, but better software than the built in file browser makes for an easier task of entering tag data. With Windows Explorer it just needs to be set to index the data stored on the relevant drives. I think this is usually on by default, but don't count on it.
Other free software exists that can simply make entering tag data easier and allow more complex search parameters.
The only point to note(as the OP asked the question here) is that these software won't index the exif data for focal length/lens used .. and stuff like that.
Other free software can do similar stuff.(tip: do a search for Wega2).

3/. for a small fee I found that IDImager was the best all rounder in that it could do everything I wanted(important to me) and import Adobe metadata info.
IDImager no longer exists and the updated software, Photosupreme, wasn't as good(back then). I need to reacquaint myself with that software to see if it's improved any too.


ps. (for Tannin). For location tag info, there are some software that can tag your images with any GPS tagged data in the image(s).
When I shoot, I always have my GPS on camera, so every image is geo tagged. Location data new set, ViewNX2(and I think there are others, like Lr too) can embed location tag info into the image to save the user from inputting it.
Can work well, limitation on the system is that it may only use the limited ability of Google Maps, and if in remote unknown places it generalises to the nearest thousand kilometers! :D
if you had a Nikon system and used ViewNX2, it also gives the option to check location via Wikipedia too .. much more accurate for remote locations. Can be handy, I've used it a few times to assist with my lack of enthusiasm for tedious processes.

[John King]

Arthur, I'm buggered today. Not a rare occurrence, unfortunately.

Just a quick comment. Windows Explorer has always natively supported CanNikon raw files via its native codecs. Most of the rest of us have to purchase third party codecs to even see the embedded thumbnails in the raw files!

I cannot speak for Windows 10. I don't use it.

My point being that it is always tricky to generalize from support for CanNikon cameras to other, smaller brands.

I have located the Bridge database store in Windows 7. It has moved from where it was stored in XP. It is very imaginatively (for Adobe ... ) called "store".

The link to it is via an explicit pathname in Bridge/Edit/Preferences/Cache. This allows the cache files to be moved and reattached to Bridge.

Bridge can be forced to rebuild the entire cache (thumbnails and metadata) either using Bridge or by simply deleting the contents of the cache folders and telling Bridge to rebuild the cache at either the existing location or a new one. Bridge extracts the data it needs from the image and XMP files.

[arthurking83]

....

Just a quick comment. Windows Explorer has always natively supported CanNikon raw files via its native codecs. ....

Kind'a sort'a!
Not in every installation, there are downloadable camera codec packs available from both M$ and some camera manufacturers(like Canon, Sony, etc)

I may have remembered the issue with Olympus, but did forget about it till you mentioned it again in this thread.

Note that there are differences between M$s codec pack download, and Nikon's codec .. so that could play a major part.

I'm thinking that maybe the M$ camera codec pack has major codec differences from the manufacturers codecs.
Most likely the M$ codec pack is at it's most basic level, simply to allow displaying of the raw formats but nothing else.
I've had it installed.
How you know you do or don't have it is:
If you use thumbnail display mode in Explorer, and open a folder containing your manufacturers raw files, the thumbnails will show the scene in each raw file. If you don't have the codec installed, the thumbnail image is a generic icon.

The latest M$ codec download is very old now, so maybe they've ceased supporting the system.

Anyhow, I'm sure there will be limitatons to the M$ codec pack compared to the manufacturers available codecs.
M$s codec pack file size if about 16Mb which contains the codecs for hundred of raw capable cameras from many manufacturers ... ie. many more codecs than just the Nikon camera database, but the Nikon codec package is larger than the entirety of the M$ codec system.

Looking into the Nikon codec system, once installed, it's 32Mb(or more), which seems to imply a degree or two more features within it than the M$ codec pack would be just for those same Nikon NEF file types.

[Tannin]

Cheers John.

When I setup my filing system, there was no such thing as geotagging, so I evolved my own method. It is very, very rare that I can't locate an image to within a smallish area, and rare not to be able to locate it as precisely as I could wish. I played briefly with GPS on cameras when I happened to buy one that had it, and quickly decided that it added little to my existing system, and had disadvantages, in particular battery life. I'm not knocking it, I hasten to add, it's just not necessary or particularly useful to me, given my highly efficient automated filing system and memory for places. (To this day, I still don't use GPS or satnav for anything.* The day I can't look at a map and memorise the salient features such that I can then go where I want is the day I'll hang my boots up and declare myself officially past it.

This made me laugh the other day. My 86-year-old father and I were travelling in north-west Victoria. Now he is, by training, an expert navigator - as pilots had to be when he started in the caper, which was 50 years before GPS and decades before widespread installation of other radio navigation aids. In his day - when he was flying DC3s around outback Queensland for example, or carting the flying doctor around to cattle stations in the Northern Territory - navigation was all about map reading and calculated speeds and drift rates on a given leg, and so on. So here we were, having to find our way to a given destination away to our north-west. One of us was doing the modern thing like any 25-year-old, looking at a tablet with Google Maps. And the other one just looked out the window, using the lie of the land, the position of the sun and the run of the riverbeds to navigate. After a couple of hundred kilometres, we got to our destination, we hadn't disagreed about any of the major navigation decisions, and we'd taken the shortest, most direct route. The joke, of course, is that it was my father using the GPS and me navigating the way that bushmen have navigated for 200 years. I like doing it that way. It demands that I know and understand the country - and if I don't know and understand the country, what the hell am I travelling there for in the first place?

* Exception: I use GPS when doing Birds Australia Bird Atlas surveys, because GPS refs are what they need for their database.

[John King]

On the contrary, if there were significant benefits, everybody would do it.

It is my suspicion that 99.99% of photographers do not even know that keywording (tagging) exists, and don't know what it is, or what benefits it might convey for them if they do!
Sad, but probably the case, Tony.


Nowhere in this thread has there been any sensible attempt to quantify the cost of keywording. Having tried it for myself, I know for a fact that in my case (and probably in 90+ percent of other cases), on a time = money basis keywording an entire collection is the equivalent of paying $1000 a year to insure your car for $100.

The time cost when even a sizeable set of images has been uploaded is negligible. Doing it while looking at what one has caught becomes second nature - a matter of selecting a keyword from a list and L/clicking on it to apply. Since about 5% of my images are of the children (the cats, AKA Household Gods), I usually just select all of these first up and tick "Cats". That fixes that group up immediately. Then seek out groups and do similarly. It really is quite fast.

Once he actually understood the benefits to him, my brother set out from scratch to keyword all his images. He's older than I am, and usually cannot be bothered doing anything like this ...

arthurking83 Arthur, I have installed XnViewMP on my Win7 Pro w/s. It will keyword RAW files, but only by creating XMP files, i.e. it does not embed the keywords into the actual RAW file. When one searches, it obligingly finds the XMP file, not the RAW file! I'm probably doing something wrong ...

At least Bridge finds the RAW file associated with the XMP file, rather than finding only the XMP file ...

[Hawthy]

It is my suspicion that 99.99% of photographers do not even know that keywording (tagging) exists, and don't know what it is, or what benefits it might convey for them if they do!

Correct!
Personally, I can think of nothing more tedious than tagging each photo with a keyword. Instead, I download my raw photos into a folder called something like ANDREW>> PHOTOS>> MAUI>>RAW FILES if I am lucky enough to have been to Maui. If not it will say ANDREW>>PHOTOS>>WYNNUM MANLY.
Then after looking at those 1,000 raw images in Bridge and finding maybe 20 keepers, I process them.
I save those into a folder like ANDREW>>PHOTOS>>MAUI>>PROCESSED PHOTOS.
I can always revisit the raw files but I really can't see myself sitting down and giving each a label like ; Lahaina : Luau: Sunset; Sunrise; Surfer; Meal; or any combination of that. The number of keepers that I get from every trip are usually so insignificant that labeling them is unnecessary.

[Tannin]

You raise a salient point, Andrew. I'll get to it in a moment. First, a not-quite-distraction.

On a good day, I might take 1000 pictures. Of those, I'll probably want to keep 100. Of those 100, there might be 5 I'll eventually want to publish (this is a good day - on a bad day it might be one, sometimes none at all) with another 15 or 20 that are genuine keepers. The remaining 80 are context - valuable and meaningful to me but seldom of interest to anybody else. (You probably have pictures of your children that you wouldn't dream of deleting, but wouldn't bother posting here or showing to your friends. Same thing. After all, I love the Australian landscape the way many people love their children.) They also serve a vital purpose insofar as they locate the 5 or 20 best pictures in time and space. For example, they might remind me that the beautiful Spotted Harrier was half an hour and 15km south of the ford over the river, not far from where the saltbush plain gave way to sandy mulga country. The whole collection of 80 or 100 forms a narrative through which I can revisit any of a hundred trips over seven states and getting on for 20 years, and does so in a way no possible tagging system could remotely reproduce.

But (getting to your point now) I am nowhere near mentally capable of reducing 1000 pictures to 100 in a single session. (I wonder how many of us are? Rather few, I suspect.)

The most I can do before my eyes glaze over and I start making bad decisions is throw out some of the worst ones. I always aim to cull 50% in one go, but more often only achieve something like 30-40%. Then I put the day's work aside to be revisited another time, when I can see it with fresh eyes. The first pass is the easiest - there are always obvious duds which need no thought at all. After that it gets harder: the easy choices are all made. But slowly, over several stages, I reduce the thousand to 600, 400, 250, 150, 100. It usually takes about that many sessions, spread over anything from several weeks to several years. Often there are fine distinctions to be made, and it's hard work. My brain overheats. So when I start struggling to make good decisions about the northern New South Wales pictures from last month, I'll flip over to Tasmania the year before last. And so on. Little by little, the collection shrinks and improves in quality. Every now and again, a shot will stand out from the crowd and demand to be published (here, or to my website, or to my private collection of favourites). Maybe I've always had my eye on that one and always intended to do the full PP thing with it when I got around to it; maybe it's one of five or forty-five very similar ones and I've only just decided that #32 is the pick of the crop; or maybe it's a little hidden treasure which has sat there on my hard drive for months or years and I've only just realised how good it is. These last are the best of all: often you had some particular vision in mind when you took it and that didn't quite work the way you wanted it to, but now, months later, you look at it afresh without preconceptions and recognise it for what it is - not what you wanted but something else as good or sometimes better. There is a special joy in these neglected gems: finding one is like finding an unexpected $50 note in your pocket - you might have had $300 in your wallet and thought nothing of it, but that bonus $50 makes your day. :)

[John King]

Good description, Tony. :nod: :D

Specially love the $50 note analogy.

My method is a bit different, in that any photo that makes it onto my HDD tends to stay there forever. Duds tend to get deleted in camera. The one below nearly met that fate as almost everything about it is "wrong", but something stopped me. It is Rosa sitting on my chest in bed. It captures something of her unique personality - specially her innate ability to be very comfortable ... ;)

https://canopuscomputing.com.au/zen2/albums/cats/_9052863_E.jpg

[John King]

Tony, that's where we are very different. On a good day, I might take 100 pics, maybe up to 200 if it's an exceptional day. I keep 95%+ of them. I am a contemplative shooter.

[Tannin]

Just so, John. In large part, this is no doubt because of our different subjects. With birds, especially small birds, there are many factors working against any one shot. You are usually working right at the technical limits of your equipment: you are pretty much always too far away, using more lens than you'd really like to, cropping harder than desirable, using higher pixel density than is ideal, pushing the ISO higher and the shutter speed lower than you'd want to, and/or shooting a few stops wider than your desired depth of field. Yes, you have IS/VR but at these focal lengths it is nevertheless imperative to keep the shutter speed up to avoid camera-movement blur and you have to skate as close to that edge as you dare lest you sacrifice too much noise or too little DOF.

And you are often doing all that in a few fleeting moments, nevertheless trying to stay relaxed and move slowly because if you don't the bird takes fright. Meanwhile, you are also guessing exposure compensation on the fly.

All that said, the biggest factor is that creatures the size of birds move rapidly and unpredictably. With something the size of a horse, a human, or a dog, you have a fighting chance of anticipating movement such that you can click the shutter just at the precise instant the horse clears the fence, or the father-in-law kisses the bride. You can do that with birds too, but with a much lower probability of success. And you have a near-zero chance of predicting the vital small movements that (for example) see a thornbill's head turned at the right angle to catch the light in its eye. This last is the key. Look over this forum's bird section: you'll soon observe that one of the biggest differences between an experienced bird photographer's work and that of a beginner is ... well, I hate the jargon term for it, but people call it "the head turn". It's the bird photography equivalent of, in humans, catching the right moment where the subject isn't blinking or looking awkwardly out of frame. And you can't predict it far enough in advance to just wait for the perfect moment.

In short, doing birds, you wind up with a vast number of pictures, many of them similar but (for example) marginally sharper or with slightly different angles, and trawling through deleting them is a slow and tedious task.

In general, I don't delete anything in-camera. There are two reasons for this. (a) You can see the angles (head-turn and such) on the camera screen, but you can't tell the difference between sharp and almost-sharp, no matter how hard you try. (b) A Canon peculiarity: if you shoot with dual cards, raw on one, JPG on the other, and delete in-camera, only the JPG gets deleted. Why? Ask Mr Canon. All Canon cameras are the same in this regard. So I can't delete in-camera on any of my best three bodies - only if I happen to be using one of my elderly spare bodies for some reason (7D and 5D II). So I just don't do it.

Managing raw + JPG collections is a topic in itself. If I remember correctly, Lightroom handles it in a fuss-free and very sensible way. (See? I just said something nice about an Adobe product, and it didn't even hurt.)

My approach is to work with and manage the JPGs, never touching the raws except to copy the odd one to a scratch folder for PPing into a finished publication-standard image. The raw files just stay in (for example) x:/2018/11-November/22-Lightning Ridge/raw, unseen and untouched. (It is far quicker and more convenient to do sorting and deleting with the JPGs only.) Then, at some convenient time when I've made good progress on the sorting, I tell the computer to make the organisation of the raw files mirror that of the JPGs. It does this using, of all things, a batch file. (Remember batch files? Am I making you feel old?) It does a few things but the primary task is a series of variations on this theme:



SET HOME=%CD%
IF NOT EXIST raw GOTO finished
:step9
IF NOT EXIST x9 GOTO step8
IF NOT EXIST raw\x9 md raw\x9
cd x9
FOR %%Z in (*.jpg) DO MOVE "%HOME%\raw\%%~nZ.cr2" "%HOME%\raw\x9"
cd %HOME%

:step8
IF NOT EXIST x8 GOTO step7
IF NOT EXIST raw\x8 md raw\x8
cd x8
FOR %%Z in (*.jpg) DO MOVE "%HOME%\raw\%%~nZ.cr2" "%HOME%\raw\x8"
cd %HOME%

.... and so on ...


It took a bit of fiddling around to get set up, but it's simplicity itself to run: just right-click on the head folder ( x:/2018/11-November/22-Lightning Ridge) and select it. From there on, the whole process is automatic. A few moments later, the raw files are organised exactly the same way as the JPGs.

Note that it doesn't delete the culls. In my book, deleting things is a human-being level decision. But it would be easy to modify. There is also a related script on the archive server which deals gracefully with partial data sets imported from the primary laptop.

File management and backup software generally is very, very good at dealing with increased data sets - for example, dealing with a folder full of documents and making sure that the backup contains all the new and modified files - but is very seldom designed to sensibly deal with workflows where the added information is negative - i.e., where the new update since last backup/archive constitutes removal of certain files rather than additions: and this is of course exactly what happens with a folder full of pictures as you gradually cull it down. This is why in the end I gave up on finding a ready-made solution and wrote my own. That was ten years or so ago and it has been a great success. I can't remember the last time I needed to update or modify it, it just works.

(PS to anyone following this discussion who uses raw + JPG: setting up a similar system is quite simple. The only technical part is writing the script part-quoted above. Sing out if you'd like a copy of it - you don't need to understand it to use it anymore than you need to know how Kym wrote the AP competition scripst in order to vote for a picture.)

[arthurking83]

....

arthurking83 Arthur, I have installed XnViewMP on my Win7 Pro w/s. It will keyword RAW files, but only by creating XMP files, i.e. it does not embed the keywords into the actual RAW file. When one searches, it obligingly finds the XMP file, not the RAW file! I'm probably doing something wrong ...

At least Bridge finds the RAW file associated with the XMP file, rather than finding only the XMP file ...

Weird issue with XnView you had.

I use XnV myself for error checking my image files.
It's one of the only programs that can view files recursively within hundreds of smaller/child directories, and with any usable performance.
I've added keyword to raw files in XnV(of course it didn't add it to the raw file), but it did find that raw file .. not just the side car file linked to the image!
But I don't really use it for that purpose anyhow .. mainly for the ability to visually check that the raw file is not corrupted.

But again, as my raw files have the tag data embedded, XnV can see that data, as can (or in my case .. could!) Bridge ... and Nikon's software, and Gallery, Explorer .. etc. etc.

[Liney]

Well folks, I must commend you all on a very interesting thread.

What I have learnt today is that a) there are photographers who take a lot of images and quite a lot of them are very good (unlike myself, I get the occasional "oh I'll keep that"), b) we all tend to hoard the images we've taken, and c) there are many different ways to skin a cat (with apologies to cat owners)...

Personally I was using Picasa for many years until it recently started throwing a hissy fit and dating recent downloads back in 2015 (don't ask, I don't know why). So I went on the hunt for another viewer, which I wish I had done after I'd read this thread! My personal modus operandi is built around the fact I have two computers, a small mac laptop that I use on a daily basis, and a larger laptop which now serves as a desktop that I use as the hub of all things photographic.

I'll confess now that I don't pay for any applications, I only use free stuff because I find it hard to justify the cost for the amount I use.

When Picassa started giving me issues I went searching for a replacement and actually settled on Bridge because it had some good reviews and was free to download. So far (three months in) it's worked reasonably well apart from a couple of intermittent issues downloading images from the camera to the laptop desktop. As I tend to stick with images in folders dated when they were taken I'm quite happy, I can roughly guess when they were taken so i can go back and usually find the one I want after a couple of guesses.

However after reading this thread I'll probably go back and check out some of the other applications mentioned. I have recently found that I need to rescale a number of images, which at the moment is time consuming as I need to open a processing application, load the image, rescale it, export it then open the next etc. When I'm focus stacking 20 or so images that takes a lot of time, so if any of these viewers allows me to do that quicker I'll be interested.

In general though I think the idea of tagging images is worth while if (and here is the condition) you have a lot of good images and you need to search for something specific and you have the time and discipline to tag them as you go along. For me at the moment none of those are true, I would be quicker having a single folder called "good stuff" that I copy the images I really like into....

If I had the time I really need to go back and delete all the dross that didn't work out and were instantly forgettable, if nothing else it would save me disk space (seeing as I have two backups of everything that counts for a lot on gash images).

Please continue the debate, seriously it is very interesting to see how everyone thinks and works.

[arthurking83]

@ Liney.
Be careful what you call dross and forgettable too.
I have a fair few images that easily fall into those categories, if not for the fact that there is something in those images that force to hold onto it for a while longer.

eg. I have 5 images of a red robin taken a couple of year back, terrible images in terms of presentability, grainy to high heaven, blurry, and not enough focal length .. etc.
But they are the only images I have of a red robin, and is the only reason I keep them(I'll cull that down to 1 image soon too).
Reason for keeping them is literally for the keyword I've added.
Now that I have that keyword and image, if ever I get a better image of red robin, I don't have to go searching for "red and black small bird with white stripe on head" on the interwebs again! :p
(reason is that; as far and bird ID goes .... well, I know what a seagull looks like! ;))

I open Photo Gallery, and on the keyword/tagged data column, I go to 'Birds' click this keyword and all the images of birds display on the page. I scroll through that and look for similar bird images.
Under the main 'Bird' keyword are all the sub keyword entries of the main bird species name(eg. red robin, eagle, seagull, etc)
ATM it takes me about 2mins to scroll through all the images as there aren't all that many ... maybe a thousand or so. I can also click each species that is sub-listed under the main Bird title.
That's the way I've set up my keywording/tagging system in my images.

So, the keywording is not only relevant for future searches, but also helps me in identifying stuff I have zero clue on to begin with.

ps. one key point with keywording is to add as much necessary data as possible, without overloading the image with fluff!
As an example of what I mean.
I have an image of a landscape(my primary interest).
Lets say I have a scene with a church in the background, it's old historic fence/gate in the foreground and a nice landscapey distant backdrop.

Obvious keywords would be church, and the location of the church.
But, if the fence or gate has a high level of appeal, then I'll also add those 'values' as keywords too .. and for any other content of interest.

[Tannin]

Which red robin?

Do you mean the Red-capped Robin?

Or the Scarlet Robin?

Mind you, the Rose Robin is pretty red too.

Not to mention the Flame Robin, which is ... er ... flame red.

[arthurking83]

Actually scarlet robin was what it was.
Went onto a bird ID site found on the interwebs .. but they do my head in.
Seagulls, albatrosses .. scarlet robins! .. they all look the same to me. :p

[jim]

Gets me the same way Arthur. “Ooh, Scarlet Robin! Aw, just another bloody albatross”.

[ameerat42]

My refrain is usually, "Oh, a sparrah!" for small birds:o

[landyvlad]

Great thread. I wish all my images were tagged as it would make it much easier to find what I want eg land rover, shotgun, motorcycle etc

[Mark L]

"Was that an albatross?" Robin asked.
"No" said Scarlet

[landyvlad]

(PS to anyone following this discussion who uses raw + JPG: setting up a similar system is quite simple. The only technical part is writing the script part-quoted above. Sing out if you'd like a copy of it - you don't need to understand it to use it anymore than you need to know how Kym wrote the AP competition scripst in order to vote for a picture.)

Mate consider this a "shout out" for the script, and how to install it (and remove it if desired).

While I don't use RAW and JPG together NOW I may well do in future so this would be good to have :)

[arthurking83]

No-one has yet satisfactorily addressed the massively time-consuming task of tagging many thousands of files. .....

There kind of are two solutions that make it less painful to do, but it's not painless.
Those two programs are Microsoft's Photo Gallery software, and IDImager's Photosupreme.
It's tedious work, but as John(and I) have said, once the process has been started, the benefits are clear.
The only point that John and I disagree on, is what is important to tag, and how it's tagged. I only think in terms of Raw file = important, any other (child) file type = unimportant .. tag the stuff that you NEED to keep in a worst case scenario.

NOTE: tagging thousand of files isn't a case where you tag each individual file singularly, tagging can be done in batches on many files at once. My only proviso is that it's done on the raw file itself.


On the contrary, if there were significant benefits, everybody would do it...
The problem with this argument is that it's 'chicken and egg' syndrome.
That you don't do it NOW, doesn't mean that you see some benefit at a point in the future. The problem as always with the future we know nothing about it!

I think of that argument in a similar manner to (say) electric cars. On the current path we travel on, it's massively obvious that electric cars are the future. Problem is they cost a fortune(to buy). Every day cost to run is much lower for those of us that rely heavily on the benefits of transportation by car. I'd much prefer for it to cost me 10c per day to travel, rather than the current $5 per day it does .. but I'm not yet willing to pay $50K for that 'benefit'


...
Bridge handles IPTC keywords and searches better/faster than any other program I have come across. ...
I really don't think you've tried enough.
You listed some timings there which are in some ways humorous to read.
The old discontinued(but still usable) and free Windows Photo Gallery is instant. Not only instantaneous, but also predictive. As you type a tagged word type it lists all the tagged data with the letter combination already typed.. ie. before you've typed the keyword you're after, it's already given to you.
Not only that, but you don't even need to type a search. all keywords are listed in a pane on one side that you scroll down too. click it, and all images for that data are displayed.
The only tradeoff is that it takes between 30-60sec for the software to load. .. but on reflection that is about half the time most Adobe software takes to load for me anyhow! :p


It wouldn't work for everyone, but I store everything - absolutely everything - by date, with folders further labelled by location. ....

I do the same, as I think many folks probably do too. It is by far the best form of organisation to begin with.
But it has failings.
Reason why I replied again, is that on another forum a comment was made re: a new train service from Maryborough to Avoca. An area Tannin probably knows very well.
I also know very well.
Those of us that know the area well enough, know that this new train service isn't new, it's being reopened. That is, there's an old line that is being renewed.
I happened to have an old photo of that old line. The buckled twisted remains of the old rail lines, easily viewed and photographed at a crossing between Avoca and Ararat. I know the road, even tho I also geotagged it, I didn't need too. I know the road well, get me into Beaufort from the north side.

In the last 3-4 years, I've 'explored' the area 5 times, and I know I have an image of the rollercoaster train line in it's dilapidated state .. but looking through 5 folders of images taken in the area over that time revealed nothing due to the quick glance method I used to view the images there.
So I searched all the folders I have of the area thinking that I may have lost track of time and the image I have of the rollercoaster was taken earlier(within the last 12 years), so searched all folders of that area I have.
Nothing! :(
Turned out that it was in one of the folder I originally glanced at, and my mistake was to view the last few images of the folders, as I thought the images of the railway were my last images for the day. I had about 15 more after the rollercoaster images.

Short version:
Moral of the story; no matter how good you think you are at remembering stuff, time is better at 'helping' you forget it all, or confusing small details about a general idea.
Keywording(or tagging) eliminates the effects of time. How valuable this resource is, comes down to the individual themselves.
Tagging images in their multiple thousands is a monumental task. The tools you use make a world of difference. now that I've started the process, which has taken me over 5 years to do less than half the actual job I'd like to do. I don't do it all the time, fits and spurts of it is probably the best way to describe my method.
I've subsequently tagged the dilapidated rollercoaster images of this train line ... stupidly tho I had tagged the other images in the folder with place names, and other stuff .. just for whatever absent minded reason, not the railway images.
I've also made a mental note to get back to this same crossing location to get more boring images of the now updated line, and have before and after shots.
But what I'm kicking myself over is not having gone back earlier .. pre line update to get proper images of the interesting looking old crossing.
Again it drive home the accurate adage about putting off what you can do now, for a later time .. sometimes it just doesn't pay.