ImageIngester now ingests up to 8 cards simultaneously

Use this new version cautiously, as it has a newly re-written ingestion engine.

Updated 5-July-2008 to reflect changes in Version 3.1.04.

I’ve just completed the OS X version (3.1.01) of one of the most significant ImageIngester updates ever. You can now load up to 8 cards at a time and ingest them together, using as many cores/CPUs as your computer has. It’s a winner even with a single core/CPU, since ImageIngester spends most of its time waiting for disk and card I/O (“input/output”) anyway.



For testing I used two powered USB hubs, each with 4 card readers, but any combination of readers will work, with any combination of connection both directly and through hubs. You can use a specialized device like the Delkin ImageRouter, but you don’t have to. (It’s much cheaper to use regular card readers.)
I haven’t tested FireWire readers, but they should work OK.
(Let me know if they don’t.)

The time to ingest, say, 4 cards at once is often less than 4 times the time to ingest one card because of overlap of CPU and I/O, and because of improved disk scheduling. Sometimes you have a disk bottleneck and there’s no time advantage. Still, it’s really convenient to load all of your cards from a shoot at once and leave the whole lot to ingest while you do other things, rather than having to keep checking so you can load in the next card.

In a test with 4 cards, I got the most favorable times when I ingested into just the primary folder, with no backup, even though that’s not how I’d suggest you use ImageIngester.
It took 2:02 (minutes and seconds) to ingest 4 cards, and 1:55 to ingest only one.
(Dual-core 1.83 GHz iMac with 1.5 GB of memory and a 500 GB FireWire drive for backup. Rebooted before each ingestion.)

With a pre-conversion backup, but no DNG conversion, there was no time advantage for ingesting multiple cards at all (just the convenience): 9:26 for 4 cards and 2:08 for one. My theory is that, of the 9:26, about 7 minutes were for copying from the backup to the primary folders, and there were only two drives, the external and the internal. So, since the I/O capacity of the drives were fully used, copying more data simply took longer and even created some bottlenecks.
Having dual cores didn’t help because the process was entirely I/O bound.
For the initial copy from the cards, however, the reads were from four independent card readers, allowing for much more overlap.

(This suggests that a different method for copying to create the pre- and post-conversion backups and the primary copy might make sense when there’s no DNG Conversion, but that’s not something I’m going to explore right away.
It’s more important right now to get this new version stabilized and to work on the Windows version.)

For ingestion with DNG Conversion, which is how I use ImageIngester, the times were 14:00 for 4 cards and 4:58 for one. This test had just as much copying as the previous one, without DNG Conversion, but a lot more CPU time, so there was more opportunity for overlap between CPU and I/O.

To use the new multi-card feature, instead of the Start button you press the Cards button:



This brings up the new Ingestion Monitor panel:



From this panel, you see up to 8 mounted cards, and you can press Refresh if the mounts occur when the panel is already up. You can press Eject to eject all the cards, but only after all ingestions are complete.
You press Close to close the panel.

You press Start to start the ingestions, and Stop to interrupt processing and stop.

Not all ImageIngester features are available when you’re ingesting from multiple cards, as this chart shows:

Feature Single-card (Start button) Multi-card (Cards button)
ingest from arbitrary disk folders, choices from the thumbnail chooser, or from the multi-camera-shoot feature yes no (actual cards only)
auto-ingest and auto-eject yes no
remove card as soon as 1st copy is complete yes no (too dangerous, as card readers are usually hard to identify)
separate number-range folders per card yes no [updated](all cards are treated as giant single card)
auto-numbers sequential within cards yes [updated] yes no (intermixed; otherwise each card would have to wait to find out what number to start with)
partitioning of number-range folders [updated] yes no (temporary limitation) [updated] yes no (temporary limitation)

[updated]As the chart indicates, I’ve temporarily disabled partitioning, but I expect to have it back soon.

The reason for not providing a way to eject single cards or not ejecting them automatically is that it’s too hard to identify the separate readers.
Removing the wrong card could be disastrous!
To see the problem, here’s the Ingestion Monitor with 4 Leica M8 and 4 Nikon D200 cards loaded.
See if you feel comfortable telling which is which within the Leica and Nikon groups.



(Someday I hope reader manufacturers will provide a way to tell one reader from another,
and camera manufacturers will put useful electronic labels on their cards.)

As usual, please give me your feedback either as comments here, in The DAM Forum ImageIngester area, or emailed directly to me.