A View to Hugh

There may be no way around it: describing, in a meaningful way, my experiments with batch scanning for mass digitization does not lend itself to short blog posts. One of the audiences we hope to reach with the Morton blog is practicing archivists, especially those exploring the realm of mass digitization. If you are not an archivist, or someone interested in scanning large quantities of material, I will not blame you if you stop reading here!

The second installation of this series on scanning slides for mass digitization discussed the issues with film scanners that produce high quality scans but at unacceptably slow speeds. Using the Nikon Coolscan film scanner was one of two approaches I explored with equipment on-hand. The second method investigated, the focus of this post, was the possible use of a good quality flatbed scanner—an Epson Expression 10000XL Photo model running Silverfast Ai scanning software, which has a batch scanning mode.

Using a flatbed scanner addresses the need to scan more than five slides at a time, a serious limiting factor for the Nikon Coolscan. (To be fair, the Coolscan was not designed for volume scanning.) The hope for the flatbed scanner was that in batch mode we might be able to produce image files that would be adequate for online access and basic reproduction requests at a higher rate of throughput. The 10000XL comes with two slide holders that, in total, hold thirty slides; bypassing the holders and laying the slides directly on the flatbed expands that number to forty-eight. Note that laying slides on a scanner’s flatbed only works if the scanner has an adjustable focusing mechanism, because not all slide mounts are the same thickness and the film does not rest on the surface of the flatbed. A fixed-focus scanner, therefore, may produce out-of-focus scans.

My test sample was a varied batch of slides from the 1950s and 1960s that has two sets of handwritten numbers on the mounts and were likely from slide presentations. This group represented a variety of subject matter, lighting conditions, film emulsions, and exposure latitudes (slightly overexposed to slightly underexposed).

I tried two methods: scanning each slide individually using the Silverfast batch mode, and scanning multiple slides as one large file. The scan above is a single image file of forty-eight slides laid out on the flatbed. The scan that follows, made from a different set of slides, uses Epson’s slide holders. (The scan below has two slide openings that are unfilled. That’s because I loaded the slides in numerical order, and two slides were missing.)

Silverfast’s batch mode requires the operator to draw a selection rectangle around each image, as seen in the detailed screen shot below. Though tedious, an advantage of selecting and then scanning individual frames is that an automatic or manual tonal and/or color adjustment can be applied to each slide. A disadvantage is that an automatic or manual tonal and/or color adjustment can be applied to each slide. Does that—being both an advantage and a disadvantage— sound like doublethink? Well, the added step to gain better image quality extends the amount of time it takes to set up each slide. A straight, unadjusted scan from the preview may be faster, but it’s usually not acceptable “as is.” Some tweaking of the preview scan—”preprocessing”—is required to get the final scan close to where you want it. You just can’t labor over it!

Note in the illustration above (made from a different set of slides) the dotted rectangular selection boxes. Note, too, the odd color cast that is the result of the film emulsion’s color dyes shifting over time. This is one reason why you need to make adjustments to raw scans, even if they are very close approximations of the original slides. Here’s an adjusted version, corrected after scanning (“post-processing”), of one of the images above:

One nice aspect about individual scans, however, is that once you click on the “Scan” button you can walk away from the scanner for forty-five minutes and do something else . . . like eat a hearty lunch.

Scanning multiple slides as one image file presents several dilemmas:

At what resolution do you scan the batch? I chose an equivalent of what the Nikon Coolscan would generate—4,000 pixels per inch. That creates a very large file, around 1.25 gigabytes for an 8-bit scan, for forty-eight slides. We experimented with downsizing the file to a presentable JPEG for Web display. Adequate, but it’s another layer of work that drags out the process.

To reduce that workload, several steps could be set up as an “action” in Photoshop where the software records individual steps as a programmed script that can be applied to subsequent batches, even whole directories of image files. You just need to be meticulous to the point that the scans are made so they will look the way you want them to look afterward.

Scanning a batch of slides laid out on a flatbed as one large file produces a lot of dead, black space where the slide mounts do not allow the light through. Using the Epson slide holders creates even more dead space, only it is a mixture of black and white dead space (see illustrations above). Using Photoshop, we experimented with cutting out the images and pasting them into new Photoshop documents. That’s better looking, easier to make tonal adjustments for individual items (see the hungry pirates), and better suited to item-level descriptive indexing, but again, more production work that defeats any gains made by batch scanning. And you have to remember to flatten the file or the resultant file size is quite a bit larger.

What if a batch to be scanned could be limited to a common subject and thus be described as a whole? That would work only so long as the batch has some underlying commonality. The disparate group of slides I scanned did not lend itself to group description. In addition to the pirates at Hatteras, I did scan a batch of slides from an Azalea Festival parade as a second sample. But each grouping had a different number of items. Small groupings compared to larger ones could potentially lead to sloppy online presentation.

While I have a bunch of facts and figures related to scan times and file sizes using both methods, the biggest dilemma of all is an inherent flaw in all but the top-of-the-line flatbed scanners: the lack of edge-to-edge sharpness. In the end, quantitative statistics are moot because of that flaw.

Most scanners have a strip running through the center of the scanning area, its “sweet spot,” that produces its best quality scan. Once you move outside of that sweet spot, image quality drops off—extremely so in the corners. On the Epson 10000XL in our shop, the sweet spot seems to be in the center, falling off gradually to its worst in the corners. Scans of slides outside the zone have a soft, blended pastel look reminiscent of looking through a shower door with textured glass.

Here is a full scan, from the upper right corner of the batch of scans in the opening illustration. The scan below is a 100 percent crop that I then reduced to 440 pixels wide:

The result is perfectly satisfactory for online display at 440 pixels wide. But the devil is in the details! Here are two 100% crops, not sharpened, displayed at 440 pixels wide:


See the flat, distorted appearance of people’s faces? Note the lack of detail in the camera and tripod? If our only intended use of the scans was online display, this batch approach would suffice; for reproduction, however, they are clearly unsatisfactory.

Batting 0-for-2, it was time to plow through the Internet for a better solution. We think we’ve found one . . . the subject of the next post.

It’s been a while since I wrote the introductory post “200,000 slides” on how we might approach the digitization of Hugh Morton’s slides. There’s been a lot happening on this front, but doing that work and then the holidays have delayed my next installment on the topic. I’ll try to keep the posts short and frequent over the upcoming days.

The close of my initial post ended with a set of questions, the last of which was, “And the most challenging and most difficult question, . . . how do you scan 200,000 slides?”

Well, how do you scan 200,000 slides?

There are at least two points to explore in order to answer that question: on what machine and to what resolution? I experimented with two approaches available to us with the equipment on-hand: a dedicated film scanner and a flatbed scanner. Today’s post focuses on the film scanner.

The best quality film scanner available to us is the Nikon Coolscan 9000 ED, which produces scans at 4,000 pixels per inch. We scan at sixteen bits using the eight-pass setting to produce a high quality scan. That resolution easily produces 20×24-inch high-quality inkjet prints on our printer, an Epson Stylus Pro 7800, at 360 dots per inch. (Did you just find yourself saying, “Huh?” If so, Scantips.com is one of many Web sites that can help you!)

Each scan takes about ninety seconds, with a short interval between scans. The scanner can handle only five slides per batch, however, so here’s the disappointing math:

200,000 slides x 90 seconds = 18,000,000 seconds or 5,000 hours, which is 625 eight-hour work days—well over three years.

And that’s just raw scan time without counting the time it takes to set up each slide’s appearance using the scanner’s software, which is considerable. Simply stated, a Nikon Coolscan is not designed to work in such a high-volume environment. High-end film scanners, such as a Hasselblad Flextight, do have batch loaders that handle up to 50 slides, but the added expense of that machine buys you higher resolution—8,000 pixels per inch—not speed. With voluminous slide collections, speed is of the essence!
Link to Part 3

What would do you do with 200,000 slides?

While trying to figure this out for myself, and posterity, I keep hearing two Paul Simon lyrics running in my head:

Kodachrome
They give us those nice bright colors
They give us the greens of summers
Makes you think all the world’s a sunny day

and

Slip sliding away, slip sliding away
You know the nearer your destination
The more you’re slip sliding away

No, I’m not going to jump off a cliff, although I did do that once—and then a bridge later that afternoon—in New Zealand during my younger years. Instead I’ve taken my adventuresome spirit to the computer keyboard and scanner. I’ve already written about six pages in a strategic plan and several longish emails about my exploration of this topic—and there’s likely more words to come. And I’ve sat in front of flatbed and film scanners running tests, because we are trying to bring 21st century technology into the solution to make processing the Morton collection more efficient. But when I try to think the process through completely, the solution seems to get more difficult. On one level—a bird’s eye view—the idea is very basic; as I start to apply more details to the picture—as I drop in elevation to reveal more nuances in the landscape—the technological solution seems to slip away.

During my afternoon break a few days ago I realized that, if I could explain the issues to those of you reading the blog, I could perhaps break this complex topic into simpler parts. So that’s what I’m going to try to do over the course of several blog entries. (I’m afraid one extremely long entry would keep you from ever wanting to return!) So let’s start from the sky and work our way down to ground level.

The ultimate goal of processing these slides is to make them accessible to users, and these days that means online via the World Wide Web in addition to the physical objects. To make the slides accessible, I first needed to sort and organize them because, initially, there was “neither rhyme nor reason” to the way we found the slides. Mostly stored in their original boxes, there were also many loose slides or small batches wrapped in paper, rubber bands, or in odd boxes. And several presentations are still in Kodak Slide Carousels or even the older Cavalcade Slide Trays. Because most of the slides, by far, have a date stamped on the mount by the photographic lab that processed the film, I sorted the original slide boxes by year, one year per shelf in the stacks. Needless to say the boxes stretch from floor to ceiling—and in any other direction I could put them. With that completed, I then started sorting the loose slides by year, then film type (usually Ektachrome and Kodachrome), then by month and, within month by the red or black ink used for the date stamp. I learned I could do this surprisingly quickly, faster than trying to sort scanned images on a screen, so long as I didn’t look at the images. As soon as I pause to look at an image, the sort stalls in the water.

You may be asking two questions: why sort by film type first, and why sort to the level of specificity of red or black date stamps? Well, for the first question there are two reasons. First, whatever scanner we use will likely have different technical settings called “profiles” for different photographic emulsions. If we batch scan a set of slides that are all Ektachrome or all Kodachrome, then the post-scan tonal corrections will be more consistent and more easily regulated. The second reason has to do with reuniting the loose slides with those still in the original box—what I’ve been calling a slide’s “mother box.” If I look at the contents of a box and see that its contents are Ektachrome slides, I can go right to that shorter stack and have fewer slides to examine to see if there any slides that can be returned to their mother box. The specificity of the red and black date stamps follows this same logic: if I open a box and all the slides within it are Kodachromes stamped “May 65” in red ink, then I’ve got far fewer slides through which I’ll have to search. I can make these distinctions much more quickly during the initial sort than I can during a subsequent search.

The traditional way photographers sort slides is on a light box or a light table. Taking a magnifier called a loupe, you look at the images and make selections based upon certain criteria like proper exposure and focus, composition, and people’s eyes being open. Archivists may do the same, calling into play more criteria, such as repetition. Photographers generally work a subject from different angles and with different lenses, and bracket their exposures when they can. This means, for example, that Hugh Morton could have shot an entire roll of film just of a single flower. How many photographs of that single flower would we need to keep? How many would a researcher want to examine? Would an archivist in a history collection keep as many as an archivist in a botany collection?

If I were to look at each slide on a light box for five seconds in order to assess them, here’s the math: 200,000 x 5 = 1,000,000 seconds.
One million seconds! That’s equivalent to about 175 eight-hour work days. (I wonder how hungry or bored I’d be by lunchtime on the first day? Or how my back would feel hunched over a light box all day?) And those five seconds do not include the time to lay the slides on the light box in a readable fashion nor the time to gather them and put them back in the original box. So in round figures, it could take one year to look at all the slides—so long as I did nothing else on those days other than eat lunch.

But what if I digitally scanned each slide without selecting in advance which ones should be scanned? Could I preview and edit images faster (and spare my back) on a large computer screen? From a labor savings standpoint, would scanning everything be more cost effective than the staff time necessary to review each slide and make a decision? What if I threw away the scans I didn’t want to keep because that was faster and cheaper than evaluating and discarding the unneeded slides? Or what if we made all the scans available on the Internet and let researchers decide what was important or useful to their needs? And the most challenging and most difficult question, . . . how do you scan 200,000 slides?

My next post on the topic will explore some of these questions. Who knows? Maybe someday I’ll be able to write a song about all this!

I know you will be relieved to hear that I am no longer suffering from the processing anxiety attacks I referred to in my first post. I’m still anxious, but it’s more like continuous fretfulness than bursts of panic.
I am now in the midst of my second “pass” through the Morton negatives. The first pass involved going through the whole lot of them (estimated at 33,000 items—strips, rolls, or individual frames) and sorting roughly by subject (e.g., Grandfather Mountain, Basketball, Hang Gliding, Nature). In my second pass, I’m refining those initial sorts into smaller categories (Grandfather Mountain—Events; Grandfather Mountain—Grounds and Facilities, etc.), getting all loose material into envelopes, and describing the resulting groupings in a spreadsheet. This, I must say, is a pretty satisfying stage—I’m bringing like materials together and starting to build descriptive tools that can be used to navigate the collection. Order is beginning to emerge!
Of course, most of my excitement is squelched when I remember that I’m less than one-third of the way through the second pass, and there’s still a third and probably fourth pass to be completed. Not to mention those pesky prints, transparencies, slides and films that still await attention. Take it easy, Hull. One pass at a time . . .