I know this is likely "TL;DR" and really, if you're not processing thousands of photos at an event, this probably won't be very interesting to you. (And even if you are dealing with thousands of photos, it might not be that interesting.)
No photos in this one. (Have I lowered your expectations enough yet?)
Throughout the first few posts, I kept noting how long each part of the process took. Now, I want to put it in the context of the bigger picture of editing photos for a whole weekend camp.
Cutting to the chase, I'll just say that it took me far too long to get these processed and released. So I'm actually presenting this as maybe the wrong approach. It just took far too many hours and consumed just about every waking hour that I was at home for approximately 7 weeks. In past years, I've usually been able to do it in 4 weeks, and even that is a bit too long to be sustainable.
The resources I'll be thinking about here are:
- My time. It really does come down to losing sleep and not having time to do other things.
- Memory CF Cards. I use 5 or 6 fairly large cards - 32 GB. Each card holds about 1100-1200 RAW photos (at 21 Megapixels). From past years I probably shoot 8000-9000 shutter clicks so I want to delete as many as I can as early as I can and maybe fit another whole contest or two on each card.
- Disk Space. Disks are fairly cheap, but I still want to conserve physical space. 6600-7200 photos works out to about 192 GB and when a hard drive says 1 TB, it really means about 966 GB.
- Memory on my computer. I only have 6 GB of RAM and when it fills up, Lightroom slows down dramatically.
| time/photo | total hrs for 800 photos release) |
total hrs for 1650 photos (smugmug release) |
total hrs for 4400 photos (after deleting in Lightroom) |
total hrs for 6500 photos (after deleting from cards) |
total hrs for 9000 photos (total shots) |
| 1 second | 0.2 | 0.5 | 1.2 | 1.8 | 2.5 |
| 10 seconds | 2.2 | 4.6 | 12.2 | 18.1 | 25.0 |
| 1 minute | 13.3 | 27.5 | 73.3 | 108.3 | 150.0 |
| 5 minutes | 66.7 | 137.5 | 366.7 | 541.7 | 750.0 |
Before the Event
I had some ideas this year for what this set would look like. Over the years, I've settled on a basic overall look with color temperature and lifting the exposure, like I discussed in Part 1. Each year, I might play a little bit with the saturation and contrast. There was one year that I was happy with (I think it was 2012), so I took that as the base for a preset.
But this year, I had the idea to push it a little further and really try to make the dancers pop a little more. In past years, I haven't used the Adjustment Brush very much. This year, I painted a mask or two on every single photo that I released. I wasn't fully aware of the resource costs until I was deep into it.
At the Event
In a typical contest, I shoot around 200-400 photos(clicks of the shutter), and publish 20-80 of them. At the end of every memory card, I have to decide to change cards and leave unused space or to fill it up and deal with taking 30-60 seconds to change the card. If I have to deal with a card change, I try to time it out so it happens in the breaks between prelim heats or between showcase couples, but it doesn't always work out that way.
Once I fill a card, I'm unlikely to be able to delete enough to fit another contest on the old card, so it's effectively locked off. My priority becomes deleting photos on the new card, and save the old card for when I get back to my room.
If there is rapid spinning or an air step, I might shoot rapid fire 3 shots at a time, which just means more I need to evaluate. I used to do this more, but now I try to be more disciplined about shooting one shot at a time.
While I'm at the contest, I am crazy freaking out about reviewing all the shots on the camera. The back of the screen is a cumbersome viewer. Pretty much every moment I was there before the evening dance, I was reviewing shots on the back of the camera (note: real photographers look down on "chimping"). Between every heat (maybe a couple minutes) and between every contest(10-15 minutes), I review and delete as many as I can. When the next heat/contest begins, I try to remember the number of the shot on the card I was last looking at and at the end of the heat/contest, I go back to that number.
For all the real shots, I shoot RAW. Between contests, I like to shoot one low resolution/quality JPG black frame. When I upload the card to the computer in Lightroom, the black frame helps me remember where to separate the contests. Unfortunately, every year it seems like there will be at least one time when I will forget to switch back to RAW, and will shoot part of a contest or performance with low resolution/quality JPG.
Back in My Room
 |
| Most folders from the first 3 days |
We all have time management issues at a camp. In my room, I also need time to eat and shower - if possible, I try to do them while I'm uploading to the computer. I cannot believe how much I sweat just shooting photos without even dancing a step. (Sorry if that's TMI.)
Taking time to review all the photos made me late to most of the dances. I was in time to shoot the contests, but I regret not getting in more dancing earlier in the evening. It also meant eating was just grabbing a few quick bites of sandwiches and salads from Trader Joe's that I kept in the room fridge, and I didn't make time to dine socially with my friends, which is another of my regrets from the weekend.
This year, I shot approximately 9000 clicks of the shutter, and took it down to around 6000 on the card before uploading. Let's say it averaged around 2 sec - 9000 shots means a total of 5 hours. I actually think it was a little more than that, but it's in the ballpark.
Usually, I would upload to a single folder and call it something like "CH2014_0e_Fri_Unsorted." Once it is all uploaded, then I'll create a folder for each contest, and move the photos to the appropriate folder. It is pretty fast to create folders and move photos once they're on the computer, but trying to find individual contests on the card takes too long. Also, I want to get the card back into the camera ASAP so I want to do all in one chunk, especially if I'm going to apply the same preset to all of them on import.
The codes like "0e" are my notation to keep everything sorted. The digit indicates the day 1-4(0 is special or organizational), and the letter is to keep the contests in chronological order. If I think there might be special processing, I will indicate that in the folder name. For instance:
- "_JPG" means I screwed up and forgot to switch back to RAW, so I can't apply the same presets, especially with color temperature.
- "_30D" is from the second camera.
I also try to recharge my camera batteries if I have time at every break, and definitely each night.
After deleting what I could off the card, I've gone from about 9000 shutter clicks to about 6500 shots uploaded to the computer. I think I filled 5 32 GB cards from the 5D, and most of 2 4GB cards.
Shooting With Two Cameras
Sometimes, I just think it's useful to quickly pick the shots from one camera or the other. I could do this with metadata filters in Library mode, but I find it faster to keep the shots from camera two in a separate folder.If I want to edit both cameras together for a contest, I can just select both folders and work on them combined.
One of the things I didn't really think about the first time I shot with two cameras was syncing the two cameras. This year, I tried to sync the camera times together before the event, but I think they were still off by a few seconds and sometimes, I might need to grab all the 30D shots and shift their metadata time by a few seconds.
Also, the image numbers are completely different.
Phase 1 - Ranking and Rejecting
I may start Phase 1 at the event if I have time, but most of it will be done when I get back home.
I tried to delete as many as I can on the camera itself, but it is difficult to judge on the tiny screen on the back of the camera. Sometimes it may still look sharp on-camera, is a little bit soft at full size. I need to review all the photos again on the computer, where I can see them a little bit bigger.
A note about machine resources:
I've found that Lightroom often takes about 2 GB of RAM. (Though this set would take more). Firefox takes around 1.5-2 GB of RAM (I have a kazillion tabs open).Slightly surprising to me is my backup service (Crash Plan) can approach 2 GB of RAM. They're pretty unapologetic about that - "but what do you expect with a million files?" Well, I expect them to cache the status of infrequently changing archives on disk or by demand instead of keeping all the filenames in memory.
On top of that, it seems like Mac Snow Leopard is really really bad with memory management when it comes to polling at a lot of files. I think it keeps some reference to them around in "Inactive Memory" just in case I want to look at it again. But for a backup program like CrashPlan or SuperDuper, it needs to look at every file, but doesn't need to hold onto anything.
What this means is: when I get ready to edit in Lightroom, I need to disable Crash Plan, Time Machine, Super Duper, and usually Firefox. That is, I can't really have anything else running.
Set up the preset
With the first set, I'll need to check out a few of them to make sure the preset is working. This is also the time for me to do any experiments or adjustments (like the blue color calibration to compensate for the LED lights). I can copy any adjustments to the rest of the shots in the first group, then add them to the presets. Choosing the right presets could save me some time later, working on each photo.Refining the Preset took about 10-15 minutes.
One part of the preset which I was unsure of was including "Luminance Smoothing." I used to do Luminance Smoothing as a final step, but for a couple years, I've included it in the preset. I still don't know whether it is too computationally intensive, or if generating a preview gets around that. In future years, I might save it for the end again.
Phase 1-a : Initial rating
- Reject - Bad shot, nothing redeeming.
- 1 - Bad shot but still something I liked about it and don't want to delete
- 2 - Has overall problems(technical and/or composition), but has some element I like
- 3 (300-700 photos) - The people in the shot might like it. I will edit it and release on Smugmug.
- 4 (500-600 photos) - I think this is a good shot and I'll release it in the smaller group on Facebook.
- 5 (200 photos) - I think this is a great shot and I'll include it in my "Favorites" or "Summary" gallery.
- Reject - It is a burden for this shot to occupy space on my hard drive.
- 2 - Shot is not that good, maybe it's the only one I have of someone and coverage is a big factor.
- 3 (1500-2000 photos) - The shot is in focus, the pose is okay, probably someone's smiling. Might want it for coverage.
- 4 (1500-2000 photos) - This shot is good or great. Coverage may be a minor factor.
In case anyone's wondering, I mark the Tongues photos by tagging them Red ('6' hotkey in Lightroom).
As I rank the photos, I try to avoid doing much editing. Sometimes, if a large number are seriously underexposed, or clearly have the wrong color temperature/tint, I might adjust the group. I try to stay in the Library module, but may need to go to Develop occasionally to do a quick crop. Sometimes, I may have an immediate gut reaction ranking. Other times, I may flip back and forth, comparing a few.
I think I average between 2 to 3 seconds per photo(6500 photos), totaling around 4-5 hours.
Pretty consistently, I rank too many shots too high. Typically, I rank more than half the total shots 3+(expect to edit and publish). Sometimes at an event, you might ask me "Did you get a lot of good shots?" and my standard response is "I hope so." That is an honest response. In the initial ranking, I don't know if I have anything good yet and don't know where to calibrate the quality until I've looked at everything. I often don't have the confidence that I got any "great" shots to rank a 4, and I tend to rank too many "good" shots as 4 and too many "okay" shots as 3.
Since I plan on spending time editing the 3+ photos that number is important to consider. At this point, I think I have 3000-3500 ranked this high. I don't want to publish that many, and I don't have time to finesse that many. This is about twice the targets stated above, so I have not been picky enough.
Phase 1-b : Mass Down-rating and Re-evaluation
Now that I've seen everything once, I have some idea of what kind of coverage I have. I also know the numbers, so I have some idea of how many need to have their rating adjusted. I can start from the current rankings and decide which to rank down, but I find it hard to lower a rating, so I spend too much time thinking about it.I've found the approach that works for me is to take EVERY photo and lower the rating by 1. That is, start from the bottom and blindly re-rate ALL of the 2s as 1s, then 3s become 2s, and 4s as 3s. I have some idea how many I want to promote and can think of it in terms of "1 in 2" or "1 in 3" or so, and localize the decision to taking the best one of a smaller group.
This pretty much means going through most of the 6500 photos again and deliberating which ones should be promoted. Unfortunately, these are usually hard decisions, and I may need to run through the set multiple times because I still want to promote too many of them.
I think I probably spent 5-10 seconds per photo doing this re-evaluation, which totals around 9-18 hours. Really, I did make some deletion decisions in the initial ranking so I'm probably down to 5500-6000 phtos at this point, so it may be the lower end of that time range.
Phase 1-c : Re-evaluate the bottom - more rejections
In Phase 1-b, I spent a lot of time thinking about which photos were the best of the set. This (hopefully) sets a higher bar for quality and I can go through the lower rated photos rated 1 through 3, but especially the 1s and 2s and re-evaluate which should be rejected (deleted). It is easier now for me to think, "I have a similar better shot that I ranked 4, so I will never ever need to see this one again."I think this meant looking over about half the photos again (around 3000-ish?) and spending 1 or 2 seconds on each, so totaling between 1 and 2 hours.
After these deletions, my total number of photos is down to about 4400.
At this stage, I haven't done any serious editing to any photo - only some minor cropping, exposure, or color tweaks if the photo was really far off. I still have a few too many (maybe 20%?) in each category, but I've been trying to emphasize: there is a time cost to both re-evaluating the photo and to processing the photo.
At some point, it actually becomes quicker to accept I will process too many photos than to go through a few more rounds of re-evaluation.
Phase 1-d : Backups and Register photos with the U.S. Copyright Office
I didn't want to waste time and disk space backing up files that I would delete. But now, I've gotten most of the large scale deletions out of the way.During the down periods when I'm not editing (for instance when I am at work at my day job or when I am sleeping), I can turn on my backup programs. I think Time Machine is fastest, so I let it run overnight, and into the next day if necessary. When it is done, I'll turn on Crash Plan, but I need to shut it down (like unload the plist file - not just dismiss the app) while I edit. I'll probably wait until the weekend and use Super Duper to backup the images to a passport drive.
Also, now is a good time to export low resolution images of EVERYTHING (even the 1-stars just in case) and register them with the U.S. Copyright Office. Since I haven't shared any of them yet, I can consider it an "unpublished" work. After doing this a couple times, I have a template and the process is pretty quick and "only" costs $55 for the whole set. There are instructions for how to do this in "The Photographer's Survival Manual: A legal guide..." by Edward C Greenberg and Jack Reznicki. Really, that was the reason I bought the book.
This export can be done overnight, then it might take an hour or so to do the registration (mostly waiting for uploads).
This may be a strange place to put a divider, but this is where the workflow in practice deviates a little from the steps I list here. Up until now, every step has been working on the entire set of photos before going on to the next step. Starting with "Phase 1e," I start working on the rest of the pipe - 1-e through 3 per-day or per-contest.
Phase 1-e : Oh, Just re-rate the 3's and 4's just one more time...
I might do this first for an entire day's photos, then do it again per-contest. When I do it per-day, I have a better sense of the big picture, and the target of having about 200 4s per day and 100-200 3s per day.In the beginning, evaluating a set of photos goes pretty quickly. It may take less than a second to move to the next photo.
But toward the end, on day 3 and day 4, I've already done a lot of editing and used the Adjustment Brush, usually many of them, in hundreds of photos. This not only balloons the size of the catalog on disk, but I think Lightroom tries to keep as much of the catalog in memory as it can, and the memory usage of Lightroom spikes above 2 GB. My Snow Leopard Mac, also uses a lot of Inactive Memory, which slows things down a lot (I think that means it's making more use of virtual memory on the hard disk).
For Day 3 and Day 4, sometimes, it would take 10-15 seconds to just advance to the next photo. I estimate I probably had between 450-500 photos per day ranked 3+ that I would look though, mostly to decide whether it was at the right rating. Not including any decision time, this means it could take an hour or so just to advance through every 3+ photo or a half hour to just advance through the 4+. This was extremely tiring and demoralizing.
Now, these were getting to be tougher decisions, so they may have been 5-10 seconds each.
I probably spent 1-2 hrs on this phase for each day of the camp before doing any actual editing. At some point, I give up and decide it is less work to just edit too many photos than to decide which ones to reject.
After doing of the Phase 1-e for each day, I have about 842 photos rated 3 and 813 rated 4+.
That is, 1655 rated 3+. These are the ones that I will take through the next steps of editing.
Phase 2 - Editing
By this point, looking through all of Phase 1, I've probably spent around 20 hrs total just rating and re-rating the photos before even getting to start editing day 1. Phase 2 is when the real editing begins.
Phase 2a - Editing and impact to resources.
Phase 2a is when I apply the ideas from my previous posts "Editing Camp H 2014 Part 1-5". Well mostly Parts 3-5.As I've written earlier, I had a vision to really push the idea of making the dancers pop through pushing exposure, saturation, contrast, clarity, etc. through the Adjustment Brush. As it turns out (maybe I should have researched this first), if you make lots of edits with the Adjustment Brush, it balloons the size of the catalog. Here are some of my lrcat files now that I'm done(It's too scary to think about the accompanying data and previews). Sizes are in MB:
310 CH2014_blog/CH2014_blog.lrcatTwo things:
1546 CampHollywood/CampHollywood-2.lrcat
3297 CampHollywood2014/CampHollywood2014.lrcat
354 Lineage_Catalog/Lineage_Catalog-2.lrcat
430 SwingCamps_Catalog/SwingCamps_Catalog-2.lrcat
364 Swing_Venues_Catalog/Swing_Venues_Catalog-2.lrcat
147 WestieCamps_Catalog/WestieCamps_Catalog.lrcat
- Not including the 2014_blog catalog, The Camp Hollywood 2014 catalog is bigger than all the rest of my dance photo catalogs combined! This is crazy! I don't have significantly more photos in 2014 than any other year that I shot Camp Hollywood. The other Camp Hollywood catalog has everything 2010-2013 (and a few from pre-2010). The SwingCamps catalog has about 5 years of Cowtown Jamborama as well as a year or two of Lindy on the Rocks and Lindy Diversion. WestieCamps has Boogie by the Bay and 2 years of the US Open. What I'm saying is: these were not small events, and yet, just this ONE YEAR of Camp Hollywood dwarfs them all.
- 3297 MB (about 3.3 GB) is a big number. One of the annoying things in Lightroom is that it does not allow catalogs on network drives. I think I read something about "blah blah sql lite blah blah unreliable over networks blah blah corrupts catalogs." So I have to keep all the catalogs on the main drive of my laptop, which is only 139 GB. (and right now, I only have 8 GB free)
While disk space may feel cheap, it can still be at a premium when I can't put the catalog on an external drive. I believe there is a correlation between the size of the lrcat file and the RAM usage, possibly pushing it into swap. I can't tell if the Luminance Smoothing from the preset and the Adjustment Brush brushstrokes impacted the CPU usage. But in either case, there was a real impact to the time it takes to review photos.
I timed myself going through the photos from a couple contests toward the end and found that I took 2-6 minutes per photo. It was rare that it took under 3 minutes, and there were a few outliers that took 8 minutes or more. The slow ones were often because something required a fine-detailed mask, such as removing sweat stains or having a lot of extended arms and legs. These times include any time it took to just open the photo.
So let's say I average 5 minutes per photo. With 1650 photos rated 3 or higher, that totals 137.5 hours.
That is really the kind of thing that makes a person say, "Never again."
Phase 2b - re-evaluate again
Now that they've been processed, it's not about seeing the potential in a photo anymore, but evaluating for final product. Though I'm well aware of a (time) cost to review the photos, I want to just check again to make sure I think everything is worthy of the rating it has. At this stage, I might actually demote a few.I'll run through them a couple times, so let's call it another 30 seconds per photo or... 13.75 hrs. (about 3-ish hrs per day of the event - I think this worked out to spending a night or two after work just going back and forth)
A few years ago, I would wait until I decided which photos to publish before applying the Luminance Smoothing. Now, I do it upfront in the Presets. I don't know if that is the right decision. I still can't tell if it impacts the CPU. I thought maybe if it was in the Import Preset, the CPU cost might get absorbed into generating the preview image. I still need to research that. It might also differ between versions of Lightroom and on the hardware.
Phase 3 - Export and Publish
Now, I'm finally ready to export and publish the photos.Usually I try to time it for when I'm out of the house or sleeping. I try to make it through as many galleries as I can, then set them up to export. Usually, it will take an hour or two. I export:
- High res (3300 pixels on the long side) for the 3-star(and higher) photos that will go up on Smugmug
- Low res (640 pixels on the long side) for the 4-star(and 5-star) photos to put on Facebook
Setting up the exports takes 5-10 minutes, then I can leave it alone for a while.
For the Facebook photos, sometimes I renumber the photos from camera 2 so they are numbered in chronological order. (Normally, the photos from camera 2 are numbered lower)
When they're done, I review the exported photos one more time. Sometimes, they are close, but do not exactly match what I saw in Lightroom. Usually things are fine. Maybe 2 or 3 need to be redone. In a couple galleries, I almost reprocessed the whole thing. This actually goes pretty quickly and there's no delay flipping through the photos.
Note: (SHAME!) I'm not using a calibrated monitor, but I keep the intensity low (6/10). Home printouts match pretty well. Professional prints might be a hair dark, but are usually pretty good.
Finally, it's time to upload the photos.
First, I upload to my Smugmug site. I create a gallery per-contest. After the upload, there's still a little bit of prep work:
- Set the price lists
- Switch the portrait-oriented photos to an alternate watermark
- Pick a cover photo
When I'm ready to publish to Facebook, I switch the Smugmug galleries to "Public" then get started with the Facebook upload.
The upload goes pretty quickly, but Facebook now strips out the copyright information, so I need to cut and paste usage guidelines in every individual photo, which may take 10-15 minutes or so. Somehow, a few photos are not in the correct order so I need to put them back. Finally, I do a lot of tagging.
Often, I'll run the last export overnight, then the next morning, spend 2-3 hours for the upload and tagging and everything.
Conclusions, Regrets, Lessons
This all took far longer than I planned.
- Phase 1(around 30 hours) and 2(around 150 hrs) each took far too long. I evaluate the photos 4 or more times and the time really adds up, and I need to work on speeding that up and making a more honest and less optimistic ranking earlier on.
- Writing these blog entries took are longer than I expected - about 4 weeks for the main bulk of that. I think it was probably about 50/50 between writing and preparing the images.
- I honestly don't know if it was worth it. I think most of the photos looked better than if I hadn't made the adjustments(well, to me anyway). But a few of them didn't, and I don't know if it was worth adding about 3 weeks to the processing time.
- Using the adjustment brush on every photo was very expensive. I may consider breaking up the catalog into smaller ones per-day if I was to do this again.
- I still need to investigate how much of a performance penalty is involved with Luminance smoothing.
- I regret so much time chimping and not dancing enough, especially when there was live music.
- This year, I didn't shoot as much social dancing, costumes, or goofing off in the hallway as I have in previous years, and I missed a few of the jam circles. I don't know if I have any regrets about neglecting that. I do regret not getting a few more shots of the bands. But that's always a difficult decision because if the band is playing, I probably want to be dancing.
- I may publish a less-than-great shot because it's the only one I have of someone in a given contest. But that really doesn't do anyone any favors, so I need to be pickier.
- Also, during the event I need more self control and should not be shooting on every 8-count. If the shot isn't there, the shot isn't there. Shooting without intent just means wasting time, sifting through a lot of mediocre photos later.
No comments:
Post a Comment