Saturday, July 24, 2010

Moving to SSD

I use Photoshop and it's little brother Lightroom for my digital workflow. Lightroom is especially handy for the massive image collection I make for Derby bouts. The usual image count after a night of Derby shooting is well north of 3000, and I need a tool that allows me to easily and quickly browse through each of those images. I copy the RAW files from my CF card directly to a queue directory on my MacPro's boot drive. Once the cards are emptied into a queue folder on in my Pictures directory on my boot drive, I set up a Lightroom import with all the keywords and file renaming rules for each bout. The Lightroom import then reads the images from the queue directory, then one by one renames them with that bout's format, then copies them to my 3TB RAID array drive where the images are stored permanently. For this import, I set Lightroom to render 1:1 previews, so I can load them into the Loupe module and view them quickly one by one, filtering out the final set that goes on for further processing. Since I use MacOS X, I have a built in Veritas based backup tool that periodically checks the drives I have assigned for backup for changes. The new images certainly meet that criteria, so when that process completes I have three copies of each bout. One on the CF cards which do not get erased until my next shoot, then the ones in the queue drive, then the copied images on my image array, then the set that gets copied into the 4TB network backup array I have set up on an old Macintosh G4 plugged directly into my router in the basement.

Lightroom uses a local database to catalog images. There are actually two database files that are created. One for the references to the images on disk, then another for the previews that are created for expedient editing. These database files exist as a MacOS X packages, which essentially is a directory that acts like a single file when viewed in the Finder. The problem comes in when the previews package grows to a massive size. Lightroom has to read and sort through this previews package to operate, and just before this exercise that package had grown to 52GB in size. This creates a number of problems, such as space consumption, but the size of the preview repository creates the real problem: Crappy performance.

When Lightroom loads, it reads through this 52GB file to re-link the images in the catalog to their corresponding previews. OS X goes through a massive number of reads during this process. These reads can be viewed in the Activity Monitor tool in the OS X Utilities folder. This read process is the real problem, because once this file gets very large, it performs very poorly. The file is so large, it takes a very long time to process, and the noise coming from my hard drive is disturbing. The disk is incredibly busy, thrashing continuously for a very long time, to the point that I think maybe I am jeopardizing its longevity. It would suck very much if this drive failed, even if I am completely backed up over my home network using Time Machine.

To solve this problem and get back to the snappy performance I need to process bouts quickly, I decided to rig up a fast SSD drive for booting. I picked up two of these. OCZ has a pretty good reputation in the business. If you're thinking SSD, right now Intel's drive and this one are good bets. I chose this one because it has insane read-write times. Check the specs on the link.

I also picked up a spanky new MacBook Pro. I him-hawed over the size for a few months before I made the jump. I also wasn't amused by the fact that the smaller MBP runs on the Core Duo architecture, rather than the Core i5 or i7 with the bigger MacBooks. Turns out there are potentially good reasons for this. If you don't feel like read Ars Technica after this windy tome, basically because of the small form factor, Apple woulda had to have stuck with Intel graphics, which suck in comparison to nVidia's offering. So, having settled that, I landed on the 13" MBP. I toyed with the concept of calling it MacJr, or LittleMac, or the terminally pseudo-cool L'il Mac, but, no such thing is happening here... I'm sticking with our internal naming, which, you can think about on your off time if you care about such things.

But the temptation really in giving this thing a pet name like that is it's an incredible little machine made even more incredible by the swapping out of its silly little 5200 RPM 250GB hard drive with one of those fast OCZ SSD drives. It's small, lighter even now, has a snappy little LED display, and boy now is it fast.

I was also nervous about opening the case up to do surgery on the thing not 24 hours after taking it out of the box. If you know anything about Hyper MacGeeks that event is a near religious experience. I'm not a HyperGeek, just a geek, so, I was able to take it out of the box without shedding tears...

Of course you want to clone the drive that came with the machine first, which took about four times as long to complete than the actual drive swap-out. I picked up a reasonable external 2.5" case at the local MicroCenter, then I seated the the SSD in it long enough to run Carbon Copy Cloner to create a full image of the original drive on the new SSD. That took about 45 minutes and created a perfect copy of the Apple installed drive on the SSD, complete with all the root and hidden files that give Unix OSes all their charm. After identifying the SSD as the Mac's boot drive, it was time to shut down and perform the outpatient procedure.

Now what's really incredible about iEngineering is the form factor of this little machine is very elegant. If you read the Ars Technical article, you know everything is pretty much crammed in the case, which makes it hard to work on. That is unless your objective is a simple hard drive swap. That turned out to be ridiculously easy, well, except for the nasty surprise that you need a Torx T6 Screwdriver to unscrew the drive-mounted anchors, then screw them back onto the replacement drive. That little bit of info wasn't mentioned in the user manual, which happily included otherwise complete instructions for the hard drive replacement process. Unscrew 10 tiny screws off the bottom, pop off the aluminum cover, unscrew the drive anchor thingy, disengage the drive from its SATA connector, pop the other one on it, screw the case back together, then prepare for Nerdgasm.

So with the operation complete, it was time for the real test. If you've ever hit the power button on a MacBook you know that can be a scary experience. MacBooks are notoriously quiet, and it takes a few seconds to cycle up to the Apple logo screen, then a little while longer to go through the spinner. On my old PowerBook, I used to hold my hand on the case to feel for some sort of vibration. Happily the MBP has a light in front that confirms operational status. So, from the time the light shone to the time I could click the first icon...

27.4 seconds.

Insane.

As it turned out, that was just the warmup. I have this bad habit of starving myself, then going on these big sprees where I bag all my stuff at once, so, having spent the load on the MacBook, why not get a new SSD for the big Hoss main computer, the Xeon based MacPro that heats up my upstairs office every night...

And so it came to pass. And, all that glorious ease of install I enjoyed on the MacBook, well, you can forget that. I have the first Intel based MacPro, which means my Optical drive bays are no SATA. Since I have all four of my drive bays occupied (one for the 250GB previous boot drive, two for a 3TB RAID array for photography, and one holding a 640GB iTunes library which feeds my hungry AppleTV, my only option for the 2.5" SSD was to mount it in the one open bay I had left, the second ATA 5.25" optical bay. Happily, there are two free SATA ports on the MOBO. Unhappily, they're stupidly wedged behind a fan casing that makes them nearly impossible to use except with a lot of fussing, sweating, and finagling, and eventually it turns out a pair of angled needle-nose pliers. Seating the 24" right-angle SATA cable in the motherboard turns out to be the easy half of the problem, for the next task is to route the thing through a nearly full rectangular opening in a metal plate that separates the optical drive bays from the motherboard. This turned out to be even more painful, because the MacPro uses these really cool drive bays with mounted SATA ports, which are jammed in the back of the case, as you would expect. You have to route the SATA cable such that it does not cross the path of the drive sled, which ultimately meant I had to unscrew one of the SATA ports to get the cable routed behind it. Happily Apple chose to use captive screws for this, which made the job way way easier.

It took some effort and the angled pliers again to complete the routing, but, once that was done, and the power connected, it became a matter of seating the drive in the 3.5" to 2.5" converter, which was mounted in turn in a 3.5" to 5.25" metal converter, then slid securely in the removable bay. Once I got that done, I could taste victory, but not before another obstacle had to be overcome.

With Missy hanging out watching the proceedings (and the sweat rolling off my nose), I tried to close the deal by reseating the drive mount. But, it would not go. Being intelligent enough not to force it, I started playing with the cables to no avail. Talking to myself didn't help either, and eventually I realized I had flipped the lock switch on the back of the case that Apple uses to secure the hard drives in their bays. Flipping that back open was the last step in the job. I was done. A week after I started, and one false start when I had to learn the hard way the SATA ports would require a special tool (or worse, removal of innards, which I had no interest in doing), I had the SSD mounted. That was last Friday night.

The MacPro spent Saturday deleting files I didn't need to clone, then actually creating the clone, then restoring the files I deleted from TimeMachine, which was a pain since I dropped over 75GB worth of data temporarily. I know, I could have created a handy Unix script to skip over that part, but, using Time Machine was faster and less risky. Having done that, by the time the guests left on Saturday it was time to see what impact the SSD had on my setup.

Boot time:

40 seconds.

Huge performance gain. Booting easily took 3 minutes before. I timed my work PC today, it took 6 minutes, which is ridiculous but I presume has to do with our network... But the real test would be real-world applications, which, in my case means Lightroom and Photoshop, and I am happy to report huge performance gains there. Do I have stats? No. But what used to take many seconds is now happening almost instantly.

Of course, SSDs are problematic given the technology's unique erase requirements. Because essentially released disk has to be zeroed before it can be rewritten, some SSDs suffer laggard performance after they've been written to and erased several times. There is a technology solution for this called TRIM, but Apple doesn't support it because they're under the impression they don't need to. At least one technical resource tends to agree...

So, who knows. Recently a reference to TRIM showed up in the latest MacOS X update, so perhaps it is coming. Perhaps it is not necessary. Stay tuned...

Monday, July 5, 2010

On the road to Atlanta


CRG062610_Sheep_2137.jpg, originally uploaded by jsevier14. The team celebrates a comeback win. In the middle of the euphoria a relieved Trauma has her hand to her face, and Cherry is midair jumping for joy.

When I first started shooting Derby generally except for the score I didn't really know what was going on in the bout I was working. I was so engrossed in the composition I usually had to look up at the scoreboard to at least see if we were winning the game or not. I did this with music too. I couldn't tell you what song was being played, I was just busily shooting faces, expressions, instruments... The same with Derby. Faces, expressions, plays, equipment... I've gotten much better about knowing what's going on as I've gotten to know the sport.

About last Saturday's trip to Atlanta... Although I've gotten better, I can't really do a decent recap. I'll leave it to Miss Print for that... I can say this much. The Lambs had an easy time of it. The Sheep did too, most of their game, but by the last 5 minutes what looked to be a rout devolved into an epic struggle with an ending that was fit for the next Derby movie. It was as tense a Derby experience as I've had, and from what I've heard from the team's players they were feeling it too. So intense it was, my heart was racing even as I worked the pictures from the last two jams, the first of which the lead was taken from CRG, only to have them come right back in the last jam and win the game. By the end, all was well, the rightful winners prevailed. As the jam and the game ended I sprinted from my spot in turn two all the way to the other end of the track where the celebration was well underway (shown above).

See the Lambs shoot here...

See the Sheep shoot here...