This post isn't really about Extensis Portfolio, iView or Lightroom - it's really another of my despairing rants - but today I was asked about getting metadata out of Portfolio and into something like iView or Lightroom.

The direct answer is that Portfolio lets you sync the metadata into jpeg and tif files, but for other file types your options are limited to using Portfolio's text file export. If your new program has a text import feature, then you can export all your data out of Portfolio and import it into the other program. Unfortunately, neither iView nor Lightroom has text file import - silly because migration inwards might make it easier to win new customers….

The other main route is via scripting, and this can be done in a number of ways:

• A Portfolio script might generate xmp sidecars, which iView or Lightroom can read. This won't work if you want to move to the Mac-limited Aperture because, though some tout its DAM features as being better than the competition, it can't read xmp metadata in sidecars or if it's embedded in the image.
• Script Portfolio and iView in tandem, so the script reads each Portfolio item and updates the corresponding item in iView. This method won't work with Lightroom because although it is marketed as “professional”, it doesn't support scripting. It might work with Aperture, but I don't know its scripting interface well enough to say.
• Export Portfolio's text file and then read it in a script that updates iView - this might even be adopted to update the SQL database behind Lightroom. Essentially this last method was how I got from Portfolio to iView, importing the text file into Excel and using its scripting engine to update iView.

In short, nothing is as easy as it should be. Thinking back, I used to do a lot of data migration of big financial accounting systems and 5+ years ago almost all of them had better text file import facilities than DAM packages which, after all, are designed for left brained Mac fanboys who really need help with big words like metadata. It's as if DAM software vendors think your archive's worthless, or you wouldn't mind re-entering all those captions and keywords. Come to think of it, the DAM program with the best text import features is probably… Portfolio.

I never claim to be an early adopter, and only recently discovered the value of Google Alerts. Now Matthew Campagna shows me the use of Google Trends by charting searches for Adobe Lightroom and Apple Aperture:

One thing to consider, however, is that Aperture remains exclusive [JB: in the sense of "limited"] to Apple computers, while Lightroom enjoys searches from users of both Mac and Windows platforms. Clearly, Lightroom holds an advantage in this regard, but the disparity is nonetheless impressively in favor of Lightroom.

I'm unconvinced that one needs to be so kind to Apple - it's not as if the results are skewed by including all photographers - but it's also interesting to see how searches for both applications compare to those including Photoshop:

After my diatribe about Lightroom writers not offering platform-limited suggestions….

While most people accept that moving to Lightroom means their Aperture adjustments will be left behind, it really surprises me that they are often resigned to losing their keywords and other descriptive metadata. There's no need to take that view because, while Aperture's metadata handling is rather old-fashioned, you can certainly get information out of it and into Lightroom without too much trouble. Here's how.

Inevitably I am not going to cover every eventuality in this summary, but first a few firm and very important words of caution - backup, test, review. And check everything afterwards - count files out, count them in. Also, I always advise people to test Lightroom thoroughly before making the big step. The grass is not always greener elsewhere and, to be brutal, for some people the real feature gap is between their ears, not in Aperture or Lightroom.

Handle raw files and JPEG originals as two distinct tasks. Looking first at raw files, you will normally need to export the masters from Aperture and choose the option to create XMP sidecar files. This means you will have duplicates of your files, but that is no bad thing when you're doing something as potentially messy as migrating data.

If you have put some effort into organizing your Aperture library around projects and folders, notice the export dialog also contains an option to make the export folder structure match the projects and folders. You can then import the new folder structure into Lightroom and it will read the metadata from the sidecar files.

While this method works well for raw files, you have to treat JPEG originals differently (this applies to DNGs too). This is because Aperture doesn't embed metadata in the JPEGs - it treats them as virgin originals and gives them XMP sidecars. Adobe products assume that these publicly-documented formats will not have such sidecars and so Lightroom won't read sidecar metadata for JPEGs.

One approach is to select all the JPEG files in Aperture, using a smart album (a feature you'll really miss once you've switched to Lightroom) to target version names containing “jpg”. Remove all their adjustments so they are back to their original state, and export versions, not masters. These JPEGs will contain embedded metadata and can then be imported into Lightroom. One possible issue is that you are essentially creating new JPEGs, rather than using your originals, but I'd contend that this is irrelevant - after all, you've got backups of your originals, haven't you?

Another method is possible if you have iView. You import all the files into an iView catalogue, then use the Annoture plugin to update the catalogue with Aperture's metadata. In iView use Action > Sync to write the metadata directly into the files, and then import the files into Lightroom. While ideal for JPEGs and DNGs, some may not want to write metadata directly into their raw files. It also works less well if you want to preserve your Aperture projects.

Although some developers of raw converters might have you think otherwise, the DNG format isn't just for Adobe. Photographer and Aperture user Micah Walters has written a DNG Export Plugin for Aperture which uses the Adobe DNG Converter's command line interface to generate DNGs from raw file.

Right now it creates a bare DNG with no metadata, but wouldn't it be neat if you could write metadata into the freshly-minted DNG? For example, one might create an xmp sidecar file from Aperture metadata, use it to get the metadata into the DNG, and kill it afterwards. I'm just thinking of IPTC initially, but one might even pass Aperture's adjustment details to the DNG. On the one hand this might allow you to create the DNG with approximate ACR adjustments, starting with some of the most important like exposure or white balance. But it might also be a way of backing up Aperture adjustments into the DNG, and a subsequent plugin might read those adjustments back into the library. The DNG would carry adjustment values for both Lightroom and Aperture. You may say I'm a dreamer….

John Nack reports a comparison of Lightroom and Aperture pro market shares:

InfoTrends recently surveyed 1,026 professional photographers in North America to determine which software they used for raw file processing. Here's what folks reported:

• 66.5% using the Photoshop Camera Raw plug-in
• 23.6% using Lightroom
• 5.5% using Aperture

To be fair to Aperture, it might be helpful to remove Windows users from the equation for a moment. Even after doing so, Lightroom's usage among Mac-based pros is still nearly double that of Aperture (26.6% vs. 14.3%).

Hold on, to be fair - include Windows users. To skew the story, then exclude them.

In my former life I did a lot of data migration, moving finance and other business information between old and new systems. Moving metadata between cataloguing systems isn't very much different and I've moved quite a number of photographers' metadata into iView (now Expression Media) using an Excel application as a bridge (from there it's an easy enough jump to Lightroom or Aperture).

I've now packaged this Excel spreadsheet up as a more generic tool that clients can use to run their own data migration. While it is for Windows users, there's an alternative route for Mac users. For more info, read this pdf file. According to the client who used the initial version to update 50000 items, “does the job beautifully“.

Until a year a two ago, mySQL seemed to be the open source database to back, but barely a week goes by without SQLite popping up behind the scenes of some program I use - it powers both Aperture and Lightroom. Today's Guardian carries an interview with SQLite's creator:

It's very clear to me that if I'd had any business sense whatsoever I could have made a lot of money, but I don't. I like to tell people that we make enough to live fine in Charlotte, North Carolina. We don't make nearly enough money to live in London or San Francisco, but we don't live there so that's OK.

PHPture is an interesting plug in for Aperture which, if I understand correctly, uses a web server on your Mac and the PHP language to run a web site which looks like Aperture. It accesses Aperture's database and shows stacks, thumbnails, and a project listing, and makes your image versions available over a network or via the web.

Don't worry if you don't have Aperture or a Mac - once things settle down, it won't be long before such a thing is practical for Lightroom. It's possible already.

Via Micah Walter's Aperture Plugged In.

I don't run Aperture seriously, more out of curiosity, but it's frustrating that metadata is stuck inside its library until you export the masters, duplicating your files. That is especially annoying because I often go through new pictures on the laptop while watching TV, like United's annihilation of Roma last night, and the same master files are often catalogued by Lightroom and iView. Why shouldn't I be able to add keywords or captions to files in Aperture, then read that metadata elsewhere?

A partial solution is Lightbox XMP, an Aperture export plug in that lets you export sidecar files. Currently you have to specify a single export folder - it really needs an option that outputs the sidecars in the same folders as the originals. And it should merge with existing sidecars. Rather like Lightroom does….

Via

Not sure when he wrote this, other than in the last 6 months, but Robert Edwards looks at Aperture or Lightroom = Neither:

I'm not suggesting there isn't a use or market for Aperture and Lightroom. Certainly their sales figures suggest otherwise. What I am stating is neither Aperture or Lightroom is the panacea digital photographers want them to be. At present no single application is going to successfully do it all for you. There is no Swiss Army Knife software for photographers. By the way have you ever tried using a Swiss Army Knife in preference of a real tool?

A suppository from Down Under?

In this post I wrote about using Lightroom's Collections to record virtual copies or versions. My thinking has changed a little since I wrote that piece, though I do still use Collections, and the key point remains that you must mark the virtual copies immediately after they are created and while LR still has them selected. Otherwise you've a big problem if you're a heavy shooter with versioning requirements.

Where you mark the virtual copies is more open to personal choice, and there's a problem if you use more than one computer - Collections aren't included in the XMP data, so you can't easily move your work across. Then again, neither is any data relating to Virtual Copies. But portability aside, Collections do have organisational benefits in that you can for example set up a Collection for a wedding shoot and then gather sub Collections for its b&w or other aspects of the job, and connect them to slideshow or web presets (Collections remember the last presets applied to them).

You could use keywords, and it's great that LR lets you keep them private. But I'm not keen in principle on abusing keywords and the downside of being private is that they aren't in the XMP for a multi computer setup (and in any case there is that stumbling block of Virtual Copies being stranded). Someone on Adobe's forum suggested using the Instructions field, and that's pretty well what I'm now doing - as well as using Collections. So I put in the Instructions something like “VC black and whites” - I can then use Find to target the field.

These are inconvenient workarounds however, and this area needs development work, so I hope Adobe will make things easier very soon. If you forget to mark the virtual copies, your only hope is if you can still sort by Edit Time - otherwise you're into manually selecting those 20 or 200 virtuals - you've about as much hope as my 8 or 9 year old classmates who chose City when some of us chose United.

Aperture manages versions better, adding a suffix to the virtual copy's name and letting you select them via “smart albums”. These let you define collections or “albums” to target the version name - eg “shots on 4/3/07 with version name including b&w”. Lightroom urgently needs smart collections, and version names that can be targeted. Here the Aperture smart album is looking for images shot over a certain date range, when I was in Rome, including the word “marmi” as in Stadio dei Marmi, and looking for the “B&W” version. Ignore Stack Groupings mkes the search look inside any stacks.

Aperture Professional is a new site dedicated to Apple Aperture with blog posts, longer articles, and forums. The underlying concept is quite interesting:

The Aperture Users Professional Network is an international user association for photographers, designers and other creatives who use Apple hardware and software to develop their artistic, commercial and editorial projects. As the name implies, we're specifically advocates of Apple's groundbreaking program, Aperture, a software tool designed to manage and simplify all phases of a photographer's workflow.

Of course, as long time Mac users, we're evangelists of the whole platform and any tool used to help facilitate the development of a user's creative projects. You'll find us working with any number of programs from Apple's Final Cut Studio to Adobe Photoshop to Microsoft Word, and every piece of shareware, commercial software and Widget in between.

The Aperture Users Professional Network provides education, training and support for Mac users working with (and around) Aperture, and we offer our services to the general public. We work with Apple in order to ensure we have the latest, most accurate information and to ensure that Apple has the feedback they need from the professional photographic community.

Platform evangelism is a bit of a turn off - aren't most photographers fed up of being talked down to by Mac fanboys - but I'm sure it's a good thing for Aperture users to discuss the product free of the shocking N Korean style censorship at Apple's own forums. No offence intended to N Korea.

O'Reilly's Lightroom vs Aperture has some interesting articles and even more interesting threads such as this discussion of Aperture's flexibility vs Lightroom's modularity. While I prefer a palette-based interface, as in Aperture, I don't think Lightroom's modular approach slows me down, and one commenter put it really well:

I'm still confused as to why modularity is a problem. To me, Aperture seems to be great for someone with a short attention span. Let me explain…

In LR, I use the Library to determine which images from the last batch of photos I wish to keep. I keyword them, tag them and rate them. I then switch to the LR Develop mode and begin tweaking my images. I start with the highest rated and work my way down. Therefore, if some of the higher rated images capture the scene better, I don't have to mess with the lesser images which may require more correction. Seems fine to me.

Now Aperture seems strange doing the same thing. I find myself sorting my images, adding keywords and then thinking hmm… what happens if I make this adjustment, or that adjustment. Meanwhile, I've stopped sorting images and started playing with them. LR eliminates this temptation and forces me to make critical decisions where they need to be made.

Having both programs, I wouldn't underestimate the difficulty of a Lightroom user comparing it with Aperture, and an Aperture user coming at it from the other direction, but that's exactly what Michael Clark on O'Reilly's Lightroom Blog and Micah Walter at O'Reilly's Aperture Blog are attempting (though Micah does seem to be writing mainly about Aperture). Already there are plenty of moments when I've thought “oh yes it does” at some comment about a feature's absence.

Coming at it from Lightroom too, I couldn't agree more with what Michael says on stacks :

Now in terms of Stacking, I was never one who stacked selects on my light table so this function seems a little strange to me but I know of a few Aperture users who swear by it. After playing with stacking in both Lightroom and Aperture, I have to say that it is much better done in Aperture because of the visual separation between stacks and the control you have as to how the stacks are made. Lightroom has similar controls to adjust and automatically form stacks but with the images all lined up right next to each other it isn't visually easy to delineate where one stack starts and another ends. For my workflow, I don't use Stacking so I'm not too worried about it. I tend to sequester my images by their star ranking and it works fine for me.

Functionally, stacks are identical in the two programs - it's simply the way they are represented visually in Aperture that makes the difference.

He's making two points, however, and and I agree with his second one every bit as much. This is regardless of who does it better - I've never really liked stacks either. Apart from the data not being portable between DAM systems, stacks have always seemed the best way to hide and lose track of that great shot that you notice later - sometimes many years later. You don't have to use them.

Now Lightroom is released, there are lots of threads comparing it with its Mac-limited competitor Aperture. See Adobe's forum and this Apple thread where one wanker (you would have had to read the thread but Apple have now deleted it) shouts that Aperture is the one because

…most of all, it's not just the features. It's also the future. Apple are a much more innovative company. Much more! As a company, they've innovated more in the last six months than Adobe have in the last twelve years. If you take a look at Apple's other pro applications, such as Final Cut Pro, they're awesome. Adobe has nothing to touch them. And then realise that Aperture belongs to the same pro family of products.

A frail line of association indeed - makes about as much sense as 9-11 conspiracy theories, the Da Vinci Code, or anyone who thinks Chelsea would have won two titles without looted Russian oil money. It's also shortsighted - apparently Apple will shortly trump Adobe. But won't Adobe have anticipated that, and trump Apple, and won't Apple anticipate that Adobe will anticipate their anticipation, and…? Rumsfordian logic?

With Lightroom about to launch, it's no harm to see what's going on across at its Mac-limited competitor, Aperture. Automator actions and stock library plug ins give you a pretty good idea of the sort of things to expect once Lightroom gets its scripting and SDK released.

Other Aperture resources:

• Bagelturf articles “written to be useful, digestible, and memorable”
• Aperture forum
• Annoture connects iView and Aperture
• O'Reilly blog
• Apple support, resources, tips, tutorials

With days left to go before Lightroom is released, Michael Tapes has a series of free videos (with a higher res DVD also available for $10.95). I got to know Michael virtually through the beta testing program and am not at al surprised that the style is professional and common sense - and I learned a few things too.

The streaming video is very impressive - somewhere I know Michael told me what he uses - and it almost makes me want to have a go at doing something similar myself in my Northern rasp. Hey, I could even inflict some background music on the photographic world - Joy Division, anyone?

It's going to be an expensive month too - today I ordered Aperture… as well.

The other day I posted about being able to examine the Lightroom database using the open source SQLite Database Browser. Well, according to Fazal Majid, Aperture uses the same underlying database….

Find the Aperture library, right click it and select “Show package contents”. The application data seems to be a folder structure with a large number of files, most of which are actually xml. There's also a file called “Library.apdb”, and this is the Aperture database.

Two thoughts:

1) So both Lightroom and Aperture databases are open and accessible via ODBC. Now, while I don't like the look of data blobs, but what's to stop someone figuring out how to pass settings between the two?

2) How long should we make do with cataloguing databases whose underlying databases are closed?

There's a new Lightroom podcast at George Jardine's iDisk (look for “1127 Podcast - Phil Clevenger and Mark Hamburg”) which initially focusses on the programs interface design. A little on the bland side - you'd hardly base a design on anything other than a “content is king” mantra - but it's interesting enough as it develops and digresses. Since using Aperture, I like Lightroom even more and the interface is a big part of that. I'm not sure about the designer Phil Clevenger's experience, but I've always felt Lightroom's look and feel is very reminiscent of Macromedia's Dreamweaver. And having used the latter for nearly 10 years, for me that's no bad thing.

“Version 1 is about where you stake your claim, version 2 is where you get it right.”

Something I forgot was Aperture's smart albums feature where you save a search definition as an album or collection of images. Each time you select that album, the query runs again. It's not exactly an earth-shattering concept - Portfolio's smart galleries are like this - and many people will never use it. But advanced users will do so, and professional applications should make the underlying database sweat. Smart collections are sorely absent from Lightroom.

There' ll be a few of these in the remaining 8 days of my trial