As I mentioned last week, something unspeakable happened to my external hard drive, resulting in pretty much total loss of data. As it happens, all my mp3s were on the drive, but I figured that at least wasn’t a problem because I could just pull them off my iPod. Which is true - but not nearly as painless as we thought it would be. So I’m going to detail some of the issues Chris and I encountered on our quest to restore the contents of my music library to their original locations from my iPod, in case you ever find yourself in the same situation.
Because Apple doesn’t want you to fully grasp the idea that your iPod is just an external hard drive with all of your music on it, audio files are stored in a hidden directory called iPod_Control - you won’t be able to see this from the Finder, even if you have the iPod’s Firewire disk mode turned on. You can, however, see it if you view the directory from the command line.
Our first task was to get the files themselves off the iPod and onto my newly reformatted hard drive, so we figured we’d just use cp to copy everything over normally. My local directory structure was not preserved on the iPod - everything gets moved to directories like F00, F01, F02, and so on - but we thought it would be simple enough to move things to their correct locations with the help of my iTunes library XML file later on. So we started copying the contents of each directory to one big directory on my external hard drive. (Note that you can’t just copy the Music directory in iPod_Control itself - the newly copied directory will also be hidden from Finder viewing, even if you delete .DS_Store.)
The first problem we encountered was that cp was, bafflingly, not copying all the files in each directory. Say F00 had 175 files in it when we listed its contents on the iPod - it would only have something like 99 when the copy to the hard drive was finished. This wasn’t a syntax problem, it wasn’t the result of not all the files being mp3s, it wasn’t a case of cp not wanting to overwrite existing files. We had and have no idea why it refused to copy something like a third of a given directory’s files.
So we decided to reveal the files in the Finder and see if good old fashioned dragging and dropping would work any better. This is easy enough to do - just install Tinker Tool and tell it to show hidden files. And as it turns out, drag-and-drop copying did get all the files that cp missed. (We don’t know why.) At that point we at least had all the data from the iPod on the external hard drive, which was a step in the right direction, but everything was still in numbered directories instead of in its correct former location.
The reason we wanted to get everything to its former location was to trick iTunes into thinking nothing had changed - I could have just dumped it all in one big directory and recreated my library, but I would have lost all my playlists and play data, which I was hoping to avoid. So to accomplish moving the files, Chris began working on a Perl script that iterated over each file in each F directory, looked up its old location in iTunes Music Library.XML, and moved it to that location, creating subdirectories as needed. In the process of doing this, he noticed that there were something like two thousand files listed in the XML file that could not be found in any of the F directories, which seemed very strange to both of us.
It turns out (and we probably should have noticed earlier) that when files are transferred to the iPod, their filenames are truncated. This means that for some of my files (which follow a band_name-song_title.mp3 naming convention), longer band names often resulted in truncation. For some of Chris’s files (which follow a band_name-album_name-song_title.mp3 naming convention), there would be five or six files with almost identical names, except that numbers had been added to the end to distinguish them on the iPod. This created a bit of a speedbump for Chris’s script, which now had to have another way to identify individual files and find their locations. We settled on size.
That was all peachy, and most of my files were happily renamed and relocated. But a few hundred were still being left behind, and this turned out to be because the size in bytes listed in the XML file did not match the size in bytes of the file on disk. Wha? Yeah, we don’t know, either. So we’re still working on getting those stray files sorted to their correct locations, and all of this has turned out to be way more of a pain in the ass than we ever thought it would be.
The moral of the story. If you ever have to rescue your music files from an iPod, don’t use command line cp. Remember that your directory structure will not be preserved, and that file names will be truncated. And bizarrely, the iTunes XML file may not be a good guide as to the size of a particular file. That’s all for now.