Files with Jan 24, 1984, 3:00am Modification Times

Well, I think I finally figured our a problem that’s been bugging me for quite a while.

Every once in a while, especially if I’m using an external drive that’s acting weird or even the now defunct Lima, I’d occasionally see files that appeared grey in directory listings. You couldn’t rename them, you couldn’t display their contents (regardless of having size), etc.

Doing a ls -@Oaeln filename, showed nothing special.

Except… the date, which was always Jan 24, 1984, 3:00AM.

Ok, yes, that’s the Mac’s birthday — and some even call it an Easter Egg. But it’s likely an Easter Egg with a purpose.

The date appears as kind of a place holder, such as momentarily when a file is downloaded. And during that time, the system knows the file is incomplete, and this is just as good as any other way to mark it as such.

Turns out this also appears to be the way that corrupted files appear too.

For example, I synchronized some files with my Lima (which stores it on a USB disk somewhere), and if the synchronization process fails it’s supposed to pick up and try later. However, if that doesn’t happen, then partially transmitted data can reside there. And Lima’s going out of business was what led me to go try to pull all data, complete or not.

Attempting to inspect the contents of such files, such as on a Lima mount, appears to confirm this, resulting in a message of read error.

So. Grey files. It’s not a Happy Birthday to Us from Mac, but a date long past being used as a magic number.

OS X: Incomplete and partial files

Grey folders inaccessible from Finder? OSStatus error -43? Incomplete or partial folders? Here’s what I think is causing them and quite possibly what you can do about them.

I recently bought a Drobo FS with a lot of storage to keep a lot of my photography and other files backed up, redundant, and available.

Even with a “small” source drive, pumping the data to the Drobo at high speed can take a while. This isn’t the fault of the Drobo, nor the network, it’s that there’s just a lot of stuff to push through the pipe.

About 25 Hours to Go

But I ran into an odd problem, and I haven’t been able to get a good answer as to what’s happening. This is a call to geeks.

The problem is when a copy operation fails.

This could be because one rebooted the Drobo during a long copy operation, one rebooted the machine during a long copy operation, or, more likely, OS X Finder just aborted for no good reason and rather than allowing one to try the operation again, resume, or skip the problem area, the whole batch stops.

TIP: It can sometimes be better to have multiple (concurrent but blocked) copy requests that were individually initiated, than one mega-copy operation. Finder seems to like smaller sized chunks, plus if something goes wrong, there’s less to deal with.

What you end up with is a situation that’s hard to describe without seeing it. It’s a destination directory that look like this:

Incomplete Files and Folders
  1. The folders appear as grey.
  2. The folders can not be selected or opened from Finder.
  3. The folders do not have the little triangle icon by them.
  4. The folders can not be renamed from Finder, but can from Terminal.
  5. The folders can be copied, but they copy as grey.
  6. The contents of the folders can be seen using Terminal.
  7. If you use Finder to copy over them, it sees the name in use and makes a similarly named folder with a number after it.
  8. The source files are not in use.

The question is, then, how to ungrey the folders and finish the copy?

So far, I haven’t found a way.

At the moment, I’m speculating if this is related to kFirstMagicBusyFiletype, kLastMagicBusyFiletype, or kMagicBusyCreationDate as shown in the Finder.h header.

Remember, if I have to delete the directory (which can take a while — if it can be done), and then re-copy everything again (which will take a long while), and still not be certain that copy will complete, it’s a huge investment that may not pay off.

Geeks, I know what you’re thinking — I thought it too:

Was the source drive clean?
Yes. I do a Disk Utility check on my source volumes before copying from them.
Was the destination drive clean?
Yes. I do a Disk Utility check on my destination volumes before copying to them. In the case of the Drobo, I had just formatted it, using the latest firmware, and its dashboard gave the all clear. I even peeked with their sshd application.
Did you check the file permissions?
They’re clean with the regular 700 permissions. Finder’s Info concurs.
# ls -ableO@dFGinpqT *
863913 drwx------ 3 501 501 - 264 Feb 13 19:09:02 2011 NormalDirectory/
863912 drwx------ 3 501 501 - 264 Feb 13 19:09:13 2011 WhyIsThisGrey/
What happens if you delete them with # rf -vrf badDir?
Sometimes that works, actually. Other times the delete command just hangs indefinitely, like the file is busy.That said, using the Path Finder shell, this is what you get if you attempt to delete a directory that’s acting up.

OSStatus Error -43

Any idea what an OSStatus error -43 is? Or why they’d be an invalid path inside the destination directory?

What about extended attributes? Type? Creator?
They’re clean, too. Verfied by Path Finder, stat, xattr, and /usr/bin/GetFileInfo.
# stat -x *
File: "NormalDirectory"
Size: 264 FileType: Directory
Mode: (0700/drwx------) Uid: ( 501/ wls) Gid: ( 501/ wls)
Device: 45,12 Inode: 863913 Links: 3
Access: Sun Feb 13 19:09:02 2011
Modify: Sun Feb 13 19:09:02 2011
Change: Sun Feb 13 19:09:02 2011
File: "WhyIsThisGrey"
Size: 264 FileType: Directory
Mode: (0700/drwx------) Uid: ( 501/ wls) Gid: ( 501/ wls)
Device: 45,12 Inode: 863912 Links: 3
Access: Sun Feb 13 19:09:13 2011
Modify: Sun Feb 13 19:09:13 2011
Change: Sun Feb 13 19:09:13 2011
# xattr -l -v -x NormalDirectory WhyIsThisGrey
(nothing returned)
# /usr/bin/GetFileInfo -a -tcdm *
# /usr/bin/GetFileInfo -a NormalDirectory
avbstclinmedz
# /usr/bin/GetFileInfo -a WhyIsThisGrey
avbstclinmedz

No Extended Attributes, according to Path Finder
What happens if you rsync?
Doing an rsync appears to work, but doing it to the “broken directory” does not fix it after it completes.
# rsync --progress -aPE source destination

Further Thoughts

I found an article that suggests there’s a lot more going on with the file system than most of us give credit for. It talks a lot about the importance of meta data.

More Metadata That I First Thought

Two blog posts, The State of Backup and Cloning Tools under Mac OS X and Extended Attributes led me to playing with the xattr and mdls commands.

xattr didn’t have much interesting.

$ xattr -l SomeGreyDir
com.apple.FinderInfo:
00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 |................|
00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000020

The mdls command presented a better clue. Check out the kMDItemFSCreationDate attribute.

$ mdls SomeGreyDir
kMDItemFSContentChangeDate = 2011-02-20 14:02:19 -0500
kMDItemFSCreationDate = 1946-02-14 03:34:56 -0500
kMDItemFSCreatorCode = ""
kMDItemFSFinderFlags = 0
kMDItemFSHasCustomIcon = 0
kMDItemFSInvisible = 0
kMDItemFSIsExtensionHidden = 0
kMDItemFSIsStationery = 0
kMDItemFSLabel = 0
kMDItemFSName = "SomeGreyDir"
kMDItemFSNodeCount = 0
kMDItemFSOwnerGroupID = 501
kMDItemFSOwnerUserID = 501
kMDItemFSSize = 0
kMDItemFSTypeCode = ""

A quick romp through my incomplete folders revealed they all had a magical creation date of 1946-02-14 03:34:56 -0500.

The solution I think I need

I’m looking for a way to locate files with the kMDItemFSCreationDate
attribute set to that magic value, and then change it to whatever is in the
kMDItemFSContentChangeDate.

My suspicion is that this will let Finder, and the Apple command line utilities,
consider the file isn’t busy anymore.

Invisible Drive on OS X

The hard drive icon on my OS X’s Finder was no longer appearing on my desktop; here’s how I fixed it.

I happened to sign on to my desktop Mac and noticed something very strange, the Harddisk icon was no longer on the desktop.

Other clever tricks for looking at the file systems showed the file was most certainly present, although Finder operations were treating the volume as if the hard disk was hidden or invisible. The drive was there when I used terminal and did $ ls -l /Volumes

Finder Preferences showed that icons should be shown, but just the drive icon wasn’t appearing.

Then I found this aritcle, which suggested firing up the Script Editor and running this script:

tell application “System Events”
set visible of disk “NameofDisk” to true
end tell
tell application “Finder” to quit
delay 1
tell application “Finder” to launch

I believe I got myself into trouble by accident when I did the last disk repair using Disk Warrior or Disk Utility. Somehow the operation marked the drive as invisible. Undoing it was as simple as asking the system to make it visible again.

OS X: Batch Rename from GUI

This is the third time I’ve had to go hunting how to enable batch filename manipulations for OS X. Now I’m documenting it so I don’t have to hunt this down again.

I keep forgetting about this trick, so I thought I’d post it in the event I have to ever do it again.

Part of the problem with a graphical GUI is that it’s very difficult to rename files in batches, for instance, prepending some text to a group of files.  This kind of thing is fairly trivial at the command line.

Apple has a facility to do this, but as it’s not something a regular user does often, it’s not enabled by default.  Here’s how to get all kinds of additional functionality out of OS X.

1) Open /Applications/AppleScript

2) Turn on Show Script Menu in menu bar

3) Optional: turn on GUI scripting, show library scripts, and choose where to show them.

You’ll notice up near the time in the menu bar a black scroll has appeared.

All the batch renaming and filename twiddling stuff is under the Finder Scripts.

UPDATE 19-Dec-2009: Upgrading to Snow Leopards deletes some useful scripts, specifically the Finder Scripts.

UPDATE 31-Aug-2010: The scripts live in “/Library/Scripts/Finger Scripts” and are

  • Add to File Names.scpt
    md5 4b0cd899acb19b5fc62ef2049d81a933 – 18114 bytes
  • Change Case of Item Names.scpt
    md5 af7429228be4d0e1a096092af5341c52 – 17808 bytes
  • Replace Text in Item Names.scpt
    md5 716493cab1c569953a7f40d76ed9a1f7 – 24328 bytes