Tag: Paritial

OS X: Incomplete and partial files

Grey folders inaccessible from Finder? OSStatus error -43? Incomplete or partial folders? Here’s what I think is causing them and quite possibly what you can do about them.

I recently bought a Drobo FS with a lot of storage to keep a lot of my photography and other files backed up, redundant, and available.

Even with a “small” source drive, pumping the data to the Drobo at high speed can take a while. This isn’t the fault of the Drobo, nor the network, it’s that there’s just a lot of stuff to push through the pipe.

About 25 Hours to Go

But I ran into an odd problem, and I haven’t been able to get a good answer as to what’s happening. This is a call to geeks.

The problem is when a copy operation fails.

This could be because one rebooted the Drobo during a long copy operation, one rebooted the machine during a long copy operation, or, more likely, OS X Finder just aborted for no good reason and rather than allowing one to try the operation again, resume, or skip the problem area, the whole batch stops.

TIP: It can sometimes be better to have multiple (concurrent but blocked) copy requests that were individually initiated, than one mega-copy operation. Finder seems to like smaller sized chunks, plus if something goes wrong, there’s less to deal with.

What you end up with is a situation that’s hard to describe without seeing it. It’s a destination directory that look like this:

Incomplete Files and Folders
  1. The folders appear as grey.
  2. The folders can not be selected or opened from Finder.
  3. The folders do not have the little triangle icon by them.
  4. The folders can not be renamed from Finder, but can from Terminal.
  5. The folders can be copied, but they copy as grey.
  6. The contents of the folders can be seen using Terminal.
  7. If you use Finder to copy over them, it sees the name in use and makes a similarly named folder with a number after it.
  8. The source files are not in use.

The question is, then, how to ungrey the folders and finish the copy?

So far, I haven’t found a way.

At the moment, I’m speculating if this is related to kFirstMagicBusyFiletype, kLastMagicBusyFiletype, or kMagicBusyCreationDate as shown in the Finder.h header.

Remember, if I have to delete the directory (which can take a while — if it can be done), and then re-copy everything again (which will take a long while), and still not be certain that copy will complete, it’s a huge investment that may not pay off.

Geeks, I know what you’re thinking — I thought it too:

Was the source drive clean?
Yes. I do a Disk Utility check on my source volumes before copying from them.
Was the destination drive clean?
Yes. I do a Disk Utility check on my destination volumes before copying to them. In the case of the Drobo, I had just formatted it, using the latest firmware, and its dashboard gave the all clear. I even peeked with their sshd application.
Did you check the file permissions?
They’re clean with the regular 700 permissions. Finder’s Info concurs.
# ls -ableO@dFGinpqT *
863913 drwx------ 3 501 501 - 264 Feb 13 19:09:02 2011 NormalDirectory/
863912 drwx------ 3 501 501 - 264 Feb 13 19:09:13 2011 WhyIsThisGrey/
What happens if you delete them with # rf -vrf badDir?
Sometimes that works, actually. Other times the delete command just hangs indefinitely, like the file is busy.That said, using the Path Finder shell, this is what you get if you attempt to delete a directory that’s acting up.

OSStatus Error -43

Any idea what an OSStatus error -43 is? Or why they’d be an invalid path inside the destination directory?

What about extended attributes? Type? Creator?
They’re clean, too. Verfied by Path Finder, stat, xattr, and /usr/bin/GetFileInfo.
# stat -x *
File: "NormalDirectory"
Size: 264 FileType: Directory
Mode: (0700/drwx------) Uid: ( 501/ wls) Gid: ( 501/ wls)
Device: 45,12 Inode: 863913 Links: 3
Access: Sun Feb 13 19:09:02 2011
Modify: Sun Feb 13 19:09:02 2011
Change: Sun Feb 13 19:09:02 2011
File: "WhyIsThisGrey"
Size: 264 FileType: Directory
Mode: (0700/drwx------) Uid: ( 501/ wls) Gid: ( 501/ wls)
Device: 45,12 Inode: 863912 Links: 3
Access: Sun Feb 13 19:09:13 2011
Modify: Sun Feb 13 19:09:13 2011
Change: Sun Feb 13 19:09:13 2011
# xattr -l -v -x NormalDirectory WhyIsThisGrey
(nothing returned)
# /usr/bin/GetFileInfo -a -tcdm *
# /usr/bin/GetFileInfo -a NormalDirectory
avbstclinmedz
# /usr/bin/GetFileInfo -a WhyIsThisGrey
avbstclinmedz

No Extended Attributes, according to Path Finder
What happens if you rsync?
Doing an rsync appears to work, but doing it to the “broken directory” does not fix it after it completes.
# rsync --progress -aPE source destination

Further Thoughts

I found an article that suggests there’s a lot more going on with the file system than most of us give credit for. It talks a lot about the importance of meta data.

More Metadata That I First Thought

Two blog posts, The State of Backup and Cloning Tools under Mac OS X and Extended Attributes led me to playing with the xattr and mdls commands.

xattr didn’t have much interesting.

$ xattr -l SomeGreyDir
com.apple.FinderInfo:
00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 |................|
00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000020

The mdls command presented a better clue. Check out the kMDItemFSCreationDate attribute.

$ mdls SomeGreyDir
kMDItemFSContentChangeDate = 2011-02-20 14:02:19 -0500
kMDItemFSCreationDate = 1946-02-14 03:34:56 -0500
kMDItemFSCreatorCode = ""
kMDItemFSFinderFlags = 0
kMDItemFSHasCustomIcon = 0
kMDItemFSInvisible = 0
kMDItemFSIsExtensionHidden = 0
kMDItemFSIsStationery = 0
kMDItemFSLabel = 0
kMDItemFSName = "SomeGreyDir"
kMDItemFSNodeCount = 0
kMDItemFSOwnerGroupID = 501
kMDItemFSOwnerUserID = 501
kMDItemFSSize = 0
kMDItemFSTypeCode = ""

A quick romp through my incomplete folders revealed they all had a magical creation date of 1946-02-14 03:34:56 -0500.

The solution I think I need

I’m looking for a way to locate files with the kMDItemFSCreationDate
attribute set to that magic value, and then change it to whatever is in the
kMDItemFSContentChangeDate.

My suspicion is that this will let Finder, and the Apple command line utilities,
consider the file isn’t busy anymore.