Tag: x

Files Gone on Drobo FS with OS X Lion? Get ’em back!

Using DroboFS and OSX Lion only to discover that your Drobo shares have no content!? Yikes! But fret not, you merely have a small corruption problem brought on by the firmware, and in moments you can force a rebuild of that database and all your files will be back safe, happy, and sound with no data loss. Here’s how.

I recently updated my DroboFS to firmware 1.2.0 and dashboard 2.0.3 when I switched to Lion, and while my volume mounted there was no data in it although the Drobo lights showed there was capacity, as did the Drobo Dashboard, and the health reports indicated everything was just fine.

I spoke with Drobo Tech Support that indicated this was a known problem they are actively addressing as high priority; the problem is with Lion and their firmware, and we can expect an updated firmware release.

What’s curious about this is that if one uses the Finder and mounts the Drobo drive with SMB, using smb://Drobo-FS/, the files are there. However afp://Drobo-FS.local/ and cifs://Drobo-FS.local/ mount but reveal nothing.

A detailed description of the problem is at the article entitled: “Missing” Data (AFP) and/or CNID DB Errors. This article then leads to a second one, but is only for the brave.

Using Dropbear (SSH) with Drobo FS to regenerate the AppleDB (CNID DB) has detailed steps for regenerating the apple database.

Walt’s More Verbose Directions

  1. Using the Drobo Dashboard login to your Drobo as Administrator.
  2. Unmount all shares.
  3. Under All Devices / Settings / Admin you’ll want to check the Enable DroboApps setting, which will mount a volume entitled DroboApps on your system.
  4. Download a copy of DropBear from the Drobo Apps page.
  5. Unzip this .zip file, resulting in instructions and a compressed dropbear.tgz file . Move the dropbear.tgz file to the root of the DroboApps directory.
  6. Restart the DroboFS by going to Capacity and Tools in the Dashboard, and selecting the Tools drop down on the right side, and selecting Restart. Or, just power off the unit physically for 20 seconds and then turn it back on.
  7. When Drobo restarts, go to the Dashboard and select All Devices / Settings… / Network. Note the IP address given to the device somewhere.
  8. From OS X’s Terminal enter the command ssh root@theIPaddressAbove
  9. The default password is root, unless you’ve used Dropbear before and followed the instructions within it.
  10. Enter the command ls /mnt/DroboFS/Shares to view a list of shares on the drive.
  11. Tech Support promises the following will not cause any data loss, but anytime you’re doing reconstruction you should always have a backup (if you don’t, question your backup policy), and double check before hitting return. For each share of yours listed above, enter the command: rm -r /mnt/DroboFS/Shares/yourShareNameHere/.AppleDB and press return. Note the period indicating it’s a hidden directory.
  12. Exit Terminal by entering exit.
  13. Using the Drobo Dashboard unmount all your shares, which should be just the DroboApps share at this point; this is under the All Devices / Shares and you just uncheck all the boxes.
  14. Restart the Drobo again (see above if you’ve already forgot how).
  15. And just as important restart any Macs connected to the Drobo.
  16. When the Drobo comes up, start the Dashboard, and test the mounts. They should be working.

OS X: Incomplete and partial files

Grey folders inaccessible from Finder? OSStatus error -43? Incomplete or partial folders? Here’s what I think is causing them and quite possibly what you can do about them.

I recently bought a Drobo FS with a lot of storage to keep a lot of my photography and other files backed up, redundant, and available.

Even with a “small” source drive, pumping the data to the Drobo at high speed can take a while. This isn’t the fault of the Drobo, nor the network, it’s that there’s just a lot of stuff to push through the pipe.

About 25 Hours to Go

But I ran into an odd problem, and I haven’t been able to get a good answer as to what’s happening. This is a call to geeks.

The problem is when a copy operation fails.

This could be because one rebooted the Drobo during a long copy operation, one rebooted the machine during a long copy operation, or, more likely, OS X Finder just aborted for no good reason and rather than allowing one to try the operation again, resume, or skip the problem area, the whole batch stops.

TIP: It can sometimes be better to have multiple (concurrent but blocked) copy requests that were individually initiated, than one mega-copy operation. Finder seems to like smaller sized chunks, plus if something goes wrong, there’s less to deal with.

What you end up with is a situation that’s hard to describe without seeing it. It’s a destination directory that look like this:

Incomplete Files and Folders
  1. The folders appear as grey.
  2. The folders can not be selected or opened from Finder.
  3. The folders do not have the little triangle icon by them.
  4. The folders can not be renamed from Finder, but can from Terminal.
  5. The folders can be copied, but they copy as grey.
  6. The contents of the folders can be seen using Terminal.
  7. If you use Finder to copy over them, it sees the name in use and makes a similarly named folder with a number after it.
  8. The source files are not in use.

The question is, then, how to ungrey the folders and finish the copy?

So far, I haven’t found a way.

At the moment, I’m speculating if this is related to kFirstMagicBusyFiletype, kLastMagicBusyFiletype, or kMagicBusyCreationDate as shown in the Finder.h header.

Remember, if I have to delete the directory (which can take a while — if it can be done), and then re-copy everything again (which will take a long while), and still not be certain that copy will complete, it’s a huge investment that may not pay off.

Geeks, I know what you’re thinking — I thought it too:

Was the source drive clean?
Yes. I do a Disk Utility check on my source volumes before copying from them.
Was the destination drive clean?
Yes. I do a Disk Utility check on my destination volumes before copying to them. In the case of the Drobo, I had just formatted it, using the latest firmware, and its dashboard gave the all clear. I even peeked with their sshd application.
Did you check the file permissions?
They’re clean with the regular 700 permissions. Finder’s Info concurs.
# ls -ableO@dFGinpqT *
863913 drwx------ 3 501 501 - 264 Feb 13 19:09:02 2011 NormalDirectory/
863912 drwx------ 3 501 501 - 264 Feb 13 19:09:13 2011 WhyIsThisGrey/
What happens if you delete them with # rf -vrf badDir?
Sometimes that works, actually. Other times the delete command just hangs indefinitely, like the file is busy.That said, using the Path Finder shell, this is what you get if you attempt to delete a directory that’s acting up.

OSStatus Error -43

Any idea what an OSStatus error -43 is? Or why they’d be an invalid path inside the destination directory?

What about extended attributes? Type? Creator?
They’re clean, too. Verfied by Path Finder, stat, xattr, and /usr/bin/GetFileInfo.
# stat -x *
File: "NormalDirectory"
Size: 264 FileType: Directory
Mode: (0700/drwx------) Uid: ( 501/ wls) Gid: ( 501/ wls)
Device: 45,12 Inode: 863913 Links: 3
Access: Sun Feb 13 19:09:02 2011
Modify: Sun Feb 13 19:09:02 2011
Change: Sun Feb 13 19:09:02 2011
File: "WhyIsThisGrey"
Size: 264 FileType: Directory
Mode: (0700/drwx------) Uid: ( 501/ wls) Gid: ( 501/ wls)
Device: 45,12 Inode: 863912 Links: 3
Access: Sun Feb 13 19:09:13 2011
Modify: Sun Feb 13 19:09:13 2011
Change: Sun Feb 13 19:09:13 2011
# xattr -l -v -x NormalDirectory WhyIsThisGrey
(nothing returned)
# /usr/bin/GetFileInfo -a -tcdm *
# /usr/bin/GetFileInfo -a NormalDirectory
avbstclinmedz
# /usr/bin/GetFileInfo -a WhyIsThisGrey
avbstclinmedz

No Extended Attributes, according to Path Finder
What happens if you rsync?
Doing an rsync appears to work, but doing it to the “broken directory” does not fix it after it completes.
# rsync --progress -aPE source destination

Further Thoughts

I found an article that suggests there’s a lot more going on with the file system than most of us give credit for. It talks a lot about the importance of meta data.

More Metadata That I First Thought

Two blog posts, The State of Backup and Cloning Tools under Mac OS X and Extended Attributes led me to playing with the xattr and mdls commands.

xattr didn’t have much interesting.

$ xattr -l SomeGreyDir
com.apple.FinderInfo:
00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 |................|
00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000020

The mdls command presented a better clue. Check out the kMDItemFSCreationDate attribute.

$ mdls SomeGreyDir
kMDItemFSContentChangeDate = 2011-02-20 14:02:19 -0500
kMDItemFSCreationDate = 1946-02-14 03:34:56 -0500
kMDItemFSCreatorCode = ""
kMDItemFSFinderFlags = 0
kMDItemFSHasCustomIcon = 0
kMDItemFSInvisible = 0
kMDItemFSIsExtensionHidden = 0
kMDItemFSIsStationery = 0
kMDItemFSLabel = 0
kMDItemFSName = "SomeGreyDir"
kMDItemFSNodeCount = 0
kMDItemFSOwnerGroupID = 501
kMDItemFSOwnerUserID = 501
kMDItemFSSize = 0
kMDItemFSTypeCode = ""

A quick romp through my incomplete folders revealed they all had a magical creation date of 1946-02-14 03:34:56 -0500.

The solution I think I need

I’m looking for a way to locate files with the kMDItemFSCreationDate
attribute set to that magic value, and then change it to whatever is in the
kMDItemFSContentChangeDate.

My suspicion is that this will let Finder, and the Apple command line utilities,
consider the file isn’t busy anymore.

My take on the (OS X) App Store so far

The App Store … it’s Terrific, Average, and Awful all at the same time.

I’ve been asked a few times now what I think about the app store. Never before has a feature scored terrific, average, and awful all at the same time for me. Here’s the break down.

What’s Terrific
Apple has made the integration as sweet and easy as System Update, with a trivial installation process that just works universally.

But that’s not the part that has me whistling Dixie — it’s that applications no longer use serial numbers nor do they phone home, instead they use cryptography. Nothing to pollute your network traffic. No software failing validation checks on a plane unable to reach a server.

And, given that I can now run software I paid for on machines owned by me, that’s a big win.

Plus, it unifies the Apple-culture a bit more in that upgrades work the way you expect: you don’t have to always reach for your wallet or buy stupid subscriptions.

What’s Average
The user interface, while simple, still isn’t hammered out in that perfect design that ever-so-well fits the problem space that Apple’s known for. For instance, under certain circumstances, buttons don’t behave the same way potentially causing one to purchase an unexpected item.

The store is missing a CONFIRM button before purchases, a CANCEL button as it’s happening, and more importantly a RETURN UNOPENED if it gets to the doc.

Many apps that have been acquired from other means don’t show up as INSTALLED, making for a very bumpy upgrade path. Others work just fine. Hopefully that will stabilize in the future.

What’s Awful
Some apps are dumbed down in terms of functional restrictions of the features we know and love due to Apple’s Terms of Use. There’s also fears of rejections by Apple, in addition to dealing with the approval process that still doesn’t feel completely worked out or entirely equitable between similar applications or that could be considered competitive.

Browsing the App Store, one notices the gross lack of a Genius mode.

Plus, while there are supposed to be a bazillion apps out there already, only a small handful appear to be browseable. This raises concerns that a popular app may drown out a cheaper and better competitor. Hopefully there will be more ways to slice and dice the browsing experience.

However, two major complaints are still outstanding and pretty valid.

One is that most of the apps on the App Store are crap — the easy distribution mechanism has flooded the market with apps of low interest of quality. It’s like when everyone published a flashlight app for the iPhone, and iFart was the talk of the town. For the historically minded, it was like when Atari opened its console system, and while it had the most game, only a handful were worth playing. Hopefully the rating system will help this problem. In the meanwhile, each ugly app drags the reputation of the App Store down a notch; unless the ratio reverses, the App Store will take on the same appeal as a yard sale.

Two is that many of the apps are priced in ways that don’t make a lot of sense. Someone produces a fantastic game, and it’s a dollar or less. Someone else produces a horrible looking monstrosity and charges $19.95 for it. The costs don’t reflect the features or quality. And, while good in the short term, and just the short term, we may see a plunge much the way that most iPhone apps cost 99 cents.

Where’s my JDK?

Caught in the Java depreciation battle under OS 10.6 and can’t find where the Java Development Kit (JDK) now resides? Here’s a command that will tell you, allowing you to still use InteliJ and other IDEs on OS X.

There’s a lot going on with Java at the moment. Oracle acquires Sun, putting the language in squarely in hands that don’t inspire trust; the same thing happened with MySQL. Meanwhile, the Apache Software Foundation quits the Java SE/EE Executive Committee. And now Apple deprecates Java, and the reason doesn’t appear to be what you’d think.

Starting with OS X 10.6.3 update, developers got caught in the back lash. Things moved on the file system.

Upgrading IntelliJ, the IDE was asking me where my JDK was, as it certainly wasn’t where the software, or I, thought it should be.

You won’t find the location using /Applications/Utilities/Java Preferences.app.

But you can find it with some digging. Thanks to this developer release note from Apple, the following command from Terminal will spew out some XML.

$ /usr/libexec/java_home –xml

Find the dictionary section that has ‘x86_64’ in it, and you’ll find an entry that has a ‘JVMHomePath’ with something like:

/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home

You want to provide this full path to the IDE as where the JDK lives.

It’s worth pointing out that Apple recommends that if you install any 3rd party JVMs they should be installed in /Library/Java/JavaVirtualMachines or ~/Library/Java/JavaVirtualMachines.

Competitors?

An opinion piece: are Apple and Microsoft competitors? This line of reasoning says no.

Who made the spoon that you used to eat your breakfast cereal this morning?

Every once in a while I’ll hear people talking about how Microsoft and Apple are competitors. As someone who’s technology agnostic, I don’t see the world in such shades of black and white. Here’s my reasoning as an outsider comparing the two.

Apple is a hardware company, their goal is selling as many units of whatever as possible, whether it be computers or mp3 players. With a focus on design and user experience, backwards compatibility takes a back seat to form and functionality. They excel at elegance and minimalism. They aim sharply at the home user. Disposability is ok. The next big thing is the future.

Microsoft is an operating system and business application vendor, their goal is to put that software on to as many pieces of hardware as possible. They are an industry leader in office applications and software development environments. They aim sharply at business and government. Backwards compatibility is highly important. They’d run on toasters if they could.

Is there overlap between the two? Sure, some. Is it a make or break difference? Not really, much is a matter of preference and familiarity.

I twitch anytime I hear someone say OS X is more “intuitive” than Windows; the word they’re looking for is consistent. Coarse application design is more similar, meaning once you get over the learning curve, subsequent new challenges have a ring of familiarity.

But more importantly, are the two environments mutually exclusive? Not at all. That’s why platform agnostics run both systems. Some things are just easier to do in some environments than others.

The operating systems wars and the phone wars are about luring people toward purchasing something from that camp. But look closer. MS-Office runs on OS X, and quite nicely. Safari and QuickTime run on Windows, and quite nicely. Once you’ve made the purchase of their crown jewel, the fighting appears to stop.

Users, on the whole, don’t care (and often don’t understand) about the underlying system. They just have a problem they want to solve, learn how to do it once and without change, and any tool that gets the job done will do.

Just like my spoon.

Camera is in use by another application

Trying to video conference with Skype and getting a message saying the camera is in use by another application with no video feed sent out? Try this.

Blind SkypeRecently I was attempting to use Skype, but it reported that “Your camera is in use by another application” and I couldn’t get any video feed, though the camera turned on.

Then I found this post which suggested removing the file
/Library/QuickTime/CamCamX5.component from the system, although I found it’s possible just to move it out of that directory.

Restarting Skype, the video conference worked perfectly.

Yahoo concurs. Here’s more on CamCamX. And here’s another thread saying to remove it.

Operating System: OS X 10.6 (Snow Leopard)

MacHeist 3: A Look At Group Purchasing Behavior

Have MacHeist sales stagnated? He’s my take on why, and what can be done to fix it, and how it has to play out… for the better!

As a glossed over quick introduction, MacHeist is a short-run sale of software packages for the Mac that has a twist. You pay $39 for a bundle of software, and some of that software is “locked.” A portion of your purchase price goes to charity, and the more money raised for charity, the more items in the bundle that get “unlocked.” Thus the more people buy, the more you continue to get. It’s a great scheme, only it isn’t working.

MacHeist 3MacHeist, at the time of this writing, is conducting their third “heist” and after some amazing fluster of activity, new sales appear to have stagnated at an alarming rate.

Alarming to bundle purchasers, because if not enough sales happen, bundle purchasers won’t get all the amazing high-cost software at the extreme end of the bundle. What’s important about that statement is that it’s never happened before, and the problem isn’t the recession.

In informal polling, there appear to be two kinds of purchasers: early adopters and frugal purchasers.

The early adopter purchases the bundle early, knowing a good value when they see it, spurred on by the fact that there are additional incentives for doing so.

The frugal purchasers have their eye on either the final packages in the bundle, or are looking at the bundle as a whole. They don’t want to purchase the bundle until they know everything in it is unlocked.

And that’s the interesting part. If no one buys it, nothing gets unlocked. If everyone takes a risk, everyone gets handsomely rewarded, guaranteed. Thus each potential purchaser is waiting on the action of everyone else — it’s crowd mentality, only the driven behavior is idleness.

The secret ingredient is momentum. By carefully crafting a set of software incentives, under ideal circumstances the early adopter crowd overlaps with the late takers. This manifests itself as a steady stream of purchases.

It might be argued that The Directorate which runs MacHeist became victims of their own success and actually caused the problem by marketing the sale too well. Based on all the pre-sale puzzles, rumors, and incentives, there was a flurry of purchases in the early hours of the sale and projections seemed rather high.

However, one of the primary packages in the bundle required what looked like a high goal to unlock, the perception was that momentum was slowing. And perception drove reality. “Hmm, that doesn’t look like it’ll get unlocked, I think I’ll wait to see if it does before I buy,” is all it took to slow the influx of unlocking purchasers.

This was ill-timed, as it also happened to coincide with the reward for the first 25,000 buyers being removed from the table as the 25,000th bundle was sold. Days later, a only mere 5,000 more have sold and questions are being raised if the final packages will be unlocked.

The up-front fast burn created enough of a gap that people who were on the fence at different points became more segregated than usual. This didn’t happen in the last two sales.

So here’s my prediction: they have to fix this. Meaning, new incentives will re-emerge, the goals will have to be re-addressed, and it’s in the best interest of MacHeist to unlock the bundles anyhow at the end of it.

Turns out before I could finish this post, a new bonus was added, and that did stir a little traffic. But the real objective here is to convey there’s movement, specifically enough that the goal could be reached. That will inspire sales again, and in turn actually unlock the software. By re-calibrating the goal levels, this would solve the problem. In fact, the easy solution is to put all the last packages into one final, achievable goal.

The truth of the matter, however, is whatever happens will be remembered, if not chronicled in Wikipedia forever. If MacHeist goes down in flames for not unlocking all it’s bundled packages, people will be ever the more skeptical, and that means early adopters turning into late purchasers. That only exacerbates the problem, killing future sales opportunities.

By contrast, if the packages do get unlocked, whether by purchasers or by The Directorate making its own donation from the profits it receives, then MacHeist will be seen as more of a sure thing in the future, sliding more of the late comers and risk adverse customers into the early adopter side. This would actually increase future sales, because more gets unlocked sooner, enticing the skeptical buyers.

As such, “betting” on MacHeist with a purchase at this point still seems like a safe move. And, even if none of my predictions happen to come true, enough is unlocked already that the $39 price tag is still an awesome buy for the collection of software provided.