Workaround – Walt-O-Matic

Case-Sensitive GNU sort

GNU sort appears to be producing case-insenstive output all the time, no matter what you use for the case-insenstive switch. Turns out there’s a little more going on behind the scenes you need to know.

Had something unexpected happen while using sort the other day: it behaved differently on two different systems.

In a nutshell, I went to go sort data that looked like this:
xyz CAT abc

The correct answer, due to case-sensitifivity, should have been:
CAT abc xyz

This, by the way is what Cygwin happens to produce.

Instead I was getting the case-insensitve answer:
abc CAT xyz

And while the sort program had a case-insenstivive switch, -f, it always seemed to be applied.

I checked for a command alias. None.

I tried a negation value, -f-, and that did nothing for me.

Eventually I figured out that the issue was with the locale specified in the environment variables.

If LC_ALL=en_US, then I got a case-insentive order, no matter what. The solution was to change it to LC_ALL=C to get the case-sensitive version.

As GNU allows explicitly for case-insensitivity, I really wish they’d also explicitly allow for case-sensitivy.

This seems to trip up a lot of people, judging by the number of posts out there about it.

Installing Hadoop on OS X

Fix the “Unable to load realm info from SCDynamicStore” error on OS Lion when using Hadoop.

Using Homebrew to install Hadoop on Lion OS X 10.7, I ran into a snag.

Attempting to do:
$ hadoop namenode -format

Resulted in this error: Unable to load realm info from SCDynamicStore

The solution was in Brandon Werner’s blog on How to Set up Haddop on OS X Lion.

As he puts it, to fix this issue, simply add the following single line to your hadoop-env.sh file:

export HADOOP_OPTS="-Djava.security.krb5.realm=OX.AC.UK -Djava.security.krb5.kdc=kdc0.ox.ac.uk:kdc1.ox.ac.uk"

Other Tips

The use of $HADOOP_HOME is now deprecated.
Homebrew uses /usr/local/bin with symbolic links to its /usr/local/Celler, so Hadoop just works from the command line. No need to add it to your path.
The Hadoop config files reside in /usr/local/Cellar/hadoop/*/libexec/conf

Files Gone on Drobo FS with OS X Lion? Get ’em back!

Using DroboFS and OSX Lion only to discover that your Drobo shares have no content!? Yikes! But fret not, you merely have a small corruption problem brought on by the firmware, and in moments you can force a rebuild of that database and all your files will be back safe, happy, and sound with no data loss. Here’s how.

I recently updated my DroboFS to firmware 1.2.0 and dashboard 2.0.3 when I switched to Lion, and while my volume mounted there was no data in it although the Drobo lights showed there was capacity, as did the Drobo Dashboard, and the health reports indicated everything was just fine.

I spoke with Drobo Tech Support that indicated this was a known problem they are actively addressing as high priority; the problem is with Lion and their firmware, and we can expect an updated firmware release.

What’s curious about this is that if one uses the Finder and mounts the Drobo drive with SMB, using smb://Drobo-FS/, the files are there. However afp://Drobo-FS.local/ and cifs://Drobo-FS.local/ mount but reveal nothing.

A detailed description of the problem is at the article entitled: “Missing” Data (AFP) and/or CNID DB Errors. This article then leads to a second one, but is only for the brave.

Using Dropbear (SSH) with Drobo FS to regenerate the AppleDB (CNID DB) has detailed steps for regenerating the apple database.

Walt’s More Verbose Directions

Using the Drobo Dashboard login to your Drobo as Administrator.
Unmount all shares.
Under All Devices / Settings / Admin you’ll want to check the Enable DroboApps setting, which will mount a volume entitled DroboApps on your system.
Download a copy of DropBear from the Drobo Apps page.
Unzip this .zip file, resulting in instructions and a compressed dropbear.tgz file . Move the dropbear.tgz file to the root of the DroboApps directory.
Restart the DroboFS by going to Capacity and Tools in the Dashboard, and selecting the Tools drop down on the right side, and selecting Restart. Or, just power off the unit physically for 20 seconds and then turn it back on.
When Drobo restarts, go to the Dashboard and select All Devices / Settings… / Network. Note the IP address given to the device somewhere.
From OS X’s Terminal enter the command ssh root@theIPaddressAbove
The default password is root, unless you’ve used Dropbear before and followed the instructions within it.
Enter the command ls /mnt/DroboFS/Shares to view a list of shares on the drive.
Tech Support promises the following will not cause any data loss, but anytime you’re doing reconstruction you should always have a backup (if you don’t, question your backup policy), and double check before hitting return. For each share of yours listed above, enter the command: rm -r /mnt/DroboFS/Shares/yourShareNameHere/.AppleDB and press return. Note the period indicating it’s a hidden directory.
Exit Terminal by entering exit.
Using the Drobo Dashboard unmount all your shares, which should be just the DroboApps share at this point; this is under the All Devices / Shares and you just uncheck all the boxes.
Restart the Drobo again (see above if you’ve already forgot how).
And just as important restart any Macs connected to the Drobo.
When the Drobo comes up, start the Dashboard, and test the mounts. They should be working.

Where’s my JDK?

Caught in the Java depreciation battle under OS 10.6 and can’t find where the Java Development Kit (JDK) now resides? Here’s a command that will tell you, allowing you to still use InteliJ and other IDEs on OS X.

There’s a lot going on with Java at the moment. Oracle acquires Sun, putting the language in squarely in hands that don’t inspire trust; the same thing happened with MySQL. Meanwhile, the Apache Software Foundation quits the Java SE/EE Executive Committee. And now Apple deprecates Java, and the reason doesn’t appear to be what you’d think.

Starting with OS X 10.6.3 update, developers got caught in the back lash. Things moved on the file system.

Upgrading IntelliJ, the IDE was asking me where my JDK was, as it certainly wasn’t where the software, or I, thought it should be.

You won’t find the location using /Applications/Utilities/Java Preferences.app.

But you can find it with some digging. Thanks to this developer release note from Apple, the following command from Terminal will spew out some XML.

$ /usr/libexec/java_home –xml

Find the dictionary section that has ‘x86_64’ in it, and you’ll find an entry that has a ‘JVMHomePath’ with something like:

/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home

You want to provide this full path to the IDE as where the JDK lives.

It’s worth pointing out that Apple recommends that if you install any 3rd party JVMs they should be installed in /Library/Java/JavaVirtualMachines or ~/Library/Java/JavaVirtualMachines.

Updates, please don’t do that…

Using a MySQL DOT-Net Conncetor and getting these error messages? Shared Memory Provider: No Process is on the other end of the pipe. Error 0. Unable to find the request .NET Framework Data Provider. It may not be installed. Or your login not working, but your password is right? Try this solution.

Using Visual Studio 2010, it’s possible to use MySQL databases. The trick is to use the MySQL DotNET Connector.

Everything was going swimmingly until for what appeared to be no reason, until one day I went to work on my project and I started getting a strange error message for what appeared to be no reason.

Shared Memory Provider: No Process is on the other end of the pipe.
Error 0.
Unable to find the request .NET Framework Data Provider. It may not be installed.

Exploring the exception shows that a connection to the database is made, but the login has failed for the username. However, the username and password work when logging in using the MySQL client. No drivers have changed, and MySQL works with all other applications just fine, just not the Microsoft development IDE. The connection string validated in a test environment, as did a raw connection with the MySqlConnection. EntityFrameworks, not so much.

Attempting to use EntityFrameworks from VS2010 results in an error dialog box that when dismissed goes back to the database wizard that brings up the dialog box again; the only way out is using Task Manager and violently killing the IDE’s process.

Attempting to use the .MySQL .NET/Connector installer to repair the installation doesn’t help, either. Yes, it updates the .dll files, yes it tweaks the registry, but no, it doesn’t help.

Making a new project and creating a new ADO.NET Data Model reveals something interesting: the MySQL providers are not listed anymore in VS2010.

The solution is to UNINSTALL the DotConnector and then re-INSTALL the same MySQL .NET/Connector.

The full installer indicates that it updates the VS2008 and VS2010 development environments. Trying the IDE again works perfectly. The ADO.NET Data Model shows the MySQL Data Provider, the Entity Frameworks doesn’t get stuck in an endless loop, and my code worked again without change.

What happened to cause all this? Turns out I had installed a minor VS2010 hotfix service pack at the end of the weekend. The result? The update silently uninstalled all non-Microsoft data providers.

Updates, please don’t do that…

Clearly, I’m not happy about that. Especially since it did it silently. If it told me that it was going to, or if it asked me to reinstall after the fact, I could live with it.

Uninstalling Intego Software

I’d rather eat an orange then brush my teeth with peppermint toothpaste than deal with cleaning up my system after using Intego’s software. If anything can bring the Windows reboot experience, coupled with the leaving of software cruft, to the Mac platform, this software does it in my opinion. Here’s how I finally got rid of it all. I hope.

I recently purchased a Mac bundle with software and it included software from Intego, consisting of the Personal Antispam and Personal Backup applications. I installed them, and from that point forward it was an experience I’ve regretted and have been trying to undo. Only now do I think I’ve made some progress toward that goal.

Frankly, I didn’t get what the backup software did for me over many of the free solutions out there, and while the personal antispam look intriguing, it was intrusive as well and I decided to fall back to Apple’s spam filter included in Mail.

Even if a product doesn’t make it into my main line of recommendations, I often will keep it around in the event I suddenly have use for it. This, for example, is how TypeIt4Me eventually won me over.

Intego went out of their way to annoy me straight from the start. How so? Every time I went to install a package from them, they felt the need to do what appeared to be a gratuitous reboot. It was like being on frickin’ Windows. And they had to install their own update manager, which had to take a glory spot in the menu bar. And it had to do updates, which required even more reboots. I was done with them at that point, but don’t even get me started on the subscription scheme that rode on top of the atrocity.

So I wrote to them asking them how to uninstall their software. Here’s the reply I got:

Proper removal of the software package requires using the Installer package located in your software bundle or disc. If you have manually attempted to remove the software, you will need to first, reinstall the software again, then use the same Installer package to properly remove the applications.

If you need to, you can re-download the installer for Internet Security Barrier X6 using the link below:

http://www.integodownload.com/en/isbx6.html

Open the installer and select to uninstall all software. Restart your computer.

Great, another reboot. Lucky for me, I hadn’t tried to go off on my own path, plus I had the original installation utility. I tried it, and it appeared to work.

Notice I said appeared?

One week later, LittleSnitch pops up and reports my system is spontaneously trying to access Intego’s update service for the very set of applications that, for all evidence I could tell, I removed and forgot about. Apparently, no so.

LittleSnitch also reveals it’s TaskManagerDaemon who’s trying to deal with Intego’s NetUpdate buried in /Library/Intego. Thank you LittleSnitch, curse you Intego.

Intego leaves cruft. Running cruft. Seems this isn’t new of them, according to Apple archives.

Part of the Mac culture is being a good citizen. In my opinion, I feel they aren’t.

After uninstalling the software in exactly the manner they prescribe, enter this this command at your terminal:
sudo find / -name Intego -print

I suspect you’ll develop a similar facial tick as it starts returning output after scanning your disk.

Go grab a root shell, you’re gonna wanna also wipe out:

/Library/Intego and everything below it.
/Library/Application Support/Intego and everything below it.
/Library/Preferences/Intego and everything below it.
/Users/wls/Library/Application Support/Intego and everything below it.

Oh, and you’ll want to Reboot as well.

…it’s not like I had other applications up or was doing anything important.

After the reboot, you’ll notice tons of console messages from launchd. Now you need to do this.
$ launchctl
launchd% remove com.intego.task.manager.notifier
launchd% remove com.intego.netupdate.agent
launchd% exit

And, you’ll need to remove some .plist files:
$ sudo rm -v /Library/LaunchAgents/*intego*
$ sudo rm -v /Library/LaunchDaemons/*intego*

And preferences, frameworks, keychains, and widgets:
$ sudo rm -vrf /Library/PreferencePanes/NetUpdate.prefPane
$ sudo rm -vrf /Library/Frameworks/IntegoiCalFramework.framework/
$ sudo rm -v /Library/Keychains/Intego.keychain
$ sudo rm -vrf /Library/Widgets/Intego\ Status.wdgt/

Reboot again.

UPDATE (12-Dec-2010): I’ve been in contact with Intego Support, support@intego.com, and they were kind enough to provide this extra information:

If there is anything left on your computer, you can remove it manually.

Can you please go into the following areas on the computer and delete any traces of Intego or VirusBarrier:

/Macintosh HD/Library/Intego
/Macintosh HD/Library/LaunchDaemons
/Macintosh HD/Library/LaunchAgents
/Macintosh HD/Applications
/Macintosh HD/Library/Preferences
/Macintosh HD/Library/Logs
/Macintosh HD/Library/Receipts
/Macintosh HD/Library/Startupitems
/Macintosh HD/Library/Widgets

Home Folder:

~/Library/Application Support
~/Library/Preferences

They were right, there’s logs, too.
$ sudo rm -rf /Library/Logs/NetUpdate/

Review: Walt gives Intego software installation TWO thumbs down. The reasons are obvious.

Camera is in use by another application

Trying to video conference with Skype and getting a message saying the camera is in use by another application with no video feed sent out? Try this.

Recently I was attempting to use Skype, but it reported that “Your camera is in use by another application” and I couldn’t get any video feed, though the camera turned on.

Then I found this post which suggested removing the file
/Library/QuickTime/CamCamX5.component from the system, although I found it’s possible just to move it out of that directory.

Restarting Skype, the video conference worked perfectly.

Yahoo concurs. Here’s more on CamCamX. And here’s another thread saying to remove it.

Operating System: OS X 10.6 (Snow Leopard)

MySQL Cascade Delete Problem

MySQL 5.1.51 introduced a nasty little bug that has the potential to really cause some production servers some ill — cascading delete and updates can sometimes fail if you’re dealing with “too much” data at once. Where “too much” is a relatively small amount. You may get bit if you have a one-to-many relationship.

A note of warning to MySQL users using 5.1.51, you may want to downgrade to 5.1.50 for a little bit.

There’s a problem with 5.1.51 in which cascading deletes or updates throw an error. Not good if you have constraints and one to many relationships.

ERROR 1030 (HY000): Got error -1 from storage engine

The mysql error log will say something like:

InnoDB: Cannot delete/update rows with cascading foreign key constraints that exceed max depth of 250

This is a confirmed bug in MySQL and is repeatable. The error will cause a transaction rollback. Not good news for people running production systems.

See MySQL Bug #357255.

While the problem was quickly identified and apparently resolved, as the defect report is closed, it does not look like the 06-Oct-2010 change has made it out to the production baseline as of the time of this writing.

Additionally, the MySQL pre-release snapshots on labs.mysql.com show there is a mysql-5.1.52 pending with a September date, and this hasn’t made it to general production yet.

That leaves one to speculate that the fix will appear in the 5.1.53 version, and we won’t be seeing that for a month or two. Yikes.

Reactive Framework conflict (SOLVED)

The type ‘System.IObservable exists in both ‘c:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.0\mscorlib.dll’ and ‘c:\Program Files (x86)\Microsoft Reactive Extensions\Redist\DesktopV2\System.Observable.dll’ (SOLVED)

During an upgrade of a C# WPF application that was using Reactive Extensions, I ran into this compile error:

The type ‘System.IObservable exists in both ‘c:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.0\mscorlib.dll’ and ‘c:\Program Files (x86)\Microsoft Reactive Extensions\Redist\DesktopV2\System.Observable.dll’

This is roughly the same kind of Defined in Multiple Assemblies problem as I faced before. (Solution provided.)

In this case, the System.Observable.dll was pinned to the 3.5 release of the Reactive Extensions.

The solution was to:

Download the .NET 4.0 version of Reactive Extensions (Rx_Net4.msi).
Install it.
Remove System.Observable from the project.
Re-add the System.Observable to the project, being super careful to select the latest version

Defined in Multiple Assemblies

The predefined type ‘System.Func’ is defined in multiple assemblies in the global alias … mscorlib.dll and System.Core.dll. SOLVED!

Today I migrated a C#/WPF project in Visual Studio 2010 from .NET 3.5 SP1 to .NET 4.0. Immediately the compile failed issuing this set of errors:

The predefined type ‘System.Func’ is defined in multiple assemblies in the global alias; using definition from ‘c:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.0\mscorlib.dll’
The predefined type ‘System.Func’ is defined in multiple assemblies in the global alias; using definition from ‘c:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.0\System.Core.dll’

To resolve this error involved understanding a little bit more about C#, .NET, and the Common Language Runtime (CLR).

According to C# 4.0 In A Nutshell from O’Reilly Press, on page 181 it says:

Some of the .NET types are used directly by the CLR and are essential for the managed hosting environment. These types reside in an assembly called mscorlib.dll and include C#’s built in types, ….

At a level above this are additional types that “flesh out” the CLR-functionality, …. These reside in System.dll, System.Xml.dll, and System.Core.dll, and together with mscorlib the provide a rich programming environment….

So mscorlib and System.Core are both needed. This begs the question, why is Func declared in both and thus causing a conflict? Or is it?

Further on in C# 4.0 In A Nutshell from O’Reilly Press, on page 183 in a general note it says:

A notable exception is the following types, which Framework 4.0 have moved from System.Core to mscorlib.dll:

The Action and Func delegates

…

This suggests a case exists where mscorlib is from our current .NET (the later one has Func), and System.Core is coming from the old one (which is where Func lives for that version).

How is this possible? It’s our projects fault.

This question on StackOverflow provides some insight; check out Simon‘s answer.

Right-click the project and select Unload Project
Right-click the project again and select Edit Project
Scroll down in the XML to find the ItemGroup element; it’ll have Reference elements insider of it.
Locate the Reference element that has Include=”System.Core” as an attribute.
If it has other qualifiers, remove them. If it has a TargetFrameworkVersion subelement remove it.
Save the XML.
Right-click the project and reload it; try a build now.

In my case, I had an entry that looked like this:

Removing the TargetFrameworkVersion, shown in red above, un-pinned the dll from the older .NET framework and things worked just fine.