NTFS on OS X

Trying to write to an NTFS volume on OS X, and all you find are third party packages and complicated config file manipulations asking you to reboot? Do this….

Every once in a while someone brings over an external NTFS formatted drive and plugs it into my Mac to exchange some large data sets. While the Mac can easily read NTFS, it doesn’t appear to have the capability to write to that file system.

Appears is the operative phrase.

If you search for a solution in the open, you’ll find companies selling commercial products, drivers, and mounters. Some people have posted complicated looking instructions telling instructing you to mess with low level system files, that may or may not exist, ending in a full system reboot that typically solves the problem for that lone drive.

NTFS on OSXHere’s the generic no-software required solution.

First connect your NTFS drive to the Mac, it’ll mount with some name like “My Disk.”

Open a Terminal window and enter the command:
$ mount

The mount command will tell you what drives are listed. Find the one that has your drive name listed in it and copy the device name:
/dev/disk1s2 on /Volumes/My Disk (hfs, local, nodev, nosuid, journaled, noowners, mounted by me)

Now eject the drive, but don’t disconnect it from the machine.

Go manually create the directory (shown in bold above); this is where the mount point will be:
$ cd /Volumes
$ mkdir My\ Disk

NOTE: The backslash escapes the next character, in this case allowing a space in the directory name.

Now, mount the drive as read-write using this command:
$ sudo mount -o rw -t ntfs /dev/disk2s1 /Volumes/My\ Disk

Make sure that you’ve used the same device location you copied down the the prior steps.

At this point your drive should appear on the desktop, and you ought to be able to read and write to it just fine. No rebooting necessary!

Note that you can get to it from the Terminal as well:
$ cd /Volumes/My\ Disk
$ ls

Note that sometimes Finder may act a little wonky with timing problems and a huge file. Apparently the underlying Unix system has no problem. You can copy a big file to the drive:
$ rsync –progress SuperBigFile.zip /Volumes/My\ Disk

Or a whole directory:
$ rsync -r –progress LargeDeepDirectory /Volumes/My\ Disk

When done, make sure you are not running any programs that are accessing the drive or have their current directory set to the drive:
$ cd /Volumes

Then, eject the drive normally, or unmount it from the command line — your choice.
$ sudo umount /Volumes/My\ Disk

Full disclosure and warnings: This was tested on OS X 10.6.5, though support has been around to do this for a while. And, any time you’re doing something that deals with questionable file system access, make sure there’s nothing on the drive you don’t mind losing. Have a backup. There’s always a slight risk, but it’s very close to zero — why state all this? Because I assume no responsibility if something goes wrong.

C# Must Haves

Resources for quickly getting up to speed on C#.

This is the page of C# resources that I wish I had when I got started.

Visual Studio 2010Snag a copy of Visual Studio, whether it’s the free Express version, or one of the uber-featured ones.

Reactive Extensions
In addition to .NET, snag the Reactive Extensions (Rx) library.


Expression Blend 4If you’re able to, grab a copy of Expression Blend as well for graphical layouts. Not mandatory, but it helps.


And you’ll want these books:

C# 4.0 in a Nutshell: The Definitive ReferenceWPF 4 UnleasedLinq in ActionEntity Frameworks

MySQL Cascade Delete Problem

MySQL 5.1.51 introduced a nasty little bug that has the potential to really cause some production servers some ill — cascading delete and updates can sometimes fail if you’re dealing with “too much” data at once. Where “too much” is a relatively small amount. You may get bit if you have a one-to-many relationship.

A note of warning to MySQL users using 5.1.51, you may want to downgrade to 5.1.50 for a little bit.

There’s a problem with 5.1.51 in which cascading deletes or updates throw an error. Not good if you have constraints and one to many relationships.

ERROR 1030 (HY000): Got error -1 from storage engine

The mysql error log will say something like:

InnoDB: Cannot delete/update rows with cascading foreign key constraints that exceed max depth of 250

This is a confirmed bug in MySQL and is repeatable. The error will cause a transaction rollback. Not good news for people running production systems.

See MySQL Bug #357255.

While the problem was quickly identified and apparently resolved, as the defect report is closed, it does not look like the 06-Oct-2010 change has made it out to the production baseline as of the time of this writing.

Additionally, the MySQL pre-release snapshots on labs.mysql.com show there is a mysql-5.1.52 pending with a September date, and this hasn’t made it to general production yet.

That leaves one to speculate that the fix will appear in the 5.1.53 version, and we won’t be seeing that for a month or two. Yikes.

Reactive Framework conflict (SOLVED)

The type ‘System.IObservable exists in both ‘c:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.0\mscorlib.dll’ and ‘c:\Program Files (x86)\Microsoft Reactive Extensions\Redist\DesktopV2\System.Observable.dll’ (SOLVED)

During an upgrade of a C# WPF application that was using Reactive Extensions, I ran into this compile error:

The type ‘System.IObservable exists in both ‘c:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.0\mscorlib.dll’ and ‘c:\Program Files (x86)\Microsoft Reactive Extensions\Redist\DesktopV2\System.Observable.dll’

This is roughly the same kind of Defined in Multiple Assemblies problem as I faced before. (Solution provided.)

In this case, the System.Observable.dll was pinned to the 3.5 release of the Reactive Extensions.

The solution was to:

  1. Download the .NET 4.0 version of Reactive Extensions (Rx_Net4.msi).
  2. Install it.
  3. Remove System.Observable from the project.
  4. Re-add the System.Observable to the project, being super careful to select the latest version

Defined in Multiple Assemblies

The predefined type ‘System.Func’ is defined in multiple assemblies in the global alias … mscorlib.dll and System.Core.dll. SOLVED!

Today I migrated a C#/WPF project in Visual Studio 2010 from .NET 3.5 SP1 to .NET 4.0. Immediately the compile failed issuing this set of errors:

  • The predefined type ‘System.Func’ is defined in multiple assemblies in the global alias; using definition from ‘c:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.0\mscorlib.dll’
  • The predefined type ‘System.Func’ is defined in multiple assemblies in the global alias; using definition from ‘c:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.0\System.Core.dll’

To resolve this error involved understanding a little bit more about C#, .NET, and the Common Language Runtime (CLR).

According to C# 4.0 In A Nutshell from O’Reilly Press, on page 181 it says:

Some of the .NET types are used directly by the CLR and are essential for the managed hosting environment. These types reside in an assembly called mscorlib.dll and include C#’s built in types, ….

At a level above this are additional types that “flesh out” the CLR-functionality, …. These reside in System.dll, System.Xml.dll, and System.Core.dll, and together with mscorlib the provide a rich programming environment….

So mscorlib and System.Core are both needed. This begs the question, why is Func declared in both and thus causing a conflict? Or is it?

Further on in C# 4.0 In A Nutshell from O’Reilly Press, on page 183 in a general note it says:

A notable exception is the following types, which Framework 4.0 have moved from System.Core to mscorlib.dll:

  • The Action and Func delegates

This suggests a case exists where mscorlib is from our current .NET (the later one has Func), and System.Core is coming from the old one (which is where Func lives for that version).

How is this possible? It’s our projects fault.

This question on StackOverflow provides some insight; check out Simon‘s answer.

  1. Right-click the project and select Unload Project
  2. Right-click the project again and select Edit Project
  3. Scroll down in the XML to find the ItemGroup element; it’ll have Reference elements insider of it.
  4. Locate the Reference element that has Include=”System.Core” as an attribute.
  5. If it has other qualifiers, remove them. If it has a TargetFrameworkVersion subelement remove it.
  6. Save the XML.
  7. Right-click the project and reload it; try a build now.

In my case, I had an entry that looked like this:

<ItemGroup>
  <Reference Include=”System.Core”>
    <TargetFrameworkVersion>3.5</TargetFrameworkVersion>
  </Reference>
</ItemGroup>

Removing the TargetFrameworkVersion, shown in red above, un-pinned the dll from the older .NET framework and things worked just fine.

Mysterious Copyright

This is clearly one of those things I did to myself as a good idea, then forgot about, only to be plagued by it later.

I noticed that all of my photographs on my camera were reporting a copyright with a 2009 year inside the exif data.

I’ve been unable to figure out where it was coming from, resorting to exiftool to remote it.

My natural thought was that perhaps it was some preference in a photo editing tool or a geospatial locator tool. But, no. Turns out I did it to myself.

The Canon EOS Utility has a nifty ability to include a value for the Copyright tag. And about a year ago when I tethered it to the computer, I must have noticed this and set it to some precanned value that includes the year.

It looked something like this:
Copyright (c) 2009 by Walt Stoneburner, All Rights Reserved.

And ever since then, my photos were stamped with that value. Which was fine, back in 2009.

Fixing the problem was as simple as tethering the camera again and firing up Canon EOS Utility. It also gave me an opportunity to update the firmware.

Strange copyright exif data: mystery solved.

Find and Replace in Word using C# .NET

Solution to how to do a global search and replace in MS-Word, including across floating text objects, in C#/.NET.

Heads up, this article contains high quantity of geek content. Non-geeks should move along.

I’ve been trying to use Microsoft.Office.Interop.Word to perform a global bulk search and replace operations across an entire document. The problem was, however, if a document contained a floating text box, which manifested itself as a shape object of type textbox, the find and replace wouldn’t substitute the text for that region. Even using Word’s capability to record a macro and show the VBA code wasn’t helpful, as the source code in BASIC wasn’t performing the same operation as inside the Word environment.

What I wanted was a simple routine to replace text anywhere inside of a document. If you Google for this you’ll get the wrong kind of textbox, the wrong language, people telling you not to use floating textboxes, and all kinds of weird story iterators.

One site seemed to have the solution; many kind thanks to Doug Robbins, Greg Maxey, Peter Hewett, and Jonathan West for coming up with this solution and explaining it so well.

However, the solution was in Visual Basic for Applications, and I needed a C# solution for a .NET project. Here’s my port, which works with Office 2010 and Visual Studio 2010 C#/.NET 4.0. I’ve left a lot of redundant qualifiers and casting on to help people searching for this article.

using System;
using System.Collections.Generic;
using System.ComponentModel;
using Microsoft.Office.Interop.Word;

// BEGIN: Somewhere in your code
Application app = null;
Document doc = null;
try
{
  app = new Microsoft.Office.Interop.Word.Application();

  doc = app.Documents.Open(filename, Missing, Missing, Missing, Missing, Missing, Missing, Missing, Missing, Missing);

  FindReplaceAnywhere(app, find_text, replace_text);

  doc.SaveAs(outfilename, Missing, Missing, Missing, Missing, Missing, Missing, Missing, Missing, Missing);
}
finally
{
  try
  {
      if (doc != null) ((Microsoft.Office.Interop.Word._Document) doc).Close(true, Missing, Missing);
  }
  finally { }
  if (app != null) ((Microsoft.Office.Interop.Word._Application) app).Quit(true, Missing, Missing);
}
// END: Somewhere in your code             



// Helper
private static void searchAndReplaceInStory(Microsoft.Office.Interop.Word.Range rngStory, string strSearch, string strReplace)
{
    rngStory.Find.ClearFormatting();
    rngStory.Find.Replacement.ClearFormatting();
    rngStory.Find.Text = strSearch;
    rngStory.Find.Replacement.Text = strReplace;
    rngStory.Find.Wrap = WdFindWrap.wdFindContinue;

    object arg1 = Missing; // Find Pattern
    object arg2 = Missing; //MatchCase
    object arg3 = Missing; //MatchWholeWord
    object arg4 = Missing; //MatchWildcards
    object arg5 = Missing; //MatchSoundsLike
    object arg6 = Missing; //MatchAllWordForms
    object arg7 = Missing; //Forward
    object arg8 = Missing; //Wrap
    object arg9 = Missing; //Format
    object arg10 = Missing; //ReplaceWith
    object arg11 = WdReplace.wdReplaceAll; //Replace
    object arg12 = Missing; //MatchKashida
    object arg13 = Missing; //MatchDiacritics
    object arg14 = Missing; //MatchAlefHamza
    object arg15 = Missing; //MatchControl

    rngStory.Find.Execute(ref arg1, ref arg2, ref arg3, ref arg4, ref arg5, ref arg6, ref arg7, ref arg8, ref arg9, ref arg10, ref arg11, ref arg12, ref arg13, ref arg14, ref arg15);
}

// Main routine to find text and replace it,
//   var app = new Microsoft.Office.Interop.Word.Application();
public static void FindReplaceAnywhere(Microsoft.Office.Interop.Word.Application app, string findText, string replaceText)
{
    // http://forums.asp.net/p/1501791/3739871.aspx
    var doc = app.ActiveDocument;

    // Fix the skipped blank Header/Footer problem
    //    http://msdn.microsoft.com/en-us/library/aa211923(office.11).aspx
    Microsoft.Office.Interop.Word.WdStoryType lngJunk = doc.Sections[1].Headers[WdHeaderFooterIndex.wdHeaderFooterPrimary].Range.StoryType;

    // Iterate through all story types in the current document
    foreach (Microsoft.Office.Interop.Word.Range rngStory in doc.StoryRanges)
    {

        // Iterate through all linked stories
        var internalRangeStory = rngStory;

        do
        {
            searchAndReplaceInStory(internalRangeStory, findText, replaceText);

            try
            {
                //   6 , 7 , 8 , 9 , 10 , 11 -- http://msdn.microsoft.com/en-us/library/aa211923(office.11).aspx
                switch (internalRangeStory.StoryType)
                {
                    case Microsoft.Office.Interop.Word.WdStoryType.wdEvenPagesHeaderStory: // 6
                    case Microsoft.Office.Interop.Word.WdStoryType.wdPrimaryHeaderStory:   // 7
                    case Microsoft.Office.Interop.Word.WdStoryType.wdEvenPagesFooterStory: // 8
                    case Microsoft.Office.Interop.Word.WdStoryType.wdPrimaryFooterStory:   // 9
                    case Microsoft.Office.Interop.Word.WdStoryType.wdFirstPageHeaderStory: // 10
                    case Microsoft.Office.Interop.Word.WdStoryType.wdFirstPageFooterStory: // 11

                        if (internalRangeStory.ShapeRange.Count &gt; 0)
                        {
                            foreach (Microsoft.Office.Interop.Word.Shape oShp in internalRangeStory.ShapeRange)
                            {
                                if (oShp.TextFrame.HasText != 0)
                                {
                                    searchAndReplaceInStory(oShp.TextFrame.TextRange, findText, replaceText);
                                }
                            }
                        }
                        break;

                    default:
                        break;
                }
            }
            catch
            {
                // On Error Resume Next
            }

            // ON ERROR GOTO 0 -- http://www.harding.edu/fmccown/vbnet_csharp_comparison.html

            // Get next linked story (if any)
            internalRangeStory = internalRangeStory.NextStoryRange;
        } while (internalRangeStory != null); // http://www.harding.edu/fmccown/vbnet_csharp_comparison.html
    }

}

Let me know if it worked for you; bug fixes and enhancements welcome.

Virtual Server Problems

I recently switched operating system vendors, and to my surprise when I went to port over the web content from one system to another, things didn’t go as smoothly as I had hoped. What used to be /etc/httpd was now /etc/apache2, and inside this directory files were organized differently that I was used to, and so forth. Still, I would have hoped moving from Apache2 on RedHat to Apache2 on Ubuntu would have been easier.

Empirical evidence was suggesting that all of my user’s virtual sites were working, but all of mine, no matter where I had them on the system, were reporting the error: You don’t have permission to access / on this server.

Here’s what was going on, primarily recounted so if I ever do this to myself in the future, I’ll know what to look for.

The virtual host files, which now appear in /etc/apache2/sites-available as *.conf files, had a slight difference. Some of them had this in their Directory directive:

    Order allow,deny
    allow from all

Mine did not. But, then again, some other websites did not as well. Turns out those directives were placed inside their .htaccess files.

Now, not all sites had the .htaccess file, and things had been working before without the explicit directive in each virtual host .conf file.

Turns out I had some how tromped on the default file, which contains a directive that looks like this:

    <Directory />

      Options FollowSymLinks
      AllowOverride None

    </Directory>

If it is not present, then all virtual hosts must explicitly allow access (via .htaccess or their .conf file).

This directive allows Apache to serve up any file the URL asks for… which one may not want to do. It seems the secure way is to edit the virtual host .conf files, and not rely on some default magic.

But, because that was in my old configuration long ago, and not in the new one, my virtual host .conf files didn’t have it, but my more modern ones for my users did. Depending on which template I used to base new sites off was how some sites worked and some didn’t.

After fixing this, I ran into a new problem. Some sites weren’t coming up still, but this time with permission errors.

When I migrated over the web content, it preserved user and group ownerships from the other system. These did not match the new Apache2 user and group on the new system.

However, I got lucky. There was no user/group mapping on the new system, which meant I could execute a find command to find and fix them. It looked something like this:

    find /home -nogroup -print
    find /home -nouser -print
    find /home -group 49 -exec chgrp www-data ‘{}’ \;

Printing in Parallels

Using Parallels virtualization, I got this pretty scary error message: pstopdffilter/pstocupsraster failed with err number -31000. I’m almost ashamed to tell you what the solution is to get past it.

Parallels is a virtualization package for the Macintosh that primarily is used for running Microsoft Windows in a virtualized environment on OS X.

At some point you’re going to run into the problem of wanting to print something from the guest OS. Do not try to install a Windows XP print driver for the device that’s connected directly to your Apple. That’s not how it works.

You have a virtual machine. Surprise, you have a virtual printer too.

To set it up is trivial:

  1. Stop your Windows VM if it’s running.
  2. Open VM Configuration Editor (Parallels Desktop menu – Edit – Virtual Machine)
  3. Add Parallel Port Printer to the VM Configuration: click “Add” – select “Parallel Port”, hit “Next” – select “Use a printer” – select the printer you have available in the Mac OS.
  4. Make sure that you are able to print using that printer from the Mac OS side.
  5. Start Windows and try printing some document using “HP Color LaserJet 8500 PS” (it’s generic driver that’s being used for printing from the Virtual Machine to any Mac OS compatible printer).

This creates a HP Color LaserJet 8500 PS printer, which then gets redirected to the host operating system’s default printer. Printing then works normally, queuing and all.

Now, I did run into this problem using Microsoft Office on Windows XP with a HP DeskJet 6980 connected wirelessly through an Apple AirPort Extreme in bridge mode:

pstopdffilter/pstocupsraster failed with err number -31000

Here’s how I solved it.
I deleted the print queue on the host operating system, then I turned the power off and back on again on the printer, and tried again.

Seriously. I power cycled the printer. That’s all that was required. Second time through, it worked like a champ.

Big scary error message, itty-bitty solution.

NOTE: You will want to scan through your document if you’re using exotic fonts. In my case apostrophes were coming out as í.

Safari Problems Downloading .DMG Files

A number of users are reporting that Safari 4 is no longer downloading .dmg files. Here’s how I fixed the problem when it started happening to me.

A while back I started having problems with Safari 4 being able to download files. Normally when one clicks on a .dmg or .zip file, Safari downloads it.

Recently, it stopped working, either doing absolutely nothing or trying to load the file into the browser itself for display. It was as if the MIME type wasn’t properly being handled.

Here’s how I fixed it.

It appears that Speed Download‘s broswer plugin is to blame. While it works amazingly well with Safari 3, it doesn’t seem to work quite right with Safari 4.0.3.

  1. Quit completely out of Safari.
  2. Go to /Library/Internet Plug-Ins directory and locate the file SpeedDownload Browser Plugin.plugin and move it out of that folder.
  3. Restart Safari.