Bash and Paths on macOS

On macOS, there’s a little more than just the order of bash scripts messing with your path. Here’s an updated diagram showing the inside scoop.

I was helping a friend diagnose why a strange script-like string was appearing in their PATH whenever they logged in or created a new bash shell window. (Note: Catalina now defaults to using zsh, not bash.)

Bash has a number of different configuration files it checks, and some of those the user may have chained together. So far, even the best resource on the matter I’ve found doesn’t tell the whole story.

On Apple machines, bash walks through these set of scripts and invokes a path modification tool, too.

macOS Bash Scripts and Path Modifications (validated with macOS Mojave v10.14.6 on 2020-04-08)

In a nutshell, /etc/profile calls a utility called path_helper, and it reads a list from two places the /etc/paths file and everything in the /etc/paths.d directory.

My friend had thought the /etc/paths file was a script and not a list, and years ago had erroneously put an export PATH= statement in there. Because it came last (for his setup), he ignored the problem.

Our false start to correct is was upon seeing an export statement in the PATH itself, we assumed it was a faulty quoting problem in a script, so primarily went looking in the script files on the left side of the graph …and wasted what felt like hours doing so.

Only after using Bash’s return statement to short-circuit script executions and using a blunt hammer of renaming scripts out of execution’s way, and the problem still persisted, did deeper exploration go down the system resources.

I hope this diagram helps get others out of trouble.


UPDATE 2020-07-08: Bob Rudis caught a typo (the text said /etc/profile, where it should have said /etc/paths); he also shared a link to GNU’s Bash Startup Files.

USB 3.0 Hub Stops Working on El Capitan

Has your USB 3.0 stopped working without explanation on your Mac? Here’s how to fix it.

I use a lot of external storage and it has been hard to find a USB hub that is fast, connects all my devices at once, and when using a device doesn’t drop other devices connected to it.

Anker 13-Port USB 3.0I finally found one that’s rock solid; it’s the Anker 13-Port USB 3.0 and it does everything I ever wanted.

Things were good until mid-January 2016 when the device started malfunctioning in strange ways. The first three ports did not recognize any device I put on it.  The other ports sometimes worked and sometimes didn’t. Reliability went out the window and I was forced to stop using it.  My guess was that something burned out. I went so far as to buy another smaller USB 3.0 hub, and well, it didn’t work either — so much so I ended up returning it.  I really wanted this hub working.

Curious, I handed the broken device to an electrical engineer and asked him if he could ascertain what was wrong with it. He took it apart, did diagnostic tests, saw nothing wrong, tried it on his computer, it worked fine, and handed it back to me fixed as just a mystery. However the story doesn’t stop there.

The Impossible Behavior

When I connected the device back up to my Mac, it behaved exactly the same way as it did before. I, of course, tried all my Mac’s USB ports.  I even tried a completely different Mac.  Identical failures.

So sure the device was working, my electrical engineer friend pulled out his Microsoft Surface Tablet, connected the hub, and instantly it worked.

We put it back on my Macs, with the same devices that just worked, it failed. Back to the Surface, it worked.  Back to the Mac, it failed.  In short, it was an electrical engineer’s WTF-nightmare.

The Common Denominator and Other Clues

At this point the problem was clearly related to the Mac.  More over, it used to work just fine, at least until mid-January.  What happened in mid-January?  El Capitan 10.11.3.

Both Macs were running El Capitan 10.11.3.

As a general rule, with Apple, the first generation hardware products have flaws, and the operating systems versions don’t usually get all the kinks out until version x.x.4 is released.  This threw immediate suspicion on the operating system, which meant it was time to check if other folks were having similar issues.

Yes they were.  (See this discussion.)

The Fix

While you’d think that one would need to go to Anker’s Driver Download page, that’s not the case.  You need to do two things:

  1. Reset the NVRAM / PRAM. (For a MacBook Pro it’s the Command-Option-P-R chord on boot.)
  2. Reset the SMC.  (For a MacBook Pro it’s the Shift-Option-Command-PowerOnButton.)

When the machine rebooted the USB hub behaved just like it used to.  Problem solved.

UPDATE (21-Mar-2016): With the introduction of El Capitan 10.11.4, it rebroke the USB 3.0 capabilities again.  The Console reports:

3/23/16 4:32:20.000 PM kernel[0]: 000227.351907 AppleUSB30Hub@14400000: AppleUSBHub::start: failed to set configuration with 0xe00002eb
3/23/16 4:32:21.000 PM kernel[0]: 000228.290970 AppleUSB30Hub@14400000: AppleUSB30Hub::start: failed to set hub depth 0 (0xe0005000)

So far, performing the above steps are not working.

OTHERS ARE HAVING IT TOO: Often the problem manifests as if the USB device, or something connected to it, is no longer working or has inadequate power, or is no longer detected by the host system.

Try your device on an older operating system (ideally the same hardware if you can), a Windows box, a Linux box, or even a Raspberry Pi — you’ll see the USB device works properly there.

YOU CAN HELP: It appears Apple may not know about the problem.

  1. Report it as a bug in OS X via the Apple Bug Reporter.
  2. Provide feedback via http://www.apple.com/feedback/

Please be kind when reporting issue, as these are the people who can help you. Give them technical details and model information to help them track it down.

UPDATE (06-May-2016): Apple has acknowledged issue 26102223 in their system and have asked for more information; I’m forwarding it to them.

UPDATE (09-Oct-2016): SOLUTION — It’s LeapMotion’s Fault!!

It seems that the Leap Motion driver may be the culprit here!

Uninstalling the driver (according to their instructions at https://support.leapmotion.com/entries/39493988-Uninstalling-the-Leap-Motion-software-on-Mac-OSX) caused the device to spring back to life without even a reboot required. Credit and thanks to David Ryskalczyk for hunting this down.

*** Between MAY and OCT, this blog suffered a large data-loss pertaining to the comments left by visitors.  I wish I had the original post by David Ryskalczyk reporting his extensive solution.  Here’s what I can manually recreate.

… I figured that maybe this was a software issue. I proceeded to clean install 10.12 on a USB drive — no issues; then 10.11 on a USB drive — *also* no issues! Seems to be software. From there I started isolating things — first with kexts, which turned out to be inconclusive, then with daemons (looking in /Library/LaunchDaemons and LaunchAgents to see what was installed and running). After removing a bunch of stuff I did additional testing and confirmation to figure out exactly what was causing the problem, and sure enough, it was the Leap Motion runtime!
Hopefully this can be fixed so the Leap Motion drivers can successfully coexist with these USB hubs. … I suspect the real cause is that the Leap Motion runtime is tripping up a bug in the Apple drivers.
I (and others) were able to confirm that David’s fix does indeed work.
Apple, after passing on this information to them, merely marked my bug report closed as a duplicate.
This information was also passed onward to Anker, who was very grateful to have the information for answering support calls about it.
I can also confirm that after months of not having the LeapMotion driver installed, my favorite Anker 3.0 USB has been working like a champ.

Low Disk Space? It Might be iTunes.

I noticed that I was freakishly low in disk space, so used DaisyDisk to identify problem directories.

Turns out, a major offender was ~/Library/Application Support/MobilSync/Backup, which contained several years worth of iPhone and iPad backups.

As tempting as it might be, don’t just go deleting the directories inside it.

Instead… use iTunes, which will keep all the internal bookkeeping correct.

Open iTunes / Preferences…, then go to the Devices tab.  You’ll see the name of the backup, an icon if it’s encrypted, and a date/time stamp of when it was made.

Select one or more of the items and press the Delete Backup button.  But beware, they won’t really disappear until you press the OK button.  Closing the dialog via the red window button or pressing Cancel will not commit the change.

This is one of those rare cases where Apple dialogs don’t reflect current state, but intended ones.  And, considering this is the safer way of dealing with data, I’m all for it.

Fixing HP Mopier, Collation, and Storage problems (WIN 7)

After upgrading to the latest HP “Universal” drivers, our HP LaserJet CP4525 printers started reporting that they were “Unable to store job at printer”, that they ran out of memory, and were “Unable to collate job at printer”.  The error messages suggested the disk wasn’t present, was full, that I needed to delete files, add RAM, or even a very expensive EIO hard disk. Didn’t get much obvious help from the HP Universal Print Driver Systems Administrator’s Guide either.

I tried adding RAM to the full capacity.  That didn’t help.  Here’s how I ended up fixing the problem, thanks to piecing together dozens of support forum posts for different printer models and a little experimentation.

  1. Go to Start / Devices and Printers (yours may say Printers and Faxes).
  2. Right click the Printer Icon of the printer giving you problems and click Printer Properties (not plain old properties that’s at the bottom).
  3. While on the General table, click the Change Properties button near the bottom left (otherwise you’ll get a read-only mode of grey items).
  4. Change over to the Device Settings tab.
  5. Ensure that the Mopier Mode is set to Disabled.
  6. Ensure that the Job Storage is also set to Disabled.
  7. Click Apply, ideally to see there were no errors in doing this.
  8. Click OK to close the dialog.

Java 8 on OS X Yosemite

I downloaded a recent copy of IntelliJ, only to discover when I went to open it, OS  Yosemite indicated I had no version of Java installed, and that I’d need to install an old version. The “More Info…” button took me to this page:  http://support.apple.com/kb/DL1572

…which didn’t load.  (UPDATE: This fixed the load issue.) Similar detailed install directions also ended up at the same broken page.

So, I attempted to download Java 8 directly from Oracle and  install install it.  The install worked fine, but IntelliJ 14 still did not open.  Same error message.

Here’s how I solved it.  Hop into terminal and do this:

$ cd "/Applications/IntelliJ IDEA 14.app/Contents"
$ cp Info.plist Info.plist-orig
   
$ vi Info.plist    # ... any text editor will do

Find the line that says 1.6* and change it to 1.8*.  Save your file, and now go open IntelliJ as normal.

This causes IntelliJ 14 to use Java JDK 8, and all is right with the world.

List iTunes Apps by Purchaser

I finally developed a way to list all your iTune applications by Purchaser, and even better, I can do it from the command line.

Last year I was helping a friend who has pretty poor internet connectivity upgrade his iPad to the latest version of iOS. To do this, I connected his iPad to my system, performed a backup in iTunes, then synchronized to update the operating system. This had the side effect of beaming some of his applications to my iTunes, which in order to continue, he authenticated against. His iPad updated great, and he was on his way.

Things on my machine seemed okay for a while, that was until some of the apps he purchased that I didn’t have wanted to update. Not having his password, I wasn’t able to update them, but even worse, Apple wasn’t announcing which apps needed updating with his account so I could simply delete them as I wanted to.

Instead, for over a year, I was greeted in iTunes by a numerical indicator saying I needed up update my apps, but when I went to do it, I was up to date. Every once in a while I’d recognize an app that I didn’t purchase (in a haystack of nearly 1,000 iPhone apps) and delete it. Only then did the number drop, but later rise again when some other app needed updating.

What I needed to do was list out all the apps by Purchaser.

One of things that really annoys me about Apple is that stuff that is trivial to implement, like putting an optional Purchaser column in iTunes, they don’t do. The feature is half heartedly there, though. Press Command-I for Info, and you can see the purchaser for an item.

Only now you have to click and inspect your whole app list. And with the number of applications I own, this doesn’t scale well at all.

Searching the web reveals that others are in a similar bind, and that Apple seems to really care less about the few handful of users with this problem. Like much on the Apple Support Site, it’s unhelpfully silent.

Frustrated, I decided to solve this problem once and for all.  Open Terminal and cut’n’paste the following in:

for f in ~/Music/iTunes/iTunes\ Media/Mobile\ Applications/*.ipa; \
 do (echo "$f" ; unzip -p "$f" "iTunesMetadata.plist" | \
 plutil -p - | egrep -i "\\"(itemName|artistName|AppleID)\\"" ) | \
 perl -e 'while (<>) { if (m!^/!) { chop; $fqn=$_; } if (m/"(.+)" => (".+")/) { $e{lc($1)}=$2; } } print "\\"${fqn}\\",$e{\\"itemname\\"},$e{\\"artistname\\"},$e{\\"appleid\\"}\n";'; \
done

Turning Web Share Back On in Mountain Lion

Apple removed Web Share from Mountain Lion. Next they’ll be removing the Start button… oh, wait… that’s someone else. Here’s how to bring back Apache for those people who use their Mac to test and design websites.

I was rather surprised and disappointed to learn that Web Sharing was removed from Mountain Lion.

According to this post, it’s possible to bring back.

Per-user Web Sharing is gone from Preferences but can be easily re-enabled via Terminal.app. Copy the following snippet into /etc/apache2/users/USER.conf:


<Directory "/Users/USER/Sites/">
 Options Indexes MultiViews FollowSymlinks
 AllowOverride All
 Order allow,deny
 Allow from all
</Directory>

and restart Apache with sudo apachectl restart.

And PHP has been hidden as well, but again thanks to this article, it can come back as well.

Uncomment these lines by removing the leading pound sign in /etc/apache2/httpd.conf:

  • LoadModule rewrite_module libexec/apache2/mod_rewrite.so
  • LoadModule perl_module libexec/apache2/mod_perl.so
  • LoadModule php5_module libexec/apache2/libphp5.so

Groking Objective-C for Modern Day Programmers

Here’s what I think trips up solid programmers about the Objective-C language.

One of the nice things about Apple’s OS X for the Macintosh is that they include a free copy of Xcode, a very sweet graphical development environment allowing developers to make Cocoa applications using Objective-C.

The problem is, jumping straight into the Cocoa framework, even with tutorials, can be a daunting task if you’re not comfortable with Objective-C.

Yes, there are a number of books out there, but today I stumbled across, quite by accident, a fairly decent tutorial on Objective-C.

That got me thinking. In a world where we can easily pick up C++, C#, Java, Python, and Ruby… what is it about Objective-C that makes it hard to follow? Here’s what I think the major stumbling blocks are:

Notation Preconceptions Clouding C as the True Origin
In the classic examples, we think in terms of an object with methods and members, often using a dot-like syntax. As such, we tend to want to force fit Objective-C’s way of doing thing into some preconceived mold, and when the syntax doesn’t express that, things fall apart.

The Solution: Back off. Drop all the way back to C. Consider Objective-C nothing more than a well written pre-processor to useful object like extensions to the C programming language. Sure your implementation may very well be a compiler, but don’t think in those terms.

You’ve Got Classes Wrong
You’re used to thinking that a class is a data type. Don’t, think of it as a thing. In fact, Objective-C only needs to know a variable is a class, it doesn’t need to know which one or what it exposes. Classes are things whose primary role is to produce instances.

It’s Supposed To Feel Like Macro Magic
When you’re working on the guts of a class, you’ve got a lot of blocks that start with @class, @interface, @end, …. If it feels like something’s acting as a pre-processor and is converting that stuff into data structures and functions, it is. You’re not going to see pretty syntax that feels integrated into the language.

Hard Crunchy Outside, Soft Gooey Inside
A class exposes an interface, this is how the outside world sees and interacts with your class. Specifically, its your methods. And they’re all public. Get over it. Test. Only your members can be protected or private.

Remember, there are class-things and instance-things.

Declarations and Definitions
With C, there’s declarations that define the type and are often used for forward references; additionally, there are definitions where the memory is actually allocated and a value assigned.

Objective-C objects work similar. There’s an @interface / @end block that describes the object, its members, and its methods. And there’s an @implementation / @end block that actually contains the real code.

void * and id
A pointer in C, indicated by a *, points at a data type. If the pointer is given a void type, the compiler does no type checking. You know this. The id type holds a object, regardless what type it is.

Messages are not Methods
Any object can be sent any message. An object maps a message to a method. However, if there’s no mapping available, it can be handed to a general handler for the object — which can be clever and analyze the message.

Extended classes… and instances.
The mapping between message and method can be extended; you can think of it as subclassing. However, it’s possible to extend an instance. Meaning, a class might not respond to a message, while instance A does and instance B doesn’t.

Categories
Because messages are bound late to an object, and the magic is done with data structures rather than types, it’s possible to create a collection of methods and add them to a class, even pre-existing ones you don’t have source code for.

Protocols
A protocol neither exposes an interface, nor does it provide an implementation. It is simply a named collection of method signatures which is attached to classes. Because any message can be sent to any object instance (or class), it is helpful to be able to “ask” if certain messages will be honored; that’s what protocols are for.

It’s Not Named Parameters
The syntax of a method may look odd, primarily because parameters appear to have additional text:
-(void) rotate:(String *)shape clockwise:(float)degrees;

In many modern languages, a signature can be overloaded. However, go back to the trick “What Would Macros Do?” In this case, any text that appears in front of a parameter is not a named parameter, but part of the method name. The above method would be the logical equivalent of void rotate_clockwise( String *shape, float degrees ) in raw C.

Constructors / Destructors
In today’s languages, performing a new allocates memory and initialized the object. Objective-C splits these into two discrete operations; this allows you to allocate memory from different heaps, as well as deferring expensive initializations – no bogus empty constructors here.

Note, you are also responsible for calling the superclass’s init, before doing your own init. If you have a method to deallocate the object, when done you need to call the superclass’s deallocation routine.

C requires you to take on more responsibility and pay attention to order; Objective-C classes require the same amount of vigilance.

Extra Data Types
Objective-C provides some helper data types, but some of them are tied quite strongly to the messaging framework.