Category: Apple

Bash and Paths on macOS

On macOS, there’s a little more than just the order of bash scripts messing with your path. Here’s an updated diagram showing the inside scoop.

I was helping a friend diagnose why a strange script-like string was appearing in their PATH whenever they logged in or created a new bash shell window. (Note: Catalina now defaults to using zsh, not bash.)

Bash has a number of different configuration files it checks, and some of those the user may have chained together. So far, even the best resource on the matter I’ve found doesn’t tell the whole story.

On Apple machines, bash walks through these set of scripts and invokes a path modification tool, too.

macOS Bash Scripts and Path Modifications (validated with macOS Mojave v10.14.6 on 2020-04-08)

In a nutshell, /etc/profile calls a utility called path_helper, and it reads a list from two places the /etc/paths file and everything in the /etc/paths.d directory.

My friend had thought the /etc/paths file was a script and not a list, and years ago had erroneously put an export PATH= statement in there. Because it came last (for his setup), he ignored the problem.

Our false start to correct is was upon seeing an export statement in the PATH itself, we assumed it was a faulty quoting problem in a script, so primarily went looking in the script files on the left side of the graph …and wasted what felt like hours doing so.

Only after using Bash’s return statement to short-circuit script executions and using a blunt hammer of renaming scripts out of execution’s way, and the problem still persisted, did deeper exploration go down the system resources.

I hope this diagram helps get others out of trouble.

UPDATE 2020-07-08: Bob Rudis caught a typo (the text said /etc/profile, where it should have said /etc/paths); he also shared a link to GNU’s Bash Startup Files.

USB 3.0 Hub Stops Working on El Capitan

Has your USB 3.0 stopped working without explanation on your Mac? Here’s how to fix it.

I use a lot of external storage and it has been hard to find a USB hub that is fast, connects all my devices at once, and when using a device doesn’t drop other devices connected to it.

Anker 13-Port USB 3.0I finally found one that’s rock solid; it’s the Anker 13-Port USB 3.0 and it does everything I ever wanted.

Things were good until mid-January 2016 when the device started malfunctioning in strange ways. The first three ports did not recognize any device I put on it.  The other ports sometimes worked and sometimes didn’t. Reliability went out the window and I was forced to stop using it.  My guess was that something burned out. I went so far as to buy another smaller USB 3.0 hub, and well, it didn’t work either — so much so I ended up returning it.  I really wanted this hub working.

Curious, I handed the broken device to an electrical engineer and asked him if he could ascertain what was wrong with it. He took it apart, did diagnostic tests, saw nothing wrong, tried it on his computer, it worked fine, and handed it back to me fixed as just a mystery. However the story doesn’t stop there.

The Impossible Behavior

When I connected the device back up to my Mac, it behaved exactly the same way as it did before. I, of course, tried all my Mac’s USB ports.  I even tried a completely different Mac.  Identical failures.

So sure the device was working, my electrical engineer friend pulled out his Microsoft Surface Tablet, connected the hub, and instantly it worked.

We put it back on my Macs, with the same devices that just worked, it failed. Back to the Surface, it worked.  Back to the Mac, it failed.  In short, it was an electrical engineer’s WTF-nightmare.

The Common Denominator and Other Clues

At this point the problem was clearly related to the Mac.  More over, it used to work just fine, at least until mid-January.  What happened in mid-January?  El Capitan 10.11.3.

Both Macs were running El Capitan 10.11.3.

As a general rule, with Apple, the first generation hardware products have flaws, and the operating systems versions don’t usually get all the kinks out until version x.x.4 is released.  This threw immediate suspicion on the operating system, which meant it was time to check if other folks were having similar issues.

Yes they were.  (See this discussion.)

The Fix

While you’d think that one would need to go to Anker’s Driver Download page, that’s not the case.  You need to do two things:

  1. Reset the NVRAM / PRAM. (For a MacBook Pro it’s the Command-Option-P-R chord on boot.)
  2. Reset the SMC.  (For a MacBook Pro it’s the Shift-Option-Command-PowerOnButton.)

When the machine rebooted the USB hub behaved just like it used to.  Problem solved.

UPDATE (21-Mar-2016): With the introduction of El Capitan 10.11.4, it rebroke the USB 3.0 capabilities again.  The Console reports:

3/23/16 4:32:20.000 PM kernel[0]: 000227.351907 AppleUSB30Hub@14400000: AppleUSBHub::start: failed to set configuration with 0xe00002eb
3/23/16 4:32:21.000 PM kernel[0]: 000228.290970 AppleUSB30Hub@14400000: AppleUSB30Hub::start: failed to set hub depth 0 (0xe0005000)

So far, performing the above steps are not working.

OTHERS ARE HAVING IT TOO: Often the problem manifests as if the USB device, or something connected to it, is no longer working or has inadequate power, or is no longer detected by the host system.

Try your device on an older operating system (ideally the same hardware if you can), a Windows box, a Linux box, or even a Raspberry Pi — you’ll see the USB device works properly there.

YOU CAN HELP: It appears Apple may not know about the problem.

  1. Report it as a bug in OS X via the Apple Bug Reporter.
  2. Provide feedback via http://www.apple.com/feedback/

Please be kind when reporting issue, as these are the people who can help you. Give them technical details and model information to help them track it down.

UPDATE (06-May-2016): Apple has acknowledged issue 26102223 in their system and have asked for more information; I’m forwarding it to them.

UPDATE (09-Oct-2016): SOLUTION — It’s LeapMotion’s Fault!!

It seems that the Leap Motion driver may be the culprit here!

Uninstalling the driver (according to their instructions at https://support.leapmotion.com/entries/39493988-Uninstalling-the-Leap-Motion-software-on-Mac-OSX) caused the device to spring back to life without even a reboot required. Credit and thanks to David Ryskalczyk for hunting this down.

*** Between MAY and OCT, this blog suffered a large data-loss pertaining to the comments left by visitors.  I wish I had the original post by David Ryskalczyk reporting his extensive solution.  Here’s what I can manually recreate.

… I figured that maybe this was a software issue. I proceeded to clean install 10.12 on a USB drive — no issues; then 10.11 on a USB drive — *also* no issues! Seems to be software. From there I started isolating things — first with kexts, which turned out to be inconclusive, then with daemons (looking in /Library/LaunchDaemons and LaunchAgents to see what was installed and running). After removing a bunch of stuff I did additional testing and confirmation to figure out exactly what was causing the problem, and sure enough, it was the Leap Motion runtime!
…
Hopefully this can be fixed so the Leap Motion drivers can successfully coexist with these USB hubs. … I suspect the real cause is that the Leap Motion runtime is tripping up a bug in the Apple drivers.
I (and others) were able to confirm that David’s fix does indeed work.
Apple, after passing on this information to them, merely marked my bug report closed as a duplicate.
This information was also passed onward to Anker, who was very grateful to have the information for answering support calls about it.
I can also confirm that after months of not having the LeapMotion driver installed, my favorite Anker 3.0 USB has been working like a champ.

Low Disk Space? It Might be iTunes.

I noticed that I was freakishly low in disk space, so used DaisyDisk to identify problem directories.

Turns out, a major offender was ~/Library/Application Support/MobilSync/Backup, which contained several years worth of iPhone and iPad backups.

As tempting as it might be, don’t just go deleting the directories inside it.

Instead… use iTunes, which will keep all the internal bookkeeping correct.

Open iTunes / Preferences…, then go to the Devices tab.  You’ll see the name of the backup, an icon if it’s encrypted, and a date/time stamp of when it was made.

Select one or more of the items and press the Delete Backup button.  But beware, they won’t really disappear until you press the OK button.  Closing the dialog via the red window button or pressing Cancel will not commit the change.

This is one of those rare cases where Apple dialogs don’t reflect current state, but intended ones.  And, considering this is the safer way of dealing with data, I’m all for it.

Java 8 on OS X Yosemite

I downloaded a recent copy of IntelliJ, only to discover when I went to open it, OS  Yosemite indicated I had no version of Java installed, and that I’d need to install an old version. The “More Info…” button took me to this page:  http://support.apple.com/kb/DL1572

…which didn’t load.  (UPDATE: This fixed the load issue.) Similar detailed install directions also ended up at the same broken page.

So, I attempted to download Java 8 directly from Oracle and  install install it.  The install worked fine, but IntelliJ 14 still did not open.  Same error message.

Here’s how I solved it.  Hop into terminal and do this:

$ cd "/Applications/IntelliJ IDEA 14.app/Contents"
$ cp Info.plist Info.plist-orig
   
$ vi Info.plist    # ... any text editor will do

Find the line that says 1.6* and change it to 1.8*.  Save your file, and now go open IntelliJ as normal.

This causes IntelliJ 14 to use Java JDK 8, and all is right with the world.

List iTunes Apps by Purchaser

I finally developed a way to list all your iTune applications by Purchaser, and even better, I can do it from the command line.

Last year I was helping a friend who has pretty poor internet connectivity upgrade his iPad to the latest version of iOS. To do this, I connected his iPad to my system, performed a backup in iTunes, then synchronized to update the operating system. This had the side effect of beaming some of his applications to my iTunes, which in order to continue, he authenticated against. His iPad updated great, and he was on his way.

Things on my machine seemed okay for a while, that was until some of the apps he purchased that I didn’t have wanted to update. Not having his password, I wasn’t able to update them, but even worse, Apple wasn’t announcing which apps needed updating with his account so I could simply delete them as I wanted to.

Instead, for over a year, I was greeted in iTunes by a numerical indicator saying I needed up update my apps, but when I went to do it, I was up to date. Every once in a while I’d recognize an app that I didn’t purchase (in a haystack of nearly 1,000 iPhone apps) and delete it. Only then did the number drop, but later rise again when some other app needed updating.

What I needed to do was list out all the apps by Purchaser.

One of things that really annoys me about Apple is that stuff that is trivial to implement, like putting an optional Purchaser column in iTunes, they don’t do. The feature is half heartedly there, though. Press Command-I for Info, and you can see the purchaser for an item.

Only now you have to click and inspect your whole app list. And with the number of applications I own, this doesn’t scale well at all.

Searching the web reveals that others are in a similar bind, and that Apple seems to really care less about the few handful of users with this problem. Like much on the Apple Support Site, it’s unhelpfully silent.

Frustrated, I decided to solve this problem once and for all.  Open Terminal and cut’n’paste the following in:

for f in ~/Music/iTunes/iTunes\ Media/Mobile\ Applications/*.ipa; \
 do (echo "$f" ; unzip -p "$f" "iTunesMetadata.plist" | \
 plutil -p - | egrep -i "\\"(itemName|artistName|AppleID)\\"" ) | \
 perl -e 'while (<>) { if (m!^/!) { chop; $fqn=$_; } if (m/"(.+)" => (".+")/) { $e{lc($1)}=$2; } } print "\\"${fqn}\\",$e{\\"itemname\\"},$e{\\"artistname\\"},$e{\\"appleid\\"}\n";'; \
done

Avoid Awesome Screenshot

I was getting some pretty bad web performance, with a little exploration I learned it was a Safari Extension that was doing things I did not like.

While doing some web page debugging, I noticed that a simple, static, html file was pulling in a ton of web resource. Most noticeably from a place called Superfish, and sucking in with it a good deal of JavaScript libraries. On. Every. Page. Load.

The culprit seems to be a Safari Extension called Awesome Screen Show 1.3.7 by Dilgo. However, the developer site isn’t coming up, and I’m not all that encouraged by what I see over at Superfish either.

Uninstall the extension, as here’s the overhead you’ll be saving:

Resetting MobileMe[ss]

Recently I noticed that my iPhone 4 started acting really, really badly. It wouldn’t unlock when a call was coming in, I couldn’t swipe between screens on the desktop, I couldn’t scroll within mail, clicks in the calendar weren’t working.

What was really strange is that it was Apple’s apps that weren’t working right, the regular store purchases were for the most part. And that’s when I noticed that things seemed to lock up hard when the phone was doing networking, even more so when going over AT&T’s instead of WiFi, though both did it.

What changed? MobileMe, specifically it’s calendars.

When I did the famous upgrade process on the MobileMe site, it never finished; I got the script is running too slow dialog. Later visits to the site threw up dialogs asking if I’d like to let Apple know there’s something wrong.

Eventually my calendar on the website came back, but there were dozens and dozens of repeating events that had happened years ago. I suspect this was a data log jam like no other, and worse, it wanted to push it all to my little phone. There appears to be no online web facility for bulk management. I tried on my laptop as well, but that too became borked as Apple synchronized its new calendar data to me.

I speculate, simply because the GUI would not respond to any input at this time, that perhaps Apple’s own applications may violate the rule about doing updates on the GUI thread. Either that, or that the message pump got so backed up, the GUI just couldn’t respond. How I wish there was an “abort” other than a violent shutdown.

While I’ve reset MobileMe in the past for other reasons, I ended up having to do a bit more.

By pulling the plug on the Internet and rebooting, I was able to come up in an isolated environment where I could shutdown all MobileMe syching and preferences. That let me peek at what was going on.

My thought: if I could recover the calendar on my laptop, I could push the valid subset back onto the MobileMe server.

With Calendar shutdown, my trip began by going to ~/Library/Calendars. Insider there are a bunch of GUID based folders. I deleted anything that had Cache in the name, as well as Calendar Sync Changes.

Then, I went through each folder. The Info.plist files had a Title entry followed by the name of the calendar. If it wasn’t a calendar I wanted, or was suspicious of, I deleted the GUID-looking folder completely.

If it did warrant keeping, I then went into the Events subfolder and looked to see how many .ics files there were. Casual browsing with Apple’s built-in previewer showed what the events were.

In my case, I found one folder related to MobileMe with over two thousand entries. Sure enough there were repeats of the same event, like a synchronization gone wrong, rather than a single event marked for repeation. So, I nuked that GUID-folder as well.

Sometimes there’d be no Events folder, but the .plist file looked like it was going to some external server. Sometimes there’s be an addition subfolder with a GUID-looking name, and maybe even an Inbox subfolder, these seemed to be more associated to GMail. I kept those.

When I restarted iCal, it knew something changed; it announced it was doing an upgrade for the folder formats, and then it presented me with some empty calendars that I knew I had gotten rid of. I deleted those from the application.

Once I was sure I could start and stop iCal without it hanging or requiring a Force Quit, I reset MobileMe.

Some fast tips here on how to do this.

  1. Hold down OPTION which clicking the Sync icon in your toolbar. You can reset the Sync services that way. Apple tells how.
  2. There’s a python script you’ll want to run at the Terminal, it’s inside that “Apple Tells How” link above.
  3. Finally, you want to deregister all your machines from MobileMe via the System Preferences, the machine you’re using last. When you do, it will offer to let you delete all the data off their servers.
  4. Then, when you re-register, you put yourself in Manual mode first, and use the Advanced… button to push all your data at their server first.
  5. Then, and only then, can you start putting things back to normal.

Andrew Robulack has a wonderful write-up about how to reset MobileMe. It’s an excellent guide.

All that said, apparently when I re-synced, there was some cruft left when I let iCal resync with MobileMe.

Then I read these horrible words from Apple: Resetting Syncing or SyncServices will not impact the new MobileMe Calendar and should not be used as a troubleshooting step for the new MobileMe Calendar.

Some support articles, after winding through their Support Express Lane wizard:

Files Gone on Drobo FS with OS X Lion? Get ’em back!

Using DroboFS and OSX Lion only to discover that your Drobo shares have no content!? Yikes! But fret not, you merely have a small corruption problem brought on by the firmware, and in moments you can force a rebuild of that database and all your files will be back safe, happy, and sound with no data loss. Here’s how.

I recently updated my DroboFS to firmware 1.2.0 and dashboard 2.0.3 when I switched to Lion, and while my volume mounted there was no data in it although the Drobo lights showed there was capacity, as did the Drobo Dashboard, and the health reports indicated everything was just fine.

I spoke with Drobo Tech Support that indicated this was a known problem they are actively addressing as high priority; the problem is with Lion and their firmware, and we can expect an updated firmware release.

What’s curious about this is that if one uses the Finder and mounts the Drobo drive with SMB, using smb://Drobo-FS/, the files are there. However afp://Drobo-FS.local/ and cifs://Drobo-FS.local/ mount but reveal nothing.

A detailed description of the problem is at the article entitled: “Missing” Data (AFP) and/or CNID DB Errors. This article then leads to a second one, but is only for the brave.

Using Dropbear (SSH) with Drobo FS to regenerate the AppleDB (CNID DB) has detailed steps for regenerating the apple database.

Walt’s More Verbose Directions

  1. Using the Drobo Dashboard login to your Drobo as Administrator.
  2. Unmount all shares.
  3. Under All Devices / Settings / Admin you’ll want to check the Enable DroboApps setting, which will mount a volume entitled DroboApps on your system.
  4. Download a copy of DropBear from the Drobo Apps page.
  5. Unzip this .zip file, resulting in instructions and a compressed dropbear.tgz file . Move the dropbear.tgz file to the root of the DroboApps directory.
  6. Restart the DroboFS by going to Capacity and Tools in the Dashboard, and selecting the Tools drop down on the right side, and selecting Restart. Or, just power off the unit physically for 20 seconds and then turn it back on.
  7. When Drobo restarts, go to the Dashboard and select All Devices / Settings… / Network. Note the IP address given to the device somewhere.
  8. From OS X’s Terminal enter the command ssh root@theIPaddressAbove
  9. The default password is root, unless you’ve used Dropbear before and followed the instructions within it.
  10. Enter the command ls /mnt/DroboFS/Shares to view a list of shares on the drive.
  11. Tech Support promises the following will not cause any data loss, but anytime you’re doing reconstruction you should always have a backup (if you don’t, question your backup policy), and double check before hitting return. For each share of yours listed above, enter the command: rm -r /mnt/DroboFS/Shares/yourShareNameHere/.AppleDB and press return. Note the period indicating it’s a hidden directory.
  12. Exit Terminal by entering exit.
  13. Using the Drobo Dashboard unmount all your shares, which should be just the DroboApps share at this point; this is under the All Devices / Shares and you just uncheck all the boxes.
  14. Restart the Drobo again (see above if you’ve already forgot how).
  15. And just as important restart any Macs connected to the Drobo.
  16. When the Drobo comes up, start the Dashboard, and test the mounts. They should be working.

1Password Woes with Safari Extension (fix)

Safari 5.1 and 1Password 3.7.x not playing well? Getting a Database error? No such column? Overview key missing? Try this… it worked for me.

I’ve been having a very nasty problem using Safari 5.1 with the latest 1Password software, while other browsers like Chrome and Firefox work just fine. I get a red ‘Problem with database’ message that says “Database error: no such column: overview_key”. It looks something like this:

1Password 3.7.1 build 21089 / Safari 5.1 (7534.48.3) / OS X Lion 10.7

Here’s how I fixed it
Start Safari, go to Safari / Preferences… / Extensions, and remove the 1Password extension if you have it. Close Safari. Re-open Safari. Close Safari using the magical sequence Command-Option-Q. Re-open Safari. Open 1Password and use it’s preferences panel to reinstall the Safari extension. Go back to Safari, and if you’re as lucky as I was, it’ll work.