On macOS, there’s a little more than just the order of bash scripts messing with your path. Here’s an updated diagram showing the inside scoop.
I was helping a friend diagnose why a strange script-like string was appearing in their PATH whenever they logged in or created a new bash shell window. (Note: Catalina now defaults to using zsh, not bash.)
Bash has a number of different configuration files it checks, and some of those the user may have chained together. So far, even the best resource on the matter I’ve found doesn’t tell the whole story.
On Apple machines, bash walks through these set of scripts and invokes a path modification tool, too.
macOS Bash Scripts and Path Modifications (validated with macOS Mojave v10.14.6 on 2020-04-08)
In a nutshell, /etc/profile calls a utility called path_helper, and it reads a list from two places the /etc/paths file and everything in the /etc/paths.d directory.
My friend had thought the /etc/paths file was a script and not a list, and years ago had erroneously put an export PATH= statement in there. Because it came last (for his setup), he ignored the problem.
Our false start to correct is was upon seeing an export statement in the PATH itself, we assumed it was a faulty quoting problem in a script, so primarily went looking in the script files on the left side of the graph …and wasted what felt like hours doing so.
Only after using Bash’s return statement to short-circuit script executions and using a blunt hammer of renaming scripts out of execution’s way, and the problem still persisted, did deeper exploration go down the system resources.
I hope this diagram helps get others out of trouble.
UPDATE 2020-07-08: Bob Rudis caught a typo (the text said /etc/profile, where it should have said /etc/paths); he also shared a link to GNU’s Bash Startup Files.
Well, I think I finally figured our a problem that’s been bugging me for quite a while.
Every once in a while, especially if I’m using an external drive that’s acting weird or even the now defunctLima, I’d occasionally see files that appeared grey in directory listings. You couldn’t rename them, you couldn’t display their contents (regardless of having size), etc.
Doing a ls -@Oaeln filename, showed nothing special.
Except… the date, which was always Jan 24, 1984, 3:00AM.
Ok, yes, that’s the Mac’s birthday — and some even call it an Easter Egg. But it’s likely an Easter Egg with a purpose.
The date appears as kind of a place holder, such as momentarily when a file is downloaded. And during that time, the system knows the file is incomplete, and this is just as good as any other way to mark it as such.
Turns out this also appears to be the way that corrupted files appear too.
For example, I synchronized some files with my Lima (which stores it on a USB disk somewhere), and if the synchronization process fails it’s supposed to pick up and try later. However, if that doesn’t happen, then partially transmitted data can reside there. And Lima’s going out of business was what led me to go try to pull all data, complete or not.
Attempting to inspect the contents of such files, such as on a Lima mount, appears to confirm this, resulting in a message of read error.
So. Grey files. It’s not a Happy Birthday to Us from Mac, but a date long past being used as a magic number.
Agressive and intrusive advertising techniques seem like they are the road to profitability, but there is a wake of evidence showing how it systematically destroys the very platforms intended to generate revenue.
The Goose That Laid The Golden Egg
If revenues are the golden eggs, and media/apps/websites the goose, then aggressive, intrusive advertising practices are the hatchet that killed the goose.
The evidence is everywhere.
Broadcast television used to have commercials between shows, then during shows, and eventually digressed to the point where the frequency of continual viewing interruption made it so that certain channels weren’t worth watching anymore. And, oddly enough, to make up for lost revenues, those channels increased the amount of advertising.
From this cable television was born, based on the the idea that if the consumer paid for content, the majority of it would be advertising-free. Although within not much time, advertising leaked in again, and eventually people were back to the old model that used to work: advertising between shows; only this time paying for the privilege.
Then a new consumer product arrived on the market, TiVo, which helped customers skip past commercials; but this early feature was all-to-quickly neutered to fast-forward-only so you’d still have the opportunity to at least see the commercials in the event you wanted to watch one.
While more modern TiVo units now have reintroduced the skip commercial ability, the once clean graphical interface now is littered with dynamic menu choices for products, shows, and offers that appear when you pause to view a frame or try to avoid the commercials.
In response to all this, broadcasters started injecting banners and overlay graphics during the show so you “had” to watch, consuming screen real-estate to announce other shows, products, or news changing the entertainment into a dashboard. Meanwhile producers started injecting product placements into actual content. In the process it means all the display pixels you paid for aren’t being used effectively, it’s distracting to watch, and can lead to some rather embarrassing sexual mishaps between show and ad for the network.
Overall, entertainment quality has gone down, presentation quality has also gone done, consumer costs have gone up, and it’s no wonder we now see a whole generation of cord cutters who have found other means to go straight to content.
The DVD market suffered a similar fate. There was a time when you bought a DVD, you got the movie and possibly special bonus material, and that was all. DVDs started inserting trailers and other commercials at the start of the movie. For a while, consumers would skip to the actual content. Then that capability was removed, and consumers fast-forwarded, which itself was challenged by the now famous “option not available” notice.
In response, streaming services appeared and consumers stopped stocking their media collections at the rate they once had. But now we see pay-for streaming media starting to inject previews and advertising in front of shows. We know where this leads: service termination and few barriers to exiting the moment a new technology surfaces. And it will, it always does.
YouTube’s success led to advertising, not surprisingly at the beginning of a video. And as viewers skipped as soon as possible, a new trend appeared: advertising pop ups over the video. (See a trend?) So while watching content, one is constantly distracted, having to dismiss dialogs; while dealing with that, one isn’t digesting the content. And like we’ve seen before, this leads to platform abandonment.
Other competing video streaming services have started to give rise. It’s surprising, but consumers will often take less content and less quality in exchange for less advertising. YouTube Red emerged as an advertising-free subscription service, just as the cable companies did before. We know where that goes next.
Netflix, Amazon Prime, and HBO were quick to discover that developing high-quality original content would lure viewers, but as more familiar advertising tendencies start to surface, enthusiasm for these platforms dwindles. At some point one starts to ask is the subscription cost justifying actual usage patterns, as consumers are slowly being pushed away. Consumers are becoming more picky while businesses look for augmenting recurring revenue.
Homes all used to have land lines, and as cell phones became cheaper and popular, house holds got them in addition to their highly reliable land lanes. However, with telemarketers invading at all hours, land lines became a dumping ground for calls we didn’t want to take. Services like Caller ID appeared, as well as the national Do Not Call list. However, due to no security in how caller ID works, numbers can be spoofed. Meanwhile, law enforcement has been ineffective at dealing to those breaking the law regarding the no call list, and advertisers for the most part ignore it. Politicians have self-servingly made themselves exempted and are often some of the worse offenders. The result, consumers have disconnected their land lines and are now mostly cell based.
However, it doesn’t take much any more for a new cell phone number to get out into the wild and make it into a robo-caller list. The net result, if someone doesn’t recognize a number, they tend not to pick up. Our cell phones are becoming spam dumps, and once direct access to someone is now fading. Meanwhile, advertisers having grown wise that people are letting content drop to voice mail, and so leave automated recorded messages, which at the end of the sales pitch, one might have the opportunity to be removed from that list (which never gets pressed), although that merely confirms the number works for other endeavors.
This has in turn led to a shift in how we even use phones to communicate, and text messaging is rapidly replacing phone calls. Even now, we’re starting to see advertising start to break into text messaging, initiated by none other than the very carriers of the service.
The home mailbox, originally intended for correspondences and packages, soon became the primary methodology of mass advertising. Consider the signal to noise ratio of your actual mailbox content between what you want verses unsolicited advertising that you have to sort through after each trip to the mail box. The difference in pile size is growing.
With so much volume introducing delay, it’s no wonder that there isn’t a measure of care during processing, and goods often arrive damaged, which in turn devalues most people’s impression of the U.S. Postal Service, their trust in it, and consequently their use.
To avoid postal mail, most people have switched to email and electronic billing; aside from the occasional package from Amazon, there’s relatively little reason to check a postal mailbox on a daily basis anymore.
We will never see postal mail delivering more than once a day as it used to not that long ago with morning and evening deliveries. And because the service is become so bad and fewer people using it, its led at least to discussions of not delivering on weekends.
Services once reserved for over night or over sized deliveries have taken on more casual load once entrusted to the post office. At the moment, advertisers find the service too expensive, so the rate structure is keeping things in check. Should that change, we know what will happen.
Email used to be a way to send messages to one another, and as the population with email addresses grew, advertisers saw it ripe for exploitation. It’s lack of encryption meant that anonymous senders could forge false from addresses and perform automated bulk mailing on a scale of unprecedented magnitude. Worse yet for the unaware receiver, the spammer can often detect if the message was read and use that information to cultivate a list of active emails. If the sender could employ social engineering successfully, it was actually possible to install advertising software on the receiver’s machine — often without them knowing. And best of all for the advertiser, under this model, the receiver pays to receive and store the message!
At this point, it is estimated that 80% of internet traffic is spam. Not emails. Internet traffic. That’s a lot of bandwidth that you’re paying for with 80% of it being used to help other people advertise at little to no cost to them. And the problem is only getting worse.
Despite spam filters, firewalls, and anti-virus software email has gotten so bad that most people have several email accounts, and use one for highly likely spam and another for personal communication. However, the moment one of our contacts shares the personal email address with some bulk email list, it’s relatively little time before it gets compromised and we’re on to another new email address, where the process starts all over, just like folks eventually changed their phone numbers for a little peace.
Websites, attempting to capitalize on traffic, started presenting advertising on the sides of their pages, and later interlaced within the articles themselves. Screaming for visual attention, ads became more gaudy, employed blinking, animation, sound and video, and pop-up dialogs, pop-under dialogs, which soon overwhelmed the value content. Visitors responded with ad blocking tools.
Websites responded by waiting a little bit and then putting a pop-over that blocks out the content below, this obnoxious behavior usually leads to site abandonment as it’s just not worth it. Sometimes, abandonment can’t be avoided: say you’re using the Flipboard app to get your news on an iPhone, it takes you the ScienceNews website, they invoke a non-mobile friendly pop-up, and you can’t see the content nor can you dismiss it. Such websites become dead to visitors.
Admittedly, this may be some interaction between 1Blocker and their advertising code, but seeing that 1Blocker can dramatically speed up load times -and- save on carrier data usage, between it and visiting such a site, 1Blocker wins hands down.
A number of websites have taken the approach that if you don’t let the advertising through, they won’t show the content. Sounds reasonable until you realize they aren’t vetting the advertising content that’s being shown. And malicious advertisers are exploiting that. The consequence of opening a rectangle of any-old content makes solid brand names incidental parties into delivering malware straight to your system. GRC provides a recent list of sites, who just by visiting, have been found to infect machines via advertising services: The New York Times, the BBC, MSN, AOL, Xfinity.com, NFL.com, Realtor.com, TheWeatherNetwork.com, TheHill.com, Newsweek.com, Answers.com, ZeroHedge, and InfoLinks.
A more disturbing practice involves hardware manufactures installing advertising malware in the hardware to make a little extra profit per unit sold. We’ve seen Dell and Gateway load up cruft on new Windows systems, but laptop maker Lenovo built mechanisms into their systems to bypass the end-user’s ability to keep the system clean, even on a pristine reinstall. This practice took them from being one of the most sought after name brands to one to be avoided, all thanks to aggressive and intrusive advertising practices.
Likewise, many software installer products use deceptive wording to get Ask and Yahoo toolbars installed, all for their own piece of the pie. Both search engines, once considered at least respectable, have lost significant ground to other players from both reputation damage and public mistrust, as well as firewall and ad-blocking rules to prevent installation of such nonsense. Browser owners are encouraged to remove Flash, Silverlight, QuickTime, and Java extensions from their browsers, as this will greatly reduce the attack vector as well as many malicious “advertisement” payloads.
iPhone apps have the ability to show a small banner on occasion or produce a pop-up in the middle of the app. Not only are these practices distracting to the end user, but are reinforced because the more they’re shown, the more revenue made. That makes advertising apps seemingly more profitable than fixed priced apps, which in turn is driving down the amount a professional app can earn.
All of this is changing consumer behavior, and we see app abandonment on the rise, diminished purchasing habits especially in impulse buying, and a noticeable dip in quality and creativity in the available selection.
Even Windows 10 has started to put advertising in its Start Menu and screen saver, and that’s not even counting the aggressive multi-gig push, annoying notifications, deceptive wording on dialog boxes to get you to install it. Such things can not be helping platform adoption.
The evidence is everywhere, aggressive and intrusive advertising practices actually hurts the very distribution platforms that are trying to benefit from it.
We need responsible advertising, not necessarily no advertising.
The Pattern is Clear
Worthwhile content is created, a distribution channel is developed for it, the audience grows. At some point, someone gets the idea that a little advertising will generate a little bit of revenue, and in the short-term it works.
Then the fallacy of if-a-little’s-good—then-a-lot’s-better kicks in, and either the distribution channel gets saturated with ads, the content gets damaged, or the ratio between the two gets so disjoint that the over all value tanks.
Desperation and greed can push the state of things beyond recovery, and what was once a sustainable system no longer has the following it once did, and usually by this time the audience has moved on to other technologies.
Synopsis
It appears we’re great at repeating history, but neglecting consequences.
Consumers are willing to pay for great content without advertising.
Consumers do not want to be distracted by anything else while engaging with content.
Advertising requires a balance, more isn’t better. It must be done responsibly.
Intrusive and aggressive advertising leads to platform abandonment.
Has your USB 3.0 stopped working without explanation on your Mac? Here’s how to fix it.
I use a lot of external storage and it has been hard to find a USB hub that is fast, connects all my devices at once, and when using a device doesn’t drop other devices connected to it.
I finally found one that’s rock solid; it’s the Anker 13-Port USB 3.0 and it does everything I ever wanted.
Things were good until mid-January 2016 when the device started malfunctioning in strange ways. The first three ports did not recognize any device I put on it. The other ports sometimes worked and sometimes didn’t. Reliability went out the window and I was forced to stop using it. My guess was that something burned out. I went so far as to buy another smaller USB 3.0 hub, and well, it didn’t work either — so much so I ended up returning it. I really wanted this hub working.
Curious, I handed the broken device to an electrical engineer and asked him if he could ascertain what was wrong with it. He took it apart, did diagnostic tests, saw nothing wrong, tried it on his computer, it worked fine, and handed it back to me fixed as just a mystery. However the story doesn’t stop there.
The Impossible Behavior
When I connected the device back up to my Mac, it behaved exactly the same way as it did before. I, of course, tried all my Mac’s USB ports. I even tried a completely different Mac. Identical failures.
So sure the device was working, my electrical engineer friend pulled out his Microsoft Surface Tablet, connected the hub, and instantly it worked.
We put it back on my Macs, with the same devices that just worked, it failed. Back to the Surface, it worked. Back to the Mac, it failed. In short, it was an electrical engineer’s WTF-nightmare.
The Common Denominator and Other Clues
At this point the problem was clearly related to the Mac. More over, it used to work just fine, at least until mid-January. What happened in mid-January? El Capitan 10.11.3.
Both Macs were running El Capitan 10.11.3.
As a general rule, with Apple, the first generation hardware products have flaws, and the operating systems versions don’t usually get all the kinks out until version x.x.4 is released. This threw immediate suspicion on the operating system, which meant it was time to check if other folks were having similar issues.
Reset the SMC. (For a MacBook Pro it’s the Shift-Option-Command-PowerOnButton.)
When the machine rebooted the USB hub behaved just like it used to. Problem solved.
UPDATE (21-Mar-2016): With the introduction of El Capitan 10.11.4, it rebroke the USB 3.0 capabilities again. The Console reports:
3/23/16 4:32:20.000 PM kernel[0]: 000227.351907 AppleUSB30Hub@14400000: AppleUSBHub::start: failed to set configuration with 0xe00002eb 3/23/16 4:32:21.000 PM kernel[0]: 000228.290970 AppleUSB30Hub@14400000: AppleUSB30Hub::start: failed to set hub depth 0 (0xe0005000)
So far, performing the above steps are not working.
OTHERS ARE HAVING IT TOO: Often the problem manifests as if the USB device, or something connected to it, is no longer working or has inadequate power, or is no longer detected by the host system.
The Hackintosh Community that’s even got an explanation and workaround. (A well deserved shout-out to TORLEY in the comments below for this one.)
Try your device on an older operating system (ideally the same hardware if you can), a Windows box, a Linux box, or even a Raspberry Pi — you’ll see the USB device works properly there.
YOU CAN HELP: It appears Apple may not know about the problem.
Please be kind when reporting issue, as these are the people who can help you. Give them technical details and model information to help them track it down.
UPDATE (06-May-2016): Apple has acknowledged issue 26102223 in their system and have asked for more information; I’m forwarding it to them.
*** Between MAY and OCT, this blog suffered a large data-loss pertaining to the comments left by visitors. I wish I had the original post by David Ryskalczyk reporting his extensive solution. Here’s what I can manually recreate.
… I figured that maybe this was a software issue. I proceeded to clean install 10.12 on a USB drive — no issues; then 10.11 on a USB drive — *also* no issues! Seems to be software. From there I started isolating things — first with kexts, which turned out to be inconclusive, then with daemons (looking in /Library/LaunchDaemons and LaunchAgents to see what was installed and running). After removing a bunch of stuff I did additional testing and confirmation to figure out exactly what was causing the problem, and sure enough, it was the Leap Motion runtime!
…
Hopefully this can be fixed so the Leap Motion drivers can successfully coexist with these USB hubs. … I suspect the real cause is that the Leap Motion runtime is tripping up a bug in the Apple drivers.
I (and others) were able to confirm that David’s fix does indeed work.
Apple, after passing on this information to them, merely marked my bug report closed as a duplicate.
This information was also passed onward to Anker, who was very grateful to have the information for answering support calls about it.
I can also confirm that after months of not having the LeapMotion driver installed, my favorite Anker 3.0 USB has been working like a champ.
I noticed that I was freakishly low in disk space, so used DaisyDisk to identify problem directories.
Turns out, a major offender was ~/Library/Application Support/MobilSync/Backup, which contained several years worth of iPhone and iPad backups.
As tempting as it might be, don’t just go deleting the directories inside it.
Instead… use iTunes, which will keep all the internal bookkeeping correct.
Open iTunes / Preferences…, then go to the Devices tab. You’ll see the name of the backup, an icon if it’s encrypted, and a date/time stamp of when it was made.
Select one or more of the items and press the Delete Backup button. But beware, they won’t really disappear until you press the OK button. Closing the dialog via the red window button or pressing Cancel will not commit the change.
This is one of those rare cases where Apple dialogs don’t reflect current state, but intended ones. And, considering this is the safer way of dealing with data, I’m all for it.
After upgrading to the latest HP “Universal” drivers, our HP LaserJet CP4525 printers started reporting that they were “Unable to store job at printer”, that they ran out of memory, and were “Unable to collate job at printer”. The error messages suggested the disk wasn’t present, was full, that I needed to delete files, add RAM, or even a very expensive EIO hard disk. Didn’t get much obvious help from the HP Universal Print Driver Systems Administrator’s Guide either.
I tried adding RAM to the full capacity. That didn’t help. Here’s how I ended up fixing the problem, thanks to piecing together dozens of support forum posts for different printer models and a little experimentation.
Go to Start / Devices and Printers (yours may say Printers and Faxes).
Right click the Printer Icon of the printer giving you problems and click Printer Properties (not plain old properties that’s at the bottom).
While on the General table, click the Change Properties button near the bottom left (otherwise you’ll get a read-only mode of grey items).
Change over to the Device Settings tab.
Ensure that the Mopier Mode is set to Disabled.
Ensure that the Job Storage is also set to Disabled.
Click Apply, ideally to see there were no errors in doing this.
I downloaded a recent copy of IntelliJ, only to discover when I went to open it, OS Yosemite indicated I had no version of Java installed, and that I’d need to install an old version. The “More Info…” button took me to this page: http://support.apple.com/kb/DL1572
So, I attempted to download Java 8 directly from Oracle and install install it. The install worked fine, but IntelliJ 14 still did not open. Same error message.
Here’s how I solved it. Hop into terminal and do this:
$ cd "/Applications/IntelliJ IDEA 14.app/Contents"
$ cp Info.plist Info.plist-orig
$ vi Info.plist # ... any text editor will do
Find the line that says 1.6* and change it to 1.8*. Save your file, and now go open IntelliJ as normal.
This causes IntelliJ 14 to use Java JDK 8, and all is right with the world.
It’s the right blend of detailed explanation combined with cookbook recipes to get going.
D3 is JavaScript library that separates data from presentation, but does so in a really cool manner. When the underlying data is updated, the components on the screen that visualize it can be made to update to reflect it. This can lead to some pretty amazing transitions and interactivity with surprisingly little code.
D3 looks at the data you have visualized already and the data you want to visualize and divides them into three piles: New Data, Updated Data, and No Longer Present Data. Transformations can be done for each set, for instance allowing new data to fade in, updated data to morph, and removed data to fade away. D3 is focused primarily on just this task, so actual visualizations are up to the coder.
However, because D3’s nature allows each element to only focus on the part of the data it needs to, and to substitute that content into the HTML, CSS, and/or SVG that’s needed, it provides a surprising amount of flexibility and features and does so with impressive speed and efficiency.
The Data Visualization with D3.js Cookbook explains the above concepts with some well formed examples, and then proceeds to demonstrate how to make bar charts, pie charts, line charts, and other more complex multi-dimensional visualizations and add interactive interfaces to them. This is where the book absolutely shines!
Each example is carefully constructed and compact, serving both as a learning tool and a starting point for your own code. As such, I’d highly recommend the book and convey that it’s very approachable, even to those with a limited JavaScript background.
I would recommend some degree of caution however, in particular with the Kindle book. This is not a fault of Packt Publishing, but with Amazon. And I’ve spoken to them at length about it. The Kindle viewer is intended of rendering pages of readable text to humans in a nicely formatted manner. However, it is terrible (as of the time of this writing) as to maintaining code formatting and spacing, which is absolutely essential for any cut’n’paste operations. Be prepared to copy code by hand, and if you’re going to go that far, perhaps the paperback version is more to your liking.
That said, I did happen to notice a few errata in the examples; nothing major, nothing you can’t move past, and nothing that isn’t rectified by going onto GitHub for the updates.
All in all, after having gone through a number of books about D3.js, this one has been my favorite and the most useful.
As I was leaving the water fountain and returning to my office, a young woman on crutches popped out of an adjoining hallway in front of me. Normally I, and others, would tend to step around — and having been on crutches myself, I can say that I didn’t feel the least bit offended. But, I had read that sometimes this bothers people, so I figured I’d just catch up and slow my pace.
That’s when she noticed me and in the brightest voice said, “Good Morning! How are you doing?”
Surprised, I equally met her energetic tone and responded “Good Morning to you too! I’m doing well, thank you for asking! How are you doing?”
She sighed, “Not that well, actually.” And I took it to mean her foot, so I opened the conversation.
“Oh no! What happened to your foot? Did you slip on the ice?”
She paused and then explained, “No. I had a stroke when I was a baby. I’m like this normally.”
An actual conversation that happened right before a meeting started:
“Hey Walt, how ya doing?”
“Sad, depressed, …beaten.”
(chuckle) “Tell us how you really feel!”
(short pause) “Surrounded by dementors.”
(awkward silence — then the meeting began)
