Overcoming Writer’s Block

Recently I discovered that one of my favorite authors put out a book on writing, called Weinberg on Writing: the Field Stone Method.

Well, on a lark, I decided to purchase the book from Amazon, and I have to say, not only have I not been able to put the book down, but it has truly inspired me about writing in ways that no English book ever could.

Basically, Weinberg draws parallels of building a wall with field stones (no mortar) to that of writing. He starts by stating that how we read, sequentially, is not necessarily how we write. Additionally, when we see today’s electronic medium, full of indexes and hyperlinks, this is merely presentation, and, again, has no bearing on the writing process.

Instead, he points back to a time in his life when he became addicted to morphine from some nasty surgeries and broke the addiction. He explains that addition is a very clever and evil process. It requires that you do something in the short term that makes you feel better, but with the side effect that it actually makes you feel worse in the long term. So, you do the activity again, and you feel better, but then after it passes, you feel even worse. Quickly you run into a terrible spiral.

However, smart and creative people can break that cycle. How so? By finding something else that makes them happy. Instead, they don’t end up with feeling worse; they simply build upon repeated successes.

He then discusses how when he’s faced with a project that’s not going well, or a deadline, or even writer’s block, that he’d distract himself (“to unblock himself”) by going for a walk, drinking a beer, having sex, watching television, cleaning out the garage, whatever. Only the problem was, when the break was over, he’d be in even a worse pickle, and have even less time. …sounds an awful lot like addition, doesn’t it?

So, he started looking for a smart and creative way to break the cycle, and he now claims he never gets writer’s block. Writers block comes from three things: not having enough ideas, having too any ideas, or having just the right amount. If there’s not enough, he goes “looking for flag stones”. If there’s too much, he organizes them. If things are just right, he polishes and shapes them. Volia, writer’s block is gone.

But what is this “looking for flagstones” he’s talking about? Turns out when you’re building a wall, like a retaining wall out of stone, the quick “city” way is to purchase a ton of rocks. However, in the country, you go out into fields, looking for stones that are the right size, shape, and color. You collect them, and after a number of years, you have your wall.

Problem is, if you just flat out collect for the wall, you run into two problems. One is that as time presses on, it gets harder and harder to find wall-specific stones. You can’t just say “I’m going to find five today.” That doesn’t work. Second, sometimes you come across other stones that have practical value or some emotional connection, but you have no immediate use for them. He says to collect those and start different piles. And with this advice, he points out with modern day cheap storage, it’s possible to collect a lot of ideas and then organize them later. Also, he spends considerable time telling how to capture ideas that might get lost such as when you’re dreaming or in a social context where it doesn’t seem appropriate at first.

Weinberg points out that a “good stone” is an idea that moves you strongly emotionally. When a passage stands out, when a well worded sentence is found, when a thought sticks out, when someone says something, when you get a flash of insight or perspective on the world, and so forth, these are good stones. Weinberg has made a career out of collecting ideas, and with it, he’s never out of material or inspiration for writing. He encourages the reader to do the same, through a series of simple observational exercises he applies in his writing classes, and I have to say… it works. My own set of blogs have been stepping up in the number of entries; I’m seeing far more reader email and comments than ever before.

Weinberg pointed out that upon a friend of his giving a review to another author’s book, praise was given that it was a gold mine. And when Wienberg asked his close friend why he never got that kind of praise. Weinberg’s friend thought and said a gold mind is something where you have to move a lot of earth, and if you’re lucky, you get a nugget. Then he proceeded to say that Weinberg’s books were more like coal mines. Continuing, that every shovel full is valuable.

And, I’d have to agree.

Weinberg draws on tons of deep and clever concepts, thoughts, and expressions, weaving them into folds of comedy and information, conveying his points effortlessly and concisely. He illustrates how the “Fieldstone Method” works for fiction, non-fiction, and technical material. And, it really does.

This simple blog entry represents my personal synopsis of the first five chapters in a twenty chapter book. I strongly encourage you to get your hands on it and read it. It will take you no time what-so-ever, as it’s less than 200 pages and quite thin. In no way does it discuss grammar, technicalities of the English language, or pseudo-science positive thinking crap.

Wienberg in the course of 40 years has produced 40 well known books and over ten times that in articles. His students are cranking out books and articles. His methodology is quite an eye opener, especially if you want to become more prolific.

Hibernate: Duplicate Mapping and Imports

<GEEK BLOG ENTRY>
I ran into a very frustrating problem this evening, causing me to stay much later than I had intended, and to miss out on some fun socializing event that I was looking forward to. Unfortunately, there was little to no useful information on the Internet as Google was coming up with few and useless results.

I hope this post saves some poor soul from the same fate.

The Problem


I’m using the Hibernate library for persistence with a JBoss EJB using JPA. My code is sprinkled with annotations, my hibernate.cfg.xml file is clean, and I have no *.hbm.xml files. My code compiles. And it runs.

However.

When I try to access something that uses the Hibernate library, I get an odd message about “Duplicate Collection Role Mapping“.

The class in question contained a Set interface and a HashSet implementation for a member.

So, I commented out this container and tried again, hoping to simplify the problem.

This time I was greeted with a “[Mappings] duplicate import” and a “DuplicateMappingException: Duplicate class/entity mapping” set of error messages.

The only related web pages was a handful of archive with people asking similar questions in various online forums.

Almost always these fell into one of three responses:

  1. You’ve got a problem with the mapping element in your hibernate config file.
  2. You’ve got annotations and and class.hbm.xml file doing something wrong.
  3. This is an old bug in JBoss.

None of the symptoms existed in my case.

Here’s How I Solved It


Turns out that Hibernate makes the recommendation that you build a HibernateUtil helper class. Inside it, you’re supposed to make a singleton of the SessionFactory (and in the case of JBoss, you should use JNDI).

A co-worker had refactored the AnnotationConfiguration() to store a single copy, however, the routine that returned it happened to call .configure() on it before returning it each time. An honest mistake, which got integrated silently into my code when I pulled the latest version from version control.

Because .configure() was being called twice, to Hibernate it did look as if I had duplicate mapping directives in my hibernate config file.

Correcting the HibernateUtil method, which handled setting up and returning the AnnotationConfiguration solved the problem.

</GEEK BLOG ENTRY>

The Best Photography Books Ever on Light

Without exception, two books leap to the front of my personal library when it comes to Photography.

Light: Science and Magic
Light: Science & Magic
Crime Scene Photography
Crime Scene Photography

Most photography books explain general principles of photography, how the camera works, and, if lucky, perhaps a simplified discussion of optics and proper metering. By the time you really understand the relationship between ISO, Shutter Speed, and Aperture, and are capable of shooting in Manual mode, you’re not done — you’ve just scratched the surface. The next subject you need to tackle is light, and it is light that allows you to get those really dramatic and interesting shots. And if you think because you own a 50mm f/1.4 lens, you know it all; think again.

Light: Science & Magic is a masterful excursion into the role that light plays with photography. It covers light sources, reflection, and angles, explaining why surfaces look the way they do. Certain objects are hard to photograph, such as glass or white subjects on white backgrounds. Either detail is lost or everything comes out gray. This resource shows how to use light to solve those hard problems. The section on diffused, direct, and glare reflections is worth the price of admission alone. Camera placement, light placement, and gobos, combined with the right metering technique, will yield stunning images. It covers methodologies of lighting portraits in fascinating ways. The book is littered with tips and tricks from front to back. It’s odd to find a book where there’s solid, approachable, directly applicable material on every page, but this book does it.

The next book is quite unexpected, Crime Scene Photography. This book goes far deeper into explaining the workings of photography, delving into the mysteries of optics, proper exposure in bad lighting conditions, and the clever use of filters. It clearly explains inverse square laws, plays with subtle differences between intensity and distance, unravels why rules of thumb actually work, shows how to get the most from a flash. And all these topics roll up to support how to draw out the details you want to capture, including from fluorescent and infrared sources. There’s tons of information to convey size and play with perspective, deal with underwater situations, and how to digitally correct severe problems when you can’t retake the photo. The book is chalked full of examples.

Frankly, if I could only own two books on photography, it would be these. I read the again, and again, and again.

Walt gives these books, two thumbs up!

Rebuilding Spotlight’s Index on OS X (Manually)

Got problems rebuilding the Spotlight index on OS X or all you’re getting is an empty list with no files found? Here’s how I reindexed my system using just the Terminal and no special software – turns out there was one small thing that had gotten in the way.

After doing a number of disk clean up and optimizations, I found myself in the circumstance of OS X’s spotlight returning no results. Whether I searched for a keyword in Mail, or by Spotlight using Command-Space, I got no results backs – just an empty list for my troubles.

It turns out there’s a neat utility out there called Rebuild Spotlight Index 2.7 that does all the grunt work for you. Problem is, it didn’t work for me.

What’s going on is actually fairly trivial, and it’s possible to simply do everything via the command line.

The metadata utilities need to run as root, so to see what your drive is up to, you’d enter something like: sudo mdutil -s /

This shows the status on the root volume.

To turn indexing on for a volume, you enter: sudo mdutil -i on /

And, to force Spotlight to rebuild its index, you simply erase the master copy of the metadata stores on the volume like this: sudo mdutil -E /

However, while I did all this, Spotlight was still not building the indexed for me.

Here’s how I solved it, using just the Terminal.
First, I wanted to see the schema file, so I printed it out using to the standard input using: sudo mdimport -X

At the bottom of the schema listing, I say a reference to a schemaLocation, and took a shot in the dark that perhaps that Spotlight’s index rebuilding needed to check data against its schema before it would start. To do that, it might need network access, if not back to the local machine.

And, for good measure, I went to check the date/timestamp on the Spotlight directory using: sudo ls -la /.Spotlight*

While most of the files had the timestamp of when I tried to delete the index, not all the files had the current date and time. Additionally, the file sizes were not growing, a good indication the index was not being rebuilt.

Thinking to myself, “what could be causing network traffic, even internally, not to be working”, I realized that I had just rebooted and PeerGuardian2 was currently active and blocking traffic. This is a great tool for blocking malware and unwanted network visitors, but occasionally it gets in the way. So, I turned it off.

Then, I did the following commands to ensure indexing was on, the spotlight metastore was really gone, and that I wanted it rebuilt:
sudo mdutil -i on /
rm -rf /.Spotlight*
sudo mdutil -E /

The moment I did the last command, this time the system sprung to life, the directory /.Spotlight-V100 was created, and the files inside it were growing quickly. Spotlight on the toolbar showed a progress bar, indicating the system would be done indexing in a bit. The big difference? I turned off the network traffic blocker for a moment.

Ubuntu and Parallels Rescue Broken XP

How I recovered a badly damaged XP box with severe Windows Update problems using Ubuntu and Parallels for OS X.

Normally, I don’t provide XP support, however, because I was the one who recommended the owner perform a Windows Update that precipitated the total incapacitation of the machine, I felt a slight guilty streak of obligation.

Because of the horrible reputation of Windows Genuine Advantage disabling legal installations, the owner of the box disabled all Windows Updates for fear his system would become disabled and he’d lose his data. As such, when I recommended keeping the system patched, there were well over 60+ patches to start with.

Frozen XP DesktopProblem was, one of those patches was for the NVIDIA GeForce Ti 4200 graphics card, and during the installation process, when the Microsoft Version was applied, the machine froze, requiring a manual reboot via the reset switch.

Naturally, after a forced shutdown one should invoke a check disk. However something insidious occurred. Explorer, and I don’t mean Internet Explorer, no I mean Explorer – the GUI shell, would lock up shortly after login. The start menu would go dead, icons didn’t function, start/run couldn’t invoke programs, applications invoked from the command line wouldn’t work, Internet Explorer wouldn’t even start, and Windows Update did nothing. Even Ctrl-Alt-Del wouldn’t work, as the Task Manager couldn’t start. Nor could the user logout or shutdown the machine. Things were bad. It was like the desktop was there, but the underlying services that made it function were dead.

I’ve had easier recoveries from the blue screen of death. If you can get past that, usually you got yourself a working system. In this case, the system would boot, and even allow a login, but once there, the interface wouldn’t function.

Of course you’d think booting and reverting to the last known good configuration would help. It didn’t. Safe mode was equally hosed. Anything past the login prompt rendered the machine in a frozen state, popping up a message about a Windows General Services failing, with an option to report the problem to Microsoft.

That’s the state of the machine as I received it prior to repair.

Here’s how I fixed it.

The detail message reported that the offending file as WUAUENG.DLL. A quick Google search showed this was the Windows Update module. It seems between going from Windows Update to Microsoft Update, the DLL got corrupted. As Windows booted after login, it accessed the DLL, and the system froze.

My goal was to replace at least this file from a working system. Problem was, I was in a catch-22. I couldn’t access the broken system, and if it was possible, the files would be in use by the operating system anyhow.

I downloaded Ubuntu and burned it to a CD using OS X. I then booted off the live CD on the broken machine, however while it could see the NTFS volume, it couldn’t write to it.

So, I enabled all the repositories by going to System / Software Sources, making sure Universe and Multi-verse were included. Then I opened up the terminal and entered sudo apt-get install ntfs-config, and installed the package that allowed writing to NTFS drives.

I added root to the fuse group, and then went to Applications / System Tools / NTFS Configuration Tool. It was quick to tell me I needed to run ntfs /dev/hda1, which fixed the volume and set it to check the disk on boot.

I shutdown Ubuntu, booted Windows, which caused a check disk, and when I finally got to the login prompt, shutdown again without ever logging in.

I booted back off the Ubuntu CD, did the same trick as before with the repositories and installation of the NTFS driver, and this time was able to mount the drive as writable.

I went to the WINDOWS\System32 directory, and found the following files, to which I renamed them, appending .old to their extension for the purposes of a backup: wuaueng.dll, wuaueng.dll.mui, and wuaueng1.dll.

Then I booted Parallels on OS X, brought up a copy of XP, went to its C:\WINDOWS\System32 directory, and copied those three files to a USB stick. I unmounted the USB stick and shutdown Parallels.

With Ubuntu still running on the broken machine, I plugged in the USB stick, which instantly appeared on the desktop, and copied over three files to the broken machine’s system32 directory.

I then shutdown Ubuntu, removed the USB stick and CD, and booted into Windows. The error message was gone, but it was obvious things were still fragile.

Back on OS X, I downloaded Windows XP Service Pack 2, burned it to CD, and stuck it in the broken machine, executing it. A bit later, it finished and I rebooted.

I was suddenly able to run Windows Update again, and that downloaded 40+ updates, effectively jump starting the process by grabbing only the critical updates. In a rise-lather-repeat cycle, I did this until all the critical updates were obtained. Then I did the same with the optional software.

Each time I came in from a mandatory reboot, I made a system restore checkpoint.

Just to confirm it was the NVIDIA driver, I downloaded just that option from Microsoft, and the machine locked up. Which, to get out of I had to hit the reset button, screwing up the disk again. No problem though, I booted, holding down F8, and booted to the last known good configuration. When it came up, I right clicked properties on the C: drive, and forced a check disk, rebooting. The machine came up fine.

Going over to NVIDIA’s site, it was a trivial matter to download the latest driver for the GeForce 4200 card, and unsurprisingly, it worked without incident.

Ubuntu saved the day for being able to repair and manipulate the NTFS volume, while Parallels made it possible to see what needed fixing, where it went, and a working copy without having to have a second dedicated Windows box.

A recovery solution wouldn’t have been possible with an disc of an OEM version of XP alone. Honestly, I don’t know why users put up with this, or how Microsoft can sleep at night.

The recover process, non-stop, took from 10am – 7pm straight. No breaks. No food. No stalling. That’s nine hours of my life I’m never getting back.

Top Ten Bad EMail Habits

With over ten years of email to sample from, here are the top ten bad habits committed by email senders.

eMailOk, I lied. They’re not ordered, and there’s more than ten. Which ones have your friends plagued you with?

Here’s a list of bad email habits that annoy recipients.

  1. When you reply to an email, don’t hit Reply-All unless you intend to send to everyone.

    For instance, when you RSVP to a party invitation, everyone who’s been invited doesn’t need to know your response.

    There’s a difference between Reply and Reply-All, learn it, and use it wisely.
     

  2. You do not need to insert your response above my email and send the whole thing back to me.

    When you hit reply, many mail clients copy the whole of the sender’s message so that you may reference it. Don’t whack a few returns, enter your response, and hit send. Delete the quoted message.

    I can’t stress how important this is for anyone who wants to maintain a sane thread of conversation. This is especially true for replying to Internet newsgroups and mailing lists.
     

  3. Do not reply by inserting your text into the quoted text, even if you make it a separate color or font.

    The most unreadable email comes when people reply to a message, and then just type after a paragraph – usually without a line break. If the recipient’s mail client can’t preserve the color or font, it becomes unclear who said what.

    Those quote levels are there for a reason.
     

  4. Reply-to-reply-to-reply-to-reply…

    You typically see this on mailing lists where someone responds with a short message, preserving the entire historical chain of messages up to that point. Stop it. If you see more than two levels of quotes, something is dreadfully wrong.

    There’s what you’ve said, there’s what everyone else has said, there’s what you’re saying now. If you see more than two levels of quoting, someone is committing at least one of these bad habits.
     

  5. Check the To and Cc fields before you hit Reply-All

    If you’ve been blind carbon copied to a message, there’s most likely a reason the sender did so — that usually involves not wanting the public recipients to know you were included.

    For instance, I maintain a list of my friends’ birthdays. Quite often, I’ll send a happy birthday greeting, but BCC their other friends as a subtle reminder. When someone hits Reply-All, it lets the birthday person know that someone else had to be reminded.

    Be considerate to the sender when that person trusts you by using BCC.
     

  6. Don’t attach a picture or video you found on the internet.

    Attachments take up space, they make getting mail slower, they take longer to download, they chew up quota. If you found something on the Internet, send the link, not the resource itself. The recipient can then use the most efficient means of getting it.
     

  7. Learn to use image compression

    If you are going to send an email with an image attachment, then at least learn to use image compression so that you have a small attachment. I can’t begin to count the number of times someone’s sent me a megabyte jpeg of something stupid.

    Like the web, try to keep images down to 32K or less, if possible. Be respectful of the other person’s INBOX space.
     

  8. Learn to upload content to a server

    Rather than clogging email with attachments, learn how to beam content up to a server, and then point the recipients at the content. The email will be smaller, often get there faster, not take as much space, and can be pulled from online faster.
     

  9. Keep your signature block small

    I don’t need random quotes. I don’t need legal disclaimers. I don’t need ASCII pictures. I don’t need colors and fonts. I don’t need your picture. I don’t need advertisements. I don’t need a notice a virus checker was used. I don’t need your slogan. I don’t need your logo.

    Plainly put, if your signature block is equal to or larger than the content of your message’s body, something’s wrong.
     

  10. Get a personal account, use it as such

    I hate automated legal disclaimer blocks, especially in signatures, and even more so if they are larger than the message content.

    “The information in this email is confidential,…”

    If you’re sending me an unsolicited personal email from your corporate email and someone thinks that legal block is somehow enforceable, forget it – you can’t just throw a legal stipulation on a person, especially if the mistake is yours. As such, I’m not bound to delete the message, either. This fluff is just annoying, and yes, most likely it comes from your work. So, get a personal account. Use it instead.

    You do know your work is legally allowed to read your private mail when you use their systems, yes? That alone should scare you.
     

  11. Stop attaching your vCard on every email

    If you’ve sent me your vCard, I’ve got it in my address book – I don’t need a copy with every email.
     

  12. Stop using backgrounds for the sake of backgrounds

    It’s one thing if your email has some functional layout and design to it, but if you’re just sending a background for the sake of adding texture, don’t. The most common occurrence I see of this is a repeating tile of textured background. Honestly, plain white is easier to read and prints better. Let’s do without the visual noise and extra attachment overhead.
     

  13. If it’s a short message, use text mode.

    Fonts, formatting, colors, and embedded images convey additional information. If you don’t need it to get your point across or add additional clarity, don’t incur the extra overhead of making an HTML message. Plain text messages are much easier to read and respond to on mobile devices.

    We’ve all seen documents and adds that look busy or appear as font soup; don’t commit the same atrocities with your emails.
     

  14. Stop putting pictures in Word and PowerPoint files

    I can’t count the number of times someone’s wanted to send me a few images, and was so clueless that they had to make an Office document to hold the picture. The amount of waste, inefficiency, and platform specific ties this incurs is mind boggling. I just can’t take people seriously who do this.
     

  15. Don’t blindly forward and email and not tell me why

    I’m not a mind reader, I just play one on TV. Yes, the information forwarded may be pertinent, but unless you establish some kind of context, it may be perceived as junk.

    Never assume the reader of your message is going to get your message in a timely manner, or will be reviewing it with the same mindset or information you have immediately at hand.
     

  16. Don’t use tiny fonts

    A number of corporate emails I get arrive as HTML documents with 6 point fonts. Yes, you might have a pretty poor monitor, and it may appear big on your screen, but if you force me to read something at a fixed size, my huge monitor will render it as the microscopic text that it really is.

    If you want me to read your email, make it readable.
     

  17. Run spell check

    If you’re typing and a word is underlined in red, double check and fix it. Additionally, avoid cell phone abbreviations like using UR for “your.” You’re not limited to 120 characters, and you’re not being charged 10 cents per message. Use enough to be clear.

    Emails are often saved, and consequently searched. If the words in your email aren’t ones entered into a search box, then you’ve made if difficult for someone to find or reference your email.

Fixing Duplicate Menu Items

When duplicate items appear on OS X’s Finder’s Open With pop up menu, there’s something you can do about it. Enter this command…

I’ve recently bumped into a problem where, on occasion, I get duplicate menu items in my right-click pop-up on the OS X desktop, when I select Open With.

Duplicate Menu Items

Turns out this is merely an indicate that the database for LaunchServices needed to be fixed.

And, in fact, this was covered over at Mac OS X Hints a while back. But, given that I had need to look up the command more than once, I present it here.

Inside Terminal, enter:

/System/Library/Frameworks/ApplicationServices.framework/Frameworks/LaunchServices.framework/Support/lsregister -kill -r -domain local -domain system -domain user

You don’t even need to enter your administrator password, nor do you need to reboot.

OnyX for OS XOnyX allows you to also rebuild the LaunchServices by going to the Maintenance / Reset panel. Though there are a number of other ways to reset launch services for each version of OS X.



Unused local variables, a gotcha that’ll getcha

Removing unused local variables from your programs is actually quite safe, makes your code more readable, and helps get faster compiler times, and can even help produce a tighter, faster, more optimized executable. However, there’s one thing you really need to watch out for…

Recently I attended the No Fluff Just Stuff conference again and learned about a free, fantastic static code analyzer for Java, called PMD. It can use be used standalone or even integrated into many popular IDEs like NetBeans or Eclipse. For the curious, I’d tell you what PMD stands for, but no one really knows; worse yet, I can’t stop myself from typing PDM.

PMD has a nifty rule that allows it to locate Unused Local Variables.

Very quickly I was able to walk through our entire code base, identify things that were assigned to, and subsequently not used, and remove them.

Gotcha #0
The first major hit of the day is that you’ll want to do a Clean on your project before you start. Believe it or not, some project building steps can build intermediate .java, files from your master source code. Problem is, if they appear anywhere inside your project’s directory structure when PMD is making the sweep, they get analyzed to. And you don’t want that.

Gotcha #1
It turns out that there’s a reverse ripple effect in performing this kind of code cleanup and operation. After you’ve removed code, you’ll want to make sure you perform subsequent passes until all concerns are removed.

Take for instance this trivial case:
  B = A;
  C = B;
  // C is unused!

What happens is that once you remove C, it turns out B may no longer be used. Remove B, and it’s even possible A may no longer be used either.

Additionally, this can eventually lead to additional Unused Imports, which dealing with those can also decrease build time.

Gotcha #2 — The Real Evil
Normally, this kind of code clean up is absolutely harmless, although there’s one error of omission that a developer can make which will create problems and cause a silent failure.

Here’s a case where a static code analyzer recommends removing a very important line of code:

boolean doSomething(int x) {
  // Do something very important with x
  return result;
}

...
  boolean result = doSomething( x ); // Do Something Important
  // result not used
...

If the return value from a method isn’t used, then the static analyzer will assume the method doesn’t need to be called, and it will recommend commenting out the line — causing your program to silently break.

This is not a error on the part of the tool!

The error was actually that of the developer for not checking the return results of the method.

To correct the problem with the source code, the developer has three options:

  1. Change the method signature to be of type void.
  2. Throw an exception from the method.
  3. Check the return value of the called method.

Otherwise, the real error is that the method may attempt to do something, fail, and communicate back to the caller than something went wrong, but the caller blindly trusts that things are okay.

If you’re not going to check a return value, you shouldn’t be incurring the overhead of sending one. If the library writer provided one for a reason, then you should be using it.

Conclusion
When a code analyzer makes a recommendation, ask yourself what implied rules about your code the analyzer is assuming. Rather than blaming the tool, the better solution is fixing the source.

It may be best to comment out, rather than deleting, code that initially seems superfluous.

Finally, after a massive code sweep, run those unit tests.

Seven Girlfriends To The Perfect Spouse

What do walking sticks, card shuffling, and movie stars have to do with finding the perfect mate? They seem to support that you can find your spouse by seriously dating seven people.

When one goes to a technical conference on Java, perhaps the last thing you’d expect the a speakers’ lunch conversation turn to is mathematical selection of a mate.

I suppose it started simply because we had been talking about change. The topic started with source code change, changes at the office, and moved into changes in people. I offered up the classic observation that women get frustrated at their men because they don’t change, but that men get frustrated at their women because they do.

One gentleman was lamenting on his situation and admiring the subtle manipulation by his wife prior to his flight:
“Honey, ” he relates she began, “you should probably lose some weight, though I’d still love you if you were fat as a house.” And, after a momentary pause, she inquires, “Do you think I need to lose weight?”

Apparently his response of “I always love you, too” wasn’t quite what she was looking for.

Another speaker spoke up and added, “Well, I don’t have that problem at all. We’ve come to an agreement. I can be either fat or bald, but not both.”

Absolutely curious, I asked how that arrangement was sought. That just isn’t the kind of thing that normally just comes up as the finer points of marital negotiation.

He pondered and explained it started by as an ultimatum by her — unfortunately for her, she was engaging with one of the brighter logical minds…

“Honey, I’ve thought about it, and you can’t grow up to be fat and bald.”
“Really? Well, I have no control over being bald.”
“Oh. Well then you can grow up to be bald, but not fat.”
“Well, if I can be bald and not fat, then it’s only fair that I can also be fat and not bald.”
“makes sense, that sounds fair.”
“Then it’s settled, I can either be bald, or fat, but not both.”
“Agreed.”

And at that point, he took another bite of cake, stroking his head of hair with the other hand.

But this started a fantastic digression about how one knows when you have the right person in your life. We’d all just come out of meetings pertaining to measuring, quantifying, trend analysis, and metrics.

Based on rough calculations, to calibrate the scale and distribution of a sample size population, to do the best you could do, in a short period of time, it was concluded that you really needed to examine at least seven samples. At that point, you have a reasonable approximation for making a reasonable judgment call, and it was just a matter of how many standard deviations you were discriminating for.

A simple example is called for.

You’re walking in the woods and you’ve been given the task to pick up the best walking stick you can find. But the rule is, once you set it down, you can’t pick it up again.

So, you pick up the first stick you come to. However, you don’t know if this is the largest, or smallest. You have nothing to compare it to. With no basis for comparison, you set it down.

So, you pick up the next stick you come to. And you start to note more attributes, such as weight, shape, type of wood, etc. And again, you drop it.

So, you pick up the next stick you come to. As the attributes that interest you start to prioritize, you find yourself become more aware of the quality. You start looking at things like balance, general utility, wear, and things that come along with the stick, such as moss or termites. Some things are alterable, others are not.

As your go from stick to stick, each time you’re refining your assessment abilities and have gained more knowledge about what you’re looking at, as well as determining what you want.

After your six stick, depending on the breadth of the distribution you’ve encountered, you’ve started to formulate an accurate picture of of what you can reasonably expect. At this point you simply decide what criteria must be met, and the next stick that meets that criteria, you take. Permanently.

Oh sure, there might be something else better out there, but the effort, cost, time, dangers, and availability will more than likely offset the value you’d currently have. Trading your walking stick for something new, only trades existing faults with new unforeseen ones.

The same mathematical application applies to the dating scene.

Young couples find themselves inexperienced for selecting long term candidates and determining personal discriminators: there are unforeseen personality clashes, other options suddenly look more attractive, and if one settles too prematurely, it usually means being treated like a doormat or being taken advantage of financially.

Who among us didn’t stumble upon infatuation and think it was true love at sixteen? Or, find an exciting girlfriend only to discover character flaws that were obvious to our friends, family, and even our own 20/20 hindsight? Or, have everything wonderful and stable only to have it all turned on its ear for not explicable reason just as she turned twenty.

Unfortunately, for many, desperation or loneliness causes some people to settle well before their calibration process is complete; these people usually learn the hard lesson that being trapped in a bad relationship is worse than being lonely.

Alternatively, there are those that spend too much time in the calibration phase, and totally miss out on the longer term joys that are more rewarding.

It’s also interesting to note that we see microcosms of discriminating selection occur around when context and locality are ignored and we forget the more global nature of the chase.

One example may be the cruise ship filled with old geezers, of which in that context, some middle-aged person who’d never catch your eye looks fairly darn attractive in light of the situation. Same goes for co-workers, people met in bars, or even at parties — the limited selection forces expectations to be lowered.

This is why online dating services that give the illusion of many candidates keeps people seeking for perfection, while speed dating narrows the options and forces a compromised choice or none at all.

Of course, one might also say that it’s the reasoning behind things being in the last place you look, for once you make the find, the search is over.

Seven, however, lets you home in on a very acceptable choice that you can be genuinely be happy with, rather than waiting forever. For you too have a shelf life, and by the time you find the ideal through an exhaustive search, you may be too old to do anything about it.

I ponder, though, if there’s something more special going on.

For example, we’ve all heard of six degrees to Kevin Bacon, at that point you’ve traversed enough paths to get where you want.

Additionally, and I think this was in a story by Wired Magazine, a number of years ago, there was a mathematician who was going on a long horseback cattle drive. To amuse himself, he brought along a deck of cards, and spent the time shuffling it. At the end of his trip, he had mathematically deduced that for a deck to be sufficiently randomized (that is any card could be assured to physically be in any position), it required the deck to be shuffled seven times.

In both cases, we see that seven got us a good sampling — and with a good sample, an intelligent choice can be made.

Using In A JavaScript Literal

Today I got bit by a very interesting bug involving the tag. If you’re writing code that generates code, you want to know about this.

I’m currently working on an application that takes content from various web resources, munges the content, stores it in a database, and on demand generates interactive web pages, which includes the ability to annotate content in a web editor. Things were humming along great for weeks until we got a stream of data which made the browser burp with a JavaScript syntax error.

Problem was, when I examined the automatically generated JavaScript, it looked perfectly good to my eyes.

So, I reduced the problem down to a very trivial case.

What would you suppose the following code block does in a browser?

<HTML>
<BODY>
  start
  <SCRIPT>
    alert( "</SCRIPT>" );
  </SCRIPT>
  finish
</BODY>
</HTML>

Try it and see.

To my eyes, this should produce an alert box with the simple text </SCRIPT> inside it. Nothing special.

However, in all browsers (IE 7, Firefox, Opera, and Safari) on all platforms (XP/Vista/OS X) it didn’t. The close tag inside the quoted literal terminated the scripting block, printing the closing punctuation.

Change </SCRIPT> to just <SCRIPT>, and you get the alert box as expected.

So, I did more reading and more testing. I looked at the hex dump of the file to see if perhaps there was something strange going on. Nope, plain ASCII.

I looked at the JavaScript documentation online, and the other thing they suggest escaping are the single and double quotes, as well as the backslash which does the escaping. (Note we’re using forward slashes, which require no escapes in a JavaScript string.)

I even got the 5th Edition of JavaScript: The Definitive Guide from O’Reilly, and on page 27, which lists the comprehensive escape sequences, there is nothing magical about the forward slash, nor this magic string.

In fact, if you start playing with other strings, you get these results:
  <SCRIPT> …works
  <A/B> …works
  </STRONG> …works
  <\/SCRIPT> …displays </SCRIPT>, and while I suppose you can escape a forward slash, there should be no need to. Ever. See prior example.
  </SCRIPT> …breaks
  </SCRIPTX> …works (note the extra character, an X)

With JavaScript, what’s in quotes is supposed to be flat, literal, uninterpreted, meaningless test.

It was after this I turned to ask for help from several security and web experts.

Security Concerns


Why security experts?

The primary concern is obviously cross site scripting. We’re taking untrusted sites and displaying portions of the data stream. Should an attacker be able to insert </SCRIPT> into the stream, a few comment characters, and shortly reopen a new <SCRIPT> block, he’d be able to mess with cookies, twiddle the DOM, dink with AJAX, and do things that compromise the trust of the server.

The Explanation


The explanation came from Phil Wherry.

As he puts it, the <SCRIPT> tag is content-agnostic. Which means the HTML Parser doesn’t know we’re in the middle of a JavaScript string.

What the HTML parser saw was this:

<HTML>
<BODY>
  start
  <SCRIPT>alert( "</SCRIPT>
  " );
  </SCRIPT>
  finish
</BODY>
</HTML>

And there you have it, not only is the syntax error obvious now, but the HTML is malformed.

The processing of JavaScript doesn’t happen until after the browser has understood which parts are JavaScript. Until it sees that close </SCRIPT> tag, it doesn’t care what’s inside – quoted or not.

Turns out, we all have seen this problem in traditional programming languages before. Ever run across hard-to-read code where the indentation conveys a block that doesn’t logically exist? Same thing. In this case instead of curly braces or begin/end pairs, it was the start and end tags of the JavaScript.

Upstream Processing


Remember, this wasn’t hand-rolled JavaScript. It was produced by an upstream piece of code that generated the actual JavaScript block, which is much more complex than the example shown.

It is getting an untrusted string. Which, to shove inside of a JavaScript string not only has to be sanitized, but also escaped in such a way that the HTML parser cannot accidentally treat the string’s contents as a legal (or illegal!) tag.

To do this we need to build a helper function to scrub data that will directly be emitted as a raw JavaScript string.


  1. Escape all backslashes, replacing \ with \\, since backslash is the JavaScript escape character. This has to be done first as not to escape other escapes we’re about to add.
  2. Escape all quotes, replacing ' with \', and " with \" — this stops the string from getting terminated.
  3. Escape all angle brackets, replacing < with \<, and > with \> — this stops the tags from getting recognized.

private String safeJavaScriptStringLiteral(String str) {

  str = str.replace(“\\”,”\\\\”); // escape single backslashes
  str = str.replace(“'”,”\\'”); // escape single quotes
  str = str.replace(“\””,”\\\””); // escape double quotes
  str = str.replace(“<“,”\\<“); // escape open angle bracket
  str = str.replace(“>”,”\\>”); // escape close angle bracket
  return str;
}

At this point we should have generated a JavaScript string which never has anything that looks like a tag in it, but is perfectly safe to an XML parser. All that’s needed next is to emit the JavaScript surrounded by a <![CDATA[]]> block, so the HTML parser doesn’t get confused over embedded angle brackets.

From a security perspective, I think this also goes to show that lone JavaScript fragment validation isn’t enough; one has to take it in the full context of the containing HTML parser. Pragmatically speaking, the JavaScript alone was valid, but once inside HTML, became problematic.