In their article, WikiHow contributors write:

Get involved in anything where you will meet people. If you are very shy, find a group for social anxiety, even if it has to be online (obviously it’s better if it’s not). Look on places like Craig’s List, or local news websites for your town for activities in your area. Volunteering can help. But don’t attend functions with the idea of making friends or meeting people. Being too demanding is a sign of loneliness. Try to go with no expectations whatsoever, and to enjoy yourself regardless of what happens. Look for activities that interest you, that also involve groups of people, like intramural sports, book clubs, church groups, political campaigns, concerts, art exhibitions, etc.

While it all “makes sense,” the WikiHow article reads like an elaborate horoscope. It’s incredibly annoying because it contains no meaningful, discrete, actionable items. Where, exactly, can I find “activities in my area”? And once I find them, how do I make sure I know about them when they are happening? And as if that wasn’t hard enough, how do I make the process workable under the extreme energy constraints that being depressed and lonely put me under? (See also: without using up too many “spoons”.)

Ironically, when I finally concocted a solution to this problem, I no longer had the time to write the blog post about solving the problem because I was so busy doing things and being social. I proceeded to pull myself out of my depression, have a pretty good (if still difficult at times) Spring and Summer, and even Fall in 2011. But now that the days are getting shorter and I’m increasingly feeling like my moods are walking on a tightrope of “happy” above a pit of bleakness, I figure it’s about time to document my process. That, and it seems people I know are running into the same problem, so hopefully sharing my own solution can really make a positive impact on others’ lives.

Creating a Cyborg’s Social Calendar

The basic problem was two-fold. First, I needed an easy way to discover local goings-on. Second, I needed a way to remember to actually attend events that I was interested in.

It turns out this is far more difficult to accomplish than one may at first believe since the set of events that I both want to attend and have the capability (energy, time, money, motivation, physical accessibility, etc.) to attend are actually relatively limited. Moreover, I also need to align the set of events that match both of those criteria with the knowledge that said event is occurring when it is occurring. It’s a bit like playing temporal Tetris.

In a nutshell, the solution I implemented was similarly two-fold. First, I cast an incredibly wide but low-cost sensor net, integrated directly into the process I already used for keeping track of my daily appointments. (See also the “no extra time” concept and its wide applicability). Second, I classified the “activities in my area” into two distinct groups: “engagements” (stuff I’ve said “yes” or “maybe” to) and “opportunities” (stuff I haven’t yet said “no” to).

Here’s what my calendar looks like after all the pieces of the system are in place:

As you can see, I have an enormous selection of activities I could participate in at any given time. Better yet, they all show up on my calendar without my ever needing to repeatedly go “look[ing] on places like Craig’s List” to find them, the events on my calendar update themselves, and I can show or hide sets of events on a whim.

The prerequisite tool for doing this is the iCalendar feed, which, in the words of Stanford University, is a popular calendar data exchange format which allows you to subscribe to a calendar and receive updates as calendar data changes. Each of those calendars under the “Subscriptions” heading in the screenshot of my iCal is actually an iCalendar feed from a remote website. iCalendar feeds are to calendars as RSS feeds are to blogs.

The first thing I did was add the event subscription feed from my Facebook. Do this:

1. Log into your Facebook account and go to the “Events” page.
2. Scroll to the very bottom of the page and click on the small “Export” link. This will reveal a personalized web address (URL) listing all upcoming Facebook events you’ve been invited to or have RSVP‘ed either “Yes” or “Maybe” to, in iCalendar feed (.ics) format. Copy that URL.
3. Back in iCal (or your calendaring application of choice), choose “Subscribe…” from the menu and paste in the URL you got from Facebook.
4. Give this calendar subscription a meaningful name. I called it “Facebook Events” (see above screenshot).
5. Set the “Refresh” interval to something that makes sense; I set it to once “every 15 minutes,” since the Facebook feed is one I check often because it changes so frequently. (For feeds from calendars that I check or that update less often, such as those of community groups, or calendars listing events that are far from home, I set the refresh rate much, much slower, such as once “every week.”)

Okay! Now, whenever a friend invites you to an event on Facebook, your calendar will be updated to reflect that event at the appropriate date and time. If you RSVP “No” to the event, it will disappear from your calendar when iCal next checks your Facebook iCalendar feed.

Repeat the same steps for any other event-management website that you use and that offers iCalendar feeds. Some services I use, such as Plancast.com and Meetup.com, actually offer two distinct iCalendar feeds, one for all of the events visible to you on the service, and one for events that you have RSVP‘ed “Yes” to. Subscribe to both; in the screenshot of my iCal window, above, you’ll note the existence of a “‘meitar’ on Plancast” calendar as well as a “Plancast Subscriptions” calendar, and similarly a “My ‘Yes/Maybe’ Meetups” calendar as well as a “My Meetups” calendar.

Now that you’ve got a bunch of subscriptions, it behooves you to organize them in a way that makes sense to you. How you can do this will depend a little bit on the tools you have at your disposal. I found Apple iCal the best choice because of its Calendar Group feature, while I found Google Calendar an incredibly frustrating tool to use.

In iCal, I first created two calendar groups. The first one was called “Social Engagements,” into which I placed all the iCalendar feeds that showed me events to which I’ve RSVP‘ed “Yes” to on the remote site. This included the Facebook, “‘meitar’ on Plancast”, and “My ‘Yes/Maybe’ Meetups” feed. The second group was called “Social Opportunities,” into which I placed all the other calendars.

Every time I learned about a new local venue, such as a nightclub, or a café, or a bookstore that had an open mic, I would scour its website to see if it offered an iCalendar feed. If it did, or if it used a tool that did, such as embedding a Google Calendar on their website,¹ I’d add their feed to my “Social Opportunities” calendar group, too. I’d do the same every time I learned of a new event aggregating website, such as the IndyBay.org calendar or the Calagator Portland Tech Community calendar, which both offer feeds.

In very short order, I became one of the go-to people to ask about what was happening ’round town—including some towns I didn’t even live in!

However, as I travelled across the country speaking at conferences, I realized that my “Social Opportunities” group was getting cluttered with events that I could not actually attend because I was literally thousands of miles away from them. To solve that problem, I created distinct “Social Opportunities” calendar groups based on geographic region, and moved the individual subscriptions to the group with which they were geographically associated; the Occupy DC calendar feed is in the “Social Opportunities – DC” calendar group, and so on. I also created an “A-geographic” group to house feeds that listed events from all over the place.²

Some event-management services let you filter by geography, making this even easier. For instance, Yahoo!’s “Upcoming” event listing website shows you events by “place,” and you can subscribe to an iCalendar feed of just those events. For instance, here are the Upcoming events in Seattle, and here is the same information in iCalendar feed format. I added the feed of each Upcoming Place to which I regularly travel to its appropriate regional calendar group.

The benefits of this set up are obvious:

• Visually overlay social opportunities on top of social engagements to ensure few conflicts, and help make the most informed choice about which events I want to go to when there are conflicts, to mitigate my social opportunity cost.
• Toggle calendars on/off to find nearby activities. Ordinarily, I simply leave all the “opportunities” calendars deselected, so I’m just looking at my personal calendars and the “Engagements” group, since this view shows me “stuff I have to do today.” When I’m bored or I’m looking for new things to do in the upcoming week, however, I simply turn on the “opportunities” calendars. Voila! In 1 click, I’m browsing a wealth of stuff to do!³
• Quickly orient oneself within the social space of a new city. If I’m taking a trip to Washington DC for a few days, all I have to do is deselect/uncheck the “Social Opportunities – SF/Bay Area” calendar group to hide all of my calendar subscriptions in that group, then select/check the “Social Opportunities – DC” calendar group and, voila, my calendar view has instantly shifted to showing me events that I can attend in Washington, DC.
• Make RSVP‘s meaningful: if I RSVP “Yes” to an event on Meetup, the event is automatically removed from my “Social Opportunities – A-geographic” calendar group and added to my “Social Engagements” calendar group.
• Easily move event information from a calendar feed to a personal calendar using copy-and-paste without ever leaving the calendaring tool of your choice.

Of course, none of this matters with regards to feeling lonely if I don’t also show up at events in physical space. Admittedly, actually mustering the physical and social energy to get up and go is by far the hardest part of this whole process. Typing on a keyboard is all fine and well (rest assured I do more than enough of it!), but there is no substitute for actually being around other human beings face-to-face. Physically vibrating the air using one’s mouth and having those vibrations move another’s ear drum (or physically moving one’s hands and letting the photons bounce off those movements and onto the retina of another’s eyes, in the case of sign language) is a vital part of the experience of being social.

This system isn’t perfect, but the imperfections are mostly due to the way sites like Facebook handle RSVP information. For my purposes, though, this workflow gets me well over 80% of the way towards my goal, and since I’m actually a human (not a machine), I can deal with a little data pollution here and there. There’s also plenty more I could write about with regards to “being a social cyborg,” such as how I use my calendar in conjunction with my contact management application (my digital rolodex) to maintain “loose” or “weak” interpersonal ties with over 1,000 people spread across the world—again, using “no extra time.” But I’ll save that for another post.

For now, hopefully this gave you a better understanding why my most frequent response to being informed of a party is something along the lines of, “Can you send me a link (to Facebook/Meetup/Google Calendar)?” and also why I’m so, so, so critical of important websites like FetLife that seem to prioritize everything but user security and interoperability.

1. Every public Google Calendar also publishes its information in an iCalendar feed. For example, rather than view the Occupy SF calendar on their website, just subscribe to the iCalendar feed provided by Google. Also, while you can create an aggregate view of multiple Google Calendars to embed on a Web page, it seems to me like this isn’t a feature offered for iCalendar feeds, so if you come across such a calendar, you’ll likely need to add the individual calendars’ feeds one by one.
2. Currently that’s just Meetup and Plancast, for me, since I’ve joined Meetup groups all over the country and I’ve subscribed to people on Plancast who live in dozens of cities.
3. Frustratingly, although Facebook also offers you a page listing events that you were not invited to but that your friends were, there seems to be no iCalendar feed of that list, forcing me to periodically check that page for events that would be “Social Opportunities” if I knew of them. Thankfully, to add them to my own calendar, I just RSVP “Yes” or (more likely) “Maybe.”

Thanks to this simple text convention in subject lines, I could trivially automate the process of sorting through the approximately 100 emails a day that the email list generates in order to single out only the emails that interest me. Here’s how I did it.

Define Your Goals

Before setting out on any task, it behooves you to take a moment and think about what it is you’re trying to accomplish. For me, with the San Francisco Freecycling Network (SFFN) email list, I wanted to achieve the following goals:

• Keep my inbox clear of email from the SFFN list unless a message was particularly interesting.
• Browse the SFFN messages when I wanted to look at them without having to go to the web site.
• Highlight particularly interesting messages in my inbox visually and play a special sound to alert me that such email has been found in case Mail was running in the background (since free stuff gets taken fast!).

I defined “particularly interesting” messages as ones that offered items of need for my recent move. With this in mind, I set out to create email rules that accomplished each goal in turn.

Step 1: Create a mailbox to store the appropriate messages

I began by creating a new mailbox to store all the SFFN messages I was getting. This alternate mailbox would be the mailbox I would shunt all SFFN email to so as to keep my inbox clear of it. I called the mailbox simply “SFFN”.

Do this:

1. From the Mailbox menu, select New Mailbox…. The New Mailbox sheet appears.
2. Select any location (“On My Mac” is fine, as is the account that receives the mailing list messages), and give it a name.
3. Click OK.

Step 2: Create an email rule to move all appropriate messages to the new mailbox

With the new mailbox created, I now needed to get all the appropriate messages in there and out of my inbox.

Apple Mail’s email rules work by looking at each incoming message and matching it against a set of conditions that you provide. If the message being evaluated matches the conditions you specify, such as “from the San Francisco Freecycler’s Network mailing list”, then an associated action is automatically performed. Every email you get is evaluated against every rule you have unless a rule moves the message to another mailbox or until you trigger the “stop evaluating rules” action.

Since moving an email message to a new mailbox ends the process of evaluating rules and moving messages to the SFFN mailbox I just created is the goal of the rule I’m creating, I decided to name the rule “END – SFFN”.

Do this:

1. From the Mail menu, select Preferences…. The Mail Preferences window opens.
2. Click the Rules button. The Rules pane appears.
3. Click the Add Rule button. The Add Rule sheet appears:
   1. Enter a meaningful description (I chose “END – SFFN”) in the Description: field.
   2. Provide the conditions you want to match. Since all SFFN emails must be addressed to the mailing list, I simply provided the email address of the mailing list (sffn@yahoogroups.com) as the condition for the To header.
   3. Provide the actions you want Mail to perform. I simply wanted to move the matched messages to the SFFN mailbox.
4. Click OK.

For me, the above configuration looked like this:

Step 3: Create an email rule to highlight a message of particular interest

At this point, any and all email I receive from the San Francisco Freecyclers’ Network is being moved to the SFFN mailbox I created for it. This is nice because it keeps my inbox clear, but it’s still not very helpful since I still have to go trudging through the SFFN mailbox in order to find anything that might be interesting to me. The whole point of this exercise is to reduce the amount of time I spend actively looking for interesting things and let my computer do that work for me. So the next step is to tell Mail what I’m looking for so it can show the interesting messages to me.

Now, as it happens I’m in need of a wireless router. Since “router” is an appropriately unique word, I’m going to tell Mail to look for that word in a subject line. However, since I only want Mail to tell me when a router is available and not when other people like me are looking for routers, I’ll also tell Mail to look for the keyword “OFFER” in the subject line. (And this is why the Freecycle guidelines tell users to format their subject lines in a conventional way.)

Finally, since I don’t want to have to go digging for the interesting email message and since my inbox is already going to be kept clear by the previous rule, I’ll simply have Mail highlight the message in a bright green color and leave the message in my inbox without moving it to the SFFN mailbox I created earlier.

Do this:

1. From the Rules pane in Mail’s preferences, click Add Rule.
2. Enter a meaningful description in the Description: field. (Since I’m looking for a router, I called it “SFFN – Search for OFFERed ‘router’”.)
3. Provide the conditions you wish to match. For me, this meant email sent to the Freecycler’s mailing list with the two words “OFFER” and “router” in the subject line.
4. Specify the actions you wish Mail to perform. I wanted Mail simply to color the message green and to leave the email go to the inbox (where it was originally destined for), so I chose “Stop evaluating rules”. (I also decided I’d want Mail to play a special sound to alert me that it had found something interesting. This is optional, of course.)
5. Click OK.

When I was done creating my rule, the above configuration looked like this:

I can now repeat this step as many times as desired to tell Mail to highlight other messages that may be of particular interest for some other reason. For instance, say instead of looking for a wireless router, I wanted to look for a toaster. I would simply need to click on “Duplicate Rule” and replace all instances of “router” with “toaster”.

Step 4: Place email rules in appropriate order

Since Mail will repeatedly check incoming email against all the active rules, we need to be sure to place the rules in the correct order. You can think of each email rule as part of large Rube Goldberg machine, each message getting funneled through some piece of the logic at each successive rule. That’s why I began the name of the first rule I created with “END,” so that I’d know it should be placed after the rest of the SFFN-related email rules.

I decided that I wanted Mail to look for anything related to cameras and, of course, to toasters. This gave me a total of 4 rules (three to search for items of interest, and one to keep my inbox clear). Since the three highlighting rules all perform the same action, it doesn’t really matter which order they go in, but it is important that all of them appear before the rule to move messages to the SFFN mailbox.

To order rules, simply click-and-drag them into the order you wish Mail to evaluate them in. When I was done, my Rules pane looked like this:

Conclusion

Mail rules are an extremely powerful feature that most email clients have, but that too few people use. They can save you enormous amounts of time and increase your productivity by automating simple yet time-consuming tasks.

The conventional, standardized subject lines that the Freecycle mailing list uses simplifies the logic required to have your computer automatically process your messages for you. This is a useful observation because it can be applied to other areas of your life where using simple conventions can help to organize otherwise overwhelming information tasks into manageable batches. Although this particular example uses stock, simple commands, you can get as fancy as you like by having an action trigger an AppleScript.

Now, hopefully, finding some additional housewares and a wireless router for my new San Francisco apartment will be as easy as checking (but not manually sorting!) my own email!

Since I’m a Subversion refugee, my only experience with different version control systems is mostly with the distinctions between the centralized versus the distributed models, not between the various tools you can use in either paradigm. What struck me when I first began using git was how conceptually similar it felt to using Subversion when I was using it by myself (as a lone developer) but how radically different it suddenly felt the moment I was sharing my code with someone else.

Now, I’m a die-hard individualist. I want things to happen my way as much as possible, and I don’t really care what happens for anyone else as long as when I interact with other people those interactions are as mutually beneficial as they can possibly be. That’s why I love DSCM tools so much.

Distributed source code management systems feel much more like translator tools between the ways in which people work as opposed to feeling like a dogma of workflow management processes, like centralized systems do. This paradigm appeals both to my preferred way to work and, as it turns out, helps more people stay more productive all at the same time.

This is also why I’m a firm believer that most of the people I’ve worked with in the past completely missed the point of using version control systems. It seems to me that most developers I’ve worked with have thought of SCM tools as “the ‘Save As…’ button on steroids.” While these developers are technically correct, their narrow view of what a VCS does means they aren’t taking advantage of the full potential of the concept.

The power of a version control system isn’t just in that it gives you the ability to easily hit the proverbial “Save As…” button as much as you want, but rather in that you get to retrieve those other versions when you’re ready for them, regardless of what your fellow developers are doing to the code on their machines. This means that a version control system’s real purpose is to insulate you from changes of any sort until you’re ready to deal with them. A good tool also does this reciprocally; it will insulate your fellow developers from the changes you’re making until they’re ready for them.

Admittedly, that’s not a very concrete “feature.” It’s more like a fundamental philosophical principle, which is probably why it’s so hard to encode into the physical manifestation of a tool. Then on top of all of that complicatedness you have to add things like usability and interoperability and resource efficiency. That’s where I learned about the majority of the distinctions between the various DSCM tools discussed in SyPy’s presentation.

However, for me, all of those things ultimately get evaluated against the following question: Does Feature X help insulate me from change (does it help in persisting my view of the state of the world until I’m ready for it to change), or not?

For example, Bazaar’s interesting notion of “nested commits” with dotted revision numbers is really intriguing because it’s much (much) more user-friendly than git’s notion of exposing SHA-1 hashes to (mere mortal) end user’s eyes. Yet, while it’s certainly less painful than copying-and-pasting hashes all over the place, there’s little fundamental difference in the way these mechanisms actually portray the state of the world to me. Any given SHA-1 will always be the exact same commit object. Any given dotted revision number will also always be the same commit (within one’s own unchanged repository).

In contrast, I learned from Martin Pool that Bazaar has a “push over SFTP” feature to let you “export” or “archive” a version of code by transmitting it over an SFTP connection. Now that really caught my attention because it’s an example of the version control tool acting like that translator I was mentioning earlier; the interoperability helps people not need to change until they want to. In this case, it means you never have to install Bazaar on a remote server to get your content there via the tool. That’s very cool—much cooler than the mundane technical fact that bzr supports the SFTP protocol out of the box.

Of course, it’s technically pretty trivial to write an expect or shell script wrapper to enable git (or whatever other tool you want to use) mimic this behavior. And that’s exactly the point: technology is always the easy part. It’s doing it right at a fundamental level that’s actually really difficult to do correctly.

I find working without an accurate awareness of the situation at large very disorienting, and software and web development are notorious for being circumstances that change rapidly. That’s why one of my favorite things about Scrum is the burn down chart. This is nothing more complex than a simple graph that depicts how much work you bit off and how far along trying to chew it you actually are. The benefit, of course, is that it’s pretty obvious pretty quickly if you’ve bit off more than you can chew. ;)

So up ’til now, my team and I have been doing this all on paper. There’s a certain tactile appreciation I have for doing this sort of thing on paper, but of course there are disadvantages, too. For instance, you can’t easily archive the information. You can’t easily share it with remote contractors. You can’t automatically mine this valuable data with software tools. You get the picture.

There are a few cool plugins to some tools like Trac that do all this, but at first blush most of these tools seem to require that you move all of your Scrum’s planning into the digital world. That is, you can’t just do the burn down chart, you have to do all your estimation (MoSCoW desirability, sizing, estimating ideal hours) through some tool. That’s a big step, and I wanted something simpler.

So naturally, I came up with a spreadsheet in Numbers.app as my solution. I mean, how much simpler can you get? Sure, it’s not exactly “well integrated” with other tools, but your non-tech-savvy boss will probably love it, and AppleScript can be used to automate data extraction. Here’s what it looks like:

An example Scrum-style burn down chart in Apple's iWork '08 Numbers spreadsheeting application, complete with an actual chart.

(Click the screenshot to get a full-size view.)

As you can see, the Numbers sheet is a simple table and a line chart. I’ve embedded instructions for how to use the chart into the example itself, which I’ll quote here:

This is a sample Scrum-style iteration burn down chart for software development created by Meitar Moscovitz. It can be used to plot a team’s progress throughout a development cycle (aka. “iteration” or “sprint”). This sample chart depicts a 3-week iteration (15 working days) with a 150-point target goal.

The X-axis represents time, and is thus labelled Time in Days, while the Y-axis represents the work to be completed, and is labelled Points.

The green line shows the team’s ideal velocity based upon the total number of points—termed the Remaining Initial Value—scheduled for completion in the graphed iteration.

The blue line shows the team’s actual velocity (or “completed work”), which is entered by the team leader (aka. Scrum Master) after each day in the Done column.

To use this chart: duplicate this sheet, enter your iteration’s total points in the Initial Value row of the Remaining column, and delete the values in the Done column except its initial value of 0. To add more days, copy and paste more rows into the table. Optionally, give the sheet and its contents new titles. ;)

Feel free to download the Example Burn Down Chart.numbers file and use it yourself. If you do use it, please leave a comment and let me know how you’re going. Thanks, and enjoy!

(Mike Cohn of Mountain Goat Software has got a similar spreadsheet for Excel you can download.)

By those definitions, I’ve been a productive twentysomething-year-old since I was a pre-teen, which just goes to show you that age has nothing to do with it. Damn straight.

If you’re not afraid of the Terminal, this can be avoided. Put the following lines in your ~/.profile (or similar):

alias rmds='find . -name ".DS_Store" -type f -print0 | xargs -0 rm'

What this does is creates a new command that you can use (rmds) which recursively finds and deletes any regular file named “.DS_Store” starting from the current directory. Thus, running this command in the folder you are about to create an archive out of will clean it first, and will prevent unnecessary confusion on the part of your archive file recipient.

Alternatively, another way to do this is to use the command-line zip program and an (admittedly more complicated) pipeline to remove the .DS_Store files after they have been added to the archive. To do that, use this series of commands:

zip -d ZIPfile.zip `unzip -l ZIPfile.zip | grep .DS_Store | awk '{print $4}'`

where, naturally, ZIPfile.zip is the ZIP archive you want to remove the .DS_Store files from. Creating an alias out of that command (and making it work for paths that contain spaces) is left as an exercise for the reader. ;)

As an aside, the alias, find and xargs commands are incredibly useful in their own right and can be used to do a lot of pretty amazing things. As always, man command will give you the nitty gritty.

Also as an aside, you can stop the Finder from creating .DS_Store files entirely when browsing network volumes (like Windows shares) with another command, documented in Apple’s Knowledge Base.

In these cases, it makes sense to be smart about when I initiate these downloads. Being something of a UNIX-head myself, I wanted to use the age-old at command to download a Linux ISO during off-peak hours, which my ISP says starts at 2 AM. Much to my chagrin, I found that at doesn’t work by default on Mac OS X and, worse, the Leopard man page leads to a dead end (though it didn’t back in Tiger…).

Turns out that the system daemon that is responsible for checking up on at jobs has been wrapped with a launchd job. This makes enabling at on your system really easy:

sudo launchctl load -w /System/Library/LaunchDaemons/com.apple.atrun.plist

Once you’ve done this, you can now use at as you normally have done. For instance, I could now schedule my downloads to happen during the off-peak hours:

Perseus:Fedora maymay$ at 2:15am tomorrow # now press return
curl -LO http://download.fedoraproject.org/pub/fedora/linux/releases/9/Fedora/x86_64/iso/Fedora-9-x86_64-DVD.iso
# now press CTRL-D.
job 1 at Tue Jul 15 02:15:00 2008
Perseus:Fedora maymay$ atq
1	Tue Jul 15 02:15:00 2008

This is also incredibly handy for scheduling just about any resource-intensive task that you don’t have to do right now. To take it one step further, you can even let the computer itself choose when to run these resource-heavy tasks by using the batch command, which will execute commands much like at but will check the system load average instead of the system clock to determine if it should start the job.

Note that with the com.apple.atrun job loaded /usr/libexec/atrun is started every 30 seconds (unless you change the StartInterval key in the plist file). Since the atrun command checks a file on disk (that it places in the /usr/lib/cron/jobs directory) to see if there is any work to do, this will probably prevent your disks from ever sleeping, which could be a major concern for battery life on portables. Also, obviously, your computer needs to be turned on and awake for the job to actually launch.

For more information, check out the result of typing man at and man launchctl at a Terminal prompt. There’s also a really good Google Tech Talk about Launchd that will teach you a lot more about job scheduling on Mac OS X.

for i in *; do
    v=`grep 'version = "' "$i/current/$i/"*.info |
      cut -d ':' -f 2 |
        sed -e 's/^version = "/v/' -e 's/"$//'`
    svn cp "$i/current" "$i/$v"
done;

It’s a bit buggy for some modules that have multiple .info files, but I’m sure a few more pipeline stages can fix that. (Which, because I’m done with this at the moment, I will leave as an exercise to the reader.)

Chalk this one up as another testament to the power of shell scripting and how it can help every developer get their job done faster.

Perseus:~ meitar$ pbpaste
Using the pbpaste and pbcopy commands, you can manipulate the contents of the Mac OS X clipboard (or more formally known as the pasteboard) right from the command line. As a brief example, just select the text of this first paragraph, copy it to your clipboard (with ?-c), and then type pbpaste in a Terminal prompt. You should see output similar to the following:Perseus:~ meitar$ 

Pretty straightforward, right? The only thing to be aware of is that the  symbol showed up in the output as a ? symbol. This is because the Terminal doesn’t support Unicode, but that’s a topic for another time.

Anyway, what’s happening here is nothing more magical than simply reading the clipboard and pasting it into a command’s standard output stream. As a result, you can construct pipelines that read from or add content to the clipboard. Here’s an example in reverse, which takes a command’s standard output and replaces the contents of the clipboard with it:

echo "Hello world! I came from the command line, but now I'm in the clipboard." | pbcopy

This command produces no output, but if we examine the contents of the clipboard (by selecting Edit → Show Clipboard from the Finder’s menu bar) we can see that the text we echoed has indeed been copied there.

Another way we can verify that this worked as expected is to simply pbpaste again:

Perseus:~ meitar$ pbpaste
Hello world! I came from the command line, but now I'm in the clipboard.
Perseus:~ meitar$ 

Anyway, this is cool, but it isn’t very useful yet. For that, we need to add some more stages to our pipeline. Let’s take the simple case of trying to count how many words the selected text contains. We’ll use the clipboard contents shown above to do this:

Perseus:~ meitar$ pbpaste | wc -w
      14

Using the word count (wc) utility, we can count words (-w) very easily. Indeed, our previous example does have exactly 14 words in it. We can also count characters (-c) or lines (-l) of text in the clipboard this way. This is like adding Microsoft Word’s “Word Count” feature to every single piece of text you can copy!

As another example take, for instance, the simple case of copying and pasting a snippet of email from Mail. Instead of pasting it back into a text file verbatim, let’s prepend ‘> ‘ to the beginning of each line. This way, when we paste our email’s snippet, we’ll know where the snippet begins and where it ends. This is a simple three-stage pipeline that uses pbpaste to take our clipboard and put it into the pipeline and then reads back the result from the pipeline back to the clipboard using pbcopy. In the middle, we use sed to insert the desired text at the start of each line:

pbpaste | sed -e 's/^/> /' | pbcopy

Now, when you paste your clipboard, you’ll have a greater-than symbol at the start of each line. Naturally, check out the manual pages for all of these commands for more detailed information. For instance, type man pbpaste for more information about the pbpaste command.

Not only was it a great article that highlights a particular XML schema for education-related material produced by the Schools Interoperability Framework Association, it paints the picture of a future I’ve already lived ten years ago. I am heartened to see that, at long last, other people are beginning to see the benefits of technology used for “distance learning.” Frankly, I can’t imagine learning any other way.

I tried to find this article online so I could link to it, but I could not. Instead, I’m reposting the article in full here. If this is available somewhere on the ‘net that I just haven’t found yet, please let me know.

Is Telecommuting the Next Wave for Education? by Kurt Cagle, Managing Editor,xml.com.

In the great analysis game, I have two particularly adept spies – my daughters. Over the years I’ve noticed that both of my daughters tend to be remarkably good barometers about the way that the wind is blowing with regard to youth trends, which in turn tend to be significant because teenagers in particular often tend to be the earliest adopters of new technologies. If something doesn’t resonate with them, no matter how big a marketing budget, it usually doesn’t fly.

One of the things I’ve noticed with my eldest daughter in particular is how many of her friends either are or have been recently “home-schooled”. Now, for many, home-schooling has long been associated with religious organizations, particularly evangelical Christians, who feel that the school system is too secular for raising their kids. However, given that the kids I know (and their parents) are generally not in that community, I was a little puzzled with what was going on.

Instead, what seems to be happening is a phenomenon that I think will have major ramifications for society, and certainly for the tech community. As the Internet was taking off around 2000, a lot of school districts began implementing a program to help those people who were often at significant distances from a school by offering certain classes online – with exercises online, video conferencing and periodic tests. At first, these classes were ones that you would expect to make the migration – science and math courses – but over time, they have extended to cover everything except those classes that require group participation – band, or choir, for example – or need physical facilities, such as wood-working. Physical education requirements could generally be met by agreeing to some form of monitored activity – swimming classes at a recreational center, for instance.

Yet a funny thing happened while setting up these distance learning programs. While remote users became enthusiastic participants in this new wave, the largest group of users have been urban or suburban kids who, for one reason or another, didn’t fit well within the school paradigm. In some cases, the people who took advantage of these courses were students who were involved in focused activities that involved travel – talented musicians who were often on the road, dancers, athletes who were often involved in activities at different schools or other events, or those whose family travelled frequently, who took to the distance courses because they were able to learn better around their other activities. In some cases, the students were people who were going through emotional issues at home – a death or divorce, for instance, though increasingly that has made its way up into those students who just couldn’t face the high-pressure world of middle-school or high-school dynamics.

In other words, the kids and teenagers who were taking advantage of these courses just recognized that it gave them an opportunity to learn in the same way that they are increasingly interacting with the rest of the world – through the computer. Teenagers are hard wired to be more alert in the evening, and typically to be sluggish in the morning, yet school as it is set up right now forces them to be capable of handling complex math and science first thing in the day, when they are generally least reponsive to learning much of anything. Then they are forced to take home dozens of pages of homework that will force them to stay up until late in the evening anyway, meaning that by the time they reach the end of the week, they are physically and mentally exhausted.

Distance Learning lets them combine the homework with the schoolwork, so they can practice new concepts when they’re presented, not after the concepts have become hazy after a full day. It moves them away from the tyranny of the timetable so that if they are having trouble learning a concept, they can spend the time they need to master it, rather than stopping abruptly halfway through because they have to move onto their next class. It also means that if they have mastered a concept, they do not have to sit around bored while others are still trying to figure out something.

It also lets them have access to the rest of the Internet to use as a research tool. While traditional academics may shudder at this notion, as a recent controversy at one university showcased when a student was expelled for setting up a study group on Facebook, the reality is that we’re moving past the point at which we need to keep a vast storehouse of information locked up in within our brains. In an era of information ubiquity, many of the skills that are taught in schools are beginning to seem increasingly quaint, and the teachers that are effective are typically the ones that have managed to incorporate this info-sphere in their own teachings. One social studies teacher of my acquaintance in particular has become quite effective at teaching using PowerPoint, Wikipedia, and other multimedia resources, and he goes out of his way to teach children not the history itself (which they can generally look up) but how to research and analyze that history and take from it any lessons that a given period may have to offer.

Distance Learning programs are also becoming more popular for the same reason that telecommuting is becoming popular – school districts are facing increasing prices for gas and food as a typical family is, but multiplied by several thousand. Many school districts are responding to this by cutting down on the routes that their school buses follow (or in some cases eliminating bus systems entirely) forcing parents to take their kids themselves, often, ironically, increasing the total gas use dramatically and certainly causing headaches for parents who have to integrate their kids into their own commuting schedules. Similarly, school cafeteria programs are being scaled back or eliminated entirely because the cost of the food is becoming painfully high. Add into that aging infrastructures for schools, in a time when the population itself is aging (and hence less likely to fund school initiatives if it affects property taxes), and what you have is a recipe for disintegrating school systems.

Given that, the idea of distance learning as an alternative is one that may be popular at the school board level, as in general, you can generally buy a whole lot of educational training and assistance for what it normally costs to move and feed kids. While it does require some retraining on the part of the teachers, they’re also attracted to it because their job frequently involves trying to keep order over thirty to as many as forty kids, many of whom simply do not want to be there – and the same time-shifting that occurs for the students typically occurs for them as well. Indeed, in many cases what ends up happening is that the teachers pre-record certain segments of their lectures (and are increasingly posting them up on YouTube), then set up one-on-one or group chat sessions with the students.

As for the parents in this equation, I’ve often suspected that one of the reasons for the decline in the quality of contemporary education has been the fact that school has become the place where parents warehouse their kids for the day. Home-schooling does require more parental involvement, but in general it is also far easier for those parents to keep a fairly close eye on what their kids are up to in this day and age when the parents are not at home, at least for kids of a certain age. What’s more, schools are in many ways like hospitals – while their ostensible purpose is education, the chance of their kids learning bad habits is far higher in school settings than they generally are at home – and the kids that are trying to be good aren’t distracted by the ones determined to make trouble.

That’s not to say that there aren’t distractions. My eldest daughter has been known to keep a chat window up talking to friends when she should be concentrating on homework, though in fairness to her, what she’s chatting about often involves that same homework, albeit in a rather disconnected fashion:

Kat: I'm :-( ):-\. Zuko shippeded Kataara. Urgh ..
Neechan: Urrgghhhhh.
Hey, what's I if volts is 10 and ohms = 3 % (can't write the squiggle for
ohms)?
Kat: Um V =IR so, uhm, I is RV ...
Neechan: No, R over V.
Kat: Oh. Right. :-) .... so 3 divided by 10 - .3. Bad Zuko! ...

Is this bad? No – it’s only different from the way that those of us who grew up pre-Internet see the world. Distance Learning does not change the amount of homework (though I suspect that it cuts down pretty dramatically on the makework that tends to deaden interest in a subject rather than reinforcing the concepts), nor does it change the need for accountability. The kids still need to test, and still need to show that they have learned, but I suspect that their retention rates will likely be considerably higher if they can learn in a way that works well for them.

Long term, I think that this will likely end up deconstructing the traditional school system, though this is a process that will take decades to happen completely. It means, for the aggressive learners, that they could in fact complete a formal curriculum in a fraction of the time necessary, though a good teacher can work with those particular students to provide additional areas of study for them to engage with. It means that slower students can learn at their own pace, and can generally be flagged for additional help if they fall sufficiently behind. Money that school districts save in terms of providing physical infrastructure in transportation can be spent on those activities that do promote socialization – music, art, theatre, sports, civic days, and so forth – and monthly activities that bring students together can keep the bond in place of school as community.

Distributed education is part of the larger process of social redistribution that is occurring because of the Internet. Modern education emerged about the same time that the modern corporation emerged – in the 1930s – during an era when the dominant forces at play involved hierarchies, centralization, consolidation and economies of scale. In this decade, the dominant forces are network related – decentralization, the economies of global localities, the disintermediation of authority and the a shift away from the geographical. Just as these forces are resisted at the corporate level despite the obvious benefits (and just as workers in places that can telecommute are increasingly doing so) , these same forces are resisted at the educational level with much the same results – students (and their parents) are taking advantage of any loophole they can to make it available to their kids because it results in a better education for them).

So far, most educational software is boutiqueware, typically Flash-like applications. This won’t necessarily change moving forward (it is hard in general to make educational software that doesn’t have a strong “games-like” component, and for the most part that game approach makes the educational software far more engaging than it would be otherwise), but one thing that will need to happen as the distributed systems move forward is for the emergence of some kind of general framework for the exchange of educational related information. One particularly promising start in this direction is the development of the Schools Interoperability Framework (SIF) (http://www.sifinfo.org/sif-specification.asp), which is an XML standard most recently updated in March 2008. Its mandate is fairly broad – providing XML schemas for describing grade-books, library services, student information, instructional services and so forth – (see Figure 1. SIF Zone Services).Figure 1. SIF Zone Services.

One of the jobs that the XML community needs to do is to reach out to the developers of educational software and insure that they are aware of the SIF standards, in order to provide better interoperability between their core applications and the growing educational educational noosphere, and to reach out to educators and education IT departments (which are all too typically the math teacher in his spare time) to make them cognizant of these same standards and to help implement solutions around these standards. As more students opt to go “virtual” the ability to maintain consistent, and more important interoperable, records becomes ever more paramount.

Long term, the move towards distributed education will shape society in some very profound ways. The kids going to school in such an environment today are more adept at the art of self-education, are usually more capable at analysis and research, and because they managed to avoid the often harsh emotional trauma of dealing with several hundred other kids of the same age daily, usually have more self-confidence than their school-based peers. They’ll likely have little patience for the Tayloresque approach to college education and will continue their lives in a similar manner by educating themselves within the college’s online environment (and will tend to shun those colleges that don’t offer such services) and when they start coming into the workforce in sufficient numbers, they will reshape the way that organizations are set up. On the flip side, I think this is likely to cause a huge amount of cultural friction between this generation and those educated more traditionally in previous generations, because their respective realities will be very different.

So far, the movement is still just a trickle, but watch this space closely – it will become a flood soon enough – and the bricks in the wall will come tumbling down (to paraphrase Pink Floyd).

Possibly the best example of this phenomenon in action is Wikipedia, which hosts several different versions of its pages. The version everyone knows about is the supremely academic one, the one Wikipedia presents by default. Here’s an excerpt of one such page’s introduction, the Wikipedia entry for the Standard Model of particle physics.

The Standard Model of particle physics is a theory that describes three of the four known fundamental interactions between the elementary particles that make up all matter. It is a quantum field theory developed between 1970 and 1973 which is consistent with both quantum mechanics and special relativity. To date, almost all experimental tests of the three forces described by the Standard Model have agreed with its predictions. However, the Standard Model falls short of being a complete theory of fundamental interactions, primarily because of its lack of inclusion of gravity, the fourth known fundamental interaction, but also because of the large number of numerical parameters (such as masses and coupling constants) that must be put “by hand” into the theory (rather than being derived from first principles).

Contrast the above with this excerpt for the same page, the Standard Model of particle physics, taken from the simple English version of Wikipedia:

The Standard Model of physics is the best idea to say how fundamental forces and elementary particles work. It uses quantum mechanics and special relativity. In physics there are many different particles and forces, the Standard Model says that all particles and forces are only two different types: fermions and bosons.

Okay, now that’s a lot easier to understand. In this example, the simple English version is a lot shorter, and at first glance that might strike you as its major distinguishing factor. However, if you read closer, you’ll notice many things specific to the language that was used that serve to give the simple English version much more accessibility than the academic one. Some of these things include:

• Simpler, more familiar vocabulary. Instead of using surgically-precise words that may not be familiar to an uninformed reader, plainer words (and no less accuracy) are used to describe concepts.
• Dense sentences are broken up into smaller chunks. When accessibility or successful communication is the primary concern, longer sentences that deliver more information in one punch may be counter-productive. Instead, it’s often better to chop up larger concepts and deliver them in smaller-sized chunks that are easier to digest.
• Specifics are introduced one at a time, and defined at each instance. Possibly the most common error writers (especially technical writers) make is introducing lots of interdependent ideas at once or without proper prior context. Rather than work your way from a complicated idea to a simple conclusion, work instead from a simple foundation to a complicated idea, building vocabulary as you go (see point the first about vocabulary).

Of course, this is always easier said than done, and it is also why simplicity is intuitively understood by lots of people to be a hard thing to create. Presenting things simply is a challenge because it requires more knowledge than simply understanding the thing; it requires understanding the thing and understanding what pieces of the thing your audience does not (yet) understand. The value of simple lies in being able to fill those gaps.

But then today I was glossing over Apple’s featured features list and I got even more excited. There are the usual, largely meaningless, fluff updates that are nice for Joe Schmo or his mother, but that power users simply don’t care about, like the new iChat support for animated buddy icons, but the list is also chock-full of really cool, really useful features.

What’s interesting is that a good deal of these features aren’t really new features at all. For instance, if you knew how to manipulate the NetInfo database on your Mac, you could already share any folder via Apple’s “Personal File Sharing” feature. (Here’s a Mac OS X Hints hint explaining how to do it.) In Leopard, however, Apple claims that this functionality is now integrated straight into a folder’s Get Info… window. If it works as smoothly as Apple claims, this is finally going to bring Mac OS X (client) into decent competition with Windows XP Professional in terms of GUI-level power-user features.

However, while all of these features are really cool, here’s a list of the ten geekiest features I will probably absolutely love, for one reason or another.

• Ruby on Rails, out of the box — The hot thing in web development right now is Ruby on Rails. Macs have already been the best personal desktop and web development platform because they have built-in support for the Apache web server and a host of other features, but now they will come with a ready-to-roll installation of Ruby on Rails, sporting Mongrel and (better yet) Capistrano! Specifically with the addition of Capistrano, which is terribly undersold as simply a Ruby on Rails deployment platform, these UNIX-y “toolbox” items are bound to make Macs that much more useful right out of the box.
• Safari’s full history search — As their recent public partnerships with Google have shown, Apple is very clearly invested in search technologies. Spotlight gets a huge number of improvements in Leopard, but none which I think are going to be more useful to more people than this one: spotlight searches on the full text of each web page in your visited history list. That’s just awesome. Also awesome: using spotlight as a calculator and as a dictionary, which also shows just how Google-like Apple is trying to be. (Google also lets you ask it arithmetic questions and a dictionary.)
• Wikipedia articles in Dictionary.app — I love Wikipedia because it’s one of the fastest ways to get (relatively) reliable information quickly. Now that Dictionary.app has built-in integration with Wikipedia, imagine the possibilities for getting that knowledge instant-gratification craving fixed. Apple has not yet announced this capability, but I can easily envision a scenario where all Cocoa text fields are instantly “wikified” (with text that matches Wikipedia articles highlighted) much in the same way that current Cocoa text fields allow you to right-click on a misspelled word and have it corrected by Dictionary.app.
• Application-based firewall — In classic Apple fashion, functionality that was previously available via third-party additions is now available from Apple itself. In this case, I have to wonder how well Apple’s updates to its firewall will obviate the need for Little Snitch, which is basically an application-based firewall, too, and a good one at that.
• Built-in guest log-in account — If you’re as paranoid about security as I am, you’ve already created a special, limited-access user on your system (called Guest or Visitor or whatever) and whenever friends are over, you tell them to use that account instead of your own. Now in Leopard, Apple has gone through the trouble of setting this up for us already. A small change that is going to have a big impact.
• Scriptable System Preferences & applications — With AppleScript, you can automate the things your computer does with scripts, as long as those things are “scriptable.” In previous versions of Mac OS X, huge gaping holes of what things shipped by Apple were scriptable existed, causing me (personally) some really annoying headaches. AppleScript GUI scripting helped me get around many of those roadblocks, but now it seems Apple is finally filling in some of the most notorious gaps in this functionality with scriptable System Preferences. Yay!
• Automator workflow variables — Automator brings the power of AppleScript I just mentioned to more people with a completely graphic programming environment. There is no need to open up a text document and write AppleScript code because Automator lets you create a script (called a Workflow in Automator jargon) using your mouse by dragging and dropping actions into the order you want them to be performed. It’s very slick, but until now it’s been very limited. With Leopard, Apple is beefing up Automator so that it includes things like variables, basic programmatic capability that was sorely lacking before. (Also majorly cool: a command-line utility to access Automator!)
• Finder.app’s path bar — Every serious Mac user knows that the Finder needs a lot of help. Now, it’s getting some. Something the Windows Explorer has had forever (as had every desktop environment for Linux, of course) is a visual cue to show you where in your filesystem tree a given folder is located when you are viewing said folder. Now the Finder gains this capability (though Apple’s description implies that it’s going to be off by default) with what Apple is calling a “Path Bar”. Finally!
• Cocoa and scripting bridges — Even though no one really seems to know about it, it has long been possible for languages other than AppleScript to do things like send Apple Events to Mac OS X applications. Specifically, Ruby and JavaScript, two of the most well-known web development languages in existence, can already do this with a single ScriptingAddition (OSAX). But now Apple is making this functionality a central feature and fully extending it to their Objective-C (and Cocoa) language and applications such as Xcode and Interface Builder. This means people like me will have a shallower learning curve before we’re able to create full-fledged, native Mac OS X applications. Now that’s exciting!
• Xcode 3 refactoring — This is something you kind of have to see to believe. I got the opportunity to see it demoed at Apple’s Leopard Tech Talks last year and I was really excited by it. With the new Xcode, Apple’s development IDE, you can do away with find-and-replace searches for things like renaming functions because Xcode understands what parts of your code are what structures and, when you tell it to “change the function named myFunction to myNewFunction,” it’ll only find-and-replace function names instead of every instance of the string “myFunction.” That’s pretty big, and if it were available for more languages, it’s almost enough to make me ditch vim.

So there you have it. Ten features you might not have already known about that are some of the most promising features I can see in Leopard. And I didn’t even get into Wide-Area Bonjour, which could make services like DynDNS or No-IP a thing of the past (and which I still want to learn more about), or the new Terminal application (finally with tabs!), or even the multiple user certificates for S/MIME encrypted email.

Note: One of the least known security features available on Mac OS X is also possibly one of the best, and the simplest. Evidently, all Intel-based Macs are shipped with the XD (aka. NX, aka. DEP) bit turned on—and thankfully there doesn’t seem to be any way for users to turn it off. However, this isn’t a silver bullet and if you want to learn why you should check out this excellent Anandtech article: A Bit About the NX Bit.

I’m tempted to give it a shot myself, for many reasons. I think using Google Docs to compose and edit my blog posts may make it easier to:

• keep copies of my blog posts somewhere other than my blog (just in case),
• work on my blog posts from anywhere with an Internet connection (and under the security of an HTTPS connection!),
• version-control all of my writing without changing my workflow the least bit,
• invite specific users to preview my drafts before they go live and even help me edit my posts so I can catch typos and grammatical errors,
• easily export my writings to a variety of formats, such as PDF or a ZIP file with the ease of a single menu,
• and of course, instantly take advantage of whatever wonderful things Google has in their pipeline.

Another blogger wrote an entire how to article showing tips for using Google Docs to blog that is quite a good read, as well.

Voila. This system is flawless. You will never be able to send me loads of spam that go anywhere but my spam box, and I hardly ever look in there.

Naturally, there is a caveat to using this technique, but I actually consider it to be an advantage. By necessity, this technique, keeps me pro-active about getting people’s contact information when I meet them (and want to talk again). If I don’t get that person’s email address, I’ll probably never see that person’s email unless I’m looking out for it. Nine times out of ten, however, that’s what I want to have happen anyway.

So this solves the problem of unwanted mail. However, what if I want to let people contact me that I don’t know ahead of time or have previous whitelisted? Well, in that case I rely on an out-of-band communication, such as an introduction from a friend, leaving a comment on my blog(s), or some other method such as an instant message to let me know that there is someone who wants to talk to me.

My contact information is so available (in so many places), and many IM services are now equipped with store-and-forward messaging that there really is no reason for email to be the first time I hear from someone. Even better, if I’m contacted over Google Talk (as an example), I automatically have an email address for that person.

Voila. Simplest. Spam. Filter. Ever.

This improved version of the post-commit hook does the same thing as the last one (that is, it logs into your web server over SSH with the given user and password, and yes, I’m aware of the scariness of embedding a password in such a way, so you should really set up SSH to use public keys for authentication for this), except now it also produces useful output.

Here’s the same script as before, but improved:

#!/usr/bin/expect -f

#
# AUTHOR: Meitar Moscovitz 
# DATE  : Thu Jun 21 16:32:42 EDT 2007
#

set HOST my.web.server
set USER someuser
set PASS xxx

# the working copy we're going to update
set WC /path/to/working/copy

# the path to the svn executable on the remote web server
set SVNBIN /usr/local/bin/svn

# our network is slow, set a long timeout
set timeout 30

##### DO NOT EDIT PAST THIS LINE! #####

# POST-COMMIT HOOK
#
# The post-commit hook is invoked after a commit.  Subversion runs
# this hook by invoking a program (script, executable, binary, etc.)
# named 'post-commit' (this file) with the
# following ordered arguments:
#
#   [1] REPOS-PATH   (the path to this repository)
#   [2] REV          (the number of the revision just committed)
#
# Note that Subversion does not provide this program with an environment
# of any kind. That means this program lacks a current working directory,
# a home directory, a $PATH, and so on.

set REPOS [lindex $argv 0]
set REV [lindex $argv 1]

# Define error codes
set E_NO_SSH     1 ;# can't find a usable SSH on our system
set E_NO_CONNECT 2 ;# failure to connect to remote server (timed out)
set E_WRONG_PASS 3 ;# password provided does not work
set E_UNKNOWN   25 ;# unexpected failure

# find the SSH binary on our system
if {[file executable /usr/bin/ssh]} {
	set SSHBIN /usr/bin/ssh
} elseif {[file executable /usr/local/bin/ssh]} {
	set SSHBIN /usr/local/bin/ssh
} else {
	send_error "Can't find a usable SSH on this system.\n"
	exit $E_NO_SSH
}

spawn $SSHBIN $USER@$HOST $SVNBIN update $WC

expect {
    "continue connecting (yes/no)? " { send "yes\r"; exp_continue; }
    -nocase "password:" { send "$PASS\r"; }
    timeout {
        send_error "\nWe have timed out after $timeout seconds while trying to connect to $HOST!\n";
        exit $E_NO_CONNECT;
    }
}

expect {
	-nocase "password:" { ;# if we are asked for the password again, then we have provided the wrong password
		send_error "\nCan not log in to $HOST because the password provided for user $USER has been rejected.\n";
		exit $E_WRONG_PASS;
	}
	-re "revision (\[0-9]+)." {
		if {$REV == $expect_out(1,string)} {
			send_user "\nSuccessfully updated $WC on $HOST to revision $REV.\n"
		} else {
			send_user "\nUpdated repository to revision $expect_out(1,string), but svn reports that we are at revision number $REV.\n"
			send_error "CAUTION: Repository updated to revision $expect_out(1,string), but committed revision $REV.\n"
		}
	}
	default {
		send_error "An unexpected error has occured. The process at spawn ID $spawn_id has produced the following output:\n"
		send_error $expect_out(buffer)
		exit $E_UNKNOWN
	}
}

Still, however, the problem was not exactly straightforward. We needed to execute a svn update command on a server other than the server on which the Subversion repository was being hosted. Shell scripts are the obvious choice for command-line automation in UNIX, but they don’t deal with interactive commands very well. So instead of writing a shell script, I wrote an expect script.

This expect script is really basic. There’s a better one in a future post on this topic.

#!/usr/bin/expect -f

#
# This expect post-commit hook connects to staging-webserver
# and updates the working copy hosted there to the latest checked-in code.
# This means that whenever code is committed to the repository, the web site hosted
# will always be running the latest version of the code.
#
# AUTHOR: Meitar Moscovitz
#

# POST-COMMIT HOOK
#
# [...]
#

set REPOS [lindex $argv 0]
set REV [lindex $argv 1]
set HOST staging-webserver
# Use update-user to log in to staging-webserver
# to update the working copy, but this can probably be improved so as not
# to expose this user's password.
set USER update-user
set PASS update-user's-password

spawn /usr/bin/ssh $USER@$HOST svn update /path/to/web/site/directory
expect "continue connecting (yes/no)? "
send "yes\r"
expect "password: "
send "$PASS\r"
expect eof

There’s one really tricky bit to this script, which is the understanding that when Subversion runs the post-commit hook, no environment information is passed to this script. As a result, there is no home directory or path information set for this executable. This is why everything is defined using absolute paths. Also, because there is no home directory for update-user, the expect script will always be prompted by SSH to re-verify the server’s identity. So rather than just expecting “password: “, we always expect “continue connecting (yes/no)? ” and say yes, and then send our password.

Note, of course, that update-user should be a user with limited access to the system, yet enough so that he may update the working copy on the web server. I’m sure there is probably a more secure way of doing this as well, so any sort of feedback on securing this or scaling it would be welcome.

I’m beginning a new project at work where, for the first time, I’m working with a large team of developers, and not just integration developers but front-end developers like myself. Since I’m the new guy, I’m finding myself required to learn the teams established vocabulary and this is making me realize just how much of my own vocabulary I have developed myself without even realizing how large it’s grown.

It all comes down to how we organize our code and that, of course, depends very much on how we are used to thinking about things. On a team of people working together for a common goal, such implementation-specific details are much more central to the success of the project than when working alone, such as in a freelance situation.

It’s interesting to me to see how many of the the established coding standards and naming conventions are obviously built off of the same kinds of ideas that I’ve already had and have been using for quite some time, and yet how different they are in implementation. This extends beyond the obvious case of using dash-separated class and ID names instead of camel case lettering or vice versa. It goes so far as to project management and workflow considerations, file name conventions, and semantics.

In some cases, optimizations are actually sacrificed on a team in favor of clarity, but this is actually a good thing. In my freelance work, I typically squeeze every last ounce of optimization into a project by using shorter names, more optimized files, and extremely dense code to ensure the best possible result. In a project with other developers however, the success of the project is far more dependant on everyone’s collective understanding of the code, so clarity is and should be prioritized above optimizations.

Well, obviously, success on a large scale means accomplish “much more” than success on a small scale. Also, success on a large scale often means working with more than one person where collaboration is a must. So what are the tools people use to accomplish this success?

The most well-known tool is management. Implement a structure in an organization that is overseen by people whose job it is to make sure success happens on varying levels of scale. The larger an organization gets the more managers it needs, not because of personnel (technology advancements and process improvements in Human Resources have significantly the amount of employees a company can keep track of) but rather to ensure that this army of personnel is not going to lose sight of whatever is deemed the successful goal in their appropriate “big picture.” For CEOs, this means the company’s bottom line. For regional managers, this means their region’s bottom line. For individual supervisors, this means success completion of the task their division was handed, and so on and so forth.

Clearly, this is an important and necessary part of doing business. But I theorize that businesses and large organizations tend to see the act of adding more management to their organization as an act of ensuring a return on their investments which is probably overestimated. A manager is someone you have to pay (often a lot more than other forms of employees), and whose job actually adds to the complexity and opaqueness and beurocracy of an organization. At some point, this must be more harmful than helpful.

So what else can be done? I would suggest that an often very underestimated and underutilized area of increasing ROI is by improving knowledge transfer and knowledge sharing between employees. Why? Because it builds on the concepts and proven methodologies of small-scale success wherein many small successes brings larger success naturally, rather than forcefully. I see the largest contributor to small-scale success (most explicitly evident in personal successes, probably the smallest scale you can get) as the increase of knowledge. My own experience has taught me that I am more productive now than I was one year ago because I can do more than I could do one year ago, and I can do more now than I could then because I know more about the world and the objects within it.

Applying this to a larger scale is simple in theory but difficult in practice. In theory, the more smart people an organization has working for it, the more collective knowledge is stored within that organization. In practice, this is actually a huge problem that the solution called management is attempting to alleviate because smart people tend to be very insular and self-driven. This leads to a phenomenon I have begun calling knowledge fracturing which ultimately results in siloed and specialized skill sets with little ability to expand beyond that specialization. This is so significant a problem in large organizations that entire industries and bodies of knowledge have been created to help alleviate workflow problems and help facilitate communication.

Now, this is good, because a prerequisite to knowledge sharing is communication. You can’t share what you know if you can’t communicate with someone who can listen. But this is only the first step. Certainly, an organization can ensure better ROI with communication tools, but the next level–which I have yet to see anyone focus on too obviously–is knowledge sharing; communication with the intent of making available the information one has accumulated to the point that enables one to share the information about one’s job function or skill set.

This is a concept that is actually extremely scary to most people. There are two oft-cited rebuttals to this. The first involves a concern for job security. If everything that I know about how to do my job and why is written down for fellow employees to see, what need is there for me? I maintain that this actually makes you as an employee more valuable, rather than less, because suddenly you are a teacher, a reference guide, an invaluable “go-to guy.” The more you share, the more clearly people understand what you know and what you’re good at. Sharing, at its roots, is all just about showcasing what you’re good at. How can that be bad?

The second involves a concern for an organizations information security, wherein too much knowledge sharing can actually be security breaches and information leaks that are unacceptable to the organization’s policies. Obviously, while this is an important issue, it is not a direct argument against the validity of knowledge sharing but rather an objection to a side-effect of it if it is done carelessly. As with all things, I think most people will agree that care should always be taken in everything you do. This is just one more example.

So why does knowledge sharing actually increase the overall ROI of an organization? I theorize, based solely on my own experiences, that the majority of time and expense that is not directly spent on progressing an organization’s goals are spent trying to navigate a system or accomplish a task that is only difficult because it is not understood. If it were understood, the procedure would take less time, or the company would not need to hire a consultant, or whatever the situation may be. The point is that if you know what you’re doing, you’ll do it more effectively and more efficiently.

Successful knowledge sharing increases the productivity, effectiveness, and efficiency of your organization by enabling your staff to accomplish orders-of-magnitude more than they would otherwise have been able to.

With that as a foundation, let’s next explore how an organization might go about successfully sharing knowledge. And as you may have guessed, I have one word that: wiki.

(I have not yet written the next part.)

That said, I have been learning a lot lately. I’ve gotten to put my hands on software I’d never have otherwise gotten to touch (like Solaris 9 and Red Hat Enterprise Linux), I’ve learned more about Subversion and how to use it than I thought I would have in the little time I’ve used it, and my practical networking knowledge with regards to things like VPN routing have been substantially increased. Even more interesting than all of this new technical skill, however, is all that I’ve experienced on a cultural level.

I am now certain that what I am experiencing is intense culture shock. Let me explain.

The first thing people notice when you move to another country is probably the way people speak. Specifically, the two things people most often comment on is the language being spoken, and how it sounds (accent, common sounds, etc.). Similarly, the first thing I noticed when I got hired was the way people spoke, and what they were saying. The heavy use of acronyms are not new to me, but at the amount of company-specific technical jargon is hard to decipher when there is no Wikipedia to search. And farbeit for there to be a single, updated glossary that is easy to reference, either, or have someone offer to expand an acronym without you asking for them to do so.

Even more bewildering than the array of new acronyms, however, was the use of what I have come to call businessese. This is a language whose apparent sole purpose is to make whatever you are saying sound very important and hard to accomplish. An example would help clarify this.

The other day I was asked to review a written document. It was written by a colleague of mine. Our mutual boss, probably wanting me to get familiar with the look and feel of the documentation, sent it to me and said something like this in an email:

Meitar, here is [document X]. Please ensure it is ready for delivery to the client.

What he actually meant was something like this:

Meitar, here is [document X]. Please proofread this for any errors and then let us know so we can pass it along to the client.

My lesson from this experience, and from numerous similar experiences lately, is that people in the corporate business world don’t tend to actually say in plain English what they want you to do.

Another interesting point of fact I’ve encountered lately is the incredible difference it makes in the technical arena when you communicate with precision. This is difficult to do, because there is often an unacceptable latency if you speak with too much precision. This is why acronyms are so useful, if you know what they mean. In other words, if you already know what a web browser is, there’s no need for me to specifically name your web browser when I tell you to go to a particular web page. But if you didn’t know what a web browser was, then I would more likely get my message successfully delivered to you if I told you to type a certain string into the address bar of a particular program, say Firefox, instead of saying “go to this web address in your browser.”

This is also nothing new to me, but this week I am working with two colleagues who are utter opposites of each other in this respect. One colleague, whom I’ll call Descriptive, speaks slowly and clearly when he talks and assumes just enough technical familiarity with the subject to speak somewhat concisely, without sounding condescending. He is also quick to offer expansions for acronyms or further explanations of specific sequences of actions when he senses they are needed, as well as omitting them when he notices the listener is loosing focus. It’s very obviously a skill.

The other colleague, whom I’ll call Snappy, speaks so quickly I can’t tell when he’s taking a breath (if at all). He absolutely never expands acronyms, nor does he provide any sort of context for what he’s saying, such as a window’s title or an a filesystem location. He reminds me more of the character Nick Burns from the SNL skits of “Your Company’s Computer Guy” than anything else. I don’t doubt his technical ability at all (he’s very obviously quite skilled), but after mere days, I find myself dreading the prospect of working with him in the future.

There are thus two ultimate conclusions I can draw from these obervations (at least right now). Since I know that I am far too much in love with precision and clarity (and usability) to tolerate any confusing imperfections in what I create, I have the potential to become very well-liked for my helpful nature if I learn enough that I can actually answer questions when they arise. This bodes well for me.

Concerning, however, is the possibility that all this businessese nonsense will ultimately get to me. I truly, truly despise it and try hard not to cringe whenever it rears its head. I can’t understand for the life of me why my bosses don’t talk like humans nor can I understand why the majority of amazingly bright people (many of whom are more technically adept than myself) can’t be bothered to make the slightest effort to think more than one or two steps ahead of themselves. There are, of course, exceptions, but the fact remains that these are exceptions and not the rule.

I am notorious for conjuring up utopian visions in my head and then not settling for anything less. Here we go again.

The whirlwind of events that lead to this new job have been the most compelling evidence I have ever encountered strengthening my arguments for the importance of relevancy. Throughout all my experiences, I have consistently been given sage-sounding advice: “Be patient. Put your time in now, and that will happen.” I’m sure you’ve been told the same thing countless times, in different contexts applicable to whatever situation you were in.

This statement, when taken literally, is in fact rather wise. Essentially, the advice is reminding you to choose your battles, and that’s a very smart (and mature) thing to do. However, most people hear two very distinct things when given this advice which can be paraphrased thusly:

1. Wait.
2. Compromise your goals.

Worse, some people actually mean these things. I will agree that the first point is indeed sage advice because it is unavoidable. I will forever continue to disagree with the second because doing so is nothing but a waste of time.

I first remember being given this advice in second grade, when I remarked that I disliked my teachers. Be patient, I was told, it would surely get better. And if it didn’t, well, next year will be different. Next year was different, but I was equally displeased with the school. Don’t worry, I was told, just put in your time at school and eventually it’ll pay off.

But that’s a lie. Putting your time into something never pays off, ever. Doing things pays off. So if you’re putting your time into something doing something you can’t see paying off, why continue to do it? That was the crux of my argument against my traditional schooling (which I eventually broke free of at 16) and was also the motivation for taking this new job opportunity.

I am notorious for being impatient. I rarely argue this fact with anyone because, by all measure, I truthfully am impatient by the standards of those who declare me to be impatient. However, all I see is a difference in priority. I am far more interested in the timeframe necessary to achieve my desired results and I am also aware of many possible paths to take to obtain said results. Many of these paths take a lot longer than others. When confronted with a choice of which path to take, I weigh my options and ask myself why I would take a longer path?

Is there some benefit to the longer path than the shorter one? Would the longer one yield a higher quality result? Is there some cost to the shorter path? Would the shorter one require too many resources, more than I can afford? Most people never get to see other people making these choices. They only see the actions one takes, and that can be misleading.

Yes, learning to take the good with the bad is an important and valuable skill. Yet I insist on reminding people that it is a fine line between that and taking the bad for the good. I would never allow myself to do that, to compromise my goals, their quality, or their timeframe, for something that just isn’t worth it. For me, one of those things that just wasn’t worth it was schooling. A more recent example of something that just wasn’t worth it was the slowness with which advancement from within Apple Computer could move my career forward and become more in line with my passions.

I’m excited, if slightly nervous, about starting with Opsware. I think (and hope) that one of the reasons I was hired was because of the very attitude I expressed during this post. If so, then I’m sure I’ll have a fantastic time and be a great asset at the same time.

Don’t expect to see too much about Opsware here. I don’t write about work-related things very often, if at all, but I sure hope that my experiences there will provide much fodder for more material in the future. And in other news, I took and passed the CompTIA Security+ certification test (the day after my Security+ prep guide came in the mail, go figure).