Everything In Between

If your project so much as pretends to have a profit motive, I will tell you to go fuck yourself and your project.

Archive for the ‘Geeky’ Category

Easy template injection in JavaScript for userscript authors, plugin devs, and other people who want to fuck with Web page content

leave a comment

The Predator Alert Tool for Twitter is coming along nicely, but it’s frustratingly slow going. It’s extra frustrating for me because ever since telling corporate America and its project managers to go kill themselves, I’ve grown accustomed to an utterly absurd speed of project development. I know I’ve only been writing and rewriting code for just under two weeks (I think—I honestly don’t even know or care what day it is), but still.

I think it also feels extra slow is because I’m learning a lot of new stuff along the way. That in and of itself is great, but I’m notoriously impatient. Which I totally consider a virtue because fuck waiting. Still, all this relearning and slow going has given me the opportunity to refine a few techniques I used in previous Predator Alert Tool scripts.

Here’s one technique I think is especially nifty: template injection. That’s a fancy way to describe changing the source code of a Web page on-the-fly. Basically, any time you need to change something on a Web page, or any time something on the Web page changes that you need to intercept (for which I use Mutation Observers). As you can imagine, this is central to the way Predator Alert Tool userscripts work, as well as most browser plugins and the like.

The idea behind template injection is to define a set of replacement strings that you can use in a “template” (itself just a string of HTML). Then, insert the template at a specific point in the DOM. The core of my simple template injection system is a function called (wait for it) “injectTemplate” and it looks like this:

PATTwitter.UI.injectTemplate = function (name, params, ref_el, injectMethod) {
    var injectMethod = injectMethod || 'appendTo';
    var t = PATTwitter.UI.Templates[name];
    var tags = PATTwitter.UI.Templates.Tags;
    for (x in tags) {
        t = t.replace(new RegExp(x, 'g'), tags[x](params));
    return jQuery(t)[injectMethod](ref_el);

You can specify in injectMethod in the fourth parameter to injectTemplate, which is expected to be a jQuery DOM manipulation function. It defaults to .appendTo() but you could also pass insertAfter, for instance, to invoke .insertAfter() on the reference element (ref_el).

The magic happens in the for loop with the call to t.replace(new RegExp(x, 'g'), tags[x](params));. The variable t is a template that I’ve previously defined, like this:

PATTwitter.UI = {};              // FlightJS components and such.
PATTwitter.UI.Templates = {};    // Stores template HTML for UI components.
PATTwitter.UI.Templates.dashboard_list_stat = '\
<li id="pat-twitter-pat-lists" class="DashboardProfileCard-stat Gric-cell u-size1of3">\
    <a class="DashboardProfileCard-statLink u-textCenter u-textUserColor u-linkClean u-block" title="%PAT_TWITTER_ITEM_COUNT% PAT Lists" href="https://twitter.com/%PAT_TWITTER_SCREEN_NAME%/lists#pat-twitter-twitter-blocklist">\
        <span class="DashboardProfileCard-statLabel u-block">PAT Lists</span>\
        <span class="DashboardProfileCard-statValue u-block">%PAT_TWITTER_ITEM_COUNT%</span>\

Notice that since the template is itself a JavaScript string, every line ends with a backslash (\). In the template, there are placeholders in the form of strings like %PAT_TWITTER_USER_SCREEN_NAME%. Each of these is the key to a property stored in an object so that the template tags reference functions that all return strings, like so:

PATTwitter.UI.Templates.Tags = { // Helpers for quickly injecting HTML.
    '%PAT_TWITTER_PREFIX%'          : function () { return 'pat-twitter-'; },
    '%PAT_TWITTER_ITEM_ID%'         : function (item_info) { return item_info.id; },
    '%PAT_TWITTER_ITEM_NAME%'       : function (item_info) { return item_info.name; },
    '%PAT_TWITTER_ITEM_NAME_DASHED%': function (item_info) {
        if (item_info.name) {
            return item_info.name.replace(/ /g, '-');
    '%PAT_TWITTER_ITEM_COUNT%': function (item_info) { return item_info.count; },
    '%PAT_TWITTER_USER_SCREEN_NAME%'     : PATTwitter.getMyTwitterScreenName,

So when we invoke injectTemplate(), we’re simply iterating over the template string with the for...in loop, and finding-and-replacing the tags with the result of the function to which the tag points,the call to tags[x](params).  For the sake of completeness, here’s the PATTwitter.getMyTwitterScreenName function:

PATTwitter.getMyTwitterScreenName = function () {
    return jQuery('.js-mini-current-user').data('screen-name');

Again, the point is that it returns a string.

Since injectTemplate() returns a jQuery collection, it can be used like this:

var lists = PATTwitter.getMyLists(); // returns an Array
var injectedElement = PATTwitter.UI.injectTemplate('dashboard_list_stat', {
    {'count': lists.length }
}, jQuery('.DashboardProfileCard-statList'))

Now injectedElement is a jQuery object holding attached DOM nodes. If you’re using something like Flight JS as Twitter does, you can create a simple component and attach your component to the injected code like so:

var myComponent = flight.component(function () {
    this.onClick = function (e) {
        alert('I was clicked!');

    this.after('initialize', function () {
        this.on('click', this.onClick);

And there you have it. The result of all this, in the current Predator Alert Tool for Twitter prototype, is that when you load your Twitter homepage, an additional “Dashboard statistic” is displayed underneath your follower count and so on. It looks like this:

The Predator Alert Tool for Twitter modifies various parts of the Twitter Web interface using a simple template injection technique. Here you can see an additional row added to user's profile Dashboard statistics displaying the number of Predator Alert Tool Lists they are subscribed to.

The Predator Alert Tool for Twitter modifies various parts of the Twitter Web interface using a simple template injection technique. Here you can see an additional row added to user’s profile Dashboard statistics displaying the number of Predator Alert Tool Lists they are subscribed to.

As Dr. Seuss might say, inject it in a loop. Inject it for a hoot. Inject it with a toot.

Um. Yeah. Okay, I’m done being nerdy now.

Written by Meitar

May 14th, 2014 at 3:43 am

On Being a Social Cyborg: How iCalendar Helps Me Fight Loneliness


Here’s a topic I’ve been meaning to write about ever since I was deeply depressed last Fall and Winter. Back then, I was incredibly lonely, and despite my best efforts I simply found it damn near impossible to do anything to improve my situation. That’s because my “best efforts” consistently lead me to dead-end resources that sounded good but that had no practical or immediately useful information; resources like WikiHow.com’s “How to Deal With Loneliness” article.

In their article, WikiHow contributors write:

Get involved in anything where you will meet people. If you are very shy, find a group for social anxiety, even if it has to be online (obviously it’s better if it’s not). Look on places like Craig’s List, or local news websites for your town for activities in your area. Volunteering can help. But don’t attend functions with the idea of making friends or meeting people. Being too demanding is a sign of loneliness. Try to go with no expectations whatsoever, and to enjoy yourself regardless of what happens. Look for activities that interest you, that also involve groups of people, like intramural sports, book clubs, church groups, political campaigns, concerts, art exhibitions, etc.

While it all “makes sense,” the WikiHow article reads like an elaborate horoscope. It’s incredibly annoying because it contains no meaningful, discrete, actionable items. Where, exactly, can I find “activities in my area”? And once I find them, how do I make sure I know about them when they are happening? And as if that wasn’t hard enough, how do I make the process workable under the extreme energy constraints that being depressed and lonely put me under? (See also: without using up too many “spoons”.)

Ironically, when I finally concocted a solution to this problem, I no longer had the time to write the blog post about solving the problem because I was so busy doing things and being social. I proceeded to pull myself out of my depression, have a pretty good (if still difficult at times) Spring and Summer, and even Fall in 2011. But now that the days are getting shorter and I’m increasingly feeling like my moods are walking on a tightrope of “happy” above a pit of bleakness, I figure it’s about time to document my process. That, and it seems people I know are running into the same problem, so hopefully sharing my own solution can really make a positive impact on others’ lives.

Creating a Cyborg’s Social Calendar

The basic problem was two-fold. First, I needed an easy way to discover local goings-on. Second, I needed a way to remember to actually attend events that I was interested in.

It turns out this is far more difficult to accomplish than one may at first believe since the set of events that I both want to attend and have the capability (energy, time, money, motivation, physical accessibility, etc.) to attend are actually relatively limited. Moreover, I also need to align the set of events that match both of those criteria with the knowledge that said event is occurring when it is occurring. It’s a bit like playing temporal Tetris.

In a nutshell, the solution I implemented was similarly two-fold. First, I cast an incredibly wide but low-cost sensor net, integrated directly into the process I already used for keeping track of my daily appointments. (See also the “no extra time” concept and its wide applicability). Second, I classified the “activities in my area” into two distinct groups: “engagements” (stuff I’ve said “yes” or “maybe” to) and “opportunities” (stuff I haven’t yet said “no” to).

Here’s what my calendar looks like after all the pieces of the system are in place:

As you can see, I have an enormous selection of activities I could participate in at any given time. Better yet, they all show up on my calendar without my ever needing to repeatedly go “look[ing] on places like Craig’s List” to find them, the events on my calendar update themselves, and I can show or hide sets of events on a whim.

The prerequisite tool for doing this is the iCalendar feed, which, in the words of Stanford University, is a popular calendar data exchange format which allows you to subscribe to a calendar and receive updates as calendar data changes. Each of those calendars under the “Subscriptions” heading in the screenshot of my iCal is actually an iCalendar feed from a remote website. iCalendar feeds are to calendars as RSS feeds are to blogs.

The first thing I did was add the event subscription feed from my Facebook. Do this:

  1. Log into your Facebook account and go to the “Events” page.
  2. Scroll to the very bottom of the page and click on the small “Export” link. This will reveal a personalized web address (URL) listing all upcoming Facebook events you’ve been invited to or have RSVP’ed either “Yes” or “Maybe” to, in iCalendar feed (.ics) format. Copy that URL.
  3. Back in iCal (or your calendaring application of choice), choose “Subscribe…” from the menu and paste in the URL you got from Facebook.
  4. Give this calendar subscription a meaningful name. I called it “Facebook Events” (see above screenshot).
  5. Set the “Refresh” interval to something that makes sense; I set it to once “every 15 minutes,” since the Facebook feed is one I check often because it changes so frequently. (For feeds from calendars that I check or that update less often, such as those of community groups, or calendars listing events that are far from home, I set the refresh rate much, much slower, such as once “every week.”)

Okay! Now, whenever a friend invites you to an event on Facebook, your calendar will be updated to reflect that event at the appropriate date and time. If you RSVP “No” to the event, it will disappear from your calendar when iCal next checks your Facebook iCalendar feed.

Repeat the same steps for any other event-management website that you use and that offers iCalendar feeds. Some services I use, such as Plancast.com and Meetup.com, actually offer two distinct iCalendar feeds, one for all of the events visible to you on the service, and one for events that you have RSVP’ed “Yes” to. Subscribe to both; in the screenshot of my iCal window, above, you’ll note the existence of a “‘meitar’ on Plancast” calendar as well as a “Plancast Subscriptions” calendar, and similarly a “My ‘Yes/Maybe’ Meetups” calendar as well as a “My Meetups” calendar.

Now that you’ve got a bunch of subscriptions, it behooves you to organize them in a way that makes sense to you. How you can do this will depend a little bit on the tools you have at your disposal. I found Apple iCal the best choice because of its Calendar Group feature, while I found Google Calendar an incredibly frustrating tool to use.

In iCal, I first created two calendar groups. The first one was called “Social Engagements,” into which I placed all the iCalendar feeds that showed me events to which I’ve RSVP’ed “Yes” to on the remote site. This included the Facebook, “‘meitar’ on Plancast”, and “My ‘Yes/Maybe’ Meetups” feed. The second group was called “Social Opportunities,” into which I placed all the other calendars.

Every time I learned about a new local venue, such as a nightclub, or a café, or a bookstore that had an open mic, I would scour its website to see if it offered an iCalendar feed. If it did, or if it used a tool that did, such as embedding a Google Calendar on their website,1 I’d add their feed to my “Social Opportunities” calendar group, too. I’d do the same every time I learned of a new event aggregating website, such as the IndyBay.org calendar or the Calagator Portland Tech Community calendar, which both offer feeds.

In very short order, I became one of the go-to people to ask about what was happening ’round town—including some towns I didn’t even live in!

However, as I travelled across the country speaking at conferences, I realized that my “Social Opportunities” group was getting cluttered with events that I could not actually attend because I was literally thousands of miles away from them. To solve that problem, I created distinct “Social Opportunities” calendar groups based on geographic region, and moved the individual subscriptions to the group with which they were geographically associated; the Occupy DC calendar feed is in the “Social Opportunities – DC” calendar group, and so on. I also created an “A-geographic” group to house feeds that listed events from all over the place.2

Some event-management services let you filter by geography, making this even easier. For instance, Yahoo!’s “Upcoming” event listing website shows you events by “place,” and you can subscribe to an iCalendar feed of just those events. For instance, here are the Upcoming events in Seattle, and here is the same information in iCalendar feed format. I added the feed of each Upcoming Place to which I regularly travel to its appropriate regional calendar group.

The benefits of this set up are obvious:

  • Visually overlay social opportunities on top of social engagements to ensure few conflicts, and help make the most informed choice about which events I want to go to when there are conflicts, to mitigate my social opportunity cost.
  • Toggle calendars on/off to find nearby activities. Ordinarily, I simply leave all the “opportunities” calendars deselected, so I’m just looking at my personal calendars and the “Engagements” group, since this view shows me “stuff I have to do today.” When I’m bored or I’m looking for new things to do in the upcoming week, however, I simply turn on the “opportunities” calendars. Voila! In 1 click, I’m browsing a wealth of stuff to do!3
  • Quickly orient oneself within the social space of a new city. If I’m taking a trip to Washington DC for a few days, all I have to do is deselect/uncheck the “Social Opportunities – SF/Bay Area” calendar group to hide all of my calendar subscriptions in that group, then select/check the “Social Opportunities – DC” calendar group and, voila, my calendar view has instantly shifted to showing me events that I can attend in Washington, DC.
  • Make RSVP’s meaningful: if I RSVP “Yes” to an event on Meetup, the event is automatically removed from my “Social Opportunities – A-geographic” calendar group and added to my “Social Engagements” calendar group.
  • Easily move event information from a calendar feed to a personal calendar using copy-and-paste without ever leaving the calendaring tool of your choice.

Of course, none of this matters with regards to feeling lonely if I don’t also show up at events in physical space. Admittedly, actually mustering the physical and social energy to get up and go is by far the hardest part of this whole process. Typing on a keyboard is all fine and well (rest assured I do more than enough of it!), but there is no substitute for actually being around other human beings face-to-face. Physically vibrating the air using one’s mouth and having those vibrations move another’s ear drum (or physically moving one’s hands and letting the photons bounce off those movements and onto the retina of another’s eyes, in the case of sign language) is a vital part of the experience of being social.

This system isn’t perfect, but the imperfections are mostly due to the way sites like Facebook handle RSVP information. For my purposes, though, this workflow gets me well over 80% of the way towards my goal, and since I’m actually a human (not a machine), I can deal with a little data pollution here and there. There’s also plenty more I could write about with regards to “being a social cyborg,” such as how I use my calendar in conjunction with my contact management application (my digital rolodex) to maintain “loose” or “weak” interpersonal ties with over 1,000 people spread across the world—again, using “no extra time.” But I’ll save that for another post.

For now, hopefully this gave you a better understanding why my most frequent response to being informed of a party is something along the lines of, “Can you send me a link (to Facebook/Meetup/Google Calendar)?” and also why I’m so, so, so critical of important websites like FetLife that seem to prioritize everything but user security and interoperability.

  1. Every public Google Calendar also publishes its information in an iCalendar feed. For example, rather than view the Occupy SF calendar on their website, just subscribe to the iCalendar feed provided by Google. Also, while you can create an aggregate view of multiple Google Calendars to embed on a Web page, it seems to me like this isn’t a feature offered for iCalendar feeds, so if you come across such a calendar, you’ll likely need to add the individual calendars’ feeds one by one. []
  2. Currently that’s just Meetup and Plancast, for me, since I’ve joined Meetup groups all over the country and I’ve subscribed to people on Plancast who live in dozens of cities. []
  3. Frustratingly, although Facebook also offers you a page listing events that you were not invited to but that your friends were, there seems to be no iCalendar feed of that list, forcing me to periodically check that page for events that would be “Social Opportunities” if I knew of them. Thankfully, to add them to my own calendar, I just RSVP “Yes” or (more likely) “Maybe.” []

Written by Meitar

November 5th, 2011 at 11:04 pm

Using Calendars from the Command Line


If you’re anything like me, you always have a terminal window open. One of the reasons I do this, of course, is because it’s fast. If I want to know anything at all about my computer, all I need do is type the question. The answer, because it’s always text-based, comes back immediately. I don’t have to wait for a window to open or for a pane to scroll. Everything comes at me from a single visual direction, the bottom of my terminal window.

However, there are some occasions when a text-based response to a complicated question isn’t very helpful because it requires so much extra work to understand. For me, the most common example of this sort of issue has always been in looking at time-based information, and more specifically, calendars. Whenever I’m on my machine, I almost always need to look at a calendar.

In the past, I used to go all the way over to iCal. Sure, I can do this using keyboard shortcuts only, but sometimes all I want is a quick answer to “what date is this upcoming Friday?” In situations like that, I’ve lately begun using the cal command, and my oh my, what a timesaver.

cal is kind of like man for dates. Of course, you can get more info by saying man cal to your prompt. The cal program, installed by default on almost all UNIX-based systems (including Mac OS X), has a ton of useful options. However, most of the time, I don’t need more than a few.

For instance, let’s say I just want a calendar of the current month. I can get get a compact, simple month view instead of going to iCal by saying just cal at the command line:

Perseus:~ maymay$ cal
     April 2008
Su Mo Tu We Th Fr Sa
       1  2  3  4  5
 6  7  8  9 10 11 12
13 14 15 16 17 18 19
20 21 22 23 24 25 26
27 28 29 30

Other options let me ask other questions of cal. Easy, simple, fast. I like it.

Written by Meitar

April 18th, 2008 at 8:47 pm

Sharing your Windows XP Virtual Machine’s Internet connection with your Mac OS X host operating system using VMware Fusion


In some situations, like the odd one I now find myself in, the only way to get Internet connectivity is to use a solution that requires a fair bit of maneuvering. In my situation, I have temporarily obtained a Vodafone 3G mobile card. Unfortunately, the Vodafone Mobile Connect software for Mac OS X as of this writing is obscenely poor. Of course, Vodafone’s software for Windows works without a hitch.

The only way I could get my Vodafone 3G card to work was to fire up a Windows XP guest inside of my MacBook Pro, using VMware Fusion. Connecting to the Internet with the 3G card using the Windows guest was smooth sailing, but that only provided the Internet connection to the Windows virtual machine. I wanted my Mac to be directly connected.

The solution is obvious, but a few gotchas really bit me hard. To get the Windows guest to share its Internet connection from the 3G card to my Mac, I would need to bridge VMware’s virtual ethernet adapter from the Windows guest to the Mac OS X host. Once bridged, both the Windows guest and the Mac OS X host would logically be on the same ethernet network segment. At this point, I can enable Windows XP’s built-in Internet Connection Sharing (stupidly dubbed “ICS” because everything needs a TLA) on the 3G connection so that Windows NATs it through to the bridged virtual ethernet card. Finally, I can connect to Vodafone’s 3G network, and all should be well.

Here’s the gotchas.

First, in order for VMware to actually initiate the network bridge when it starts up, it must detect that a physical link is active on your Mac. In other words, Mac OS X’s Network System Preferences pane must show you a yellow dot next to at least one physical networking device (probably either your “Built-in Ethernet” or your “AirPort” ports). VMware Fusion will give you no errors or warnings that a bridge is unavailable until you try to connect your virtual machine’s network while set to bridge, in which case VMware Fusion will complain with an error that reads: “The device on /dev/vmnet0 is not running.”

Obviously, if you have no other devices to connect to, you need to fake one. The easiest way to do this is to set up a Computer-to-Computer network using AirPort. Just go to your AirPort menu bar item and select “Create Network…” and create the network (preferably encrypted). If you check System Preferences now, you should see a that AirPort has a yellow dot next to it and reads as having a “Self-Assigned IP Address.” Now that you have a physical link on your AirPort card, you should be able to start the VMware Fusion virtual machine with bridged networking mode without incident.

However, if you do encounter the above error anyway, you need to restart the VMware network bridge. You can do this either by shutting down VMware completely (turn off your guest operating systems, and quit the VMware Fusion application), or you can run the following commands as an administrator in Terminal, which will stop any bridge currently running (or do nothing if no bridge is running) and then restart it, providing the output as shown:

sudo killall vmnet-bridge
sudo "/Library/Application Support/VMware Fusion/vmnet-bridge" -D vmnet0 ''
Entering event loop...
Examining network configuration...
Turning on bridge with host network interface en1...

Obviously, you may be asked for your password as you perform this procedure. Note that the trailing two apostrophes are single quotes with no space. This is (almost) how the VMware Fusion boot.sh script starts and stops the network bridge. Specifically, you’re telling the vmnet-bridge application to run in Debug mode and to bridge vmnet0 to whatever is the current primary networking interface. In the example output shown above, this is en1, or my AirPort card connected to the computer-to-computer network I created in the previous step.

Hopefully you won’t have to mess with the vmnet-bridge application, as this should happen on its own when you start up VMware Fusion if you have any physical link on a network device. Nevertheless, I’ve found this is sometimes unreliable, so just in case it doesn’t now you know how to bring up the bridge on your own. (Tip: once it’s up, you can CTRL-Z to pause it, re-start it with fg %1 and then quit Terminal if you like. The bridge will still be up.)

Now that the AirPort card has a physical link, and the VMware network bridge is running, the next step is to configure your virtual machine to use bridged networking. Just go to Virtual Machine → Network → Bridged as normal. Make sure Connected is also selected. Now start up your Windows guest.

Once Windows boots, go to the Network Connections window by selecting Start → Connections → Show all connections. At this point, your “Local Area Connection” in Windows probably has a warning sign on it and reads as having “Little or no connectivity.” It probably has a self-assigned IP address just like your AirPort card. That’s fine—as long as it’s not “unplugged,” we’re in good shape.

Next, select whatever other connection you want to share the Internet from (in my case, the 3G modem, but it could also just be any other connection in the window), right-click it and select Properties. Go to the Advanced tab and make sure “Allow other network users to connect through this computer’s Internet connection” is checked. The other boxes won’t matter.

What this does is turns on Windows’ own NAT service that configures the one connection (the one your sharing) as the WAN side of (yet another) virtual networking device and the Local Area Connection (the one we’ve bridged to our AirPort or Built-in Ethernet card on our Mac) as the LAN side. Hit OK as many times as is necessary to close the network connection properties windows and wait a few moments. Sometimes this can take up to 30 seconds or so, but eventually you’ll see Windows announce that “Local Area Connection is now connected.” If you inspect it, you’ll see that the IP address configuration has been automatically assigned as a “Manual Configuration” with the address of, a subnet mask of, and no default gateway.

As a last step, now we can actually connect to the Internet using whatever service we have. In my case, this is when I hit the “connect” button on my Vodafone Mobile Connect software. Once the connection is established and the Windows XP virtual machine can see Internet, it takes up to another minute or two (or three) for the Mac’s connection to get an IP address from the Windows guest, but it invariably works.

If the Windows side of things is giving you any trouble, the most reliable solution I’ve found is to simply disable, then re-enable whatever connection isn’t behaving as desired. If after all of this your Mac still doesn’t get an IP address from the Windows XP guest, disconnect and then re-connect the virtual machine’s ethernet card (by toggling the “Connected” menu item in the Virtual Machine → Network menu). Also, of course, be doubly sure that your AirPort is set to “Use DHCP.”

Phew! So simple…and yet so much harder than it had to be. I found the following two PDF documents very helpful in understanding all of this. You might too:

  1. VMware Fusion Network Settings — a super-brief, but excellent introduction to VMware’s network setting internals. It’s also a PDF download attached to the linked forum thread.
  2. Share Windows XP Guest Internet Connection with OS X Host HOWTO — This basically describes the same thing this post does, but it does so using absolute step-by-step instructions. It’s also a PDF download attached to the linked forum thread.

Written by Meitar

March 31st, 2008 at 4:06 am

Steven Pinker’s ‘The Stuff of Thought’

one comment

This video, which is one of the recent TED Talk videos, is of Steven Pinker’s talk called The Stuff of Thought. This is simply brilliant. So brilliant, in fact, that those who know me well are about to be utterly astounded by what I am going to say:

I now understand the value of indirect communication. And it is immense.

I also understand why I never saw it before: the benefits are reaped solely through language’s social applications, not its analytical ones. See for yourself by watching the video.

An incredible interview with this Harvard professor is available on Google Video.

Written by Meitar

December 28th, 2007 at 4:21 am

We should re-instate that old USENET warning

leave a comment

From the everything-you-say-can-and-will-be-used-against-you department:

I’ve been doing this for years, and my solution is pretty simple: no regrets.

As an aside, these days when you punch in “privacy concern” into Googlepedia, you get the Wikipedia entry for Facebook. I was kind of expecting the entry for “US Government,” but whatever.

Written by Meitar

November 27th, 2007 at 3:31 pm

The Map Of The Internet, Circa 2007

leave a comment

This map of the Internet is pretty neat. It even gives you a “you are here” signpost when you first arrive.

More information on the map is available from the ISI ANT Census home page.

Written by Meitar

October 9th, 2007 at 5:23 pm

Moving personal data from one Mac to another


I recently purchased a new iMac. It’s aluminum. And glass. And shiny. And it’s so much faster than my old G4 workstation I feel a little bit like I’ve just come out of the stone age.

Anyway, the experience of moving one’s data from one machine to another is always a bit of a hassle. There are so many little things you’re sure you’ve forgotten, preferences you don’t want to have to recreate, and small gotchas that, if you’re not careful, will mean you’ve lost something like your old email or your contacts—sometimes for good.

Thankfully, with a remarkably few easy planning steps, a lot of this trouble can be avoided all together. Much of this is thanks to the brilliant architecture of Mac OS X, which does a superb (though not perfect) job of separating data from applications. The other major benefactor of this kind of easy transition is server-based data stores, or hosted services (even if those services are ones I’ve hosted myself).

So here’s what I wanted to move over from my old mac to my new one:

  • Personal documents, pictures, movies, and music, etc.
  • Preferences for all applications, such as my dock layout, the Finder’s and Mail’s toolbar button arrangements, and so on.
  • PIM databases, such as my Address Book contacts, iCal calendars, my email, and the like.

This could really be a much longer list, especially if I were going to name every single item of personal importance. Nevertheless, each item falls into one of these three main categories of stuff that I want to move.

The first time you turn on a new Mac it will ask if you want to transfer your data from your old computer to your new computer. It does this using the Migration Assistant application located in your Mac’s /Applications/Utilities folder. While this works well for the most part, I’ve seen this program fail one too many times—especially for cross-architecture, PPC to Intel, transfers—that I’m simply not a fan of using it. Besides, as you’ll see in a moment, it’s trivially easy to do all of that yourself.

Of course, all of your personal information should be somewhere in your Home folder. You could just copy the entire directory from one Mac to the other, but you’ll probably end up deleting some (albeit probably non-critical) items from your new machine. When you drag-and-drop in the Finder, the Finder first removes anything of the same name on the destination as the source of the transfer. This is not what you want.

So here’s how I dealt with moving each type of item, after ensuring I had a good backup:

Transferring personal documents

rsync is a godsend for any situation where you want to ensure that you’re copying files from one location to another. It compares the contents of folders and, by default, only adds missing items to the destination while leaving everything else alone. It also overwrites identically-named files on the destination with files from the source.

To use rsync, I first needed to turn on the destination Mac’s “Remote Login” feature; this turns on the SSH server. (Find this in System Preferences → Sharing → Services.) Next, simply issue this relatively simple command at the command line, from your home directory:

rsync -aEve ssh Documents UserShortName@DestinationMac:Documents

This tells rsync to copy files from the local machine’s Documents directory to the DestinationMac‘s Documents directory in archive mode (the -a option), which preserves numerous file meta-data such as its modification times and access permissions, along with the file’s extended attributes and resource forks, if any exist (-E), reporting verbosely (-v) as it does so, executing over SSH (-e ssh). The UserShortName is the DestinationMac‘s user short name, obviously, and the DestinationMac is either its hostname (such as mynewmacintosh.local) or its IP address (such as

I repeated the above step for the Desktop, Movies, Music, Pictures, Public, and Sites folders. Most of my music is actually on an external hard drive, so that was of course a cinch, and rsync took care of the rest.

Transferring preferences

Mac OS X stores its user-specific preferences in each user’s home Library/Preferences folder. The set up is elegant and beautifully implemented. Just like the folders above, I could have simply rsync‘ed my preferences over, but I wanted to be more selective. So I simply scrolled through the list finding the preference files for the applications I still use and copied them over. I used the Finder’s drag-and-drop, but you could use whatever method you want.

These files are named in reverse-DNS style notation, so for instance, my Mail preferences were stored in the file com.apple.mail.plist.

This was a super-simple step.

Transferring PIM databases

Like preferences, user-specific data is often stored in that user’s home Library/Application Support folder. Inside that folder is a folder for each application that has some data to store. All of iCal’s information, for example, is thus stored in your home Library/Application Support/iCal folder.

rsync to the rescue again:

rsync -aEve ssh Library/Application\ Support/iCal "UserShortName@DestinationMac:Library/Application\ Support/iCal"

In this case, because of the space in the Application Support folder’s name, the quotations and the backslashes are required to help rsync find the proper folders. Other than that, the command is the same as before.

I selectively copied over all the data from the applications I wanted to keep and, voila, the next time I opened iCal or Address Book, my data was there.

The Mail application and Safari both have special ways of handling their information, however, so their datastores couldn’t be found in the Application Support folder. Instead, they keep their folders directly inside the Library folder named (predictably) Mail and Safari, respectively.

Also, since I use IMAP for email and shared calendars in the form of .ics subscriptions whenever possible, the natural sync process for these databases saved a lot of time.

Most other programs in non-standard locations could be found with a little digging inside my Library folder.

It should be noted that for this to work reliably, you should always move all the preference files along with the Application Support directory for any program whose Application Support directory you’re grabbing!

Cleaning up

All told, the whole thing took maybe two hours and most of that time was spent transferring data. It was really quite painless. Because this was a surgical transfer, instead of a full “migration,” there’s very little to clean up.

However, for some reason, Apple’s .Mac Sync service had quite a bit of trouble keeping up with my new Mac’s new additions. The fix was simple enough: unregister every sync’ed computer, then reset all of the .Mac Sync data with the data on the new Mac. Then, re-sync each device either merging its data during its syncing process or replacing the data on it with the new, known-good data on .Mac. (Again, make sure your backup works before you do this.)

And that’s pretty much it. Now I have a new iMac, but it looks exactly like my old one on the screen. :)

Written by Meitar

September 13th, 2007 at 1:26 pm

Stay awake and dream

leave a comment

Wow. I’m having an incredible amount of trouble staying awake today. Yesterday, too. I’m not sure what’s up exactly but I’m going to take the obvious assumption as fact right now in that I am very, very tired. It’s been weeks (literally) since I’ve gotten a single night’s rest with more than 5 hours of sleep. I definitely need a chunk of restful time to get some of my energy back.

I have a number of personal projects that I’m working on and would love to have the energy for, so feeling like I’m exhausted all the time is beginning to wear thin. Yesterday I had a lot of fun going rock climbing at the City Climber’s Club with Sara and another couple of friends but it completely wore me out. I’m still feeling the exhaustion today. On the flip side, however, it feels so good to experience that kind of fun (and social!) work out at a gym again. I forgot how much I missed that.

The job craziness and apathy things might also be contributing to my lack of energy, because I find myself spending time searching job boards and talking to recruiters rather than focusing my downtime on things like adding features to my programs.

Just a bit earlier today actually, I was speaking to a colleague of mine about exactly that. He’s interested in moving away from the IT industry because, he says, he’s not interested in using computers for the sake of computing but rather using computers as tools to create something else. I wholeheartedly agree. There’s very little interesting things about computers themselves. The reason they’re attractive to me is how good they are at enabling other things to come into existence, and what makes me passionate about them is the fact that I can be extremely expressive through the medium (ala, web development and design).

Certainly, however, it is important and infinitely helpful to have operational skill with the tool you use to create something in order to enable you to create something better. Case in point, in web development, it is my designer friends who are constantly asking me operations questions like “How do I create a redirect on Apache for all but one file,” or things like that. The fact that I have had the administration experience to be able to do this means I can create a better-implemented web site than most of them, however we are both driven by the same interests: to express our creativity using chosen medium.

So, y’know…I’d like to be able to find some way of making that desire self-sustaining and financially viable. Here’s hoping this upcoming trip to Seattle proves fruitful in that regard.

Written by Meitar

March 15th, 2007 at 12:39 pm

Data lives forever; so does geekery!

one comment

A long, long time ago, on a server far, far away, I started my very first web site. It’s since moved through various places, and it’s funny that each step in the redirect process still exists. What’s even funnier is that not only do the redirects exist, but so too do my old geeky writings.

Written by Meitar

February 20th, 2007 at 9:54 am

Posted in Geeky,General,Personal