This should be really simple, as the correct command line is plain as day (where the string of colon-separated 00′s is your preferred MAC address):

sudo ifconfig en1 lladdr 00:00:00:00:00:00

There are numerous blog posts all over the ‘net that tell you this time and again, but each one seems to have comments from users complaining that it doesn’t work on their system. I ran into a similar problem not long ago when my MacBook Pro didn’t do what I expected. Just like others, whenever I tried to run the above command, nothing seemed to happen:

ifconfig | grep ether # Determine current MAC addresses
sudo ifconfig en1 lladdr 00:00:00:00:00:00 # Try changing MAC address for en1 (usually Airport)
ifconfig | grep ether # Confirm change; but uh-oh! Output is the same as before! Why?

Here’s how I fixed this problem.

The thing to know is that there seem to be a number of conditions that will prevent Mac OS X from successfully changing a NIC‘s MAC address. Some are obvious and some are not. As far as I can tell, these conditions are:

• having the interface “down” (i.e., if you’ve recently run ifconfig en0 down or an equivalent),
• being associated with (i.e., connected to) a Wi-Fi network with your Airport card,
• having the System Preferences application running,
• forgetting to “unstick” the current system configuration set.

It’s the last one that bit me. Mac OS X has a feature called “system configuration sets” or “locations,” as it’s termed in much of the GUI. These can be accessed via the Network pane in System Preferences, or via the scselect command from Terminal; it’s that scselect command which offers the key to changing a Mac’s MAC address.

On my MacBook Pro (which, for the record and if it matters, is running Mac OS X 10.6.7), I need to do all of the following before running ifconfig, as shown above:

• If I’m changing my Airport card’s MAC address, I need to disassociate from any network. (This can most easily be done by invoking airport -z from Terminal. If you don’t have this command, see my tips on where to find airport.)
• Quit System Preferences if it’s open.
• Tell the operating system to “delay changing the system’s ‘location’ until the next system boot” by running: scselect -n.

According to the man page for scselect:

scselect provides access to the system configuration sets, commonly referred to as “locations”. When invoked with no arguments, scselect displays the names and associated identifiers for each defined “location” and indicates which is currently active. scselect also allows the user to select or change the active “location” by specifying its name or identifier. Changing the “location” causes an immediate system re-configuration, unless the -n option is supplied.

[…]

-n Delay changing the system’s “location” until the next system boot (or the next time that the system configuration preferences are changed).

Once I perform the above rigmarole, I can then change my MAC address without issue. But I have to be ludicrously careful. As soon as I open the Network System Preferences pane or otherwise do something to change the system configuration preferences, I have to run through that rigmarole again before changing my MAC address will work as expected.

Mac OS X ships with a command-line tool called airport that can do all sorts of nifty things with Wi-Fi networks. Unfortunately, it’s so squirreled away that most people don’t seem to know about it. The utility is part of the Apple80211 Private Framework used to power your Mac’s Airport menubar icon.

Invoking the utility without arguments prints a useful (if incomplete) usage message. At a Terminal command prompt, type:

/System/Library/PrivateFrameworks/Apple80211.framework/Versions/Current/Resources/airport

The tool let’s you do a number of interesting things, so it’s worth playing around with. While you’re playing, you may as well create a symlink (a shortcut) to the utility so you don’t have to type that long path name all the time:

sudo ln -s /System/Library/PrivateFrameworks/Apple80211.framework/Versions/Current/Resources/airport /usr/bin/airport

Among the easiest things you can do is print a list of the Wi-Fi networks within range of your computer, but unlike the Airport menubar item, this report shows you a bunch of extra, precise data, such as which encryption protocol (if any) is being used on the network:

$ airport en1 scan
                            SSID BSSID             RSSI CHANNEL HT CC SECURITY (auth/unicast/group)
                       moscohome 00:22:6b:8b:86:51 -61  10      N  -- WPA2(PSK/AES/AES)
                     PUBLIC-455H 00:15:6d:60:95:d1 -82  1       N  -- NONE
                    Alex Network 00:1e:e5:24:c4:4f -86  1       Y  TW WPA(PSK/TKIP,AES/TKIP) WPA2(PSK/TKIP,AES/TKIP)
                   linksysELNIDO 00:21:29:a3:fd:99 -90  6       N  -- WPA(PSK/AES,TKIP/TKIP) WPA2(PSK/AES,TKIP/TKIP)
                        2WIRE024 00:18:3f:02:2f:49 -88  6       N  US WEP
                        2WIRE940 00:12:88:d9:85:41 -93  6       N  US WEP

If I wanted to see which of my neighbors still haven’t upgraded from WEP, I could just filter using grep:

airport en1 scan | grep WEP

More awesome, perhaps, is the tool’s ability to actually perform traffic sniffing and capture packets. Tell airport to sniff, and optionally provide a channel (which you now know thanks to your ability to scan). You need to be an administrator (i.e., you need sudo privileges) to do this:

sudo airport en1 sniff 6

This creates a file called airportSniffXXXXXX.cap in the /tmp directory, where XXXXXX is a string for uniqueness. You can then feed this file into your favorite network analyzer such as Wireshark to examine the traffic offline.

Thanks to this simple text convention in subject lines, I could trivially automate the process of sorting through the approximately 100 emails a day that the email list generates in order to single out only the emails that interest me. Here’s how I did it.

Define Your Goals

Before setting out on any task, it behooves you to take a moment and think about what it is you’re trying to accomplish. For me, with the San Francisco Freecycling Network (SFFN) email list, I wanted to achieve the following goals:

• Keep my inbox clear of email from the SFFN list unless a message was particularly interesting.
• Browse the SFFN messages when I wanted to look at them without having to go to the web site.
• Highlight particularly interesting messages in my inbox visually and play a special sound to alert me that such email has been found in case Mail was running in the background (since free stuff gets taken fast!).

I defined “particularly interesting” messages as ones that offered items of need for my recent move. With this in mind, I set out to create email rules that accomplished each goal in turn.

Step 1: Create a mailbox to store the appropriate messages

I began by creating a new mailbox to store all the SFFN messages I was getting. This alternate mailbox would be the mailbox I would shunt all SFFN email to so as to keep my inbox clear of it. I called the mailbox simply “SFFN”.

Do this:

1. From the Mailbox menu, select New Mailbox…. The New Mailbox sheet appears.
2. Select any location (“On My Mac” is fine, as is the account that receives the mailing list messages), and give it a name.
3. Click OK.

Step 2: Create an email rule to move all appropriate messages to the new mailbox

With the new mailbox created, I now needed to get all the appropriate messages in there and out of my inbox.

Apple Mail’s email rules work by looking at each incoming message and matching it against a set of conditions that you provide. If the message being evaluated matches the conditions you specify, such as “from the San Francisco Freecycler’s Network mailing list”, then an associated action is automatically performed. Every email you get is evaluated against every rule you have unless a rule moves the message to another mailbox or until you trigger the “stop evaluating rules” action.

Since moving an email message to a new mailbox ends the process of evaluating rules and moving messages to the SFFN mailbox I just created is the goal of the rule I’m creating, I decided to name the rule “END – SFFN”.

Do this:

1. From the Mail menu, select Preferences…. The Mail Preferences window opens.
2. Click the Rules button. The Rules pane appears.
3. Click the Add Rule button. The Add Rule sheet appears:
   1. Enter a meaningful description (I chose “END – SFFN”) in the Description: field.
   2. Provide the conditions you want to match. Since all SFFN emails must be addressed to the mailing list, I simply provided the email address of the mailing list (sffn@yahoogroups.com) as the condition for the To header.
   3. Provide the actions you want Mail to perform. I simply wanted to move the matched messages to the SFFN mailbox.
4. Click OK.

For me, the above configuration looked like this:

Step 3: Create an email rule to highlight a message of particular interest

At this point, any and all email I receive from the San Francisco Freecyclers’ Network is being moved to the SFFN mailbox I created for it. This is nice because it keeps my inbox clear, but it’s still not very helpful since I still have to go trudging through the SFFN mailbox in order to find anything that might be interesting to me. The whole point of this exercise is to reduce the amount of time I spend actively looking for interesting things and let my computer do that work for me. So the next step is to tell Mail what I’m looking for so it can show the interesting messages to me.

Now, as it happens I’m in need of a wireless router. Since “router” is an appropriately unique word, I’m going to tell Mail to look for that word in a subject line. However, since I only want Mail to tell me when a router is available and not when other people like me are looking for routers, I’ll also tell Mail to look for the keyword “OFFER” in the subject line. (And this is why the Freecycle guidelines tell users to format their subject lines in a conventional way.)

Finally, since I don’t want to have to go digging for the interesting email message and since my inbox is already going to be kept clear by the previous rule, I’ll simply have Mail highlight the message in a bright green color and leave the message in my inbox without moving it to the SFFN mailbox I created earlier.

Do this:

1. From the Rules pane in Mail’s preferences, click Add Rule.
2. Enter a meaningful description in the Description: field. (Since I’m looking for a router, I called it “SFFN – Search for OFFERed ‘router’”.)
3. Provide the conditions you wish to match. For me, this meant email sent to the Freecycler’s mailing list with the two words “OFFER” and “router” in the subject line.
4. Specify the actions you wish Mail to perform. I wanted Mail simply to color the message green and to leave the email go to the inbox (where it was originally destined for), so I chose “Stop evaluating rules”. (I also decided I’d want Mail to play a special sound to alert me that it had found something interesting. This is optional, of course.)
5. Click OK.

When I was done creating my rule, the above configuration looked like this:

I can now repeat this step as many times as desired to tell Mail to highlight other messages that may be of particular interest for some other reason. For instance, say instead of looking for a wireless router, I wanted to look for a toaster. I would simply need to click on “Duplicate Rule” and replace all instances of “router” with “toaster”.

Step 4: Place email rules in appropriate order

Since Mail will repeatedly check incoming email against all the active rules, we need to be sure to place the rules in the correct order. You can think of each email rule as part of large Rube Goldberg machine, each message getting funneled through some piece of the logic at each successive rule. That’s why I began the name of the first rule I created with “END,” so that I’d know it should be placed after the rest of the SFFN-related email rules.

I decided that I wanted Mail to look for anything related to cameras and, of course, to toasters. This gave me a total of 4 rules (three to search for items of interest, and one to keep my inbox clear). Since the three highlighting rules all perform the same action, it doesn’t really matter which order they go in, but it is important that all of them appear before the rule to move messages to the SFFN mailbox.

To order rules, simply click-and-drag them into the order you wish Mail to evaluate them in. When I was done, my Rules pane looked like this:

Conclusion

Mail rules are an extremely powerful feature that most email clients have, but that too few people use. They can save you enormous amounts of time and increase your productivity by automating simple yet time-consuming tasks.

The conventional, standardized subject lines that the Freecycle mailing list uses simplifies the logic required to have your computer automatically process your messages for you. This is a useful observation because it can be applied to other areas of your life where using simple conventions can help to organize otherwise overwhelming information tasks into manageable batches. Although this particular example uses stock, simple commands, you can get as fancy as you like by having an action trigger an AppleScript.

Now, hopefully, finding some additional housewares and a wireless router for my new San Francisco apartment will be as easy as checking (but not manually sorting!) my own email!

Everyone has sensitive files that they keep on their computer and, fortunately for Mac OS X Users, Apple has made it ridiculously easy to create a cryptographically secure containers for such files. You can think of a container like this, which is just a standard Mac OS X disk image (.dmg) file, like a vault that you open, put stuff you want to keep safe inside, and then close again.

Here’s how you go about making and using one.

Create the container, an encrypted disk image

1. First, open up your copy of Disk Utility.app, which is located in your computer’s /Applications/Utilities folder. (As an aside, this program is a bit like a swiss army knife for handling disk operations in Mac OS X. You should definitely find out what else it can do).
2. Next, select the File → New → Blank Disk Image… option. This will cause the New Blank Image window to appear.
3. Fill in the typical details such as the disk image file’s name and where you want to save it to. In addition, you’ll be presented with a number of options such as Volume Name, Volume Size, and Image Format. The defaults are usually adequate except for Volume Name, which you should customize so that when you mount the disk image the disk label is meaningful for you, and the Image Format, which I recommend you switch to “sparse disk image.”

   Sparse disk images can start small and grow automatically as you write more files into them. If what you want to keep secure in this manner are very large files, say gigantic high resolution PhotoShop documents, then you might consider the sparse bundle disk image format instead.

   Also, obviously, set the Encryption to a value other than “None.”

   Here’s an example screenshot from my Mac:

   

   Screenshot of the New Blank Image window showing meaningful values entered, Encryption field set to 128-bit, and Image Format field set to sparse disk image.

4. Press the “Create” button and you’ll be presented with a standard password selection dialogue. This is the password you’ll use to mount the disk image and is analogous to the idea of setting the combination on your vault’s lock. It’s critical that the password you choose is a good one. Ideally, your password is a totally random string that may include any printable character. Since that’s hard to remember, you can have the Mac OS X keychain manage your passwords for you.

Encrypt some files by writing them to the disk image

Now that you have an encrypted disk image, a secure container for your sensitive data, you can make use of it just as you might any other disk image on Mac OS X. For instance, say I have a top secret file called “My Killer Business Plan.pages” and I don’t want anyone to get at it. All I need to do is copy the file into my encrypted disk image, as the following screenshot shows:

It should go without saying that you want to delete the original, unencrypted copy of the file you’re copying into the encrypted disk image, but I’ll say that anyway. Don’t leave unprotected copies of your files lying around. Also, be certain to unmount (eject) the disk image when you’re done using it because the only thing the password protects is opening the disk image, not the files contained within it.

External references

Here are some additional places where this technique is discussed. Check out these additional articles about this topic elsewhere for more information and other perspectives:

• Mac OS X: How to create a password-protected (encrypted) disk image
• Mac OS X: About Encrypted Disk Images
• MacOSXHints.com: Create an encrypted disk image that grows as required

I spent about an hour looking into this this afternoon and it turns out that much of this information is readily available on the filesystem in the Apple WikiServer’s data store. I whipped up the following shell script to extract this information in CSV format, exactly as requested.

I’m posting this script here in case someone else wants similar “export a list of WikiServer pages to a comma-separated values (CSV) file” functionality but isn’t sure how to go about getting it. To use this, just edit the line that reads http://my-server.example.com/groups/wiki/ so that it refers to the wiki base URI of your own server.

Update: The latest version of this script is now available at its Github-hosted repository. You should probably use that instead of the script below.

#!/bin/sh -
#
# Script to extract data from an Apple WikiServer's data store by querying the
# filesystem itself. Creates a 'wikipages.csv' file that's readable by any
# spreadsheeting application, such as Numbers.app or Microsoft Excel.app.
#
# USAGE:   To use this script, change to the WikiServer's pages directory, then
#          just run this script. A file named wikipages.csv will be created in
#          your current directory. For instance:
#
#              cd /Library/Collaboration/Groups/mygroup/wiki  # dir to work in
#              wikipages2csv.sh                               # run the script
#              cp wikipages.csv ~/Desktop                     # save output
#
# WARNING: Since the WikiServer's files are only accessible as root, this script
#          must be run as root to function. Additionally, this is not extremely
#          well tested, so use at your own risk.
#
# Author:  Meitar Moscovitz
# Date:    Mon Sep 22 15:03:54 EST 2008

##### CONFIGURE HERE ########

# The prefix to append to generated links. NO SPACES!
WS_URI_PREFIX=http://my-server.example.com/groups/wiki/

##### END CONFIGURATION #####
# DO NOT EDIT PAST THIS LINE
#############################

WS_CSV_OUTFILE=wikipages.csv
WS_PAGE_IDS_FILE=`mktemp ws-ids.tmp.XXXXXX`

function extractPlistValueByKey () {
    head -n \
      $(expr 1 + `grep -n "<key>$1</key>" page.plist | cut -d ':' -f 1`) page.plist | \
        tail -n 1 | cut -d '>' -f 2 | cut -d '<' -f 1
}

function linkifyWikiServerTitle () {
    echo $1 | sed -e 's/ /_/g' -e 's/&amp;/_/g' -e 's/&gt;/_/g' -e 's/&lt;/_/g' -e 's/\?//g'
}

function formatISO8601date () {
    echo $1 | sed -e 's/T/ /' -e 's/Z$//'
}

function csvQuote () {
    echo $1 | grep -q ',' >/dev/null
    if [ $? -eq 0 ]; then
        echo '"'$1'"'
    else
        echo $1
    fi
}

ls -d [^w]*.page | \
  sed -e 's/^\([a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]\)\.page$/\1/' > $WS_PAGE_IDS_FILE

echo "Title,ID,Date Created,Last Modified,URI" > $WS_CSV_OUTFILE
while read id; do
    cd $id.page
    title=$(extractPlistValueByKey title)
    created_date="$(formatISO8601date $(extractPlistValueByKey createdDate))"
    modified_date="$(formatISO8601date $(extractPlistValueByKey modifiedDate))"
    link=$WS_URI_PREFIX"$id"/`linkifyWikiServerTitle "$title"`.html
    cd ..
    echo `csvQuote "$title"`,$id,$created_date,$modified_date,`csvQuote "$link"` >> $WS_CSV_OUTFILE
done < $WS_PAGE_IDS_FILE
rm $WS_PAGE_IDS_FILE

For those new to the Wiki Server, this introduction to the Apple WikiServer for web developers may be of interest.

If you’re not afraid of the Terminal, this can be avoided. Put the following lines in your ~/.profile (or similar):

alias rmds='find . -name ".DS_Store" -type f -print0 | xargs -0 rm'

What this does is creates a new command that you can use (rmds) which recursively finds and deletes any regular file named “.DS_Store” starting from the current directory. Thus, running this command in the folder you are about to create an archive out of will clean it first, and will prevent unnecessary confusion on the part of your archive file recipient.

Alternatively, another way to do this is to use the command-line zip program and an (admittedly more complicated) pipeline to remove the .DS_Store files after they have been added to the archive. To do that, use this series of commands:

zip -d ZIPfile.zip `unzip -l ZIPfile.zip | grep .DS_Store | awk '{print $4}'`

where, naturally, ZIPfile.zip is the ZIP archive you want to remove the .DS_Store files from. Creating an alias out of that command (and making it work for paths that contain spaces) is left as an exercise for the reader. ;)

As an aside, the alias, find and xargs commands are incredibly useful in their own right and can be used to do a lot of pretty amazing things. As always, man command will give you the nitty gritty.

Also as an aside, you can stop the Finder from creating .DS_Store files entirely when browsing network volumes (like Windows shares) with another command, documented in Apple’s Knowledge Base.

Apple has been on point, however, providing good security utilities built right into the operating system and easily available to end users. Of most common use is probably “Secure Empty Trash” which securely deletes files that you put into the trash. The counterpart to this function available in the Finder is, too few Mac users know, the srm or secure remove command-line utility.

srm can be thought of as simply a version of rm that overwrites file data before unlinking it from the file system. It comes with a few more options than rm comes with all geared towards tweaking just how it overwrites files. My favorite is -m, which the manual page says:

overwrite the file with 7 US DoD compliant passes (0xF6, 0×00, 0xFF, random, 0×00, 0xFF, random)

I had the perfect occasion to use srm today: I was transporting my SSH private key from one laptop to another via a temporary drive. I wanted to securely remove all traces of the private key file from the temporary drive after installing it in the new computer. (See this SSH public key tutorial if you don’t know why this might be important.)

After copying the private key file over, removing it securely looks like this:

srm -m private_key_file

It’s that easy.

To be confident that your file is truly overwritten with garbage, you can use the -n option. This is one way to retain a file, but completely corrupt it. Observe:

Meitar:~ meitar$ cat testfile
Hello world.
Meitar:~ meitar$ srm -mn testfile
Meitar:~ meitar$ cat testfile
?
 ?)c?I
      P?Meitar:~ meitar$

That garbage you see after the second invocation of cat shows that the file really was trashed, that is, overwritten with garbage data. Now, a simple rm testfile can do the rest of the work.

As always, man srm will give you all the other juicy details.

In my office of approximately twenty-some-odd employees, we’ve just begun using the groups’ blog feature to replace all-staff emails for interesting items that are not business related. This has actually been a huge boon for several reasons, not least of which is the productivity boost we can enjoy thanks to moving from a push system (relatively annoying, if occasionally interesting emails) to a pull system (web browsing, RSS feeds, all generated from the Apple WikiServer group blog). Out of the box, only one feature was missing from the group blog: user profile pictures (“avatars”).

If you only turn on the group wikis and blogs features in Apple’s WikiServer, you’ll find that whenever someone posts a comment to a wiki page or a blog post, a generic profile picture will appear next to their comment. If you give that person’s user account a profile picture in Workgroup Manager, you’ll see that generic profile icon turn into a broken question mark. It turns out that this is because the user profile pictures are served by a completely different web service than the group’s wiki and blog is served so if that server isn’t running none of these images will be served up to the browser.

Fixing that is simple enough: simply turn on the appropriate server—the User Weblog server—by opening Server Admin, navigating to the Web Service settings, and enabling the “Blogs” service for users under your web site, then clicking save. For the default web site (*), all that checkbox technically does is remove the comment in the /etc/apache2/sites/0000_any_80_.conf file that reads:

#        Include /etc/apache2/httpd_users.conf

The /etc/apache2/httpd_users.conf file enables the use of your web site’s /users URL paths. In practice, this means that you’ve now allowed anyone with a user account in your Open Directory database to create a new hosted, personal weblog on your server. This may be what you want, but it wasn’t what I wanted—all I wanted was user profile pictures on the groups features.

As it happens, everything behind the /users URL is actually a completely different web server (really an instance of Twisted Python) that’s accessed via a ProxyPass directive. This turns out to be really handy, because it means we can intercept requests for these URLs and redirect them before they ever get to the Twisted “User Weblog” server.

By examining the source of the wiki page on which a user’s profile picture icon appears, we can see that the URL path to the user’s image is retrieved by accessing a URL that looks like /users/username/icon.jpg (where username is the user’s full Unix username). So, with the following lines of Apache RewriteRule magic, we can enable only the serving of these user profile avatars but not let users create their own personal blogs:

#### We are ONLY using the /etc/apache2/httpd_users.conf file to
#### enable per-user avatar icons sourced from our OpenDirectory
#### user database. So to avoid the messy instance where people
#### create their own blogs we will redirect anything except the
#### image icons themselves to a 403 Forbidden error page.
<IfModule mod_rewrite.c>
        RewriteEngine On
        RewriteCond %{REQUEST_URI} ^/users
        RewriteRule !^/users/[A-Za-z]+/icon.jpg [F,L]
</IfModule>

The magic happens in the lines that begin with RewriteCond and RewriteRule. The RewriteCond rule examines the incoming URI and only continues processing if it begins with “/users”. That’s important because the next line, the RewriteRule returns a 403 Forbidden error for any and all requests that do not match a URI that starts out like /users/username/icon.jpg. In other words, without the RewriteCond directive, the entire web site would only be able to serve user profile pictures, and without the RewriteRule, all the URLs of the User Weblog server would be available (such as those to create new personal weblogs).

With both in place, however, I can get exactly what I want out of the Weblog Server. No more and no less.

In these cases, it makes sense to be smart about when I initiate these downloads. Being something of a UNIX-head myself, I wanted to use the age-old at command to download a Linux ISO during off-peak hours, which my ISP says starts at 2 AM. Much to my chagrin, I found that at doesn’t work by default on Mac OS X and, worse, the Leopard man page leads to a dead end (though it didn’t back in Tiger…).

Turns out that the system daemon that is responsible for checking up on at jobs has been wrapped with a launchd job. This makes enabling at on your system really easy:

sudo launchctl load -w /System/Library/LaunchDaemons/com.apple.atrun.plist

Once you’ve done this, you can now use at as you normally have done. For instance, I could now schedule my downloads to happen during the off-peak hours:

Perseus:Fedora maymay$ at 2:15am tomorrow # now press return
curl -LO http://download.fedoraproject.org/pub/fedora/linux/releases/9/Fedora/x86_64/iso/Fedora-9-x86_64-DVD.iso
# now press CTRL-D.
job 1 at Tue Jul 15 02:15:00 2008
Perseus:Fedora maymay$ atq
1	Tue Jul 15 02:15:00 2008

This is also incredibly handy for scheduling just about any resource-intensive task that you don’t have to do right now. To take it one step further, you can even let the computer itself choose when to run these resource-heavy tasks by using the batch command, which will execute commands much like at but will check the system load average instead of the system clock to determine if it should start the job.

Note that with the com.apple.atrun job loaded /usr/libexec/atrun is started every 30 seconds (unless you change the StartInterval key in the plist file). Since the atrun command checks a file on disk (that it places in the /usr/lib/cron/jobs directory) to see if there is any work to do, this will probably prevent your disks from ever sleeping, which could be a major concern for battery life on portables. Also, obviously, your computer needs to be turned on and awake for the job to actually launch.

For more information, check out the result of typing man at and man launchctl at a Terminal prompt. There’s also a really good Google Tech Talk about Launchd that will teach you a lot more about job scheduling on Mac OS X.

No matter what kind of writing you’re doing, using good tools to make your writing technically better is an incredibly handy thing. Letting the computers do the technical stuff—the stuff they’re good at—let’s you focus on the creative stuff: writing great content. Which is why, if you use a Mac, you’ll be happy to hear that any application’s text field let’s you do a number of really cool things (as long as it’s a Cocoa application, of course).

1. Auto-complete unfinished words

Try this out:

1. Open TextEdit, from your /Applications directory. A new blank document will open.
2. Type Hel and then press the ESC key. A drop-down menu will suddenly appear with an alphabetically sorted auto-complete list of suggestions, sourced from your computer’s current language dictionary. It looks like this:

This feature works with both Pages and, for those of you still using it for some reason (I know you’re out there), TextEdit, too. Also, if you’re a developer and a writer as well (like I am), you’ll be happy to hear that this feature also works with Xcode‘s Code Sense feature, and suggests completions for variable, function, class, and method names in your code.

2. Spellcheck as you type

A Cocoa text input field also has a number of other tricks up its sleeve. For instance, in many applications you can elect to turn on the “Check spelling as you type” feature, which will cause words you misspell (words not in the computer’s dictionary) to appear with a dotted red underline. If you right-click on these words, the contextual menu that appears will offer spelling corrections.

However, sometimes we use words like those in slang or colloquial language that isn’t in a proper dictionary. These words will still appear to be “misspelled” when you type, so in these cases, we can tell Mac OS X to “Learn Spelling” (also from the contextual menu). When you select this option, you append that spelling to your personal dictionary. (This is really just a plain-text file located at ~/Library/Spelling/lang file, where lang is the language code you’re typing in. For Enlgish, this file is ~/Library/Spelling/en.)

3. Look up word definitions and search for text in Google or Spotlight

Last, but certainly not least, another neat thing you can do with text on your Mac is look them up with Dictionary.app. Simply highlight some selectable text on screen, right-click and select “Look up in Dictionary”. This will cause Dictionary.app to open and display the definition of the selected word.

Since Dictionary.app can also look up articles in Wikipedia, this is also a very quick way to go to a Wikipedia article without ever having to open up a Web browser.

Also, from the very same menu, you can open a Web browser. Simply select “Search in Google” to cause your default Web browser to launch a Google search for the highlighted text.

However, if the services you typically access via Bonjour use static IP addresses, then there is one age-old networking technique you can use to simulate Bonjour-style naming conventions without actually using Bonjour. This, of course, is the /etc/hosts file.

The /etc/hosts is a simple, static, text-based mapping of computer names to IP addresses. It does exactly what Bonjour does except it doesn’t keep itself up to date when things change. Of course, if you’re using static IPs for the services you want access to, you can pretty safely assume that things aren’t going to be changing frequently anyway. Long-time sysadmins will laugh at this, but I say let them laugh. This is remarkably useful and very easy to implement.

Let’s assume I’m running a personal web server on my home network, and I can access my home network via a VPN. On my home network, my web server’s IP address is, say, 192.168.2.100, and I usually access it as http://server.local/. All I need to do is open a Terminal prompt and run the following commands as an administrative user:

sudo echo "192.168.2.100	server.local" >> /etc/hosts

That’s it. What this does is hard-wire the name server.local so that it always resolves to the IP address 192.168.2.100. Now, anytime anything on my computer tries to access server.local, it’ll always access 192.168.2.100 directly instead of ever needing to make an mDNS query on the network. The net effect is that we can trick our computer into thinking that Bonjour is working, even when it’s not—such as over a VPN connection.

Note that in default cases, hard-wiring an IP address like this completely prevents your computer from ever asking other computers (such as DNS servers) what the current IP address for this name is. That means if the IP address of the remote server changes, you won’t be notified, and things will just not work. So be mindful that you’ve made this change, and revert it as a first step in troubleshooting procedures.

By the way, Windows users can do the very same thing simply by editing their etc/hosts. They can find this file at C:\WINDOWS\system32\drivers\etc\hosts and can edit it with Notepad. They will also need to install Bonjour for Windows to get Bonjour working in the first place, of course.

Perseus:~ meitar$ pbpaste
Using the pbpaste and pbcopy commands, you can manipulate the contents of the Mac OS X clipboard (or more formally known as the pasteboard) right from the command line. As a brief example, just select the text of this first paragraph, copy it to your clipboard (with ?-c), and then type pbpaste in a Terminal prompt. You should see output similar to the following:Perseus:~ meitar$ 

Pretty straightforward, right? The only thing to be aware of is that the  symbol showed up in the output as a ? symbol. This is because the Terminal doesn’t support Unicode, but that’s a topic for another time.

Anyway, what’s happening here is nothing more magical than simply reading the clipboard and pasting it into a command’s standard output stream. As a result, you can construct pipelines that read from or add content to the clipboard. Here’s an example in reverse, which takes a command’s standard output and replaces the contents of the clipboard with it:

echo "Hello world! I came from the command line, but now I'm in the clipboard." | pbcopy

This command produces no output, but if we examine the contents of the clipboard (by selecting Edit → Show Clipboard from the Finder’s menu bar) we can see that the text we echoed has indeed been copied there.

Another way we can verify that this worked as expected is to simply pbpaste again:

Perseus:~ meitar$ pbpaste
Hello world! I came from the command line, but now I'm in the clipboard.
Perseus:~ meitar$ 

Anyway, this is cool, but it isn’t very useful yet. For that, we need to add some more stages to our pipeline. Let’s take the simple case of trying to count how many words the selected text contains. We’ll use the clipboard contents shown above to do this:

Perseus:~ meitar$ pbpaste | wc -w
      14

Using the word count (wc) utility, we can count words (-w) very easily. Indeed, our previous example does have exactly 14 words in it. We can also count characters (-c) or lines (-l) of text in the clipboard this way. This is like adding Microsoft Word’s “Word Count” feature to every single piece of text you can copy!

As another example take, for instance, the simple case of copying and pasting a snippet of email from Mail. Instead of pasting it back into a text file verbatim, let’s prepend ‘> ‘ to the beginning of each line. This way, when we paste our email’s snippet, we’ll know where the snippet begins and where it ends. This is a simple three-stage pipeline that uses pbpaste to take our clipboard and put it into the pipeline and then reads back the result from the pipeline back to the clipboard using pbcopy. In the middle, we use sed to insert the desired text at the start of each line:

pbpaste | sed -e 's/^/> /' | pbcopy

Now, when you paste your clipboard, you’ll have a greater-than symbol at the start of each line. Naturally, check out the manual pages for all of these commands for more detailed information. For instance, type man pbpaste for more information about the pbpaste command.

Whenever you tell an application to “Remember this password in my keychain,” what you’re doing is writing a new encrypted entry into your user account’s ~/Library/Keychains/login.keychain file. Then, the next time the application needs to access a restricted resource, it just asks Mac OS X to get the password for it. Of course, all of this happens automatically, so except for that single checkbox most users probably don’t know that the keychain even exists.

What’s even more awesome than all of this automagic password storing action, though, is the fact that Apple has also provided an easy-to-use application to manipulate the keychain yourself. What good does this do us? Plenty! Observe.

Say you’ve just signed up with a new ISP. They send you a username and a password to log on to their ADSL network with. Of course, they send this password to you on paper—how insecure! Instead, after changing the password to something else first (something other than mypassword, which is the example password I’ll use here), we can use Mac OS X Keychain to securely store the password and retrieve it later.

1. First, launch the Keychain Access application located in the /Applications/Utilities folder of your startup drive.
2. Next, click the “Create a new Keychain item” button (the +) button near the lower left-hand corner of the window. The Add Keychain Item sheet appears.
3. Enter a meaningful name, such as “ADSL ISP Account” in my example, in the Keychain Item Name field.
4. Enter the username or account name associated with this password in the Account Name field.
5. Enter the password into the Password field.
6. Click the Add button.

That’s all there is to it. To later retrieve your password if, say, you ever forget it:

1. Launch the Keychain Access application.
2. Locate and double-click the keychain item that stores the account and password information you want to retrieve.
3. Tick the “Show password” checkbox. You’ll be presented with a dialogue box that asks for your keychain’s master password. Unless you’ve already set it to something else, this is the same password you use to log in to your Mac OS X user account.
4. Enter your keychain password and click “Allow.” If you click “Always Allow” instead, Keychain Access will not prompt you for your login keychain’s password the next time you ask to see this particular password. I never press that button.
5. Your password’s plaintext is now visible.

This effectively obviates the need for third-party applications such as Password Gorilla, PasswordWallet or KeePassX which are great programs, but all suffer from a lack of a good user interface. Furthermore, there’s no reason why we can’t store short arbitrary strings of sensitive information in the keychain temporarily. Sure, it might clutter up your keychain, but you can always search the entries using the standard Mac OS X filter search bar at the top right of the window.

In fact, Apple’s been kind enough to offer an interface to do just that in an even more effective way, called Secure Notes. These are simply plain text strings of arbitrary length that can be stored securely inside your keychain, and that use the same interface to access (requiring your password to view). The only real difference is that instead of a single line, you’re given a fully scrollable text area in which to type your secure note.

Moreover, because keychains can be synced to multiple Macs with .Mac Sync (or a third-party synchronization solution), you can always have access to all your passwords regardless of which physical Mac you’re using. Best of all, since you never have to remember another password ever again, you can quit using the same password for multiple accounts, and you can always use really hard-to-crack passwords.

With that caveat out of the way, here’s how I managed to move all the pages in an Apple WikiServer group wiki and blog from one group to another.

Dear God, please have a backup!

Okay, step 1 is mundane, but seriously, please of please have a backup of your data before you do any of these things. To make a back up of your entire WikiServer’s data store, simply:

sudo tar -cvzf backup-file-name.tgz /Library/Collaboration

If anything goes wrong, you can restore from your backup just as simply:

sudo tar -C /Library/Collaboration -xvzf backup-file-name.tgz

Okay, with that out of the way, next make sure absolutely nobody is using any of the Group services for the group you are going to perform the move from, or to. People can work on other group’s wikis, it’s just the ones you’ll be touching you want people to avoid. You can enforce this with some Apache redirects, which is left as an exercise to the reader.

Step 1: Rename or create a new Group in Workgroup Manager

First, you need to either rename or create a new group in Workgroup Manager. If you make a new group, be sure to enable all the services that the old group used.

If you’re simply renaming a group, then you might not even have to go through this trouble. You merely need to change the group’s “Name” (as opposed to its “Short Name”) and then stop and start the Web Service from Server Admin to see the group’s name change. However, if you also want to change the group’s “Short Name” (i.e., the group’s POSIX group account symbolic name), then you will need to perform these steps.

Once you have the new group ready to go in Workgroup Manager and you have stopped and started your Web Service in Server Admin, continue to the next step.

Step 2: rsync all your files from the old group to the new group

This is simple. Just run:

sudo rsync -avzE --progress /Library/Collaboration/Groups/old-group/ /Library/Collaboration/Groups/new-group/

Note that the trailing slashes on the directory names in this command are quite important, as they tell rsync to take the contents of the first directory and place those items as the contents of the second directory. Without the trailing slashes, rsync will make extraneous directories for you, which Apple WikiServer won’t understand. See man rsync for more information.

Step 3: Update the plists for all your WikiServer pages in the new group

Now, WikiServer has all the content of your old group but all of the internal references are wrong, since they still point to the old group. What you need to do is rewrite all those references so that they point to the new group address. The group references are stored in property list files. See my older blog entries for details about the filesystem structure of Apple’s WikiServer data storage layout.

To do this is a relatively simple procedure. As root, do the following, and do it carefully because as root you can easily mess up (and there is no undo button on the command line!):

sudo su -
cd /Library/Collaboration/Groups/new-group
grep -ri "old-group" * | grep '^[^B]' | cut -d ':' -f 1 | grep 'plist$' | sort | uniq > /tmp/list_of_plist_files_to_edit
for i in `cat /tmp/list_of_plist_files_to_edit`; do sed -e 's/<string>groups\/old-group/<string>groups\/new-group/' $i > $i.new; mv "$i.new" "$i"; chown teamsserver:teamsserver "$i"; chmod o-rwx "$i" done;

In English, this means:

1. Become root.
2. Change to the /Library/Collaboration/Groups/new-group directory
3. Find all plist files that have the the string old-group in them, sort them, and write this list of files to the /tmp/list_of_plist_files_to_edit file.
4. For each of the files listed in the /tmp/list_of_plist_files_to_edit, find the text string <string>groups/old-group and replace that text with <string>groups/new-group, and save this change as the name of the file with .new appended. Finally, replace the original file with the file we modified, and give them the appropriate permissions.

Step 4: Create an Apache redirect to make sure no HTML links are broken

Okay, at this point your new group is up and running and it should be working. However, if you had any links at all in any of your group’s pages, they are now all broken because they still point to the old group. Rather than going through the HTML itself and cleaning this up right now (because that’s very error-prone indeed, even with automated tools), it’s much easier to just tell Apache to redirect all requests for the old group to the new group.

To do this, edit the Apache configuration file of whatever Virtual Host you have been serving the WikiServer from. Most of the time, this will be at /etc/httpd/sites/0000_any_80_.conf.

The end of that file probably looks something like this::

#       Include /etc/httpd/httpd_users.conf
#       Include /etc/httpd/httpd_directory.conf
        Include /etc/httpd/httpd_groups.conf
        Include /etc/httpd/httpd_teams_required.conf
        LogLevel warn
        ServerAlias *
</VirtualHost>

Right above these Include directives, simply add the following:

        <Location /groups>
            <IfModule mod_alias.c>
                Redirect 301 /groups/old-group http://your.server.address/groups/new-group
            </IfModule>
        </Location>
#       Include /etc/httpd/httpd_users.conf
#       Include /etc/httpd/httpd_directory.conf
        Include /etc/httpd/httpd_groups.conf
        Include /etc/httpd/httpd_teams_required.conf
        LogLevel warn
        ServerAlias *
</VirtualHost>

There you have it. New Teams Server group, old group’s data.

Caveats

Note that this does not update any of the SQLite databases used to store things like revision history and so forth. These things are, for the most part, not really necessary to update but it would be ideal if the old plist revisions could be changed in there, too. That’s not so much more extra work, really, but I’ve found it typically unnecessary except in fringe cases, so I leave that as an exercise to the reader.

If you do this to your WikiServer and do not update the SQLite databases as well, just be mindful of that fact so that you’re not surprised if something goes wonky down the line.

1. Be certain you have the git-core package installed on your system and that this package includes the git-cvsimport command. You can run git help -a | grep cvsimport to verify this.
2. Be certain you have the cvsps command-line tool installed. This does not come with the git suite of tools, so you’ll need to get it separately. If you’re a lazy Mac OS X user, like me, you can use MacPorts: sudo port install cvsps. Otherwise, get it from the source.
3. Prepare your CVS login information for the remote server before you run git cvsimport. You need to do this so that the git tool will be able to log you in to the CVS server automatically. The command for this looks like:

   CVSROOT=:cvs-login-method:cvs-user-name@cvs.server.name:/path/to/CVS/root cvs login

   For example, if you’re pulling code from the anonymous CVS server that runs on Drupal.org, you might use this: CVSROOT=:pserver:anonymous@cvs.drupal.org:/cvs/drupal-contrib cvs login. This command will prompt you for the password for the user you specified at the server you specified (for anonymous access, the password is almost always anonymous) and will hash this in the ~/.cvspass file for future use by CVS

4. Finally, run the git cvsimport tool, and specify the proper options. Using the Drupal example above, your command might look like this:

   git cvsimport -v -d :pserver:anonymous@cvs.drupal.org:/cvs/drupal-contrib contributions/modules/module-name

   This would login to cvs.drupal.org using the CVS‘s pserver login method, provide the username anonymous and the password you specified in the previous step that is hashed in ~/.cvspass, set the CVS document root to /cvs/drupal-contrib, and pull the code located at contributions/modules/module-name into the current working directory as a git repository.

This works pretty nicely, and creates a git repository just as though you’d created it with git init in the current working directory.

If you get an error that looks like this:

AuthReply: cvs [pserver aborted]: descramble: unknown scrambling method

then you’ve most likely specified the CVS document root incorrectly. Most notably, git cvsimport does not understand a CVS document root wherein the password is specified in the document root URL itself. So, for example, git cvsimport -d :pserver:password:username@cvs.server.name:/path/to/CVS/root code/to/checkout will not work. Omitting the password and the separating colon from the URL should fix it.

I’m going in!

Apple’s WikiServer keeps each group’s data segregated. There’s currently no way in the GUI to search across multiple wikis from a single wiki, or move one wiki page in one wiki to another wiki. Luckily, because the Apple WikiServer is a rather lightweight application, it’s not that hard to move things around. Doing so doesn’t seem to cause any problems with the application, either—a testament to its apparent simplicity.

Most simply, say you have created a number of different wikis for different groups, and in so doing you’ve created segregated spaces for different parts of your organization. Now say you’ve got a wiki page in one wiki that you’d rather place in a different one, or that you’d like to copy. Sure, you can edit the first wiki’s page, copy the WYSIWYG editor’s text, and paste it into a new page on the second wiki. But where’s the fun in that?

Instead, let’s move some files around to get a sense for how the Apple WikiServer stores its data.

First, recall that the Apple WikiServer stores its content inside the /Library/Collaboration directory on the computer hosting the wiki web site. Inside of this directory you’ll find four items:

$ cd /Library/Collaboration/
$ ls -la
total 8
drwxrwxr-x   6 _teamsserver  _teamsserver   204 Mar  2 13:50 .
drwxrwxr-t@ 60 root          admin         2040 Mar  3 19:16 ..
drwxr-xr-x   2 _teamsserver  _teamsserver    68 Mar  2 13:50 ArchivedGroups
drwxr-xr-x   6 _teamsserver  _teamsserver   204 Apr  4 18:48 Groups
drwxr-xr-x   2 _teamsserver  _teamsserver    68 Mar  2 13:50 Users
-rw-r--r--   1 _teamsserver  _teamsserver   249 Mar  2 13:50 dataVersion.plist

First, note that all of these files are owned solely by the _teamserver user, and only that user can access these files in any way. This keeps people like you and me from accidentally tampering with the Apple WikiServer data store. However, for the purposes of this exercise, we’re going to tamper anyway. To do so, you’ll need root permissions, which comes with a big caveat: be careful whenever you are root—one typo could erase your hard drive (and there is no undo)!

Become root by saying:

sudo su -

and supplying your password. Once you are root, you can explore further into the depths of the Apple WikiServer.

Inside the Apple WikiServer data store for wiki pages

The Apple WikiServer, which runs under the username _teamserver, is really just a bunch of organized flat files, supplemented with a number of SQLite databases (also flat files, really), and glued together with a bunch of Python scripts running under the Twisted networking framework. We’re not going to cover Python or Twisted in this post, so for further information check out their respective homepages. There’s also a great introduction to Twisted at ONLAMP.com.

To make the wiki go, the WikiServer runs these scripts to read and write data to these flat files. Let’s dig into this data store. Since the wikis are each associated with an existing group, all of the wiki files are inside the Groups subdirectory. In this directory, you’ll find one directory for each group that has a wiki. Each group directory, in turn, contains 6 other directories, 2 regular files, and 1 SQLite database:

# pwd
/Library/Collaboration/Groups/chiefs
# ls -p
discussion/	index.db	metadata.plist	resources/	wiki/
extrainfo	mailinglist/	public/		weblog/

The two regular files are metadata.plist and extrainfo. The metadata.plist file maintains the preferences of the individual wiki web site that you set from the “Group Settings” page on the wiki web site itself. That is, these settings are all editable via the web directly at http://your-server.local/groups/your-group-name/settings/. That page is also accessible to an admin user from the wiki home page under the “Admin functions” header in the sidebar of the web page (the key names in the plist file are pretty self-explanatory and map easily to the GUI options).

At the moment, the extrainfo file is a bit of mystery. It stores single lines of plain ASCII text in the form:

groups/your-group-name/web-service/page-id

where your-group-name is the name of the associated group for the file, web-service is one of wiki, weblog, and I assume can also be mailinglist or discussion, although I’ve only seen the first two on my server in practice.

What we’re most interested in for the purposes of wiki pages is, unsurprisingly, the wiki directory. This directory stores as many subdirectories as there are pages on your wiki. Every time you create a new page, a new directory is added here. If you delete that page, however, this directory is not deleted outright.

Each directory is named with the unique page identifier that the Apple WikiServer automatically generates followed by a .page extension. The exception is the welcome.page directory, special in that it contains the content of the wiki’s main (front) page. The Mac OS X Server Web Technologies Administration for Version 10.5 Leopard manual explains the contents of this directory as follows:

• /Library/Collaboration/Groups/groupname/wiki/pagename.page/ contains the component files of a wiki page.
• /Library/Collaboration/Groups/groupname/wiki/pagename.page/page.html contains the main text of the wiki (html content).
• /Library/Collaboration/Groups/groupname/wiki/pagename.page/page.plist contains the metadata for the wiki page.
• /Library/Collaboration/Groups/groupname/wiki/pagename.page/revisions.db contains the version history database for that wiki page.
• /Library/Collaboration/Groups/groupname/pagename.page/images/ contains the images for that wiki page.
• /Library/Collaboration/Groups/groupname/pagename.page/attachments/ contains all attachments for that wiki page.

Here are some additional details:

• The “page.html” file contains all of the (X)HTML that appears inside the WYSIWYG editor field when you click the “Switch to HTML View” button (the arrow brackets). Nothing more, and nothing less.
• The “page.plist” file maintains the metadata that associates a single wiki page to its group as far as the Apple WikiServer is concerned. If you take a closer look at it, you’ll find property list keys such as commentUID and uid whose values are both strings—URLs, in fact—that the Apple WikiServer uses to, for example, call the appropriate wiki theme to render around the page content. This file also stores self-explanatory data such as author, createdDate, page kind, a last modifiedDate and lastModifiedAuthor and, of course, the page title. The most interesting key in this file is the versioned key, which is a boolean that can turn versioning on or off. By default, wiki pages have this key set to true and (for example) weblog pages have this set to false. However, you can manually turn off versioning for a specific wiki page by changing this key yourself. It won’t delete past revisions, it will just stop new revisions from being saved. This is especially cool, because it means you can version-control weblog pages simply by editing the page’s associated plist file.
• The revisions.db is the SQLite database in which all revisions are stored. The database contains a single table, revisions, that looks like this:

  CREATE TABLE revisions(revision INTEGER PRIMARY KEY AUTOINCREMENT, editType, comment, lastModifiedBy, content, time DEFAULT CURRENT_TIMESTAMP);

  The interesting bit is the content field, which itself is just a string—but a string stored as an Apple property list, in full. Some keys in the property list inside the SQLite database are the same as those inside the page.plist file for the page, but with some differences. For instance, there is no versioned key in the plist in the database (for obvious reasons), and there is a tags key (which is missing from the filesystem equivalent). Most importantly, there is a content key, which stores the content of the page.html file.

• The revisions.db file is created automatically only if the versioned key in the page’s page.plist file is set to true.

Finally: move a wiki page from one wiki to another

Now that we understand some of the internals of the Apple WikiServer data store, let’s mess with it just a little by moving that one test wiki page from one wiki into another. Since almost everything is flat files, it’s remarkably easy to do:

# mv /Library/Collaboration/Groups/your-group-name/wiki/page-id.page /Library/Collaboration/Groups/new-group-name/

Be certain that there isn’t already an existing page in the new-group-name wiki with the same page-id as the one you’re moving. So far, Apple WikiServer appears to generate unique page IDs across the entire system (as opposed to just per-group), but I haven’t verified this yet.

With the wiki page files physically moved into the new wiki, you can now access the page at the new wiki URL. Trying to access it in the old URL will result in an HTTP 404 “Not Found” error. However, the page at the new wiki URL has the same theme as the other, old wiki. To correct this, you need to edit the page.plist file in two places: the uid string value and the commentUID string value relative URLs need their your-group-name changed to the new-group-name, whatever it is.

After you save this this change, simply reload the wiki page in your browser. Voila. Page moved.

However, if you find that the .local host name no longer works but the IP address still does, the problem may be in a corrupt or outdated local DNS cache. Luckily, the solution is incredibly simple. Just run:

dscacheutil -flushcache

at a Terminal prompt, and try again. If you’re still running Mac OS X 10.4 Tiger and you experience this problem, the solution is just as simple. Instead of the above line, simply run:

lookupd -flushcache

at a Terminal prompt.

Apple WikiServer: Mac OS X Server’s built-in Intranet builder

At work, I’m finally getting the opportunity to try the Apple WikiServer out. Its strongest asset, by far, is the integration it has with Apple’s Mac OS X permission scheme. Apple WikiServer makes heavy use of the OS‘s built-in user accounts to define users and groups, and the permissions those users and groups have to edit, view, and comment on pages in the wiki you create with it. And, because Leopard Server has full support for Access Control Lists, those permissions schemes can be as complex as you like.

This is very important because many large (and small) organizations have sensitive material that they’d like to keep private, or restricted to certain groups. Historically, wikis are a free-for-all. Anyone and everyone who has access to any part of the wiki can change any other part of the wiki. Recent wiki implementations such as later version of MediaWiki and some other wiki software have implemented permissions systems to allow administrative users to control access rights, but these are often complicated or require code-level configurations.

With Apple’s WikiServer, all of these permissions can be managed via the Workgroup Manager application, and because you can take advantage of the built-in ACL support, you can model your organizations permissions scheme directly in the Server OS permissions structure, giving you a much easier way to control information access. Take note, however, that like almost all other things that have to do with your Apache configurations, your Server’s Web Service will likely need to be stopped and started again for any changes you make to a wiki’s permissions take effect.

The Workgroup Manager application is also where you go to create new wikis for groups. To enable a wiki, you need to already have created a group and assigned users to that group. For instance, I created a Developer Wiki where all of the in-house developers can share tech tips, so I created a group called “Developers” and assigned individual developers, as well as the company executives (by way of the “Executives” group) to that group. The group-within-a-group technique is key, because if the company executives change, the members of the Developers group does not need to change, too. In all of Apple’s publications, Apple refers to the wikis hosted by WikiServer as an “intranet website.”

It’s clear that Apple intended this product for use within small companies, and not necessarily out on the open Internet. What follows are just a few notes I’ve compiled about how the Apple WikiServer works.

Front-end Code Generation from the Apple WikiServer WYSIWYG Editor

The Apple wikis are very nice to use. Their functionality is relatively straightforward to find and activate. However, the HTML code that the Apple wikis generate can be a little confusing. By default, new page text is entered into a semantically meaningless <div> element. This can be changed by highlighting text and then selecting “Paragraph” from the formatting toolbar. Subsequent paragraphs that are typed seem to then use <p> elements. However, some paragraphs revert back to <div>s when I used it, and I’m still not sure why or when this occurred.

On the plus side, so far, all the browsers I’ve used with the Apple WikiServer function the same way. This include Firefox 2, Safari 3.1, and Internet Explorer 6 and 7.

Typing actual code and having it marked up as such can’t be done in the GUI formatting toolbar to select a <code> element. The “Monospace” item in the text formatting toolbar creates <pre> elements and <pre> elements only. However, Apple does provide a “Switch to HTML view” button (the arrow brackets button) and one can enter standard HTML, including <code>…</code> elements in that view, and then switch back. This behaves perfectly on all browsers except Internet Explorer, in which your text area field shows no line breaks whatsoever.

Apple’s WYSIWYG editor handles escaping special characters when those special characters have HTML entity reference equivalents, such as double quotes (“), arrow brackets (< and >), and ampersands (&). It does not seem to handle Unicode characters, such as the ellipses in the prior paragraph. However, such Unicode characters need not be escaped as long as the document’s character set is UTF-8 (or UTF-16), which the Apple WikiServer specifies and supports out of the box.

Pressing the Return key twice causes the Apple wiki to generate an empty <div> or <p> element with an explicit break (<br />) inside of it. One can deduce that this is a design choice in order to help transition users who are used to plain <textarea /> inputs to Apple’s WYSIWYG editor. It’s also the only way to space paragraphs properly if the user hasn’t selected the “Paragraph” option in the text formatting toolbar. Otherwise, simply hitting the Return key once is enough to space paragraphs apart properly (i.e., the functionality is equivalent to the way Microsoft Word or Pages handles paragraph breaks).

Interestingly, the “Enter URL…” functionality form the toolbar is smarter than one might first assume. For instance, it recognizes email address and prepends a mailto: scheme to the link if it finds one. This is contrary to what the Apple-provided manual states, which tells you to enter the “mailto:” portion as part of the URL. In fact, you should omit this, else your final mailto link will actually read “mailto:mailto:your.email@address.com”.

This means linking to “mailto: links” is as simple as typing an email address. Similarly, the WYSIWYG doesn’t complain if your fully-qualified URL doesn’t include a scheme, so you can enter //apple.com/ and the subsequent link is generated as <a href="//apple.com/">Link text</a>. This is one step above and beyond even WordPress’s new WYSIWYG editor, which forcefully prepends an http: scheme to URLs without one.

For the most part, copy-and-paste works as expected, except in cases where the WYSIWYG editor does not understand the current formatting, such as a specific font and (and this is a biggie) for links. At first, the editor will appear to show that the formatting (including links) is saved, but when you actually save the page, only the formatting that the WYSIWYG editor understands is actually saved. Worse, all your links are turned into underlined—but unlinked—text. In short, this means that if you are copying and pasting page content that contains links, you need to do so in the HTML view of the page editor.

Inexplicably, the editor generates <i> and <b> tags for italics and bold, instead of the preferred <em> and <strong> elements. I’m not sure I understand why this is the case. There does exist a an “emphasis” option in the toolbar, as does an “important” option, but these generate strange spans instead. The “Important” item wraps the selected text inside a <span class=”Apple-style-span custom_forecolor_important”>…</span> and the “Emphasis” item wraps the selected text inside a <span class=”Apple-style-span custom_forecolor_emphasis”>…</span> element.

There’s also one other item, “Highlight,” which wraps the selected text inside of a <span class=”Apple-style-span custom_backcolor_highlight”>…</span> element.

The only explanation that makes sense to me, after much speculation, is that perhaps Apple does not want to encode semantic information such as what <em> and <strong> would imply from users who use a WYSIWYG editor. This shows either a blatant distrust of users or incredible foresight. I’m not sure which.

Page names and URLs

Currently, the search functionality built into Apple’s WikiServer only searches on the text of a page’s title.

Apple’s WikiServer generates unique page names for each new wiki page. These names consist of two parts, but only one seems to make any difference. For example, if you create a new Wiki page called “Hello”, you might get an address in your web browser’s location bar that looks like this:

http://your-server.local/groups/your-group-name/wiki/1d06a/Hello.html

Most of this is standard URL stuff (the protocol, the server address), and most of the rest is self-explanatory (the group name, the wiki section). The important bits in this URL are the last two:

• 1d06a
• Hello.html

Obviously, “Hello.html” came from the fact that you named your new page “Hello”. WikiServer appended “.html” on its own. The other bit, the short string of random characters, is a unique identifier used across all of this group’s web services (the wiki, blog, calendar, and mail list, if these other services are enabled) to uniquely identify this page. What’s interesting about this is that only the unique string of characters seems to matter in regards to accessing the page. That is, if you next ask for:

http://your-server.local/groups/your-group-name/wiki/1d06a/Some-Random-Page.html

you’ll still get the same “Hello.html” page from before, even though you’re seemingly asking for “Some-Random-Page.html”. In fact, it doesn’t seem to matter what you replace “Some-Random-Page” with. So long as there is some text in that part of the URL, that the URL ends with “.html” and that the unique identifier remains untouched, you’ll always end up retrieving the “Hello.html” page.

This means that if you change this page’s name later to, for instance, “Hello world”, old links that point to “…/1d06a/Hello.html” will continue to work, even while new links will start to point at “…/1d06a/Hello_world.html”. From a usability perspective, this is simple and effective; it ensures that users have a reminder of what the page is about by looking at the last part of the URL. However, once page names change, it becomes a bit non-optimal, because the same page can be referred to by multiple names—a “no-no” in the SEO world and a practice discouraged by most semantic-web types.

I would imagine that Apple made this design decision because the company envisioned their WikiServer to be used, again, primarily in intranet and SOHO environments, and as a result are not too concerned with search engine optimization. As an aside, this unique string is also how Apple’s WikiServer identifies the stored content on the filesystem. Read on for more details.

Hacking the Apple WikiServer

There isn’t a lot of information out on the Web right now about how to work with the WikiServer, especially for developers. Therefore, some digging is needed. After a bit of research, I discovered the following key directories that the WikiServer uses. They are as follows:

• /usr/share/collaboration - This has a few developer tool support files as well as the majority of the client-side code for the Wiki (javascripts, etc).
• /usr/share/wikid - This directory holds the Python sources and compiled bytecode for all the “Teams” components (including wiki, blog, calendar, etc.). It seems to run on Twisted and a number of other familiar-sounding components.
• /Library/Application Support/Apple/WikiServer - This is where most the data is stored, inside of plist files and a few others. The Themes subdirectory here is where Apple recommends that look-and-feel changes be made.
• /Library/Collaboration – This is the default data storage location for all the “Teams” components. The actual content of the wikis and blogs will be kept somewhere in this directory, which means that this is the directory you want to backup to backup the content of your wikis. This location is the only user-configurable one of the bunch. To change it, change the “Data Store” value in Server Admin. (A more detailed listing of this directory hierarchy is available on page 62 of the Mac OS X Server Web Technologies Administration For Version 10.5 Leopard manual.)

If you take a peek at the /Library/Collaboration directory, and follow that into the Groups/your-group-name/wiki directory, you’ll find a list of all the pages in your wiki stored as .page bundles, identified with the unique character string WikiServer generated when it first created the page.

It should be noted that anything in the /usr/share directory will likely be overridden whenever Apple releases an update that modifies the WikiServer. As a result, any and all changes you make to WikiServer’s templates or themes should be done by creating new files in the /Library/Application Support/Apple/WikiServer/Themes directory.

It’s interesting to note that the WikiServer seems to use Python for its back-end processing. This may open up some interesting integration possibilities for Python programmers in the future.

More help eslewhere

Even though Apple WikiServer is relatively new, there’s a load of helpful information about it on the web. Most of the good stuff is on Apple’s own Discussions boards, but more and more info is beginning to show up on blog posts. A Google search should give you what you need. For the really lazy, however, here are a few helpful items:

• Server Admin 10.5 Help: Customizing Wiki Themes and Layouts
• Apple Discussions: How to customize Apple WikiServer templates and themes
• Mac OS X Server Web Technologies Administration For Version 10.5 Leopard

This was just a brief introduction to WikiServer from some notes I’ve been collecting in my experimentations, but I hope it’s helpful to someone somewhere. Cheers. Or, continue to Part 2.

The only way I could get my Vodafone 3G card to work was to fire up a Windows XP guest inside of my MacBook Pro, using VMware Fusion. Connecting to the Internet with the 3G card using the Windows guest was smooth sailing, but that only provided the Internet connection to the Windows virtual machine. I wanted my Mac to be directly connected.

The solution is obvious, but a few gotchas really bit me hard. To get the Windows guest to share its Internet connection from the 3G card to my Mac, I would need to bridge VMware’s virtual ethernet adapter from the Windows guest to the Mac OS X host. Once bridged, both the Windows guest and the Mac OS X host would logically be on the same ethernet network segment. At this point, I can enable Windows XP’s built-in Internet Connection Sharing (stupidly dubbed “ICS” because everything needs a TLA) on the 3G connection so that Windows NATs it through to the bridged virtual ethernet card. Finally, I can connect to Vodafone’s 3G network, and all should be well.

Here’s the gotchas.

First, in order for VMware to actually initiate the network bridge when it starts up, it must detect that a physical link is active on your Mac. In other words, Mac OS X’s Network System Preferences pane must show you a yellow dot next to at least one physical networking device (probably either your “Built-in Ethernet” or your “AirPort” ports). VMware Fusion will give you no errors or warnings that a bridge is unavailable until you try to connect your virtual machine’s network while set to bridge, in which case VMware Fusion will complain with an error that reads: “The device on /dev/vmnet0 is not running.”

Obviously, if you have no other devices to connect to, you need to fake one. The easiest way to do this is to set up a Computer-to-Computer network using AirPort. Just go to your AirPort menu bar item and select “Create Network…” and create the network (preferably encrypted). If you check System Preferences now, you should see a that AirPort has a yellow dot next to it and reads as having a “Self-Assigned IP Address.” Now that you have a physical link on your AirPort card, you should be able to start the VMware Fusion virtual machine with bridged networking mode without incident.

However, if you do encounter the above error anyway, you need to restart the VMware network bridge. You can do this either by shutting down VMware completely (turn off your guest operating systems, and quit the VMware Fusion application), or you can run the following commands as an administrator in Terminal, which will stop any bridge currently running (or do nothing if no bridge is running) and then restart it, providing the output as shown:

sudo killall vmnet-bridge
sudo "/Library/Application Support/VMware Fusion/vmnet-bridge" -D vmnet0 ''
Entering event loop...
Examining network configuration...
Turning on bridge with host network interface en1...

Obviously, you may be asked for your password as you perform this procedure. Note that the trailing two apostrophes are single quotes with no space. This is (almost) how the VMware Fusion boot.sh script starts and stops the network bridge. Specifically, you’re telling the vmnet-bridge application to run in Debug mode and to bridge vmnet0 to whatever is the current primary networking interface. In the example output shown above, this is en1, or my AirPort card connected to the computer-to-computer network I created in the previous step.

Hopefully you won’t have to mess with the vmnet-bridge application, as this should happen on its own when you start up VMware Fusion if you have any physical link on a network device. Nevertheless, I’ve found this is sometimes unreliable, so just in case it doesn’t now you know how to bring up the bridge on your own. (Tip: once it’s up, you can CTRL-Z to pause it, re-start it with fg %1 and then quit Terminal if you like. The bridge will still be up.)

Now that the AirPort card has a physical link, and the VMware network bridge is running, the next step is to configure your virtual machine to use bridged networking. Just go to Virtual Machine → Network → Bridged as normal. Make sure Connected is also selected. Now start up your Windows guest.

Once Windows boots, go to the Network Connections window by selecting Start → Connections → Show all connections. At this point, your “Local Area Connection” in Windows probably has a warning sign on it and reads as having “Little or no connectivity.” It probably has a self-assigned IP address just like your AirPort card. That’s fine—as long as it’s not “unplugged,” we’re in good shape.

Next, select whatever other connection you want to share the Internet from (in my case, the 3G modem, but it could also just be any other connection in the window), right-click it and select Properties. Go to the Advanced tab and make sure “Allow other network users to connect through this computer’s Internet connection” is checked. The other boxes won’t matter.

What this does is turns on Windows’ own NAT service that configures the one connection (the one your sharing) as the WAN side of (yet another) virtual networking device and the Local Area Connection (the one we’ve bridged to our AirPort or Built-in Ethernet card on our Mac) as the LAN side. Hit OK as many times as is necessary to close the network connection properties windows and wait a few moments. Sometimes this can take up to 30 seconds or so, but eventually you’ll see Windows announce that “Local Area Connection is now connected.” If you inspect it, you’ll see that the IP address configuration has been automatically assigned as a “Manual Configuration” with the address of 192.168.0.1, a subnet mask of 255.255.255.0, and no default gateway.

As a last step, now we can actually connect to the Internet using whatever service we have. In my case, this is when I hit the “connect” button on my Vodafone Mobile Connect software. Once the connection is established and the Windows XP virtual machine can see Internet, it takes up to another minute or two (or three) for the Mac’s connection to get an IP address from the Windows guest, but it invariably works.

If the Windows side of things is giving you any trouble, the most reliable solution I’ve found is to simply disable, then re-enable whatever connection isn’t behaving as desired. If after all of this your Mac still doesn’t get an IP address from the Windows XP guest, disconnect and then re-connect the virtual machine’s ethernet card (by toggling the “Connected” menu item in the Virtual Machine → Network menu). Also, of course, be doubly sure that your AirPort is set to “Use DHCP.”

Phew! So simple…and yet so much harder than it had to be. I found the following two PDF documents very helpful in understanding all of this. You might too:

1. VMware Fusion Network Settings — a super-brief, but excellent introduction to VMware’s network setting internals. It’s also a PDF download attached to the linked forum thread.
2. Share Windows XP Guest Internet Connection with OS X Host HOWTO — This basically describes the same thing this post does, but it does so using absolute step-by-step instructions. It’s also a PDF download attached to the linked forum thread.

First things first. Once you have an idea of what work you want to do, you should give yourself a space to do this work without disturbing anybody else’s work. Do this by making a new, personal branch. Unlike Subversion and some other centralized version control systems, git makes it possible to do make all kinds of changes to your repository, including making branches, and even save those changes without having to republish everything back to the central repository server at each step. In other words, if you thought Subversion branches were “cheap,” you’ll love git’s branches.

Also unlike Subversion, which stores its branches in completely separate pathnames, git keeps all branches in the same filesystem tree separated only with metadata (in .git/refs/remotes for remote branches and in .git/refs/heads for local branches, to be a bit more precise), so you don’t have to create lots of different directories for all your branches (unless you want to). With no branches defined, you’re working in the “master branch,” or “the trunk” by default.

git branch
* master

By saying git branch you ask git to print a list of all (local) branches. The one with the asterisk marks the current branch, the one you’re using at the moment you run the command. Since there are no branches, you’re currently in the “master” branch. But we don’t want to make changes here, we want to make changes in our own private branch, so we’ll make a new one.

git checkout -b new_branch_name

This will create a new branch with the name new_branch_name and immediately switch to it. Notice that you’ve made absolutely no changes to the filesystem itself; only the git metadata has been altered. Saying git branch again will show you the change:

git branch
  master
* new_branch_name

Also note that since we haven’t comitted any changes, we don’t need a commit message (or “log message”) for creating this branch. We’ll add one later, when we need it. Now go ahead and write some code in your new branch. At any time, you can create a new branch in the same way you added the first. Each new branch is created at the HEAD (“latest”) revision of whatever branch you’re currently working in.

git checkout -b another_branch
Switched to a new branch "another_branch"
git branch
  master
  new_branch_name
* another_branch

To switch back to any other branch, simply git checkout that branch again:

git checkout master
Switched to branch "master"

If you want to delete a branch you don’t like, that’s easy too:

git branch -d another_branch
Deleted branch another_branch.

Keep in mind that throughout all of these branch creation and deletion actions, the only thing that’s being altered is the git metadata. That’s why it’s so cheap to create new branches. If you ever have a new idea you’re working on, it’s recommended that you create a branch for it, even if that branch is so short-lived it never gets published.

So you have a new local branch, and you’ve been working as you normally do for a few minutes, creating files, editting them, and so on. Running git status now will ask git to show you the changes you’ve made to your filesystem. If you’ve created any files in new directories that git doesn’t know about, it will simply report that directory. If you’ve made new files in directories git does know about, it will list all those files explicitly.

git status
# On branch cartoon_contests
# Untracked files:
#   (use "git add ..." to include in what will be committed)
#
#	sites/default/modules/cartoon_contests/
#	sites/default/modules/factiva/.factiva.module.swp
#	sites/default/modules/testfile
#	sites/default/settings.php
nothing added to commit but untracked files present (use "git add" to track)

In the above sample output, I’m working on a new Drupal module for a web site. I’ve created a new directory, sites/default/modules/cartoon_contests/ (note the trailing slash), and I have several untracked files. One is my vim swap file for a different module, .factiva.module.swp, one’s an unimportant testfile, and the last is the Drupal configuration file, settings.php.

The only thing I want to commit is the new cartoon_contests directory, and all the files within it. Like Subversion, I have to tell git that I want to track this directory, which is done simply by saying git add cartoon_contests. Unlike Subversion, future invocations of git status let me see everything that git is going to go in the next commit.

git status
# On branch cartoon_contests
# Changes to be committed:
#   (use "git reset HEAD ..." to unstage)
#
#	new file:   sites/default/modules/cartoon_contests/cartoon_contests.info
#	new file:   sites/default/modules/cartoon_contests/cartoon_contests.module
#
# Untracked files:
#   (use "git add ..." to include in what will be committed)
#
#	sites/default/modules/factiva/.factiva.module.swp
#	sites/default/modules/testfile
#	sites/default/settings.php

The “Changes to be committed” section of the output is called the staging area, or the index. In this way, I can prepare all the changes I want to commit before I do so, making sure they’re perfect before I actually commit them to the git repository. At any time before I commit, I can make additional modifications, such as git adding more files or directories, git reseting to unstage all (or some) of my changes, etc. git help reset also has a number of handy explanations with examples for different things you might need to do at this point.

If you were using svn:ignore, equivalent functionality exists in git. Simply append file glob patterns, one per line, to the $GIT_DIR/info/exclude file in your git repository. Like so:

echo -e "*.swp\nsites/default/settings.php" >> .git/info/exclude ; git status
# On branch cartoon_contests
# Changes to be committed:
#   (use "git reset HEAD ..." to unstage)
#
#	new file:   sites/default/modules/cartoon_contests/cartoon_contests.info
#	new file:   sites/default/modules/cartoon_contests/cartoon_contests.module
#

If you’ve got many of these, you can use git svn show-ignore >> .git/info/exclude to search through your old Subversion repository and look for any and all ignores, automatically adding them to git’s exclude list. (Checkout Tsuna’s blog entry on learning git for more tips like this.)

Finally, after you’ve done some of your work and you’ve finished staging your changes, you’re ready to commit them to the repository. On the other hand, if you hate what you’ve done and want to undo it all, you can say git reset --hard HEAD to throw away all your local changes. To throw away changes to just a single file, just checkout that file again by saying git checkout filename. This is the equivalent of Subversions svn revert filename command.

Before you actually make your first commit, however, you should properly introduce yourself to git. You don’t have to do this because git will try to figure out who you are by itself (details explained in detail here), but you probably should at least create global defaults for your user (which git will store in ~/.gitconfig). If you want to, you can also create per-repository defaults (which git will store in .git/config), or even system-wide defaults for all users of this computer (which git will store in /etc/gitconfig). To do so, say this:

git config --global user.name "Your Full Name"
git config --global user.email "you@example.com"

This will create the file ~/.gitconfig if it doesn’t alreay exist and will write your name into it. You can also just edit the file directly yourself instead of using git config commands.

Like Subversion, committing will create a saved, fixed point-in-time that marks the changes you have made to your files. Like branches, commits are also very cheap in git, so go ahead and commit at any time you like. Remember to stage your files (by git adding them), and then just git commit.

git add filepattern
git commit -m "My very frst git commit!"
Created commit ef483c1: My very frst git commit!
 5 files changed, 58 insertions(+), 12 deletions(-)

If you forget to specify a commit log message (-m "log message") on the command line, or if you want to enter a multi-line commit log, git will prompt you for it in your favorite $EDITOR. You can view a history, including all the log messages, for your project with git log. You can even the view logs in a number of pretty formats. Check git help log for more information.

If you want to change anything about the commit you just made, such as the author, you can just run git commit again with the --amend flag added to the command. Notice the typo in the commit log? Fixing it is really easy:

git commit --amend -m "My very first git commit!"
Created commit 88602f6: My very first git commit!
 5 files changed, 58 insertions(+), 12 deletions(-)

Finally, with your new code commited to your local git branch, it’s time to share that code with your colleagues who are (for some inexplicable reason) still using Subversion. This is also extremely simple. Just say git svn dcommit. (That’s right, dcommit, not just commit. I don’t know why….)

Not sharing your changes via Subversion, but with a patch instead? git diff -p will generate patches for you. See git help diff-files and look for the “GENERATING PATCHES WITH -P” section.

If you found this helpful, you may also enjoy an alternative tour for beginners from Carl Worth.