Everything In Between

Nary a day goes by when I don’t use my computer for some extremely personal stuff. I would consider it a Very Bad Thing if some of this information (my bank account details or private SSH keys, for instance) fell out of my control.

Everyone has sensitive files that they keep on their computer and, fortunately for Mac OS X Users, Apple has made it ridiculously easy to create a cryptographically secure containers for such files. You can think of a container like this, which is just a standard Mac OS X disk image (.dmg) file, like a vault that you open, put stuff you want to keep safe inside, and then close again.

Here’s how you go about making and using one.

Create the container, an encrypted disk image

1. First, open up your copy of Disk Utility.app, which is located in your computer’s /Applications/Utilities folder. (As an aside, this program is a bit like a swiss army knife for handling disk operations in Mac OS X. You should definitely find out what else it can do).
2. Next, select the File → New → Blank Disk Image… option. This will cause the New Blank Image window to appear.
3. Fill in the typical details such as the disk image file’s name and where you want to save it to. In addition, you’ll be presented with a number of options such as Volume Name, Volume Size, and Image Format. The defaults are usually adequate except for Volume Name, which you should customize so that when you mount the disk image the disk label is meaningful for you, and the Image Format, which I recommend you switch to “sparse disk image.”

   Sparse disk images can start small and grow automatically as you write more files into them. If what you want to keep secure in this manner are very large files, say gigantic high resolution PhotoShop documents, then you might consider the sparse bundle disk image format instead.

   Also, obviously, set the Encryption to a value other than “None.”

   Here’s an example screenshot from my Mac:

   

   Screenshot of the New Blank Image window showing meaningful values entered, Encryption field set to 128-bit, and Image Format field set to sparse disk image.

4. Press the “Create” button and you’ll be presented with a standard password selection dialogue. This is the password you’ll use to mount the disk image and is analogous to the idea of setting the combination on your vault’s lock. It’s critical that the password you choose is a good one. Ideally, your password is a totally random string that may include any printable character. Since that’s hard to remember, you can have the Mac OS X keychain manage your passwords for you.

Encrypt some files by writing them to the disk image

Now that you have an encrypted disk image, a secure container for your sensitive data, you can make use of it just as you might any other disk image on Mac OS X. For instance, say I have a top secret file called “My Killer Business Plan.pages” and I don’t want anyone to get at it. All I need to do is copy the file into my encrypted disk image, as the following screenshot shows:

It should go without saying that you want to delete the original, unencrypted copy of the file you’re copying into the encrypted disk image, but I’ll say that anyway. Don’t leave unprotected copies of your files lying around. Also, be certain to unmount (eject) the disk image when you’re done using it because the only thing the password protects is opening the disk image, not the files contained within it.

External references

Here are some additional places where this technique is discussed. Check out these additional articles about this topic elsewhere for more information and other perspectives:

• Mac OS X: How to create a password-protected (encrypted) disk image
• Mac OS X: About Encrypted Disk Images
• MacOSXHints.com: Create an encrypted disk image that grows as required

When you’re administering a web site, sometimes you need to make changes that for whatever reasons require that the web site be temporarily unavailable for normal visitors. One obvious example is database maintenance. Unless you have the resources to do full-blown load balancing across a server cluster, you probably have to accept that your site is going to be down for a short period of time.

When this happens, it’s generally a good idea to show your visitors a web page that briefly explains the situation. This is typically a page that politely explains that the “site is temporarily down for maintenance” and so on. I’ve taken to calling such a page a “curtain” because it’s a little like putting a curtain up in front of construction work.

You put the curtain up, do whatever you need to do to fix or upgrade or maintain your web site in the background, then take the curtain down. For delicate servers, this has the added benefit of dramatically reducing server load while you do your maintenance tasks. You can even allow access to specific visitors, such as QA testers or remote admins while this curtain is up, while still redirecting normal users to the “down for maintenance” page.

Obviously, the first thing you need is the page that explains your site is down for maintenance reasons. This can be anything you like, but it’s simplest to make it a static HTML page and place any and all resources you need for this page (like images) into the same folder. I often use a directory named down-for-maintenance and I place an index.html file in that folder to use as my “curtain” page. Images go straight into the down-for-maintenance directory, too.

Once you have that, you can then use the following Apache configurations to create an “on/off switch” for putting your curtain up and taking it down.

# To take the web site into a maintenance mode, create a file named
# maintenance-mode-on at the document (website) root, such as this:
#
#     touch maintenance-mode-on
#
# To bring the web site back, remove or rename the file, such as this:
#
#    mv maintenance-mode-on maintenance-mode-off
#
# To enable the site for your IP address only but nobody else's
# uncomment the second RewriteCond directive.
<IfModule mod_rewrite.c>
    RewriteEngine On
    RewriteCond %{DOCUMENT_ROOT}/maintenance-mode-on -f
    #RewriteCond %{REMOTE_ADDR} !^your.IP.address.here$
    RewriteRule !^down-for-maintenance/.*$ /down-for-maintenance/ [R,L]
</IfModule>

What this does, step by step, is:

1. Determines whether or not you have mod_rewrite enabled. If you don’t, then nothing happens.
2. Enables the mod_rewrite rewriting engine.
3. Checks for the existence of a file called maintenance-mode-on at your site’s root. If such a file does not exist, nothing happens.
4. With the second RewriteCond directive uncommented, it also checks the visitor’s IP address and if it does match the one listed nothing special happens. If it does not match the one listed, the next line, which is the redirect, is executed.
5. Checks the requested URI and if it does not begin with down-for-maintenance, a temporary redirect (HTTP status code 302) is issued that points browsers to the down-for-maintenance directory you created earlier. Obviously, if you named this directory something else, you should change this RewriteRule.

This isn’t perfect. For example, if a visitor is filling out a multi-page form then they might get interrupted half way through when you enable the curtain since the curtain takes effect starting at the next HTTP request after you enable it. That said, the only way to do truly graceful maintenance with zero downtime is load balancing, and that is beyond the capability of most simple sites, but this curtain is extremely simple and extremely effective.

In a lot of places in the world, many people still have to pay for bandwidth costs. I’m one of those people who just can’t afford to download lots of stuff during peak hours when my bandwidth might quickly get shaped or, worse, I’ll get charged. Nevertheless, there are often plenty of legit reasons to initiate huge downloads.

In these cases, it makes sense to be smart about when I initiate these downloads. Being something of a UNIX-head myself, I wanted to use the age-old at command to download a Linux ISO during off-peak hours, which my ISP says starts at 2 AM. Much to my chagrin, I found that at doesn’t work by default on Mac OS X and, worse, the Leopard man page leads to a dead end (though it didn’t back in Tiger…).

Turns out that the system daemon that is responsible for checking up on at jobs has been wrapped with a launchd job. This makes enabling at on your system really easy:

sudo launchctl load -w /System/Library/LaunchDaemons/com.apple.atrun.plist

Once you’ve done this, you can now use at as you normally have done. For instance, I could now schedule my downloads to happen during the off-peak hours:

Perseus:Fedora maymay$ at 2:15am tomorrow # now press return
curl -LO http://download.fedoraproject.org/pub/fedora/linux/releases/9/Fedora/x86_64/iso/Fedora-9-x86_64-DVD.iso
# now press CTRL-D.
job 1 at Tue Jul 15 02:15:00 2008
Perseus:Fedora maymay$ atq
1	Tue Jul 15 02:15:00 2008

This is also incredibly handy for scheduling just about any resource-intensive task that you don’t have to do right now. To take it one step further, you can even let the computer itself choose when to run these resource-heavy tasks by using the batch command, which will execute commands much like at but will check the system load average instead of the system clock to determine if it should start the job.

Note that with the com.apple.atrun job loaded /usr/libexec/atrun is started every 30 seconds (unless you change the StartInterval key in the plist file). Since the atrun command checks a file on disk (that it places in the /usr/lib/cron/jobs directory) to see if there is any work to do, this will probably prevent your disks from ever sleeping, which could be a major concern for battery life on portables. Also, obviously, your computer needs to be turned on and awake for the job to actually launch.

For more information, check out the result of typing man at and man launchctl at a Terminal prompt. There’s also a really good Google Tech Talk about Launchd that will teach you a lot more about job scheduling on Mac OS X.

Without doubt, the most common use of computers today is to create written content of some kind. Blogs are an obvious example, but written content can take a number of forms. Writing manuscripts for publication is another example.

No matter what kind of writing you’re doing, using good tools to make your writing technically better is an incredibly handy thing. Letting the computers do the technical stuff—the stuff they’re good at—let’s you focus on the creative stuff: writing great content. Which is why, if you use a Mac, you’ll be happy to hear that any application’s text field let’s you do a number of really cool things (as long as it’s a Cocoa application, of course).

1. Auto-complete unfinished words

Try this out:

1. Open TextEdit, from your /Applications directory. A new blank document will open.
2. Type Hel and then press the ESC key. A drop-down menu will suddenly appear with an alphabetically sorted auto-complete list of suggestions, sourced from your computer’s current language dictionary. It looks like this:

This feature works with both Pages and, for those of you still using it for some reason (I know you’re out there), TextEdit, too. Also, if you’re a developer and a writer as well (like I am), you’ll be happy to hear that this feature also works with Xcode’s Code Sense feature, and suggests completions for variable, function, class, and method names in your code.

2. Spellcheck as you type

A Cocoa text input field also has a number of other tricks up its sleeve. For instance, in many applications you can elect to turn on the “Check spelling as you type” feature, which will cause words you misspell (words not in the computer’s dictionary) to appear with a dotted red underline. If you right-click on these words, the contextual menu that appears will offer spelling corrections.

However, sometimes we use words like those in slang or colloquial language that isn’t in a proper dictionary. These words will still appear to be “misspelled” when you type, so in these cases, we can tell Mac OS X to “Learn Spelling” (also from the contextual menu). When you select this option, you append that spelling to your personal dictionary. (This is really just a plain-text file located at ~/Library/Spelling/lang file, where lang is the language code you’re typing in. For Enlgish, this file is ~/Library/Spelling/en.)

3. Look up word definitions and search for text in Google or Spotlight

Last, but certainly not least, another neat thing you can do with text on your Mac is look them up with Dictionary.app. Simply highlight some selectable text on screen, right-click and select “Look up in Dictionary”. This will cause Dictionary.app to open and display the definition of the selected word.

Since Dictionary.app can also look up articles in Wikipedia, this is also a very quick way to go to a Wikipedia article without ever having to open up a Web browser.

Also, from the very same menu, you can open a Web browser. Simply select “Search in Google” to cause your default Web browser to launch a Google search for the highlighted text.

If you’re a Mac user who often uses VPN connections, you’ll notice one very disappointing thing about connecting to your corporate or personal network over such tunneled connections: typically, Bonjour-style addresses (such as “computer-name.local”) don’t work. This is because multicast DNS (or mDNS) doesn’t work over a tunnel. Though there are ways to get it functional, they are pretty complicated and require that you have a lot of esoteric networking knowledge.

However, if the services you typically access via Bonjour use static IP addresses, then there is one age-old networking technique you can use to simulate Bonjour-style naming conventions without actually using Bonjour. This, of course, is the /etc/hosts file.

The /etc/hosts is a simple, static, text-based mapping of computer names to IP addresses. It does exactly what Bonjour does except it doesn’t keep itself up to date when things change. Of course, if you’re using static IPs for the services you want access to, you can pretty safely assume that things aren’t going to be changing frequently anyway. Long-time sysadmins will laugh at this, but I say let them laugh. This is remarkably useful and very easy to implement.

Let’s assume I’m running a personal web server on my home network, and I can access my home network via a VPN. On my home network, my web server’s IP address is, say, 192.168.2.100, and I usually access it as http://server.local/. All I need to do is open a Terminal prompt and run the following commands as an administrative user:

sudo echo "192.168.2.100	server.local" >> /etc/hosts

That’s it. What this does is hard-wire the name server.local so that it always resolves to the IP address 192.168.2.100. Now, anytime anything on my computer tries to access server.local, it’ll always access 192.168.2.100 directly instead of ever needing to make an mDNS query on the network. The net effect is that we can trick our computer into thinking that Bonjour is working, even when it’s not—such as over a VPN connection.

Note that in default cases, hard-wiring an IP address like this completely prevents your computer from ever asking other computers (such as DNS servers) what the current IP address for this name is. That means if the IP address of the remote server changes, you won’t be notified, and things will just not work. So be mindful that you’ve made this change, and revert it as a first step in troubleshooting procedures.

By the way, Windows users can do the very same thing simply by editing their etc/hosts. They can find this file at C:\WINDOWS\system32\drivers\etc\hosts and can edit it with Notepad. They will also need to install Bonjour for Windows to get Bonjour working in the first place, of course.

I can’t explain why this happened because in my several-year-long history with Subversion, I’ve never experienced this issue once. However, today, I fell into the (arguably) unfortunate circumstance of running into a most disturbing error from SVN. When trying to commit my changes, SVN barfed at me and complained of a “checksum mismatch”. It looked something like this:

Transmitting file data ..svn: Commit failed (details follow):
svn: Checksum mismatch for '/Users/maymay/Sites/path/to/subversion/working/copy/.svn/text-base/working-file.php.svn-base'; expected 'cde4d8fbd5c623da3a8b1a343aa7b3f4', actual: '270b2f20804a5fcdbd15eec5910f6e3f'

Of course, the path/to/subversion/working/copy bit was the path to my working copy file’s parent directory and the working-file.php was an actual file in my working directory.

I think what Subversion was trying to tell me is that its hashed copy of the working-file.php file and the copy I was asking it to commit weren’t the same. It would be nice if it would actually tell me why that happened, but it’s clearly more temperamental than that.

Anyway, to fix this issue (at least for now…?) I simply checked out a new working copy of this directory, examined the .svn/entries file from it and sure enough, found the actual checksums in there, just as Subversion reported expecting. I simply copied those expected checksums into the .svn/entries overwriting the old actual checksums and, voila, Subversion has been fooled. After that, I could commit my changes.

Step by step (because I’m sure someone, somewhere, somehow, will run into this again—if it’s not me that is!), this procedure looked like this:

1. Copy the “expected” and “actual” checksums Subversion reports to you to a new text file so you can refer to them later. Note which one is the expected and which is the actual checksum.
2. Go to where the problem is (that is, cd path/to/broken-files-parent-dir/.svn)
3. Open the entries for editing (for example, vim entries)
4. Search the file for the actual checksum.
5. Replace it with the expected checksum. Be careful not to change any other part of the file.
6. Save the file.
7. Try to svn commit again.
8. Lather, rinse, and repeat for any other files Subversion barfs at you about.

I’m sure this is not an elegant or even the recommended solution to this problem. The truth is I never bothered to look up what the recommended solution is, because it seems to me that any code repository that can’t guarantee what I get out of it is the same as what I put into it isn’t a versioning system I really want to trust the “recommended” solution of, anyway.

Also known as: this is another reason why I like git better now.

When hacking away at WordPress sites, often times you’ll find yourself in a situation where you need to filter out certain posts from displaying on some pages, such as the home page. There are a lot of ways to do this, but few are perfect. Recently, I had the need to do this and went searching for pre-existing solutions.

I came across Vaibhav’s post on the topic and noted that his solution uses the query_posts() function to alter WordPress’s query object before The Loop has run. While this is a great solution if your exclusion criteria is simple enough to be supported directly by the WordPress query object, other times the query_posts() function doesn’t provide you with the hook you need.

In these cases, you can run the original query, note any modifications you need to make, and then create a new, modified query and display the results you get from running that one instead. For instance, you might need to do this if you need to exclude posts based on category and, say, the beginning of their title, or their category and a certain piece of content in the post itself, or all three, or any other combination you can think of.

Another advantage of this technique over simpler ones is that this method maintains the same behavior you’d expect to see in every other way. Most notably, this means that if you’ve told WordPress to display the 10 most recent posts on the home page (in the WordPress settings), you’ll still see ten posts on that page even after you exclude some of them.

To do something like excluding posts if they are in the “Uncategorized” category (traditionally the category with an ID of 1 in WordPress) and their title begins with “Some title”, you can do this:

// original query runs in The (real) Loop first
while ( have_posts() ) : the_post();
    // detect pots matching our exclusion criteria
    if (in_category(1) && (0 === strpos(the_title('', '', false), 'Some title')) ) {
        $wp_query->post_count++; // increment the post counter
        continue;
    }
    endif;
endwhile;
// now make a new query and show the posts for real, with the adjusted post count and filtering
$my_new_query = new WP_Query($query_string.'&showposts='.$wp_query->post_count);
// do another The Loop (and display the results this time)
while ( $my_new_query->have_posts() ) : $my_new_query->the_post();
    // detect and exclude these same posts
    if (in_category(1) && (0 === strpos(the_title('', '', false), 'Some title')) ) { continue; }

// ...the rest of the WordPress template goes here...

This is neat because it gives you the capability to define arbitrarily complex exclusion patterns and directly modify your new query object however you like before you execute it. Once you know this works, you’ll probably want to extract the filtering code into a function. Using the above example, your new code might look like this:

// define criteria for filtering
function matches_filtering_criteria () {
    if (in_category(1) && (0 === strpos(the_title('', '', false), 'Some title')) ) {
        return true;
    } else {
        return false;
    }
}
// original query runs in The (real) Loop first
while ( have_posts() ) : the_post();
    // detect pots matching our exclusion criteria
    if (matches_filtering_criteria()) {
        $wp_query->post_count++; // increment the post counter
        continue;
    }
    endif;
endwhile;
// now make a new query and show the posts for real, with the adjusted post count and filtering
$my_new_query = new WP_Query($query_string.'&showposts='.$wp_query->post_count);
// do another The Loop (and display the results this time)
while ( $my_new_query->have_posts() ) : $my_new_query->the_post();
    //
    // detect and exclude these same posts
    if (matches_filtering_criteria()) { continue; }

// ...the rest of the WordPress template goes here...

For more information on these functions, see:

• The Loop
• in_category()
• the_title()
• WP_Query()

Using the pbpaste and pbcopy commands, you can manipulate the contents of the Mac OS X clipboard (or more formally known as the pasteboard) right from the command line. As a brief example, just select the text of this first paragraph, copy it to your clipboard (with -c), and then type pbpaste in a Terminal prompt. You should see output similar to the following:

Perseus:~ meitar$ pbpaste
Using the pbpaste and pbcopy commands, you can manipulate the contents of the Mac OS X clipboard (or more formally known as the pasteboard) right from the command line. As a brief example, just select the text of this first paragraph, copy it to your clipboard (with ?-c), and then type pbpaste in a Terminal prompt. You should see output similar to the following:Perseus:~ meitar$ 

Pretty straightforward, right? The only thing to be aware of is that the  symbol showed up in the output as a ? symbol. This is because the Terminal doesn’t support Unicode, but that’s a topic for another time.

Anyway, what’s happening here is nothing more magical than simply reading the clipboard and pasting it into a command’s standard output stream. As a result, you can construct pipelines that read from or add content to the clipboard. Here’s an example in reverse, which takes a command’s standard output and replaces the contents of the clipboard with it:

echo "Hello world! I came from the command line, but now I'm in the clipboard." | pbcopy

This command produces no output, but if we examine the contents of the clipboard (by selecting Edit → Show Clipboard from the Finder’s menu bar) we can see that the text we echoed has indeed been copied there.

Another way we can verify that this worked as expected is to simply pbpaste again:

Perseus:~ meitar$ pbpaste
Hello world! I came from the command line, but now I'm in the clipboard.
Perseus:~ meitar$ 

Anyway, this is cool, but it isn’t very useful yet. For that, we need to add some more stages to our pipeline. Let’s take the simple case of trying to count how many words the selected text contains. We’ll use the clipboard contents shown above to do this:

Perseus:~ meitar$ pbpaste | wc -w
      14

Using the word count (wc) utility, we can count words (-w) very easily. Indeed, our previous example does have exactly 14 words in it. We can also count characters (-c) or lines (-l) of text in the clipboard this way. This is like adding Microsoft Word’s “Word Count” feature to every single piece of text you can copy!

As another example take, for instance, the simple case of copying and pasting a snippet of email from Mail. Instead of pasting it back into a text file verbatim, let’s prepend ‘> ‘ to the beginning of each line. This way, when we paste our email’s snippet, we’ll know where the snippet begins and where it ends. This is a simple three-stage pipeline that uses pbpaste to take our clipboard and put it into the pipeline and then reads back the result from the pipeline back to the clipboard using pbcopy. In the middle, we use sed to insert the desired text at the start of each line:

pbpaste | sed -e 's/^/> /' | pbcopy

Now, when you paste your clipboard, you’ll have a greater-than symbol at the start of each line. Naturally, check out the manual pages for all of these commands for more detailed information. For instance, type man pbpaste for more information about the pbpaste command.

Since Mac OS X 10.2 Jaguar, Mac users have been accustomed to the ease of use of Apple’s very cool Keychain Services technology. The Mac OS X Keychain basically a secure database of all your passwords, sorted into files called (unsurprisingly enough) “keychains.” Each user account on a Mac OS X system has a login.keychain, and the system itself also has a system.keychain.

Whenever you tell an application to “Remember this password in my keychain,” what you’re doing is writing a new encrypted entry into your user account’s ~/Library/Keychains/login.keychain file. Then, the next time the application needs to access a restricted resource, it just asks Mac OS X to get the password for it. Of course, all of this happens automatically, so except for that single checkbox most users probably don’t know that the keychain even exists.

What’s even more awesome than all of this automagic password storing action, though, is the fact that Apple has also provided an easy-to-use application to manipulate the keychain yourself. What good does this do us? Plenty! Observe.

Say you’ve just signed up with a new ISP. They send you a username and a password to log on to their ADSL network with. Of course, they send this password to you on paper—how insecure! Instead, after changing the password to something else first (something other than mypassword, which is the example password I’ll use here), we can use Mac OS X Keychain to securely store the password and retrieve it later.

1. First, launch the Keychain Access application located in the /Applications/Utilities folder of your startup drive.
2. Next, click the “Create a new Keychain item” button (the +) button near the lower left-hand corner of the window. The Add Keychain Item sheet appears.
3. Enter a meaningful name, such as “ADSL ISP Account” in my example, in the Keychain Item Name field.
4. Enter the username or account name associated with this password in the Account Name field.
5. Enter the password into the Password field.
6. Click the Add button.

That’s all there is to it. To later retrieve your password if, say, you ever forget it:

1. Launch the Keychain Access application.
2. Locate and double-click the keychain item that stores the account and password information you want to retrieve.
3. Tick the “Show password” checkbox. You’ll be presented with a dialogue box that asks for your keychain’s master password. Unless you’ve already set it to something else, this is the same password you use to log in to your Mac OS X user account.
4. Enter your keychain password and click “Allow.” If you click “Always Allow” instead, Keychain Access will not prompt you for your login keychain’s password the next time you ask to see this particular password. I never press that button.
5. Your password’s plaintext is now visible.

This effectively obviates the need for third-party applications such as Password Gorilla, PasswordWallet or KeePassX which are great programs, but all suffer from a lack of a good user interface. Furthermore, there’s no reason why we can’t store short arbitrary strings of sensitive information in the keychain temporarily. Sure, it might clutter up your keychain, but you can always search the entries using the standard Mac OS X filter search bar at the top right of the window.

In fact, Apple’s been kind enough to offer an interface to do just that in an even more effective way, called Secure Notes. These are simply plain text strings of arbitrary length that can be stored securely inside your keychain, and that use the same interface to access (requiring your password to view). The only real difference is that instead of a single line, you’re given a fully scrollable text area in which to type your secure note.

Moreover, because keychains can be synced to multiple Macs with .Mac Sync (or a third-party synchronization solution), you can always have access to all your passwords regardless of which physical Mac you’re using. Best of all, since you never have to remember another password ever again, you can quit using the same password for multiple accounts, and you can always use really hard-to-crack passwords.

As I’ve been blogging about, I’ve been playing a lot with Apple’s new WikiServer (or “Teams Server”) at work. We’re still evaluating what we’d like to use it for, but as part of the experiments, I’ve been finding myself having to do some pretty crazy things with the WikiServer. This one is pretty bizarre, and is probably not only very dangerous for the content of your group’s wiki and blog, but also almost certainly not a best-practice.

With that caveat out of the way, here’s how I managed to move all the pages in an Apple WikiServer group wiki and blog from one group to another.

Dear God, please have a backup!

Okay, step 1 is mundane, but seriously, please of please have a backup of your data before you do any of these things. To make a back up of your entire WikiServer’s data store, simply:

sudo tar -cvzf backup-file-name.tgz /Library/Collaboration

If anything goes wrong, you can restore from your backup just as simply:

sudo tar -C /Library/Collaboration -xvzf backup-file-name.tgz

Okay, with that out of the way, next make sure absolutely nobody is using any of the Group services for the group you are going to perform the move from, or to. People can work on other group’s wikis, it’s just the ones you’ll be touching you want people to avoid. You can enforce this with some Apache redirects, which is left as an exercise to the reader.

Step 1: Rename or create a new Group in Workgroup Manager

First, you need to either rename or create a new group in Workgroup Manager. If you make a new group, be sure to enable all the services that the old group used.

If you’re simply renaming a group, then you might not even have to go through this trouble. You merely need to change the group’s “Name” (as opposed to its “Short Name”) and then stop and start the Web Service from Server Admin to see the group’s name change. However, if you also want to change the group’s “Short Name” (i.e., the group’s POSIX group account symbolic name), then you will need to perform these steps.

Once you have the new group ready to go in Workgroup Manager and you have stopped and started your Web Service in Server Admin, continue to the next step.

Step 2: rsync all your files from the old group to the new group

This is simple. Just run:

sudo rsync -avzE --progress /Library/Collaboration/Groups/old-group/ /Library/Collaboration/Groups/new-group/

Note that the trailing slashes on the directory names in this command are quite important, as they tell rsync to take the contents of the first directory and place those items as the contents of the second directory. Without the trailing slashes, rsync will make extraneous directories for you, which Apple WikiServer won’t understand. See man rsync for more information.

Step 3: Update the plists for all your WikiServer pages in the new group

Now, WikiServer has all the content of your old group but all of the internal references are wrong, since they still point to the old group. What you need to do is rewrite all those references so that they point to the new group address. The group references are stored in property list files. See my older blog entries for details about the filesystem structure of Apple’s WikiServer data storage layout.

To do this is a relatively simple procedure. As root, do the following, and do it carefully because as root you can easily mess up (and there is no undo button on the command line!):

sudo su -
cd /Library/Collaboration/Groups/new-group
grep -ri "old-group" * | grep '^[^B]' | cut -d ':' -f 1 | grep 'plist$' | sort | uniq > /tmp/list_of_plist_files_to_edit
for i in `cat /tmp/list_of_plist_files_to_edit`; do sed -e 's/<string>groups\/old-group/<string>groups\/new-group/' $i > $i.new; mv "$i.new" "$i"; chown teamsserver:teamsserver "$i"; chmod o-rwx "$i" done;

In English, this means:

1. Become root.
2. Change to the /Library/Collaboration/Groups/new-group directory
3. Find all plist files that have the the string old-group in them, sort them, and write this list of files to the /tmp/list_of_plist_files_to_edit file.
4. For each of the files listed in the /tmp/list_of_plist_files_to_edit, find the text string <string>groups/old-group and replace that text with <string>groups/new-group, and save this change as the name of the file with .new appended. Finally, replace the original file with the file we modified, and give them the appropriate permissions.

Step 4: Create an Apache redirect to make sure no HTML links are broken

Okay, at this point your new group is up and running and it should be working. However, if you had any links at all in any of your group’s pages, they are now all broken because they still point to the old group. Rather than going through the HTML itself and cleaning this up right now (because that’s very error-prone indeed, even with automated tools), it’s much easier to just tell Apache to redirect all requests for the old group to the new group.

To do this, edit the Apache configuration file of whatever Virtual Host you have been serving the WikiServer from. Most of the time, this will be at /etc/httpd/sites/0000_any_80_.conf.

The end of that file probably looks something like this::

#       Include /etc/httpd/httpd_users.conf
#       Include /etc/httpd/httpd_directory.conf
        Include /etc/httpd/httpd_groups.conf
        Include /etc/httpd/httpd_teams_required.conf
        LogLevel warn
        ServerAlias *
</VirtualHost>

Right above these Include directives, simply add the following:

        <Location /groups>
            <IfModule mod_alias.c>
                Redirect 301 /groups/old-group http://your.server.address/groups/new-group
            </IfModule>
        </Location>
#       Include /etc/httpd/httpd_users.conf
#       Include /etc/httpd/httpd_directory.conf
        Include /etc/httpd/httpd_groups.conf
        Include /etc/httpd/httpd_teams_required.conf
        LogLevel warn
        ServerAlias *
</VirtualHost>

There you have it. New Teams Server group, old group’s data.

Caveats

Note that this does not update any of the SQLite databases used to store things like revision history and so forth. These things are, for the most part, not really necessary to update but it would be ideal if the old plist revisions could be changed in there, too. That’s not so much more extra work, really, but I’ve found it typically unnecessary except in fringe cases, so I leave that as an exercise to the reader.

If you do this to your WikiServer and do not update the SQLite databases as well, just be mindful of that fact so that you’re not surprised if something goes wonky down the line.