Everything In Between

Last time, we checked out the Apple WikiServer from the user’s side of things. We learned about the code it generates, how it handles page name changes, and what the key filesystem locations the Apple WikiServer looks at are. This time, let’s delve a little deeper into the WikiServer’s internals by (safely) messing around with a running instance and seeing how we can effect changes that aren’t available in the GUI.

I’m going in!

Apple’s WikiServer keeps each group’s data segregated. There’s currently no way in the GUI to search across multiple wikis from a single wiki, or move one wiki page in one wiki to another wiki. Luckily, because the Apple WikiServer is a rather lightweight application, it’s not that hard to move things around. Doing so doesn’t seem to cause any problems with the application, either—a testament to its apparent simplicity.

Most simply, say you have created a number of different wikis for different groups, and in so doing you’ve created segregated spaces for different parts of your organization. Now say you’ve got a wiki page in one wiki that you’d rather place in a different one, or that you’d like to copy. Sure, you can edit the first wiki’s page, copy the WYSIWYG editor’s text, and paste it into a new page on the second wiki. But where’s the fun in that?

Instead, let’s move some files around to get a sense for how the Apple WikiServer stores its data.

First, recall that the Apple WikiServer stores its content inside the /Library/Collaboration directory on the computer hosting the wiki web site. Inside of this directory you’ll find four items:

$ cd /Library/Collaboration/
$ ls -la
total 8
drwxrwxr-x   6 _teamsserver  _teamsserver   204 Mar  2 13:50 .
drwxrwxr-t@ 60 root          admin         2040 Mar  3 19:16 ..
drwxr-xr-x   2 _teamsserver  _teamsserver    68 Mar  2 13:50 ArchivedGroups
drwxr-xr-x   6 _teamsserver  _teamsserver   204 Apr  4 18:48 Groups
drwxr-xr-x   2 _teamsserver  _teamsserver    68 Mar  2 13:50 Users
-rw-r--r--   1 _teamsserver  _teamsserver   249 Mar  2 13:50 dataVersion.plist

First, note that all of these files are owned solely by the _teamserver user, and only that user can access these files in any way. This keeps people like you and me from accidentally tampering with the Apple WikiServer data store. However, for the purposes of this exercise, we’re going to tamper anyway. To do so, you’ll need root permissions, which comes with a big caveat: be careful whenever you are root—one typo could erase your hard drive (and there is no undo)!

Become root by saying:

sudo su -

and supplying your password. Once you are root, you can explore further into the depths of the Apple WikiServer.

Inside the Apple WikiServer data store for wiki pages

The Apple WikiServer, which runs under the username _teamserver, is really just a bunch of organized flat files, supplemented with a number of SQLite databases (also flat files, really), and glued together with a bunch of Python scripts running under the Twisted networking framework. We’re not going to cover Python or Twisted in this post, so for further information check out their respective homepages. There’s also a great introduction to Twisted at ONLAMP.com.

To make the wiki go, the WikiServer runs these scripts to read and write data to these flat files. Let’s dig into this data store. Since the wikis are each associated with an existing group, all of the wiki files are inside the Groups subdirectory. In this directory, you’ll find one directory for each group that has a wiki. Each group directory, in turn, contains 6 other directories, 2 regular files, and 1 SQLite database:

# pwd
/Library/Collaboration/Groups/chiefs
# ls -p
discussion/	index.db	metadata.plist	resources/	wiki/
extrainfo	mailinglist/	public/		weblog/

The two regular files are metadata.plist and extrainfo. The metadata.plist file maintains the preferences of the individual wiki web site that you set from the “Group Settings” page on the wiki web site itself. That is, these settings are all editable via the web directly at http://your-server.local/groups/your-group-name/settings/. That page is also accessible to an admin user from the wiki home page under the “Admin functions” header in the sidebar of the web page (the key names in the plist file are pretty self-explanatory and map easily to the GUI options).

At the moment, the extrainfo file is a bit of mystery. It stores single lines of plain ASCII text in the form:

groups/your-group-name/web-service/page-id

where your-group-name is the name of the associated group for the file, web-service is one of wiki, weblog, and I assume can also be mailinglist or discussion, although I’ve only seen the first two on my server in practice.

What we’re most interested in for the purposes of wiki pages is, unsurprisingly, the wiki directory. This directory stores as many subdirectories as there are pages on your wiki. Every time you create a new page, a new directory is added here. If you delete that page, however, this directory is not deleted outright.

Each directory is named with the unique page identifier that the Apple WikiServer automatically generates followed by a .page extension. The exception is the welcome.page directory, special in that it contains the content of the wiki’s main (front) page. The Mac OS X Server Web Technologies Administration for Version 10.5 Leopard manual explains the contents of this directory as follows:

• /Library/Collaboration/Groups/groupname/wiki/pagename.page/ contains the component files of a wiki page.
• /Library/Collaboration/Groups/groupname/wiki/pagename.page/page.html contains the main text of the wiki (html content).
• /Library/Collaboration/Groups/groupname/wiki/pagename.page/page.plist contains the metadata for the wiki page.
• /Library/Collaboration/Groups/groupname/wiki/pagename.page/revisions.db contains the version history database for that wiki page.
• /Library/Collaboration/Groups/groupname/pagename.page/images/ contains the images for that wiki page.
• /Library/Collaboration/Groups/groupname/pagename.page/attachments/ contains all attachments for that wiki page.

Here are some additional details:

• The “page.html” file contains all of the (X)HTML that appears inside the WYSIWYG editor field when you click the “Switch to HTML View” button (the arrow brackets). Nothing more, and nothing less.
• The “page.plist” file maintains the metadata that associates a single wiki page to its group as far as the Apple WikiServer is concerned. If you take a closer look at it, you’ll find property list keys such as commentUID and uid whose values are both strings—URLs, in fact—that the Apple WikiServer uses to, for example, call the appropriate wiki theme to render around the page content. This file also stores self-explanatory data such as author, createdDate, page kind, a last modifiedDate and lastModifiedAuthor and, of course, the page title. The most interesting key in this file is the versioned key, which is a boolean that can turn versioning on or off. By default, wiki pages have this key set to true and (for example) weblog pages have this set to false. However, you can manually turn off versioning for a specific wiki page by changing this key yourself. It won’t delete past revisions, it will just stop new revisions from being saved. This is especially cool, because it means you can version-control weblog pages simply by editing the page’s associated plist file.
• The revisions.db is the SQLite database in which all revisions are stored. The database contains a single table, revisions, that looks like this:

  CREATE TABLE revisions(revision INTEGER PRIMARY KEY AUTOINCREMENT, editType, comment, lastModifiedBy, content, time DEFAULT CURRENT_TIMESTAMP);

  The interesting bit is the content field, which itself is just a string—but a string stored as an Apple property list, in full. Some keys in the property list inside the SQLite database are the same as those inside the page.plist file for the page, but with some differences. For instance, there is no versioned key in the plist in the database (for obvious reasons), and there is a tags key (which is missing from the filesystem equivalent). Most importantly, there is a content key, which stores the content of the page.html file.

• The revisions.db file is created automatically only if the versioned key in the page’s page.plist file is set to true.

Finally: move a wiki page from one wiki to another

Now that we understand some of the internals of the Apple WikiServer data store, let’s mess with it just a little by moving that one test wiki page from one wiki into another. Since almost everything is flat files, it’s remarkably easy to do:

# mv /Library/Collaboration/Groups/your-group-name/wiki/page-id.page /Library/Collaboration/Groups/new-group-name/

Be certain that there isn’t already an existing page in the new-group-name wiki with the same page-id as the one you’re moving. So far, Apple WikiServer appears to generate unique page IDs across the entire system (as opposed to just per-group), but I haven’t verified this yet.

With the wiki page files physically moved into the new wiki, you can now access the page at the new wiki URL. Trying to access it in the old URL will result in an HTTP 404 “Not Found” error. However, the page at the new wiki URL has the same theme as the other, old wiki. To correct this, you need to edit the page.plist file in two places: the uid string value and the commentUID string value relative URLs need their your-group-name changed to the new-group-name, whatever it is.

After you save this this change, simply reload the wiki page in your browser. Voila. Page moved.