Everything In Between

Using the pbpaste and pbcopy commands, you can manipulate the contents of the Mac OS X clipboard (or more formally known as the pasteboard) right from the command line. As a brief example, just select the text of this first paragraph, copy it to your clipboard (with -c), and then type pbpaste in a Terminal prompt. You should see output similar to the following:

Perseus:~ meitar$ pbpaste
Using the pbpaste and pbcopy commands, you can manipulate the contents of the Mac OS X clipboard (or more formally known as the pasteboard) right from the command line. As a brief example, just select the text of this first paragraph, copy it to your clipboard (with ?-c), and then type pbpaste in a Terminal prompt. You should see output similar to the following:Perseus:~ meitar$ 

Pretty straightforward, right? The only thing to be aware of is that the  symbol showed up in the output as a ? symbol. This is because the Terminal doesn’t support Unicode, but that’s a topic for another time.

Anyway, what’s happening here is nothing more magical than simply reading the clipboard and pasting it into a command’s standard output stream. As a result, you can construct pipelines that read from or add content to the clipboard. Here’s an example in reverse, which takes a command’s standard output and replaces the contents of the clipboard with it:

echo "Hello world! I came from the command line, but now I'm in the clipboard." | pbcopy

This command produces no output, but if we examine the contents of the clipboard (by selecting Edit → Show Clipboard from the Finder’s menu bar) we can see that the text we echoed has indeed been copied there.

Another way we can verify that this worked as expected is to simply pbpaste again:

Perseus:~ meitar$ pbpaste
Hello world! I came from the command line, but now I'm in the clipboard.
Perseus:~ meitar$ 

Anyway, this is cool, but it isn’t very useful yet. For that, we need to add some more stages to our pipeline. Let’s take the simple case of trying to count how many words the selected text contains. We’ll use the clipboard contents shown above to do this:

Perseus:~ meitar$ pbpaste | wc -w
      14

Using the word count (wc) utility, we can count words (-w) very easily. Indeed, our previous example does have exactly 14 words in it. We can also count characters (-c) or lines (-l) of text in the clipboard this way. This is like adding Microsoft Word’s “Word Count” feature to every single piece of text you can copy!

As another example take, for instance, the simple case of copying and pasting a snippet of email from Mail. Instead of pasting it back into a text file verbatim, let’s prepend ‘> ‘ to the beginning of each line. This way, when we paste our email’s snippet, we’ll know where the snippet begins and where it ends. This is a simple three-stage pipeline that uses pbpaste to take our clipboard and put it into the pipeline and then reads back the result from the pipeline back to the clipboard using pbcopy. In the middle, we use sed to insert the desired text at the start of each line:

pbpaste | sed -e 's/^/> /' | pbcopy

Now, when you paste your clipboard, you’ll have a greater-than symbol at the start of each line. Naturally, check out the manual pages for all of these commands for more detailed information. For instance, type man pbpaste for more information about the pbpaste command.

Since Mac OS X 10.2 Jaguar, Mac users have been accustomed to the ease of use of Apple’s very cool Keychain Services technology. The Mac OS X Keychain basically a secure database of all your passwords, sorted into files called (unsurprisingly enough) “keychains.” Each user account on a Mac OS X system has a login.keychain, and the system itself also has a system.keychain.

Whenever you tell an application to “Remember this password in my keychain,” what you’re doing is writing a new encrypted entry into your user account’s ~/Library/Keychains/login.keychain file. Then, the next time the application needs to access a restricted resource, it just asks Mac OS X to get the password for it. Of course, all of this happens automatically, so except for that single checkbox most users probably don’t know that the keychain even exists.

What’s even more awesome than all of this automagic password storing action, though, is the fact that Apple has also provided an easy-to-use application to manipulate the keychain yourself. What good does this do us? Plenty! Observe.

Say you’ve just signed up with a new ISP. They send you a username and a password to log on to their ADSL network with. Of course, they send this password to you on paper—how insecure! Instead, after changing the password to something else first (something other than mypassword, which is the example password I’ll use here), we can use Mac OS X Keychain to securely store the password and retrieve it later.

1. First, launch the Keychain Access application located in the /Applications/Utilities folder of your startup drive.
2. Next, click the “Create a new Keychain item” button (the +) button near the lower left-hand corner of the window. The Add Keychain Item sheet appears.
3. Enter a meaningful name, such as “ADSL ISP Account” in my example, in the Keychain Item Name field.
4. Enter the username or account name associated with this password in the Account Name field.
5. Enter the password into the Password field.
6. Click the Add button.

That’s all there is to it. To later retrieve your password if, say, you ever forget it:

1. Launch the Keychain Access application.
2. Locate and double-click the keychain item that stores the account and password information you want to retrieve.
3. Tick the “Show password” checkbox. You’ll be presented with a dialogue box that asks for your keychain’s master password. Unless you’ve already set it to something else, this is the same password you use to log in to your Mac OS X user account.
4. Enter your keychain password and click “Allow.” If you click “Always Allow” instead, Keychain Access will not prompt you for your login keychain’s password the next time you ask to see this particular password. I never press that button.
5. Your password’s plaintext is now visible.

This effectively obviates the need for third-party applications such as Password Gorilla, PasswordWallet or KeePassX which are great programs, but all suffer from a lack of a good user interface. Furthermore, there’s no reason why we can’t store short arbitrary strings of sensitive information in the keychain temporarily. Sure, it might clutter up your keychain, but you can always search the entries using the standard Mac OS X filter search bar at the top right of the window.

In fact, Apple’s been kind enough to offer an interface to do just that in an even more effective way, called Secure Notes. These are simply plain text strings of arbitrary length that can be stored securely inside your keychain, and that use the same interface to access (requiring your password to view). The only real difference is that instead of a single line, you’re given a fully scrollable text area in which to type your secure note.

Moreover, because keychains can be synced to multiple Macs with .Mac Sync (or a third-party synchronization solution), you can always have access to all your passwords regardless of which physical Mac you’re using. Best of all, since you never have to remember another password ever again, you can quit using the same password for multiple accounts, and you can always use really hard-to-crack passwords.

As I’ve been blogging about, I’ve been playing a lot with Apple’s new WikiServer (or “Teams Server”) at work. We’re still evaluating what we’d like to use it for, but as part of the experiments, I’ve been finding myself having to do some pretty crazy things with the WikiServer. This one is pretty bizarre, and is probably not only very dangerous for the content of your group’s wiki and blog, but also almost certainly not a best-practice.

With that caveat out of the way, here’s how I managed to move all the pages in an Apple WikiServer group wiki and blog from one group to another.

Dear God, please have a backup!

Okay, step 1 is mundane, but seriously, please of please have a backup of your data before you do any of these things. To make a back up of your entire WikiServer’s data store, simply:

sudo tar -cvzf backup-file-name.tgz /Library/Collaboration

If anything goes wrong, you can restore from your backup just as simply:

sudo tar -C /Library/Collaboration -xvzf backup-file-name.tgz

Okay, with that out of the way, next make sure absolutely nobody is using any of the Group services for the group you are going to perform the move from, or to. People can work on other group’s wikis, it’s just the ones you’ll be touching you want people to avoid. You can enforce this with some Apache redirects, which is left as an exercise to the reader.

Step 1: Rename or create a new Group in Workgroup Manager

First, you need to either rename or create a new group in Workgroup Manager. If you make a new group, be sure to enable all the services that the old group used.

If you’re simply renaming a group, then you might not even have to go through this trouble. You merely need to change the group’s “Name” (as opposed to its “Short Name”) and then stop and start the Web Service from Server Admin to see the group’s name change. However, if you also want to change the group’s “Short Name” (i.e., the group’s POSIX group account symbolic name), then you will need to perform these steps.

Once you have the new group ready to go in Workgroup Manager and you have stopped and started your Web Service in Server Admin, continue to the next step.

Step 2: rsync all your files from the old group to the new group

This is simple. Just run:

sudo rsync -avzE --progress /Library/Collaboration/Groups/old-group/ /Library/Collaboration/Groups/new-group/

Note that the trailing slashes on the directory names in this command are quite important, as they tell rsync to take the contents of the first directory and place those items as the contents of the second directory. Without the trailing slashes, rsync will make extraneous directories for you, which Apple WikiServer won’t understand. See man rsync for more information.

Step 3: Update the plists for all your WikiServer pages in the new group

Now, WikiServer has all the content of your old group but all of the internal references are wrong, since they still point to the old group. What you need to do is rewrite all those references so that they point to the new group address. The group references are stored in property list files. See my older blog entries for details about the filesystem structure of Apple’s WikiServer data storage layout.

To do this is a relatively simple procedure. As root, do the following, and do it carefully because as root you can easily mess up (and there is no undo button on the command line!):

sudo su -
cd /Library/Collaboration/Groups/new-group
grep -ri "old-group" * | grep '^[^B]' | cut -d ':' -f 1 | grep 'plist$' | sort | uniq > /tmp/list_of_plist_files_to_edit
for i in `cat /tmp/list_of_plist_files_to_edit`; do sed -e 's/<string>groups\/old-group/<string>groups\/new-group/' $i > $i.new; mv "$i.new" "$i"; chown teamsserver:teamsserver "$i"; chmod o-rwx "$i" done;

In English, this means:

1. Become root.
2. Change to the /Library/Collaboration/Groups/new-group directory
3. Find all plist files that have the the string old-group in them, sort them, and write this list of files to the /tmp/list_of_plist_files_to_edit file.
4. For each of the files listed in the /tmp/list_of_plist_files_to_edit, find the text string <string>groups/old-group and replace that text with <string>groups/new-group, and save this change as the name of the file with .new appended. Finally, replace the original file with the file we modified, and give them the appropriate permissions.

Step 4: Create an Apache redirect to make sure no HTML links are broken

Okay, at this point your new group is up and running and it should be working. However, if you had any links at all in any of your group’s pages, they are now all broken because they still point to the old group. Rather than going through the HTML itself and cleaning this up right now (because that’s very error-prone indeed, even with automated tools), it’s much easier to just tell Apache to redirect all requests for the old group to the new group.

To do this, edit the Apache configuration file of whatever Virtual Host you have been serving the WikiServer from. Most of the time, this will be at /etc/httpd/sites/0000_any_80_.conf.

The end of that file probably looks something like this::

#       Include /etc/httpd/httpd_users.conf
#       Include /etc/httpd/httpd_directory.conf
        Include /etc/httpd/httpd_groups.conf
        Include /etc/httpd/httpd_teams_required.conf
        LogLevel warn
        ServerAlias *
</VirtualHost>

Right above these Include directives, simply add the following:

        <Location /groups>
            <IfModule mod_alias.c>
                Redirect 301 /groups/old-group http://your.server.address/groups/new-group
            </IfModule>
        </Location>
#       Include /etc/httpd/httpd_users.conf
#       Include /etc/httpd/httpd_directory.conf
        Include /etc/httpd/httpd_groups.conf
        Include /etc/httpd/httpd_teams_required.conf
        LogLevel warn
        ServerAlias *
</VirtualHost>

There you have it. New Teams Server group, old group’s data.

Caveats

Note that this does not update any of the SQLite databases used to store things like revision history and so forth. These things are, for the most part, not really necessary to update but it would be ideal if the old plist revisions could be changed in there, too. That’s not so much more extra work, really, but I’ve found it typically unnecessary except in fringe cases, so I leave that as an exercise to the reader.

If you do this to your WikiServer and do not update the SQLite databases as well, just be mindful of that fact so that you’re not surprised if something goes wonky down the line.

This should be straightforward, but it’s not. To import (not track, but just import) code from a remote CVS repository to a local git repository, you need to do the following:

1. Be certain you have the git-core package installed on your system and that this package includes the git-cvsimport command. You can run git help -a | grep cvsimport to verify this.
2. Be certain you have the cvsps command-line tool installed. This does not come with the git suite of tools, so you’ll need to get it separately. If you’re a lazy Mac OS X user, like me, you can use MacPorts: sudo port install cvsps. Otherwise, get it from the source.
3. Prepare your CVS login information for the remote server before you run git cvsimport. You need to do this so that the git tool will be able to log you in to the CVS server automatically. The command for this looks like:

   CVSROOT=:cvs-login-method:cvs-user-name@cvs.server.name:/path/to/CVS/root cvs login

   For example, if you’re pulling code from the anonymous CVS server that runs on Drupal.org, you might use this: CVSROOT=:pserver:anonymous@cvs.drupal.org:/cvs/drupal-contrib cvs login. This command will prompt you for the password for the user you specified at the server you specified (for anonymous access, the password is almost always anonymous) and will hash this in the ~/.cvspass file for future use by CVS

4. Finally, run the git cvsimport tool, and specify the proper options. Using the Drupal example above, your command might look like this:

   git cvsimport -v -d :pserver:anonymous@cvs.drupal.org:/cvs/drupal-contrib contributions/modules/module-name

   This would login to cvs.drupal.org using the CVS‘s pserver login method, provide the username anonymous and the password you specified in the previous step that is hashed in ~/.cvspass, set the CVS document root to /cvs/drupal-contrib, and pull the code located at contributions/modules/module-name into the current working directory as a git repository.

This works pretty nicely, and creates a git repository just as though you’d created it with git init in the current working directory.

If you get an error that looks like this:

AuthReply: cvs [pserver aborted]: descramble: unknown scrambling method

then you’ve most likely specified the CVS document root incorrectly. Most notably, git cvsimport does not understand a CVS document root wherein the password is specified in the document root URL itself. So, for example, git cvsimport -d :pserver:password:username@cvs.server.name:/path/to/CVS/root code/to/checkout will not work. Omitting the password and the separating colon from the URL should fix it.

When we left off, you had just finished transforming a remote Subversion repository into a git repository and optimizing it to save you some space. Now that you have a git repository, what do you do?

First things first. Once you have an idea of what work you want to do, you should give yourself a space to do this work without disturbing anybody else’s work. Do this by making a new, personal branch. Unlike Subversion and some other centralized version control systems, git makes it possible to do make all kinds of changes to your repository, including making branches, and even save those changes without having to republish everything back to the central repository server at each step. In other words, if you thought Subversion branches were “cheap,” you’ll love git’s branches.

Also unlike Subversion, which stores its branches in completely separate pathnames, git keeps all branches in the same filesystem tree separated only with metadata (in .git/refs/remotes for remote branches and in .git/refs/heads for local branches, to be a bit more precise), so you don’t have to create lots of different directories for all your branches (unless you want to). With no branches defined, you’re working in the “master branch,” or “the trunk” by default.

git branch
* master

By saying git branch you ask git to print a list of all (local) branches. The one with the asterisk marks the current branch, the one you’re using at the moment you run the command. Since there are no branches, you’re currently in the “master” branch. But we don’t want to make changes here, we want to make changes in our own private branch, so we’ll make a new one.

git checkout -b new_branch_name

This will create a new branch with the name new_branch_name and immediately switch to it. Notice that you’ve made absolutely no changes to the filesystem itself; only the git metadata has been altered. Saying git branch again will show you the change:

git branch
  master
* new_branch_name

Also note that since we haven’t comitted any changes, we don’t need a commit message (or “log message”) for creating this branch. We’ll add one later, when we need it. Now go ahead and write some code in your new branch. At any time, you can create a new branch in the same way you added the first. Each new branch is created at the HEAD (“latest”) revision of whatever branch you’re currently working in.

git checkout -b another_branch
Switched to a new branch "another_branch"
git branch
  master
  new_branch_name
* another_branch

To switch back to any other branch, simply git checkout that branch again:

git checkout master
Switched to branch "master"

If you want to delete a branch you don’t like, that’s easy too:

git branch -d another_branch
Deleted branch another_branch.

Keep in mind that throughout all of these branch creation and deletion actions, the only thing that’s being altered is the git metadata. That’s why it’s so cheap to create new branches. If you ever have a new idea you’re working on, it’s recommended that you create a branch for it, even if that branch is so short-lived it never gets published.

So you have a new local branch, and you’ve been working as you normally do for a few minutes, creating files, editting them, and so on. Running git status now will ask git to show you the changes you’ve made to your filesystem. If you’ve created any files in new directories that git doesn’t know about, it will simply report that directory. If you’ve made new files in directories git does know about, it will list all those files explicitly.

git status
# On branch cartoon_contests
# Untracked files:
#   (use "git add ..." to include in what will be committed)
#
#	sites/default/modules/cartoon_contests/
#	sites/default/modules/factiva/.factiva.module.swp
#	sites/default/modules/testfile
#	sites/default/settings.php
nothing added to commit but untracked files present (use "git add" to track)

In the above sample output, I’m working on a new Drupal module for a web site. I’ve created a new directory, sites/default/modules/cartoon_contests/ (note the trailing slash), and I have several untracked files. One is my vim swap file for a different module, .factiva.module.swp, one’s an unimportant testfile, and the last is the Drupal configuration file, settings.php.

The only thing I want to commit is the new cartoon_contests directory, and all the files within it. Like Subversion, I have to tell git that I want to track this directory, which is done simply by saying git add cartoon_contests. Unlike Subversion, future invocations of git status let me see everything that git is going to go in the next commit.

git status
# On branch cartoon_contests
# Changes to be committed:
#   (use "git reset HEAD ..." to unstage)
#
#	new file:   sites/default/modules/cartoon_contests/cartoon_contests.info
#	new file:   sites/default/modules/cartoon_contests/cartoon_contests.module
#
# Untracked files:
#   (use "git add ..." to include in what will be committed)
#
#	sites/default/modules/factiva/.factiva.module.swp
#	sites/default/modules/testfile
#	sites/default/settings.php

The “Changes to be committed” section of the output is called the staging area, or the index. In this way, I can prepare all the changes I want to commit before I do so, making sure they’re perfect before I actually commit them to the git repository. At any time before I commit, I can make additional modifications, such as git adding more files or directories, git reseting to unstage all (or some) of my changes, etc. git help reset also has a number of handy explanations with examples for different things you might need to do at this point.

If you were using svn:ignore, equivalent functionality exists in git. Simply append file glob patterns, one per line, to the $GIT_DIR/info/exclude file in your git repository. Like so:

echo -e "*.swp\nsites/default/settings.php" >> .git/info/exclude ; git status
# On branch cartoon_contests
# Changes to be committed:
#   (use "git reset HEAD ..." to unstage)
#
#	new file:   sites/default/modules/cartoon_contests/cartoon_contests.info
#	new file:   sites/default/modules/cartoon_contests/cartoon_contests.module
#

If you’ve got many of these, you can use git svn show-ignore >> .git/info/exclude to search through your old Subversion repository and look for any and all ignores, automatically adding them to git’s exclude list. (Checkout Tsuna’s blog entry on learning git for more tips like this.)

Finally, after you’ve done some of your work and you’ve finished staging your changes, you’re ready to commit them to the repository. On the other hand, if you hate what you’ve done and want to undo it all, you can say git reset --hard HEAD to throw away all your local changes. To throw away changes to just a single file, just checkout that file again by saying git checkout filename. This is the equivalent of Subversions svn revert filename command.

Before you actually make your first commit, however, you should properly introduce yourself to git. You don’t have to do this because git will try to figure out who you are by itself (details explained in detail here), but you probably should at least create global defaults for your user (which git will store in ~/.gitconfig). If you want to, you can also create per-repository defaults (which git will store in .git/config), or even system-wide defaults for all users of this computer (which git will store in /etc/gitconfig). To do so, say this:

git config --global user.name "Your Full Name"
git config --global user.email "you@example.com"

This will create the file ~/.gitconfig if it doesn’t alreay exist and will write your name into it. You can also just edit the file directly yourself instead of using git config commands.

Like Subversion, committing will create a saved, fixed point-in-time that marks the changes you have made to your files. Like branches, commits are also very cheap in git, so go ahead and commit at any time you like. Remember to stage your files (by git adding them), and then just git commit.

git add filepattern
git commit -m "My very frst git commit!"
Created commit ef483c1: My very frst git commit!
 5 files changed, 58 insertions(+), 12 deletions(-)

If you forget to specify a commit log message (-m "log message") on the command line, or if you want to enter a multi-line commit log, git will prompt you for it in your favorite $EDITOR. You can view a history, including all the log messages, for your project with git log. You can even the view logs in a number of pretty formats. Check git help log for more information.

If you want to change anything about the commit you just made, such as the author, you can just run git commit again with the --amend flag added to the command. Notice the typo in the commit log? Fixing it is really easy:

git commit --amend -m "My very first git commit!"
Created commit 88602f6: My very first git commit!
 5 files changed, 58 insertions(+), 12 deletions(-)

Finally, with your new code commited to your local git branch, it’s time to share that code with your colleagues who are (for some inexplicable reason) still using Subversion. This is also extremely simple. Just say git svn dcommit. (That’s right, dcommit, not just commit. I don’t know why….)

Not sharing your changes via Subversion, but with a patch instead? git diff -p will generate patches for you. See git help diff-files and look for the “GENERATING PATCHES WITH -P” section.

If you found this helpful, you may also enjoy an alternative tour for beginners from Carl Worth.

Let’s just jump into it.

Using Mac OS X, first make sure you have the git core installed as well as the git-svn components. Unlike other version control systems, git is not one monolithic program but a collection of smaller utilities, and the Subversion conduits are a subset of these utilities. The git-Subversion utilities themselves depend on having the Perl::Subversion bindings installed for Perl 5.

For the lazy MacPorts user, simply run:

sudo port install git-core +svn

This will install the git-core, the git-svn packages, as well as all the documentation for git and any required dependencies you don’t already have. Note that the documentation (man pages) is installed in /opt/local/man, which may not be in your default $MANPATH, so be sure to add that directory if man git returns a “No manual entry for git” error.

Alternatively, if you don’t want to use MacPorts, you can download a pre-compiled Mac OS X binary that includes git, the git docs, and the git-svn package from the git web site that comes complete with a standard GUI installation procedure.

Next, initialize a new git repository that tracks a remote Subversion one. This allows you to work privately using git, but to collaborate with other people who are using Subversion. Running

git svn init svn://my.svn.server/path/to/svn/repo workingcopy

will give you an empty git working copy named workingcopy configured to use the remote Subversion repository at svn://my.svn.server/path/to/svn/repo. This step is analogous to the need to run svnadmin create repo_db, to initialize a new repository database (where all the file versioning information will be stored). Unlike Subversion, git’s distributed database means that the working copy itself is also the location of the repository database, so there’s no need to deal with two filesystem paths anymore.

Next, change directory to your working copy, and run

cd workingcopy
git svn fetch

to populate your new, empty git working copy (and repository) with all the files from the remote Subversion repository.

Now that you have filled your git repository with a lot of data, if you want to, you can now also save a significant chunk of disk space by repacking that data into a more single, efficient, native packed git archive format (a .pack file). The git-repack -a command is used to do this, and its manual page says:

Instead of incrementally packing the unpacked objects, pack everything referenced into a single pack. Especially useful when packing a repository that is used for private development and there is no need to worry about people fetching via dumb protocols from it.

According to some sources, this can turn a 353MB Subversion repository into 31MB of git pack. Say:

git repack -a -d -f
Generating pack...
Counting objects: 284
Done counting 4475 objects.
Deltifying 4475 objects...
 100% (4475/4475) done
Writing 4475 objects...
 100% (4475/4475) done
Total 4475 (delta 1876), reused 0 (delta 0)
Pack pack-a115c320ff9c9968248bd250bdfea3110d0f0c1a created.
Removing unused objects 100%...
Done.

to perform the compression. For even greater savings, increase the compression by stipulating a high --window option, such as git repack -a -d -f --window=100. (--window defaults to 10—the higher the window, the harder git-repack will try to compress your stuff.) This turned a 36MB Subversion repository in one of my projects to a 20MB git pack. As always, your mileage may vary.

Congratulations, you have transformed your old Subversion repository to a git repository. All that’s left to do now is to get to the coding. Perhaps start by making a local (cheaper than cheap!) git branch….

That was easy, right? Most things in git are, in fact, that easy. If you’ve never used other version control systems before, be grateful. If you have, you can breathe a sigh of relief. Still not convinced? Don’t take my word for it…ask Linus Torvalds (or Randall Schwartz).

I have a suspicion that there are a lot of web developers out there who, like me, are eager to start learning more about Ruby but who are stunted by the incredible amount of unfamiliar conventions that are used in Rails. For many of us, our first introduction to Ruby was (or is) through Rails and the challenge of learning both Ruby and Rails at once should not be underestimated; frankly it’s damn hard.

Thankfully, there’s an easier way. Especially if you’re accustomed to “old-school” development workflows that you learned through, say, PHP or perhaps Perl scripting, then ditching Rails and starting purely with Ruby might be the way to go. Doing things on your own without the “magic” of Rails will let you learn Ruby in a simple, yet fully-featured environment, much like the way you might have learned classic CGI programming.

Specifically, by using eRuby, the Embedded Ruby interpreter, you can make Ruby web development feel just like PHP development. Install it, upload an HTML file with some embedded Ruby code in it, and—voila—you have a single-page web application.

Why this tutorial?

Hivemind’s Getting Started with Ruby on the Web is an excellent resource that described this process from start-to-finish for Windows users. In fact, it was after finding that article that I realized web development with Ruby didn’t have to be governed by Rails and, being a web developer like the authors of Hiveminds, I wanted an easy way to use my Ruby knowledge on the Web.

However, I couldn’t find any similar tutorial or walkthrough for those of us who use Macs. Even the Linux walkthroughs are lacking (or at least my Googling skill for them is). After spending a little while successfully getting eRuby set up on my Mac running Mac OS X 10.4 Tiger, I thought I’d share what I learned with others, so I wrote this article. Its content mirrors the one from Hiveminds to a great degree but focuses on getting eRuby installed on Mac OS X instead of Windows (obviously). I imagine these instructions should be pretty similar if not identical for any other *nix-like platform.

The Setup

Getting eRuby installed and configured with your web server is actually pretty straightforward and won’t take an experienced web developer much time at all. All we’re going to need is the Apache web server and the Ruby programming language to start, both of which are already pre-installed on Mac OS X 10.4 Tiger. (And in Mac OS X 10.5 Leopard, Apple is even shipping new Macs with Ruby On Rails preconfigured, but that’s a subject for another time.) However, Apple typically ships an older copy of Ruby than is readily available, so you might choose to use a newer version of Ruby that you download yourself.

The easiest way to get a copy of eRuby (and an updated Ruby, if you want it) is with a package manager such as MacPorts. With MacPorts installed (which is a simple, standard package installation downloadable from the MacPorts home page), simply run

sudo port install eruby

from your terminal to download the latest stable Ruby and eRuby in one fell swoop. After this runs, you’ll have a new Ruby in /opt/local/bin/ruby and an eRuby in /opt/local/bin/eruby. Alternatively, of course, you could compile and install eRuby yourself. Instructions for that are on the eRuby homepage.

As a quick test to see if everything is installed and working correctly, create a plain text file with your favorite editor that contains the following code:

Hello world! The time is now <%= Time.now %>.

All this does is print out a “Hello world!” greeting and then announces the current time (as reported by your system clock, of course).

If you named the above file helloworld.rhtml then you can feed this file to the eruby interpreter as follows:

eruby helloworld.rhtml

Note that the file extension doesn’t actually matter. You could use .erb, if you like, or any other arbitrary file extension. The file extension is, as you’ll see later, only used for telling Apache which kind of file it is. So once you come up with file extension you like, stick with it. The standard file extensions are either .rhtml or .erb, so I’d recommend using one of those. (I prefer .erb, myself, since I don’t necessarily only want to create HTML pages. I might want to create XML documents like news feeds or even XML databases, so .erb seems a more reasonable file name extension, though most of the eRuby documentation uses .rhtml.)

If things worked properly, you should see output similar to the following:

Hello world! The time is now Wed Oct 24 21:58:54 -0400 2007.

Serve it up!

Now that eRuby is installed and working, the next step is to set it up as a CGI so that when your web server (Apache, in my case) fetches the file it will first feed it through eRuby for processing before sending it back to the browser. The simplest way to do this is to configure Apache to use a specific directory somewhere on your filesystem as a CGI directory—that is, a directory whose contents are all treated as CGI programs. You do this via Apache’s ScriptAlias directive.

Conveniently, standard installations of Mac OS X already come with a CGI directory. It’s located at /Library/WebServer/CGI-Executables. This directory is just like any other, except that Apache treats it specially. It treats it specially because Apache’s configuration file, /etc/httpd/httpd.conf contains a line of text that reads as follows. (On a standard installation of Mac OS X client, this is line 671 of the file.)

ScriptAlias /cgi-bin/ "/Library/WebServer/CGI-Executables/"

This line basically just tells apache, “Treat every file in the /Library/WebServer/CGI-Executables directory as though it were a CGI program.” So, since that’s how we’re treating eRuby, we’re going to want to put our eruby binary in that directory. However, we don’t actually want to make copies of the binary. Having too many copies of a program on your system can easily get confusing. So instead, we’ll make a symbolic link (or an alias to use the classic Mac terminology) to the binary.

ln -s /opt/local/bin/eruby /Library/WebServer/CGI-Executables/eruby

With our eruby CGI executable in the proper place for Apache to find it, we can now tell Apache which files we want it to send to this program. We do this using the AddHandler and Action directives.

AddHandler simply tells Apache to treat files with a certain extension in a certain way. For instance, you can say something like, “Treat all files ending in .php as PHP scripts.” Similarly, you can (and will) tell Apache, “Treat all files ending in .rhtml or .erb as eRuby CGI pages.”

To do this, we add the following line to our /etc/httpd/users/username.conf file somewhere inside the <Directory> and </Directory> block. Naturally, replace username with, of course, your Mac OS X user’s short name:

AddHandler rubypage .erb .rhtml

Again, this just tells Apache that files ending in .erb and .rhtml should be treated the same way other rubypages are treated. But how are eRuby pages supposed to be treated? They’re supposed to be handed off to the eRuby program, of course, so we need an Action directive that applies to the rubypage handler we just defined.

Right below the above line, add the following line, too:

Action rubypage /cgi-bin/eruby

This line tells Apache that any file that it is treating as a “rubypage” file should be sent to the program at the URL accessible at /cgi-bin/eruby, which we’ve defined earlier to be our symbolic link to the eruby interpreter.

Your completed user-specific Apache configuration file should now look something like this:

<Directory "/Users/username/Sites/">
…
    # Let eRuby files get parsed through the proper CGI binary
    AddHandler rubypage .erb .rhtml
    Action rubypage /cgi-bin/eruby
</Directory>

Are You Being Served?

That’s it, everything should now be in place. To make sure, move your earlier helloworld.rhtml file into your user’s Sites folder, and now simply point your browser to http://localhost/~username/helloworld.rhtml (again, replacing username with your Mac OS X user short name, of course). Congratulations, you’re serving up Ruby-coded applications via eRuby.

Additional Reading

This is just the set up, of course. Now the real fun begins: learning Ruby. Here are some suggested places to start.

• Programming Ruby: Ruby and the Web — A chapter out of the free book Programming Ruby all about Ruby as a CGI programming language.
• Hiveminds: eRuby: How to use Ruby CGI and ERB templating for Web Pages — a quick’n'dirty article explaining the basics of using Ruby on web pages in classic CGI style where program logic and HTML templates are kept separated, as opposed to classic PHP style, where the two are mingling together on one page.
• COOLNAMEHERE: Simple Ruby CGI — Like the Hiveminds example above, another quick’n'dirty Ruby CGI how-to.
• cgi: Ruby Standard Library Documentation — Ruby’s standard library contains a rich set of features, and the CGI objects are among them. They are well-documented in the official Ruby Documentation Project.

Naturally, feel free to leave comments if you have more resources that you found helpful.

I wanted to use my blog entries, created in WordPress to generate content for pages on my site that were not in my blog. This seemed simple enough, and it is, but without some guidance this can be a really confusing situation. While WordPress supports a pseudo-templating system that can be used to easily customize the look and feel of a WordPress blog, getting that blog-portion of the site to play nice with the other parts of a site (or the converse, as the case may be) is often not as easy as it seems.

I believe this is one of the reasons why WordPress’s popularity has not yet reached its enormous potential.

But without further ado, here’s what I did to get WordPress to generate pages in a directory completely outside WordPress’s own installation directory.

The Solution

The solution involves replacing all occurences of WordPress’s erroneous absolute link-creation output and replace it with output appropriate to the /newsection structure. This was done with PHP‘s output buffering. Then, to make it work with the permalink structure, I made some edits to the WordPress-generated RewriteRules. In the hopes that this will be of use to someone, here’s what I did:

1. In the top of your new section’s template (/newsection/index.php), following the line with the require() statement, put a new line that reads:

   ob_start();

   This will start the output buffer.

2. At the very bottom of your template, after the </html> add the following lines:

   <?php
   $page_content = ob_get_contents();
   $replace_this = 'href="http://mysite.com/blog';
   $with_this = 'href="http://mysite.com/newsection';
   $modified_page_content = str_replace($replace_this, $with_this, $page_content);
   ob_end_clean();
   echo $modified_page_content;
   ?>

   These do the replacements you need. Obviously, change the mysite.com to whatever your site domain is, change the /blog to whatever path your WordPress install is at, and change the /newsection to the path of the new section you’re creating.

   With these lines wrapped around your template, you can use all of WordPress’s original template tags like the_permalink() and wp_list_cats() just as you would if you were using them in the original section.

   Note to regex wizards: this is obviously a very crude string replacement, and I don’t know how well it will work in situations not like my own. I’m not very good with regular expressions, however, so if you can think of one that would work better with preg_replace() please let me know.

3. If you’re using WordPress’s virtual site structure (pretty-permalinks): Copy the rules WordPress wrote for you at Options->Permalinks (or from your existing .htaccess file) and paste them into a new text document. This will become your new section’s .htaccess file.

   1. In the line that begins with RewriteBase replace the old base with the new base path to your section. If the line read RewriteBase /blog/ then you would replace /blog/ with /newsection/.

   2. Finally, for the second, third, and fourth lines that begins with RewriteRule replace the old actual path with the actual path to your new section. So, continuing the example above, if you have a line that reads like this,

      RewriteRule ^category/?(.*) /blog/index.php?category_name=$1 [QSA]

      replace it with this:

      RewriteRule ^category/?(.*) /newsection/index.php?category_name=$1 [QSA]

   3. Save this file as a new .htaccess file and upload it to your server so that it resides at http://mysite.com/newsection/.htaccess

No matter what section/directory of your site you’re putting WordPress templates in, you can use this method to point links to the current section, and thus load WordPress content in the templates within that section/directory. One important caveat to this method is that every time a template using this method is accessed, your server works a little over-time to do the str_replace() on your content. This won’t be noticeable on small or short pages, but could really become an issue on oft-visited, long, long pages. As always, use at your own risk.

Improvements/warnings/suggestions/hate mail are welcome.

External Resources

• The original support forum thread.
• The Wiki HowTo node I wrote.
• WordPress Customization Notes.

Here’s a useful little snippet of CSS2 lifted straight off a personal project I’ve been working on. The following four rules display, find, and insert appropriate punctuation for making unordered lists read like English prose.

ul, ul li {
    display: inline; list-style: none; margin: 0; padding: 0;
}
li:after { content: ", "; }
li:last-child:after { content: "."; }
li+li:last-child:before { content: "and "; }

Here’s what’s happening.

1. First they display the entire list and each list item as an inline element. This causes them to show up right up against any other text around them.
2. Then they add a comma and a space after each list item.
3. The last list item, however, gets a period and no space.
4. Finally, the word “and” followed by a space is inserted before the last list item if, and only if, there is a list item before it (it has a sibling).

There you have it. List markup that looks just like a normal paragraph, complete with proper punctuation! A great little tip that makes lists a whole lot more flexible!

Note that as of this writing this won’t work in Internet Explorer because it can’t understand the last three rules. Since it does understand the first, the result will merely be an inline box for the whole list, sans punctuation.