Most notably, custom fields and templates’ linked files are not copied. This is a deal-breaker for any large installation that uses the built-in MT “Clone” feature.

To get around this limitation, I wrote a stupid, quick ‘n’ dirty PHP script to finish the cloning process, called finishclone.php. It takes only 1 argument: the “new ID” you are cloning to. If all goes well, you’ll see output like this:

[root@dev www]$ php finishclone.php 28
Cloning process complete.
[root@dev www]$ 

In this example, 28 is the newly created blog’s ID. The blog you want to clone from is set as a constant within the script. I’ll leave modifying the script to support more flexible command line arguments as an exercise for the reader.

<?php
/**
 * Ease the final steps in cloning a Movable Type blog.
 *
 * Description:   This script should be run after Movable Type's "Clone Blog"
 *                function has completed and before the cloned blog is used.
 *
 * Author:        "Meitar Moscovitz" <meitar@maymay.net>
 */

// Set constants.
define('MT_ORIG_BLOG', 0); // the ID of the blog you are cloning from
define('MYSQL_HOST', 'localhost');
define('MYSQL_USER', 'movabletype');
define('MYSQL_PASS', 'PASSWORD_HERE');
define('MYSQL_DB', 'movabletype');

// Get command line arguments.
if (2 > $_SERVER['argc']) { die('Tell me the ID of the blog to clone into.'); }

$blog_id = (int) $argv[1];

// Connect to db
if ( !mysql_pconnect( MYSQL_HOST, MYSQL_USER, MYSQL_PASS ) ) {
	die( 'Connection to the database has failed: ' . mysql_error( ) );
}
mysql_select_db( MYSQL_DB );

// Clone custom fields.
$result = mysql_query('SELECT * FROM mt_field WHERE field_blog_id='.MT_ORIG_BLOG.';');
while ($row = mysql_fetch_object($result)) {
	mysql_query(
		sprintf("INSERT INTO mt_field ("
			."field_basename,"
			."field_blog_id,"
			."field_default,"
			."field_description,"
			."field_name,"
			."field_obj_type,"
			."field_options,"
			."field_required,"
			."field_tag,"
			."field_type) "
			."VALUES('%s','%s','%s','%s','%s','%s','%s','%s','%s','%s');",
			mysql_real_escape_string($row->field_basename),
			mysql_real_escape_string($blog_id),
			mysql_real_escape_string($row->field_default),
			mysql_real_escape_string($row->field_description),
			mysql_real_escape_string($row->field_name),
			mysql_real_escape_string($row->field_obj_type),
			mysql_real_escape_string($row->field_options),
			mysql_real_escape_string($row->field_required),
			mysql_real_escape_string($row->field_tag),
			mysql_real_escape_string($row->field_type)
		)
	) OR print mysql_error() . "\n";
}

// Link template files to filesystem.
$arr = array();
$result = mysql_query('SELECT template_name,template_linked_file FROM mt_template WHERE template_blog_id='.MT_ORIG_BLOG.';');
while ($row = mysql_fetch_object($result)) {
	$arr[$row->template_name] = $row->template_linked_file;
}
foreach ($arr as $k => $v) {
	mysql_query("UPDATE mt_template SET template_linked_file='$v' WHERE template_blog_id=$blog_id AND template_name='$k';");
}

print "Cloning process complete.\n";

A few nights ago, I received an email from Editor of EdenFantasys’s SexIs Magazine, Judy Cole, asking me to modify this Kink On Tap brief I published that cites Lorna D. Keach’s writing. Judy asked me to “provide attribution and a link back to” SexIs Magazine. An ordinary enough request soon proved extraordinarily unethical when I discovered that EdenFantasys has invested a staggering amount of time and money to develop and implement a technology platform that actively denies others the courtesy of link reciprocity, a courtesy on which the ethical Internet is based.

While what they’re doing may not be illegal, EdenFantasys has proven itself to me to be an unethical and unworthy partner, in business or otherwise. Its actions are blatantly hypocritical, as I intend to show in detail in this post. Taking willful and self-serving advantage of those not technically savvy is a form of inexcusable oppression, and none of us should tolerate it from companies who purport to be well-intentioned resources for a community of sex-positive individuals.

For busy or non-technical readers, see the next section, Executive Summary, to quickly understand what EdenFantasys is doing, why it’s unethical, and how it affects you whether you’re a customer, a contributor, or a syndication partner. For the technical reader, the Technical Details section should provide ample evidence in the form of a walkthrough and sample code describing the unethical Search Engine Optimization (SEO) and Search Engine Marketing (SEM) techniques EdenFantasys, aka. Web Merchants, Inc., is engaged in. For anyone who wants to read further, I provide an Editorial section in which I share some thoughts about what you can do to help combat these practices and bring transparency and trust—not the sabotage of trust EdenFantasys enacts—to the market.

EXECUTIVE SUMMARY

Internet sex toy retailer Web Merchants, Inc., which bills itself as the “sex shop you can trust” and does business under the name EdenFantasys, has implemented technology on their websites that actively interferes with contributors’ content, intercepts outgoing links, and alters republished content so that links in the original work are redirected to themselves. Using techniques widely acknowledged as unethical by Internet professionals and that are arguably in violation of major search engines’ policies, EdenFantasys’s publishing platform has effectively outsourced the task of “link farming” (a questionable Search Engine Marketing [SEM] technique) to sites with which they have “an ongoing relationship,” such as AlterNet.org, other large news hubs, and individual bloggers’ blogs.

Articles published on EdenFantasys websites, such as the “community” website SexIs Magazine, contain HTML crafted to look like links, but aren’t. When visited by a typical human user, a program written in JavaScript and included as part of the web pages is automatically downloaded and intercepts clicks on these “link-like” elements, fetching their intended destination from the server and redirecting users there. Due to the careful and deliberate implementation, the browser’s status bar is made to appear as though the link is legitimate, and that a destination is provided as expected.

For non-human visitors, including automated search engine indexing programs such as Googlebot, the “link” remains non-functional, making the article a search engine’s dead-end or “orphan” page whose only functional links are those whose destination is EdenFantasys’s own web presence. This makes EdenFantasys’ website(s) a self-referential black hole that provides no reciprocity for contributors who author content, nor for any website ostensibly “linked” to from article content. At the same time, EdenFantasys editors actively solicit inbound links from individuals and organizations through “link exchanges” and incentive programs such as “awards” and “free” sex toys, as well as syndicating SexIs Magazine content such that the content is programmatically altered in order to create multiple (real) inbound links to EdenFantasys’s websites after republication on their partner’s media channels.

How EdenFantasys’s unethical practices have an impact on you

Regardless of who you are, EdenFantasys’s unethical practices have a negative impact on you and, indeed, on the Internet as a whole.

• If you’re an EdenFantasys employee, people will demand answers from you regarding the unethical practices of your (hopefully former) employer. While you are working for EdenFantasys, you’re seriously soiling your reputation in the eyes of ethical Internet professionals. Ignorance is no excuse for the lack of ethics on the programmers’ part, and it’s a shoddy one for everyone else; you should be aware of your company’s business practices because you represent them and they, in turn, represent you.
• If you’re a partner or contributor (reviewer, affiliate, blogger), while you’re providing EdenFantasys with inbound links or writing articles for them and thereby propping them up higher in search results, EdenFantasys is not returning the favor to you (when they are supposed to be doing so). Moreover, they’re attaching your handle, pseudonym, or real name directly to all of their link farming (i.e., spamming) efforts. They look like they’re linking to you and they look like their content is syndicated fairly, but they’re actually playing dirty. They’re going the extra mile to ensure search engines like Google do not recognize the links in articles you write. They’re trying remarkably hard to make certain that all roads lead to EdenFantasys, but none lead outside of it; no matter what the “link,” search engines see it as stemming from and leading to EdenFantasys. The technically savvy executives of Web Merchants, Inc. are using you without giving you a fair return on your efforts. Moreover, EdenFantasys is doing this in a way that preys upon people’s lack of technical knowledge—potentially your own as well as your readership’s. Do you want to keep doing business with people like that?
• If you’re a customer, you’re monetarily supporting a company that essentially amounts to a glorified yet subtle spammer. If you hate spam, you should hate the unethical practices that lead to spam’s perpetual reappearance, including the practices of companies like Web Merchants, Inc. EdenFantasys’s unethical practices may not be illegal, but they are unabashedly a hair’s width away from it, just like many spammers’. If you want to keep companies honest and transparent, if you really want a “sex shop you can trust,” this is relevant to you because EdenFantasys is not it. If you want to purchase from a retailer that truly strives to offer a welcoming, trustworthy community for those interested in sex positivity and sexuality, pay close attention and take action. For ideas about what you can do, please see the “What you can do” section, below.
• If you’ve never heard about EdenFantasys before, but you care about a fair and equal-opportunity Internet, this is relevant to you because what EdenFantasys is doing takes advantage of non-tech-savvy people in order to slant the odds of winning the search engine game in their favor. They could have done this fairly, and I personally believe that they would have succeeded. Their sites are user-friendly, well-designed, and solidly implemented. However, they chose to behave maliciously by not providing credit where credit is due, failing to follow through on agreements with their own community members and contributors, and sneakily utilizing other publishers’ web presences to play a very sad zero-sum game that they need not have entered in the first place. In the Internet I want, nobody takes malicious advantage of those less skilled than they are because their own skill should speak for itself. Isn’t that the Internet and, indeed, the future you want, too?

TECHNICAL DETAILS

What follows is a technical exploration of the way the EdenFantasys technology works. It is my best-effort evaluation of the process in as much detail as I can manage within strict self-imposed time constraints. If any of this information is incorrect, I’d welcome any and all clarifications provided by the EdenFantasys CTO and technical team in an appropriately transparent, public, and ethical manner. (You’re welcome—nay, encouraged—to leave a comment.)

Although I’m unconvinced that EdenFantasys understands this, it is the case that honesty is the best policy—especially on the Internet, where everyone has the power of “View source.”

The “EF Framework” for obfuscating links

Article content written by contributors on SexIs Magazine pages is published after all links are replaced with a <span> element bearing the class of linklike and a unique id attribute value. This apparently happens across any and all content published by Web Merchants, Inc.’s content management system, but I’ll be focusing on Lorna D. Keach’s post entitled SexFeed:Anti-Porn Activists Now Targeting Female Porn Addicts for the sake of example.

These fake links look like this in HTML:

And according to Theresa Flynt, vice president of marketing for Hustler video, <span class="linklike" ID="EFLink_68034_fe64d2">female consumers make up 56% of video sales.</span>

This originally published HTML is what visitors without JavaScript enabled (and what search engine indexers) see when they access the page. Note that the <span> is not a real link, even though it is made to look like one. (See Figure 1; click it to enlarge.)

Figure 1:

In a typical user’s browser, when this page is loaded, a JavaScript program is executed that mutates these “linklike” elements into <a> elements, retaining the “linklike” class and the unique id attribute values. However, no value is provided in the href (link destination) attribute of the <a> element. See Figure 2.

Figure 2:

The JavaScript program is downloaded in two parts from the endpoint at http://cdn3.edenfantasys.com/Scripts/Handler/jsget.ashx. The first part, retrieved in this example by accessing the URI at http://cdn3.edenfantasys.com/Scripts/Handler/jsget.ashx?i=jq132_cnf_jdm12_cks_cm_ujsn_udm_stt_err_jsdm_stul_ael_lls_ganl_jqac_jtv_smg_assf_agrsh&v_14927484.12.0, loads the popular jQuery JavaScript framework as well as custom code called the “EF Framework”.

The EF Framework contains code called the DBLinkHandler, an object that parses the <span> “linklike” elements (called “pseudolinks” in the EF Framework code) and retrieves the real destination. The entirety of the DBLinkHandler object is shown in code listing 1, below. Note the code contains a function called handle that performs the mutation of the <span> “linklike” elements (seen primarily on lines 8 through 16) and, based on the prefix of each elements’ id attribute value, two key functions (BuildUrlForElement and GetUrlByUrlID, whose signatures are on lines 48 and 68, respectively) interact to set up the browser navigation after responding to clicks on the fake links.

var DBLinkHandler = {
    pseudoLinkPrefix: "EFLink_",
    generatedAHrefPrefix: "ArtLink_",
    targetBlankClass: "target_blank",
    jsLinksCssLinkLikeClass: "linklike",
    handle: function () {
        var pseudolinksSpans = $("span[id^='" + DBLinkHandler.pseudoLinkPrefix + "']");
        pseudolinksSpans.each(function () {
            var psLink = $(this);
            var cssClass = $.trim(psLink.attr("class"));
            var target = "";
            var id = psLink.attr("id").replace(DBLinkHandler.pseudoLinkPrefix, DBLinkHandler.generatedAHrefPrefix);
            var href = $("<a></a>").attr({
                id: id,
                href: ""
            }).html(psLink.html());
            if (psLink.hasClass(DBLinkHandler.targetBlankClass)) {
                href.attr({
                    target: "_blank"
                });
                cssClass = $.trim(cssClass.replace(DBLinkHandler.targetBlankClass, ""))
            }
            if (cssClass != "") {
                href.attr({
                    "class": cssClass
                })
            }
            psLink.before(href).remove()
        });
        var pseudolinksAHrefs = $("a[id^='" + DBLinkHandler.generatedAHrefPrefix + "']");
        pseudolinksAHrefs.live("mouseup", function (event) {
            DBLinkHandler.ArtLinkClick(this)
        });
        pseudolinksSpans = $("span[id^='" + DBLinkHandler.pseudoLinkPrefix + "']");
        pseudolinksSpans.live("click", function (event) {
            if (event.button != 0) {
                return
            }
            var psLink = $(this);
            var url = DBLinkHandler.BuildUrlForElement(psLink, DBLinkHandler.pseudoLinkPrefix);
            if (!psLink.hasClass(DBLinkHandler.targetBlankClass)) {
                RedirectTo(url)
            } else {
                OpenNewWindow(url)
            }
        })
    },
    BuildUrlForElement: function (psLink, prefix) {
        var psLink = $(psLink);
        var sufix = psLink.attr("id").toString().substring(prefix.length);
        var id = (sufix.indexOf("_") != -1) ? sufix.substring(0, sufix.indexOf("_")) : sufix;
        var url = DBLinkHandler.GetUrlByUrlID(id);
        if (url == "") {
            url = EF.Constants.Links.Url
        }
        var end = sufix.substring(sufix.indexOf("_") + 1);
        var anchor = "";
        if (end.indexOf("_") != -1) {
            anchor = "#" + end.substring(0, end.lastIndexOf("_"))
        }
        url += anchor;
        return url
    },
    ArtLinkClick: function (psLink) {
        var url = DBLinkHandler.BuildUrlForElement(psLink, DBLinkHandler.generatedAHrefPrefix);
        $(psLink).attr("href", url)
    },
    GetUrlByUrlID: function (UrlID) {
        var url = "";
        UrlRequest = $.ajax({
            type: "POST",
            url: "/LinkLanguage/AjaxLinkHandling.aspx",
            dataType: "json",
            async: false,
            data: {
                urlid: UrlID
            },
            cache: false,
            success: function (data) {
                if (data.status == "Success") {
                    url = data.url;
                    return url
                }
            },
            error: function (xhtmlObj, status, error) {}
        });
        return url
    }
};

Once the mutation is performed and all the content “links” are in the state shown in Figure 2, above, an event listener has been bound to the anchors that captures a click event. This is done using prototypal extension, aka. classic prototypal inheritance, in another part of the code, the live function on line 2,280 of the (de-minimized) jsget.ashx program, as shown in code listing 2, here:

        live: function (G, F) {
            var E = o.event.proxy(F);
            E.guid += this.selector + G;
            o(document).bind(i(G, this.selector), this.selector, E);
            return this
        },

At this point, clicking on one of the “pseudolinks” triggers the EF Framework to call code set up by the GetUrlByUrlID function from within the DBLinkHandler object, initiating an XMLHttpRequest (XHR) connection to the AjaxLinkHandling.aspx server-side application. The request is an HTTP POST containing only one parameter, called urlid, and its value matches a substring from within the id value of the “pseudolinks.” In this example, the id attribute contains a value of EFLink_68034_fe64d2, which means that the unique ID POST’ed to the server is 68034. This is shown in Figure 3, below.

Figure 3:

The response from the server, shown in Figure 4, is also simple. If successful, the intended destination is retrieved by the GetUrlByUrlID object’s success function (on line 79 of Code Listing 1, above) and the user is redirected to that web address, as if the link was a real one all along. The real destination, in this case to CNN.com, is thereby only revealed after the XHR request returns a successful reply.

Figure 4:

All of this obfuscation effectively blinds machines such as the Googlebot who are not JavaScript-capable from seeing and following these links. It deliberately provides no increased Pagerank for the link destination (as a real link would normally do) despite being “linked to” from EdenFantasys’s SexIs Magazine article. While the intended destination in this example link was at CNN.com, it could just as easily have been—and is, in other examples—links to the blogs of EdenFantasys community members and, indeed, everyone else linked to from a SexIs Magazine article or potentially any website operated by Web Merchants, Inc. that makes use of this technology.

The EdenFantasys Outsourced Link-Farm

In addition to creating a self-referential black hole with no gracefully degrading outgoing links, EdenFantasys also actively performs link-stuffing through its syndicated content “relationships,” underhandedly creating an outsourced and distributed link-farm, just like a spammer. The difference is that this spammer (Web Merchants, Inc. aka EdenFantasys) is cleverly crowd-sourcing high-value, high-quality content from its own “community.”

Articles published at SexIs Magazine are syndicated in full to other large hub sites, such as AlterNet.org. Continuing with the above example post by Lorna D. Keach, Anti-Porn Activists Now Targeting Female Porn Addicts, we can see that this content was republished on AlterNet.org shortly after original publication through EdenFantasys’ website on May 3^rd at http://www.alternet.org/story/146774/christian_anti-porn_activists_now_targeting_female_. However, a closer look at the HTML code of the republication shows that each and every link contained within the article points to the same destination: the same article published on SexIs Magazine, as shown in Figure 5.

Figure 5:

Naturally, these syndicated links provided to third-party sites by EdenFantasys are real and function as expected to both human visitors and to search engines indexing the content. The result is “natural,” high-value links to the EdenFantasys website from these third-party sites; EdenFantasys doesn’t merely scrounge pagerank from harvesting the sheer number of incoming links, but as each link’s anchor text is different, they are setting themselves up to match more keywords in search engine results, keywords that the original author likely did not intend to direct to them. Offering search engines the implication that EdenFantasys.com contains the content described in the anchor text, when in fact EdenFantasys merely acts as an intermediary to the information, is very shady, to say the least.

In addition to syndication, EdenFantasys employs human editors to do community outreach. These editors follow up with publishers, including individual bloggers (such as myself), and request that any references to published material provide attribution and a link back to us, to use the words of Judy Cole, Editor of SexIs Magazine in an email she sent to me (see below), and presumably many others. EdenFantasys has also been known to request “link exchanges,” and offer incentive programs that encouraged bloggers to add the EdenFantasys website to their blogroll or sidebar in order to help raise both parties search engine ranking, when in fact EdenFantasys is not actually providing reciprocity.

More information about EdenFantasys’s unethical practices, which are not limited to technical subterfuge, can be obtained via AAGBlog.com.

EDITORIAL

It is unsurprising that the distributed, subtle, and carefully crafted way EdenFantasys has managed to crowd-source links has (presumably) remained unpenalized by search engines like Google. It is similarly unsurprising that nontechnical users such as the contributors to SexIs Magazine would be unaware of these deceptive practices, or that they are complicit in promoting them.

This is no mistake on the part of EdenFantasys, nor is it a one-off occurrence. The amount of work necessary to implement the elaborate system I’ve described is also not even remotely feasible for a rogue programmer to accomplish, far less accomplish covertly. No, this is the result of a calculated and decidedly underhanded strategy that originated from the direction of top executives at Web Merchants, Inc. aka EdenFantasys.

It is unfortunate that technically privileged people would be so willing to take advantage of the technically uneducated, particularly under the guise of providing a trusted place for the community which they claim to serve. These practices are exactly the ones that “the sex shop you can trust” should in no way support, far less be actively engaged in. And yet, here is unmistakable evidence that EdenFantasys is doing literally everything it can not only to bolster its own web presence at the cost of others’, but to hide this fact from its understandably non-tech-savvy contributors.

On a personal note, I am angered that I would be contacted by the Editor of SexIs Magazine, and asked to properly “attribute” and provide a link to them when it is precisely that reciprocity which SexIs Magazine would clearly deny me (and everyone else) in return. It was this request originally received over email from Judy Cole, that sparked my investigation outlined above and enabled me to uncover this hypocrisy. The email I received from Judy Cole is republished, in full, here:

From: Judy Cole <luxuryholmes@gmail.com>
Subject: Repost mis-attributed
Date: May 17, 2010 2:42:00 PM PDT
To: kinkontap+viewermail@gmail.com
Cc: Laurel <laurelb@edenfantasys.com>

Hello Emma and maymay,

I am the Editor of the online adult magazine SexIs (http://www.edenfantasys.com/sexis/). You recently picked up and re-posted a story of ours by Lorna Keach that Alternet had already picked up:

http://kinkontap.com/?s=alternet

We were hoping that you might provide attribution and a link back to us, citing us as the original source (as is done on Alternet, with whom we have an ongoing relationship), should you pick up something of ours to re-post in the future.

If you would be interested in having us send you updates on stories that might be of interest, I would be happy to arrange for a member of our editorial staff to do so. (Like your site, by the way. TBK is one of our regular contributors.)

Thanks and Best Regards,

Judy Cole
Editor, SexIs

Judy’s email probably intended to reference the new Kink On Tap briefs that my co-host Emma and I publish, not a search result page on the Kink On Tap website. Specifically, she was talking about this brief: http://KinkOnTap.com/?p=676. I said as much in my reply to Judy:

Hi Judy,

The URL in your email doesn’t actually link to a post. We pick up many stories from AlterNet, as well as a number from SexIs, because we follow both those sources, among others. So, did you mean this following entry?

http://KinkOnTap.com/?p=676

If so, you should know that we write briefs as we find them and provide links to where we found them. We purposefully do not republish or re-post significant portions of stories and we limit our briefs to short summaries in deference to the source. In regards to the brief in question, we do provide attribution to Lorna Keach, and our publication process provides links automatically to, again, the source where we found the article. :) As I’m sure you understand, this is the nature of the Internet. Its distribution capability is remarkable, isn’t it?

Also, while we’d absolutely be thrilled to have you send us updates on stories that might be of interest, we would prefer that you do so in the same way the rest of our community does: by contributing to the community links feed. You can find detailed instructions for the many ways you can do that on our wiki:

http://wiki.kinkontap.com/wiki/Community_links_feed

Congratulations on the continued success of SexIs.

Cheers,
-maymay

At the time when I wrote the email replying to Judy, I was perturbed but could not put my finger on why. Her email upset me because she seemed to be suggesting that our briefs are wholesale “re-posts,” when in fact Emma and I have thoroughly discussed attribution policies and, as mentioned in my reply, settled on a number of practices including a length limit, automated back linking (yes, with real links, go see some Kink On Tap briefs for yourself), and clearly demarcating quotes from the source article in our editorializing to ensure we play fair. Clearly, my somewhat snarky reply betrays my annoyance.

In any event, this exchange prompted me to take a closer look at the Kink On Tap brief I wrote, at the original article, and at the cross-post on AlterNet.org. I never would have imagined that EdenFantasys’s technical subterfuge would be as pervasive as it has proven to be. It’s so deeply embedded in the EdenFantasys publishing platform that I’m willing to give Judy the benefit of the doubt regarding this hypocrisy because she doesn’t seem to understand the difference between a search query and a permalink (something any laymen blogger would grok). This is apparent from her reply to my response:

From: Judy Cole <luxuryholmes@gmail.com>
Subject: Re: Repost mis-attributed
Date: May 18, 2010 4:57:59 AM PDT
[…redundant email headers clipped…]

Funny, the URL in my email opens the same link as the one you sent me when I click on it.

Maybe if you pick up one of our stories in future, you could just say something like “so and so wrote for SexIs.” ?

As it stands, it looks as if Lorna wrote the piece for Alternet. Thanks.

Judy

That is the end of our email exchange, and will be for good, unless and until EdenFantasys changes its ways. I will from this point forward endeavor never to publish links to any web property that I know to be owned by Web Merchants, Inc., including EdenFantasys.com. I will also do my best to avoid citing any and all SexIs Magazine articles from here on out, and I encourage everyone who has an interest in seeing honesty on the Internet to follow my lead here.

As some of my friends are currently contributors to SexIs Magazine, I would like all of you to know that I sincerely hope you immediately sever all ties with any and all Web Merchants, Inc. properties, suppliers, and business partners, especially because you are friends and I think your work is too important to be sullied by such a disreputable company. Similarly, I hope you encourage your friends to do the same. I understand that the economy is rough and that some of you may have business contracts bearing legal penalties for breaking them, but I urge you to nevertheless consider looking at this as a cost-benefit analysis: the sooner you break up with EdenFantasys, the happier everyone on the Internet, including you, will be (and besides, you can loose just as much of your reputation, money, and pagerank while being happy as you can being sad).

What you can do

• If you are an EdenFantasys reviewer, a SexIs Magazine contributor, or have any other arrangement with Web Merchants, Inc., write to Judy Cole and demand that content you produce for SexIs Magazine adheres to ethical Internet publication standards. Sever business ties with this company immediately upon receipt of any non-response, or any response that does not adequately address every concern raised in this blog post. (Feel free to leave comments on this post with technical questions, and I’ll do my best to help you sort out any l33t answers.)
• EdenFantasys wants to stack the deck in Google. They do this by misusing your content and harvesting your links. To combat this effort, immediately remove any and all links to EdenFantasys websites and web presences from your websites. Furthermore, do not—I repeat—do not publish new links to EdenFantasys websites, not even in direct reference to this post. Instead, provide enough information, as I have done, so visitors to your blog posts can find their website themselves. In lieu of links to EdenFantasys, link to other bloggers’ posts about this issue. (Such posts will probably be mentioned in the comments section of this post.)
• Boycott EdenFantasys: the technical prowess their website displays does provide a useful shopping experience for some people. However, that in no way obligates you to purchase from their website. If you enjoy using their interface, use it to get information about products you’re interested in, but then go buy those products elsewhere, perhaps from the manufacturers directly.
  • On the recommendation of my friend Dr. Charlie Glickman, I suggest Good Vibrations.

  • On the recommendation of my friend Shanna Katz, I also recommend Fascinations.
• Watch for “improved” technical subterfuge from Web Merchants, Inc. As a professional web developer, I can identify several things EdenFantasys could do to make their unethical practices even harder to spot, and harder to stop. If you have any technical knowledge at all, even if you’re “just” a savvy blogger, you can keep a close watch on EdenFantasys and, if you notice anything that doesn’t sit well with you, speak up about it like I did. Get a professional programmer to look into things for you if you need help; yes, you can make a difference just by remaining vigilant as long as you share what you know and act honestly, and transparently.

If you have additional ideas or recommendations regarding how more people can help keep sex toy retailers honest, please suggest them in the comments.

Update: To report website spamming or any kind of fraud to Google, use the authenticated Spam Report tool.

Update: Google provides much more information about why the kinds of practices EdenFantasys is engaged in degrade the overall web experience for you and me. Read Cloaking, sneaky Javascript redirects, and doorway pages at the Google Webmaster Tools help site for additional SEO information. Using Google’s terminology, EdenFantasys’s unethical technology is a very skilled mix of social engineering and “sneaky JavaScript redirects.”

During testing, I discovered that the odd-ball task failed to run, and found the following error in the system log:

sudo: sorry, you must have a tty to run sudo

I traced this error to a line trying to invoke a perl command as a user called dynamic:

sudo -u dynamic /usr/bin/perl run-periodic-tasks --load 5 --randomly

A simple Google search turned up an obvious solution to the error: use visudo to disable sudo’s tty requirement, allowing sudo to be invoked from any shell lacking a tty (including cron). This would have solved my problem, but it just felt wrong, dirty, and most troublingly insecure.

One reason why sudo ships with the requiretty option enabled by default is, among other reasons, to prevent remote users from exposing the root password over SSH. Disabling this security precaution for a simple maintenance task already running as root seemed totally unnecessary, not to mention irresponsible. Moreover, client‘s script didn’t even need a tty.

Thankfully, there’s a better way: use su --session-command and send the whole job to the background.

su --session-command="/usr/bin/perl run-periodic-tasks --load 5 --randomly" dynamic &

This line launches a new, non-login shell (typically bash) as the other user in a separate, background process and runs the command you passed using the shell’s -c option. Sending the command to the background (using &) continues execution of the rest of the cron job.

A process listing would look like this:

root     28109     1  0 17:10 ?        00:00:00 su --session-command=/usr/bin/perl run-periodic-tasks --load 5 --randomly dynamic
dynamic  28110 28109  0 17:10 ?        00:00:00 bash -c /usr/bin/perl run-periodic-tasks --load 5 --randomly

Note the parent process (PID 28109) is owned by root but the actual perl process (PID 28110) is being run as dynamic.

This in-script solution that replaces sudo -u user cmd with su --session-command=cmd user seems much better than relying on a change in sudo‘s default (and more secure) configuration to me.

One reason why you might use git-svn because your project actually resides in a Subversion repository and other people need to access it using Subversion-only tools. Another might be because you have multiple projects, some that use git and others that use Subversion, and you’re tired of switching between svn and git commands—like me. For us, it’s far easier to simply use git as a Subversion client and never have to call svn directly.

As an important aside, please note that I would strongly discourage people who are new to git from learning about it by using git-svn. Although you may think that moving to git from Subversion would be eased by using the git-svn bridge, I really don’t think that’s the case. You’re much, much better off simply using git by itself right off the bat, and you can do this even if your fellow committers are using subversion.

Also, I’m going to assume you’ve already got a Subversion repository set up somewhere.

First, checkout the subversion repository. In Subversion you would do this:

svn checkout http://example.com/path/to/svn/repo

With git-svn, you do this:

git svn clone http://example.com/path/to/svn/repo

This will cause git-svn to create a new directory called repo, switch to it, initialize a new git repository, configure the Subversion repository at http://example.com/path/to/svn/repo as a remote git branch (confusingly called git-svn by default, although you can specify your name by passing a -Rremote_name or --svn-remote=remote_name option), and then does a checkout.

The output of this command will be a little awkward. Here’s a sample from one my repositories:

r14 = dbd7266f328ef2ad061ea4532f39ce7cebaba0c5 (git-svn)
	M	trunk/Chapter 6/Chapter 6.doc
	M	trunk/Chapter 6/code examples/6.1.html
	A	trunk/Chapter 6/code examples/6.2.html
r15 = 4cca08341ab0600069cece77ce67afc449caca68 (git-svn)
	M	trunk/Chapter 6/Chapter 6.doc
	A	trunk/Chapter 6/code examples/print.css
	A	trunk/Chapter 6/code examples/screen.css
	M	trunk/Chapter 6/code examples/6.1.html
	M	trunk/Chapter 6/code examples/6.2.html
r16 = 7b2f3e0ccfd79be61b527b6ba325f8689475dc01 (git-svn)
	M	trunk/Chapter 5/Chapter 5.doc
r17 = a319764855361d92bb6e006cfd18a51319046cae (git-svn)
	M	trunk/Chapter 5/Chapter 5.doc
r18 = 4cd5cb43d33b2dd45bd39b9a2b7ea9416f9e3d8f (git-svn)
	M	trunk/Chapter 6/Chapter 6.doc
	M	trunk/Chapter 6/code examples/screen.css
	M	trunk/Chapter 6/code examples/6.1.html

As you can see, git-svn is associating specific Subversion revisions with particular git commit objects. Due to this required mapping, the initial cloning process of a Subversion repository may take some time. This is a good opportunity for your morning coffee break.

When this process is done, you’ll have a typical git repository with a local master branch and one remote branch for the Subversion repository:

Perseus:repo maymay$ git branch
* master
Perseus:repo maymay$ git branch -r
  git-svn

You can now treat the Subversion repository as though it were a remote branch of sorts. Say you’ve done a bunch of work and, as you typically do with git, you commit this work to your topic branch.

Perseus:repo maymay$ git checkout -b awesome-feature
Switched to a new branch "awesome-feature"
Perseus:repo maymay$ vim awesome-feature-stylesheet.css
Perseus:repo maymay$ git add awesome-feature-stylesheet.css
Perseus:repo maymay$ git commit -m "Now I'm perty."
Created commit 07ee832: Now I'm perty.
 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 100644 awesome-feature-stylesheet.css

Right now your changes are still in the topic branch (called awesome-feature in the above example). To get them to Subversion, you merely need to say git svn dcommit:

Perseus:repo maymay$ git svn dcommit
Committing to http://example.com/path/to/svn/repo ...

Note that pesky extra “d” in the command. This is the equivalent of Subversion’s svn commit, but the commit message used is the one from the previous command, which in this case was git commit -m "Now I'm perty.". Also interesting to note here is that because Subversion doesn’t understand git branches, any change on any branch can be “pushed” to Subversion at any time using git svn dcommit—the git commits don’t have to be on any specific branch, since all git-svn does is map a git commit object to a Subversion revision and vice versa.

Similarly, you can at any time run the equivalent of svn update to get the latest changes from the Subversion repository into your Subversion branch.

• To do this, without affecting your working tree—that is, to only fetch the latest changes but not write them to the filesystem, just to the git-svn metadata area and the remote git branch—use git svn fetch. To apply these changes to your local branch, you simply merge: git checkout master; git merge git-svn.
• If you do want to write out the changes to the filesystem (as svn update would do), use git svn rebase, which automatically linearizes your local git commit history after the commit history of the incoming Subversion changesets. Very slick.

If your fetching/rebasing causes a conflict, you’ll be notified and will have to resolve it as per usual. If your “pushes” to the svn repo causes a Subversion conflict, you’ll be notified and you should again edit the appropriate files to resolve it, but this time make sure you run a git svn rebase before you try dcommit-ing again (since, remember, Subversion can only handle linear commit history).

As always, saying man git-svn or git help svn to your shell will give you all the other details. Among these, the most likely you’ll probably want to learn about is how to track multiple Subversion branches as normal git branches.

Clickjacking does not rely on bugs in any software. Instead, the technique is simply an abuse of the growing graphical capabilities that advanced web standards like CSS provide to web browsers. A good introduction to clickjacking is provided by Steve Gibson and Leo Laporte on their Security Now! podcast.

As far as I’m aware, only Firefox when combined with the NoScript add-on and Internet Explorer when combined with the GuardedID product provide any measure of protection against clickjacking attacks. To date no other browser can detect, alert, or otherwise help you to avoid or mitigate the risks of clickjacking attacks.

That said, there’s gotta be something users of other browsers can do. Well, it may not be as much as what NoScript can do, but there is something: use a user style sheet to help expose common clickjacking attack attempts.

clickjane.css helps detect clickjacking attacks for all browsers

Until browser manufacturers provide built-in protections against clickjacking attacks in their software (which is arguably the best place for such logic in the first place), I’ve started putting together a user style sheet I’m calling clickjane.css that attempts to instantly reveal common clickjacking attempts. Since it’s a CSS user style sheet, this approach should be cross-browser compatible so that users of any browser including Safari, Opera, and other browsers that don’t have other means of protecting against clickjacking attacks can use it.

I’ve only recently learned about this class of exploits and so I’m not supremely well-informed on the topic. As a result, the clickjane.css file is relatively sparse and currently only reveals what I’m sure is a small set of clickjacking attmpts. However, as I research the topic further and learn more about the actual underlying HTML and CSS that clickjacking uses, I’ll be updating the clickjane.css code to reveal those attempts as well.

Naturally, contributions and assistance in any form are most welcome! Learn more about clickjane.css as well as how to use it at the Clickjane CSS Github wiki.

Before and after clickjane.css

Here are two example screenshots of a benign clickjacking demo.

1. Before:
   

   Screenshot of Safari before clickjane.css is used to expose clickjacking attempts.

2. After:
   

   Screenshot of Safari after clickjane.css is used to expose clickjacking attempts.

Good habits you should get into to mitigate clickjacking risks

Here is a list of behaviors that you should make habitual while you browse the web. Engaging in these behaviors can dramatically reduce the likelihood that you will be victimized by a clickjacking attack.

• Explicitly log out of any service you have logged in to when you are done. That log-out button is there for a reason: use it!
• Avoid providing your browser with “Auto-Complete” information for critical sites, such as your bank.
• Make sure you are running Flash Player 10 or greater, which mitigates this vulnerability for Adobe Flash content.

More resources to learn about clickjacking

• Hackademix.net – More clickjacking links to the OWASP presentation, the white paper, and a blog post showing several CSS-based exploits.

Translations of this article:

• Belorussian (thanks, Bohdan Zograf)
• Ukrainian (thanks, Alyona Lompar)

Here’s the summary of my argument.

Adding many “programmatic” features to the CSS language such as variables, macros, or flow control is a mistake. However, CSS‘s failure to simply encode visual relationships (instead of merely typographic properties)—a severe deficiency in the core language itself—requires the addition of delegation features. With the additional capability to reference an arbitrary element’s computed value regardless of its hierarchical context, CSS will be more accessible to both amateur and professional web designers, more capable, and will more forcefully promote the semantic Web and its ideals.

In this corner: CSS variables are harmful

Bert does a great job of summarizing the conclusion of his argument himself. In his essay, Bert says:

Adding any form of macros or additional scopes and indirections, including symbolic constants, is not just redundant, but changes CSS in ways that make it unsuitable for its intended audience. Given that there is currently no alternative to CSS, these things must not be added.

As we all know, one of the wonderful things about CSS is that the core language itself is remarkably simple. (What’s not simple is the spectacular way browser manufacturers have destroyed everyone’s hope that implementing CSS-based designs in the real world will ever be easy, but that’s a whole different can of worms.) Fundamentally, CSS‘s syntax can be explained with a mere three major components: property/value pairs, declaration blocks, and rule sets.

What this means is that CSS as a language is stupidly easy to learn. I think everyone would agree that it’s certainly easier to learn than, say, JavaScript or XSL. Now, that’s important because, without putting too fine a point on it, Bert mentions multiple times that CSS‘s “intended audience” are the diverse and likely relatively technically ignorant content authors that are responsible for the overwhelming majority of web pages on the public Internet today.

He makes the very good point that The value of the semantic Web isn’t defined by how well structured the best documents are, but by how well structured the vast majority of documents are. In other words, CSS needs to remain instantly useable and reusable to these untrained, amateur web content publishers for the benefits of self-describing documents (i.e., the semantic Web) to see mass adoption.

To wit:

reusing other people’s style sheets is more difficult if those style sheets contain user-defined names. Class names are an example. Their names may suggest why the author created them (assuming they are in a language you understand), but typically you will have to look at the document to see where they occur and why. Symbolic constants make that problem worse.

And, later:

For many people, style sheets with constants will thus simply not be usable. It is too difficult to look in two places at once, the place where a value is used and the place where it is defined, if you don’t know why the rule is split in this way. Many people are confused by indirection anyway and adding an extra one, in addition to the element and class names, has the same effect as obfuscating the style sheet.

Whether or not you believe Bert Bos is underestimating the average web designer, it’s pretty clear that these are really good points. Nobody wants CSS to be obfuscated, hard to learn, or hard to reuse. That’d just be crazy talk.

In the other corner: CSS variables are a real-world requirement

The more features you add to an application, a programming language, or indeed any software, the more difficult it becomes to grok it. As the Python people would say, the larger a language gets the more difficult it is to hold all of it in your head. Nevertheless, adding “features” is sometimes the only way to add capabilities, and I don’t think anyone in their right mind would argue that, once written, software should never change. (That’d just be crazy talk, too.)

In his opposing arguments, Matt Wilcox recognizes this when he says, Yes, the syntax should be simple, but the capabilities of CSS should not. What he’s alluding to without verbalizing it is the balance between adding necessary capabilities without unnecessarily growing the “size of the language.”

However, Matt says that modern web design methodologies (e.g., separation of concerns between structure, presentation, and behavior) dictate that CSS needs more capabilities than it currently has:

CSS lacks capabilities to allow truly flexible design, requiring layer upon layer of ‘tricks’ to accomplish certain objectives, requiring content to be structured ‘just so’ to achieve a display objective, or in the case of some designs proving instead to be completely incapable.

[…]

CSS’s positioning is a cludge. It’s a cludge because you can only position relative to the last positioned parent container. Well, that limitation in itself dictates that all positioning relies upon how the content is structured. And that means the presentation and the content are not truly separable.

To align CSS‘s capabilities with the requirements of real-world web design objectives, he says, CSS needs to be capable of describing relationships between semantically and structurally arbitrary but visually related elements.

Visual design is fundamentally about relationships between elements. For all of the artistic flourishes and creativity, it’s about relationships. ‘That yellow’ only grabs your attention because of its contrasting relationship with ‘that blue’. ‘This heading’ only works as a heading because of it’s exaggerated relationship to the size of the body text. […] CSS has no clue about relationships, period. And that’s why CSS as it stands right now, is not good enough. That’s why CSS without variables (true variables), without basic logic, without maths, can never be as flexible as we need it to be.

This is what web designers have been complaining about for (what feels like hundreds of) years. The fact that CSS has no capability to describe presentational relationships between elements in addition to directly describing an individual element’s presentational properties is a gaping hole that sorely degrades its ability to be a media-agnostic styling language. Every single web designer I’ve worked with has gasped at this omission, and though at first I didn’t understand why, the more I understood the principles behind graphic design the more I came to realize how fundamentally problematic this omission really is.

Adding delegation makes CSS easier for designers

As Matt eloquently stated, design is all about relationships. Good web designers create designs by constructing visual elements that have strong, often exacting relationships with other visual elements. There are many names and examples for this: visual language, visual hierarchy, the golden ratio, the grid, visual balance, the typographer’s scale, and so on.

What happens when the designer tries to define a relationship between elements? “How do I say that the whitespace between element A and element B should always be the same? How do I define element A’s height as half of element B’s?” These definitions, which are natural and necessary to the way designers work in both their mind and their mediums, are impossible to encode in CSS.

The closest you can get is declaring the same values to each element’s properties, not describing the relationship itself. This suffices only so long as these values are known ahead of time and are the same as one another, which severely limits the design possibilities we are capable of (without resorting to what Matt calls “tricks”). That’s why achieving simple visual effects are actually very complex and so, sadly, that’s where you’ll find the majority of indirection and obfuscation in CSS today. (I’m looking at you, faux columns.)

So who wins?

Both Bert Bos and Matt Wilcox have made some great points. Bert rightfully wishes to keep CSS lean and simple, even at the expense of some arguably beneficial styling power. Matt, on the other hand, argues that our needs as web designers have evolved faster than the technology to the point where CSS is too limited, fundamentally so.

The truth is, they’re both right. And they’re both wrong. Or rather, they are each taking a position that is too extreme. Bert’s absolutely correct when says that many of these proposed extensions are redundant and harmful, and yet Matt’s also correct that CSS lacks some fundamental capabilities that designers expect to be present.

Bert says that the CSS capabilities everyone’s asking for can be implemented using techniques that don’t rely on CSS whatsoever. These techniques, he says, make things like true CSS variables “redundant.”

There are examples of CSS with constants to satisfy all styles of programming, e.g.: David Walsh (in PHP), Tedd Sperling (in PHP), Digital Web Magazine (in PHP), Eco Consulting (in SSI), and Christian Heilmann (SSI and PHP).

Quite simply, he’s correct in stating that programmatic features need not be added to CSS proper to achieve desired results, but he’s incorrect in his apparent thinking that designers will be able to use these other tools to leverage CSS. Take, for instance, the probably more familiar (though not linked above) notion of using JavaScript to manipulate CSS values.

var x = document.getElementById('SideBar'); // get #SideBar element
var y = document.getElementById('MainColumn'); // get #MainColumn
var z = document.defaultView.getComputedStyle(y, '').getPropertyValue('height'); // get computed height of #MainColumn
x.style.height = ( parseInt(z) / 2 ) + 'px'; // set #SideBar's height 1/2 of #MainColumn's

This is an example of programmatic code that uses variables and expressions. It sets the element with the ID of SideBar to half the pixel height of the element with the ID of MainColumn. It does this by obtaining the MainColumn‘s height (at the time this code runs) and saving it in a variable, then performs some trivial math to half the value and use the result as the pixel height of the SideBar.

Doing this is currently impossible with CSS alone, yet it’s something that clearly belongs with whatever other “presentational” code exists and not in “programmatic” scripts that would otherwise be charged with defining “functionality.” As Matt states, using JavaScript to “script” solutions to CSS‘s shortcomings like this is not an acceptable answer.

CSS doesn’t have [basic logic or maths]. Nor is it the job of JavaScript to make up for this lack of abilities. JavaScript is about interaction behaviour, and what we are talking about here is pure display logic. Not interaction logic.

Moreover, the place designers expect to put code like this is, of course, into a CSS style sheet. The way designers expect to put code like this into CSS is by adding delegation features. Requiring designers to learn JavaScript (or any other programming language) to encode such design relationships is nothing short of ridiculous. In what world is that easier for untrained laymen to understand than CSS?

Adding delegation to CSS is worth the effort

One of Bert’s arguments against such additions to CSS is that implementations would become harder to create, and that we’ll (almost certainly) see more bugs.

extending CSS makes implementing more difficult and programs bigger, which leads to fewer implementations and more bugs. That has to be balanced against the usefulness of the extension.

Although I do agree with his statement that an extension’s usefulness has to be balanced against its potential costs, I think something so fundamental to design methodology as delegation greatly overcompensates for the cost of such implementation efforts. Moreover, if I understand Bert correctly and as he also discusses, the majority of implementations that would need to implement such delegation already have relatively complex internal structures to make the implementation effort somewhat easier:

There is no scoping [in proposals that only define global constants]. That means that an implementation needs a symbol table, but no stack. A stack would require a little bit more memory, but mostly it would make implementations more complex. (Although every programmer has, one hopes, learnt to program a symbol table with lexical scope during his training.) Constants in CSS are thus easier than, e.g., XML Namespaces, which are lexically scoped.

It is different for those CSS implementations that provide a CSS Object Model (an API for manipulating a style sheet in memory). Those implementations do need to keep track of scope in some way, because adding or removing a line of the style sheet can make a previously redundant definition become meaningful.

In order to use JavaScript to solve many of the shortcomings of CSS, as huge numbers of professional web developers do routinely, we use the very CSS Object Model whose prior implementation already exists for us to build upon.

CSS delegation doesn’t grow the size of the language

For the sake of argument, let’s simplify our requirement somewhat so that our somewhat contrived example of design intent is to create a relationship between the MainColumn and the SideBar elements such that they are of equal height. This is more informally known as “making columns.”

Here’s what a natural, hypothetical snippet of CSS would look like if the language supported delegation features such that it could encode visual relationships.

#SideBar { height: #MainColumn; }

This code theoretically says almost the exact same thing as the JavaScript shown earlier (save for the division, of course); it takes the computed value of the MainColumn element’s height property and applies that value to the SideBar element’s height property. In other words, “The SideBar’s (element B’s) height is always the same as the MainColumn’s (element A’s).” (Of course, this is a parse error in reality today.)

This extremely trivial example has some remarkably far-reaching implications, and yet there is really nothing radical about its syntax. Making this a reality significantly expands the capabilities of CSS without dramatically increasing the size of the language. This capability would not only beat the pants off “CSS tables,” it also potentially obsoletes the arguably misguided efforts of the CSS3 Advanced Layout and Grid Positioning modules, too.

We’ve long since abandoned table layouts because they force us to use presentational markup. That’s still what “CSS tables” force us to do, too. In other words, with display: table, the SideBar needs to be a child of the MainColumn element or, maybe worse and more likely, a child of a semantically meaningless wrapper element.

CSS positioning was introduced with the promise of freeing us from source-order-dependent styling, without which there is no hope of efficiently abstracting presentation away from structure. Moreover, abstracting presentation away from structure is the single most important prerequisite needed to improve document reusability and strengthen the semantic Web. Absolute positioning works, but limitations elsewhere in CSS mean its use is problematic for many designs, so in practice it doesn’t gain widespread adoption.

Here’s a theoretical solution to a two-column and a footer layout using CSS delegation with this semantic HTML:

<body>
    <div id="MainColumn">I'm the main column.</div>
    <div id="SideBar">I'm the right-hand sidebar.</div>
    <div id="Legalese">No one will read me.</div>
</body>

The CSS would look extremely familiar, possibly like this:

#MainColumn { margin: 0 25% 1em 0; float: left; }
#SideBar { width: 25%; min-height: #MainColumn; }

Using the same HTML, the same solution using the CSS3 Advanced Layout module would look something more like this, although to be frank I’m not certain I fully understand this syntax even after staring at it for months:

body {
    display: "a  b"
             ".  ." /1em
             "c  c"
             75% 25%
}
#MainColumn { position: a; }
#SideBar { position: b; }
#Legalese { position: c; }

Not only does there seem to me to be far more indirection in this method than there would be using CSS delegation, there is also an enormous increase to the size of the CSS language: a new (ASCII-art?!) value to the display property whose syntax is clunky at best. A similar story can be said of the CSS3 Grid Positioning module, which does lots more than just add a new (already complex) gr CSS unit.

The upshot is that the Advanced Layout and the Grid Positioning modules are doing some of the right things in many of the wrong ways. Both those modules add unnecessary complexity to CSS without giving designers a natural way to say what they mean. They do more to introduce obfuscation and indirection than simple delegation would, and they aren’t as broadly capable. Both of them try to solve a specific problem instead of dealing with fundamental deficiencies in the toolset designer’s have to work with.

Designers want relationships via delegation, not variables

Adding delegation such as that I’ve just shown is a natural, necessary addition to CSS because it is how designers create visual components—such as grids—in their designs. Variables (and constants, and macros, etc.), which simply reuse and modify pre-defined statements aren’t what designers care about. Adding them will bloat CSS without adding useful functionality.

“Okay,” you may be saying to yourself, “but delegation is itself a kind of variable, isn’t it?” Technically yes, however adding delegation resolves the core deficiency in the CSS language that designers need to use every day. Yes, it’s technically a form of variable, but that’s not how designers think of it. To say that one element’s visual properties is like another makes a variable only by creating a logical and visually appropriate mapping from the first element’s property to the second independent of markup, thereby avoiding indirection in the form of a variable name or other unfamiliar symbol.

Delegation like this doesn’t require the addition of anything other than what already exists in CSS. Class names and ID values are identifiers whose indirection people already have to deal with. Using them for delegation (to reference another element’s style) doesn’t increase the cognitive load any more than using them to reference HTML elements does. Though untested, the cognitive load might actually be even less since the CSS delegation’s references could be in the same (style sheet) file.

Moreover, delegation will increase the likelihood of document reusability by enabling style sheets to be more self-describing, more self-referential, in a similar way as good markup is. It satisfies a very fundamental need that designers have to define graphical relationships between elements. At the same time, it does so in a way that is natural to both their way of thinking and beneficial to the separation of concerns principle on which the “web stack” (the trifecta of HTML, CSS, and JavaScript) is based.

I spent about an hour looking into this this afternoon and it turns out that much of this information is readily available on the filesystem in the Apple WikiServer’s data store. I whipped up the following shell script to extract this information in CSV format, exactly as requested.

I’m posting this script here in case someone else wants similar “export a list of WikiServer pages to a comma-separated values (CSV) file” functionality but isn’t sure how to go about getting it. To use this, just edit the line that reads http://my-server.example.com/groups/wiki/ so that it refers to the wiki base URI of your own server.

Update: The latest version of this script is now available at its Github-hosted repository. You should probably use that instead of the script below.

#!/bin/sh -
#
# Script to extract data from an Apple WikiServer's data store by querying the
# filesystem itself. Creates a 'wikipages.csv' file that's readable by any
# spreadsheeting application, such as Numbers.app or Microsoft Excel.app.
#
# USAGE:   To use this script, change to the WikiServer's pages directory, then
#          just run this script. A file named wikipages.csv will be created in
#          your current directory. For instance:
#
#              cd /Library/Collaboration/Groups/mygroup/wiki  # dir to work in
#              wikipages2csv.sh                               # run the script
#              cp wikipages.csv ~/Desktop                     # save output
#
# WARNING: Since the WikiServer's files are only accessible as root, this script
#          must be run as root to function. Additionally, this is not extremely
#          well tested, so use at your own risk.
#
# Author:  Meitar Moscovitz
# Date:    Mon Sep 22 15:03:54 EST 2008

##### CONFIGURE HERE ########

# The prefix to append to generated links. NO SPACES!
WS_URI_PREFIX=http://my-server.example.com/groups/wiki/

##### END CONFIGURATION #####
# DO NOT EDIT PAST THIS LINE
#############################

WS_CSV_OUTFILE=wikipages.csv
WS_PAGE_IDS_FILE=`mktemp ws-ids.tmp.XXXXXX`

function extractPlistValueByKey () {
    head -n \
      $(expr 1 + `grep -n "<key>$1</key>" page.plist | cut -d ':' -f 1`) page.plist | \
        tail -n 1 | cut -d '>' -f 2 | cut -d '<' -f 1
}

function linkifyWikiServerTitle () {
    echo $1 | sed -e 's/ /_/g' -e 's/&amp;/_/g' -e 's/&gt;/_/g' -e 's/&lt;/_/g' -e 's/\?//g'
}

function formatISO8601date () {
    echo $1 | sed -e 's/T/ /' -e 's/Z$//'
}

function csvQuote () {
    echo $1 | grep -q ',' >/dev/null
    if [ $? -eq 0 ]; then
        echo '"'$1'"'
    else
        echo $1
    fi
}

ls -d [^w]*.page | \
  sed -e 's/^\([a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]\)\.page$/\1/' > $WS_PAGE_IDS_FILE

echo "Title,ID,Date Created,Last Modified,URI" > $WS_CSV_OUTFILE
while read id; do
    cd $id.page
    title=$(extractPlistValueByKey title)
    created_date="$(formatISO8601date $(extractPlistValueByKey createdDate))"
    modified_date="$(formatISO8601date $(extractPlistValueByKey modifiedDate))"
    link=$WS_URI_PREFIX"$id"/`linkifyWikiServerTitle "$title"`.html
    cd ..
    echo `csvQuote "$title"`,$id,$created_date,$modified_date,`csvQuote "$link"` >> $WS_CSV_OUTFILE
done < $WS_PAGE_IDS_FILE
rm $WS_PAGE_IDS_FILE

For those new to the Wiki Server, this introduction to the Apple WikiServer for web developers may be of interest.

• parameter-based post limit to define how many posts the plugin function will print
• parameter-based format option to output the posts in real <li> elements

The wp-category-posts.php patch file is available for download here. To apply the patch, run the following commands at your shell promp:

cd path/to/wordpress/installation/wp-content/plugins/wordpress-category-posts
patch -p0 < path/to/downloaded/wp-category-posts.patch

I’m hoping this will get integrated as the next version of the plugin, perhaps version 2.1.

I find working without an accurate awareness of the situation at large very disorienting, and software and web development are notorious for being circumstances that change rapidly. That’s why one of my favorite things about Scrum is the burn down chart. This is nothing more complex than a simple graph that depicts how much work you bit off and how far along trying to chew it you actually are. The benefit, of course, is that it’s pretty obvious pretty quickly if you’ve bit off more than you can chew. ;)

So up ’til now, my team and I have been doing this all on paper. There’s a certain tactile appreciation I have for doing this sort of thing on paper, but of course there are disadvantages, too. For instance, you can’t easily archive the information. You can’t easily share it with remote contractors. You can’t automatically mine this valuable data with software tools. You get the picture.

There are a few cool plugins to some tools like Trac that do all this, but at first blush most of these tools seem to require that you move all of your Scrum’s planning into the digital world. That is, you can’t just do the burn down chart, you have to do all your estimation (MoSCoW desirability, sizing, estimating ideal hours) through some tool. That’s a big step, and I wanted something simpler.

So naturally, I came up with a spreadsheet in Numbers.app as my solution. I mean, how much simpler can you get? Sure, it’s not exactly “well integrated” with other tools, but your non-tech-savvy boss will probably love it, and AppleScript can be used to automate data extraction. Here’s what it looks like:

An example Scrum-style burn down chart in Apple's iWork '08 Numbers spreadsheeting application, complete with an actual chart.

(Click the screenshot to get a full-size view.)

As you can see, the Numbers sheet is a simple table and a line chart. I’ve embedded instructions for how to use the chart into the example itself, which I’ll quote here:

This is a sample Scrum-style iteration burn down chart for software development created by Meitar Moscovitz. It can be used to plot a team’s progress throughout a development cycle (aka. “iteration” or “sprint”). This sample chart depicts a 3-week iteration (15 working days) with a 150-point target goal.

The X-axis represents time, and is thus labelled Time in Days, while the Y-axis represents the work to be completed, and is labelled Points.

The green line shows the team’s ideal velocity based upon the total number of points—termed the Remaining Initial Value—scheduled for completion in the graphed iteration.

The blue line shows the team’s actual velocity (or “completed work”), which is entered by the team leader (aka. Scrum Master) after each day in the Done column.

To use this chart: duplicate this sheet, enter your iteration’s total points in the Initial Value row of the Remaining column, and delete the values in the Done column except its initial value of 0. To add more days, copy and paste more rows into the table. Optionally, give the sheet and its contents new titles. ;)

Feel free to download the Example Burn Down Chart.numbers file and use it yourself. If you do use it, please leave a comment and let me know how you’re going. Thanks, and enjoy!

(Mike Cohn of Mountain Goat Software has got a similar spreadsheet for Excel you can download.)

cd ~/                          # change to home directory
test -d ~/src || mkdir ~/src   # if there isn't already a ~/src directory, create it
cd ~/src                       # then change to that directory
curl -O http://www.kernel.org/pub/software/scm/git/git-1.5.6.4.tar.gz # download
tar -xvzf git-1.5.6.4.tar.gz   # and extract the git source code
cd git-1.5.6.4                 # change to the source code directory
./configure --prefix=$HOME     # configure build to install into $HOME
make                           # do the build
make install                   # move the built binaries to the right places
echo "export PATH=\$PATH:$HOME/bin" >> ~/.bashrc # make sure non-interactive shells can find git

Of special note is the last line, which sets up the necessary $PATH specifically for non-interactive bash shells for use with git-push or git-pull. With out that, you’ll run into the infamous “bash: git-receive-pack: command not found” error.

Also, of course, lines 4 through 6 are referring to version 1.5.6.4 of the git tarballs, so you may want to change these to point to whatever is now the most recent version.

If you’re not afraid of the Terminal, this can be avoided. Put the following lines in your ~/.profile (or similar):

alias rmds='find . -name ".DS_Store" -type f -print0 | xargs -0 rm'

What this does is creates a new command that you can use (rmds) which recursively finds and deletes any regular file named “.DS_Store” starting from the current directory. Thus, running this command in the folder you are about to create an archive out of will clean it first, and will prevent unnecessary confusion on the part of your archive file recipient.

Alternatively, another way to do this is to use the command-line zip program and an (admittedly more complicated) pipeline to remove the .DS_Store files after they have been added to the archive. To do that, use this series of commands:

zip -d ZIPfile.zip `unzip -l ZIPfile.zip | grep .DS_Store | awk '{print $4}'`

where, naturally, ZIPfile.zip is the ZIP archive you want to remove the .DS_Store files from. Creating an alias out of that command (and making it work for paths that contain spaces) is left as an exercise for the reader. ;)

As an aside, the alias, find and xargs commands are incredibly useful in their own right and can be used to do a lot of pretty amazing things. As always, man command will give you the nitty gritty.

Also as an aside, you can stop the Finder from creating .DS_Store files entirely when browsing network volumes (like Windows shares) with another command, documented in Apple’s Knowledge Base.

designing for the web is a lot less about making something dazzle and a lot more about making it work. The design decisions that matter pertain directly to the constraints of the materials. What form elements to use. What font sizes. What composition. What flow. Those decisions are poorly made at an arm’s length.

I’ve worked with many web designers in the past who only did abstractions and then handed over pictures to be chopped and implemented by “HTML monkeys”. It never really gelled well. The things that got strong attention were all the things that Photoshop did well. Imagery, curvy lines, and the frame. All the around stuff, never the it stuff.

In other words, to do great web design you have to design in the Web, not in some other medium for the Web. I mean, serious magazine firm employs designers who don’t understand how to work with page layout programs like InDesign. Why, then, do so many web design agencies employ designers who don’t know how to work with web technologies, or even how to use programs like Dreamweaver? It doesn’t really make any sense, and it’s no wonder that the resulting implementation is rarely top-notch work.

But if you’re a graphic designer who doesn’t know much about Web technologies, what are you to do? Well, as a first step, I think you should pick up my new book, Foundation Website Creation. It’s available from all good booksellers (and probably some crappy ones) as of today. The book is targeted towards all manner of web professionals, including graphic designers and website producers, who want to learn more about what it takes to actually implement a site.

If I do say so myself, the chapters on XHTML and CSS are exceptionally thorough. The book doesn’t try to turn you into an exceptional programmer. Instead, it will explain the foundational concepts you need to know to understand how XHTML and CSS actually work, and in so doing will enable you to use the tools you already know to solve problems and get things done.

I think this book will be an excellent starting point for lots of designers and other web professionals. However, it is not going to take you from zero to hero—no book can. That’s why I recommend that, after you read Foundation Website Creation and have a solid grasp of what the technology can do for you and how it actually does it, you next take a look at these excellent books:

• DOM Scripting by Jeremy Keith — if you’re a designer that needs to add a behavioral layer with JavaScript and Ajax to your pages, you need to read this book next.
• Mastering CSS with Dreamweaver CS3 – if you’re familiar with Dreamweaver and want to keep using it to create standards-based web sites, then I recommend you follow Foundation Website Creation with this book by Stephanie Sullivan and Greg Rewis to take your Dreamweaver skills to the next level.

As always, most of all, have fun. Because if you’re not having fun, you’re not going to make good web sites no matter what you know.

Note: As of this writing, the book listing on Amazon still publishes the wrong author list, which is very frustrating but out of my hands. At least the image of our book’s front cover lists the correct authors.

Transmitting file data ..svn: Commit failed (details follow):
svn: Checksum mismatch for '/Users/maymay/Sites/path/to/subversion/working/copy/.svn/text-base/working-file.php.svn-base'; expected 'cde4d8fbd5c623da3a8b1a343aa7b3f4', actual: '270b2f20804a5fcdbd15eec5910f6e3f'

Of course, the path/to/subversion/working/copy bit was the path to my working copy file’s parent directory and the working-file.php was an actual file in my working directory.

I think what Subversion was trying to tell me is that its hashed copy of the working-file.php file and the copy I was asking it to commit weren’t the same. It would be nice if it would actually tell me why that happened, but it’s clearly more temperamental than that.

Anyway, to fix this issue (at least for now…?) I simply checked out a new working copy of this directory, examined the .svn/entries file from it and sure enough, found the actual checksums in there, just as Subversion reported expecting. I simply copied those expected checksums into the .svn/entries overwriting the old actual checksums and, voila, Subversion has been fooled. After that, I could commit my changes.

Step by step (because I’m sure someone, somewhere, somehow, will run into this again—if it’s not me that is!), this procedure looked like this:

1. Copy the “expected” and “actual” checksums Subversion reports to you to a new text file so you can refer to them later. Note which one is the expected and which is the actual checksum.
2. Go to where the problem is (that is, cd path/to/broken-files-parent-dir/.svn)
3. Open the entries for editing (for example, vim entries)
4. Search the file for the actual checksum.
5. Replace it with the expected checksum. Be careful not to change any other part of the file.
6. Save the file.
7. Try to svn commit again.
8. Lather, rinse, and repeat for any other files Subversion barfs at you about.

I’m sure this is not an elegant or even the recommended solution to this problem. The truth is I never bothered to look up what the recommended solution is, because it seems to me that any code repository that can’t guarantee what I get out of it is the same as what I put into it isn’t a versioning system I really want to trust the “recommended” solution of, anyway.

Also known as: this is another reason why I like git better now.

In JavaScript, this is thankfully pretty trivial. My solution is to just parse the version strings with a simple function and return them as objects with appropriate properties whose values are integers. Once in this form, we can compare them with simple math.

function parseVersionString (str) {
    if (typeof(str) != 'string') { return false; }
    var x = str.split('.');
    // parse from string or default to 0 if can't parse
    var maj = parseInt(x[0]) || 0;
    var min = parseInt(x[1]) || 0;
    var pat = parseInt(x[2]) || 0;
    return {
        major: maj,
        minor: min,
        patch: pat
    }
}

Using this new object, we can now compare two versions really simply:

var running_version = parseVersionString('3.5.2');
var latest_version = parseVersionString('3.4.5');
if (running_version.major < latest_version.major) {
    // A major new update is available!
} else if (running_version.minor < latest_version.minor || running_version.patch < latest_version.patch) {
    // A new minor or patch update is available.
} else {
    // We are running the latest version! No need to update.
}

I came across Vaibhav’s post on the topic and noted that his solution uses the query_posts() function to alter WordPress’s query object before The Loop has run. While this is a great solution if your exclusion criteria is simple enough to be supported directly by the WordPress query object, other times the query_posts() function doesn’t provide you with the hook you need.

In these cases, you can run the original query, note any modifications you need to make, and then create a new, modified query and display the results you get from running that one instead. For instance, you might need to do this if you need to exclude posts based on category and, say, the beginning of their title, or their category and a certain piece of content in the post itself, or all three, or any other combination you can think of.

Another advantage of this technique over simpler ones is that this method maintains the same behavior you’d expect to see in every other way. Most notably, this means that if you’ve told WordPress to display the 10 most recent posts on the home page (in the WordPress settings), you’ll still see ten posts on that page even after you exclude some of them.

To do something like excluding posts if they are in the “Uncategorized” category (traditionally the category with an ID of 1 in WordPress) and their title begins with “Some title”, you can do this:

// original query runs in The (real) Loop first
while ( have_posts() ) : the_post();
    // detect pots matching our exclusion criteria
    if (in_category(1) && (0 === strpos(the_title('', '', false), 'Some title')) ) {
        $wp_query->post_count++; // increment the post counter
        continue;
    }
    endif;
endwhile;
// now make a new query and show the posts for real, with the adjusted post count and filtering
$my_new_query = new WP_Query($query_string.'&showposts='.$wp_query->post_count);
// do another The Loop (and display the results this time)
while ( $my_new_query->have_posts() ) : $my_new_query->the_post();
    // detect and exclude these same posts
    if (in_category(1) && (0 === strpos(the_title('', '', false), 'Some title')) ) { continue; }

// ...the rest of the WordPress template goes here...

This is neat because it gives you the capability to define arbitrarily complex exclusion patterns and directly modify your new query object however you like before you execute it. Once you know this works, you’ll probably want to extract the filtering code into a function. Using the above example, your new code might look like this:

// define criteria for filtering
function matches_filtering_criteria () {
    if (in_category(1) && (0 === strpos(the_title('', '', false), 'Some title')) ) {
        return true;
    } else {
        return false;
    }
}
// original query runs in The (real) Loop first
while ( have_posts() ) : the_post();
    // detect pots matching our exclusion criteria
    if (matches_filtering_criteria()) {
        $wp_query->post_count++; // increment the post counter
        continue;
    }
    endif;
endwhile;
// now make a new query and show the posts for real, with the adjusted post count and filtering
$my_new_query = new WP_Query($query_string.'&showposts='.$wp_query->post_count);
// do another The Loop (and display the results this time)
while ( $my_new_query->have_posts() ) : $my_new_query->the_post();
    //
    // detect and exclude these same posts
    if (matches_filtering_criteria()) { continue; }

// ...the rest of the WordPress template goes here...

For more information on these functions, see:

• The Loop
• in_category()
• the_title()
• WP_Query()

My co-authors on the book, called Foundation Web Standards Foundation Website Creation (you can pre-order now) and published by Friends of ED, an Apress company, are Jonathan Lane of Industry Interactive, Inc. and Joe Lewis, who blogs at Sanbeiji.com. I’m not going to say much more until after the book is released in late July.

For the eager, here’s the description of the book posted on the Friends of ED website:

Foundation Website Creation explores the process of constructing a web site from start to finish. There is more to the process than just knowing HTML! Designers and developers must follow a proper process to flush out goals and objectives and determine requirements both prior to, and during project development.

Large Web projects are rarely completed by a single person. Producers, project managers, designers, developers, writers, and editors all play critical parts in a project’s evolution. This book provides an overview of the entire process, and also shows project development from the perspective of these different roles. It introduces the key concepts and duties performed by every member of such a team, and gives you the skills necessary to tackle projects like a professional.

It’s quite exciting getting a book out, and it’s quite a bit more work than I’d have ever originally thought. That being said, it’s extremely rewarding. There’s a lot more work I need to do on it between now and the time it gets released to publishing, so, well…back to work I go.

Now you all know where I’ve been spending my time writing.

for i in *; do
    v=`grep 'version = "' "$i/current/$i/"*.info |
      cut -d ':' -f 2 |
        sed -e 's/^version = "/v/' -e 's/"$//'`
    svn cp "$i/current" "$i/$v"
done;

It’s a bit buggy for some modules that have multiple .info files, but I’m sure a few more pipeline stages can fix that. (Which, because I’m done with this at the moment, I will leave as an exercise to the reader.)

Chalk this one up as another testament to the power of shell scripting and how it can help every developer get their job done faster.

1. Be certain you have the git-core package installed on your system and that this package includes the git-cvsimport command. You can run git help -a | grep cvsimport to verify this.
2. Be certain you have the cvsps command-line tool installed. This does not come with the git suite of tools, so you’ll need to get it separately. If you’re a lazy Mac OS X user, like me, you can use MacPorts: sudo port install cvsps. Otherwise, get it from the source.
3. Prepare your CVS login information for the remote server before you run git cvsimport. You need to do this so that the git tool will be able to log you in to the CVS server automatically. The command for this looks like:

   CVSROOT=:cvs-login-method:cvs-user-name@cvs.server.name:/path/to/CVS/root cvs login

   For example, if you’re pulling code from the anonymous CVS server that runs on Drupal.org, you might use this: CVSROOT=:pserver:anonymous@cvs.drupal.org:/cvs/drupal-contrib cvs login. This command will prompt you for the password for the user you specified at the server you specified (for anonymous access, the password is almost always anonymous) and will hash this in the ~/.cvspass file for future use by CVS

4. Finally, run the git cvsimport tool, and specify the proper options. Using the Drupal example above, your command might look like this:

   git cvsimport -v -d :pserver:anonymous@cvs.drupal.org:/cvs/drupal-contrib contributions/modules/module-name

   This would login to cvs.drupal.org using the CVS‘s pserver login method, provide the username anonymous and the password you specified in the previous step that is hashed in ~/.cvspass, set the CVS document root to /cvs/drupal-contrib, and pull the code located at contributions/modules/module-name into the current working directory as a git repository.

This works pretty nicely, and creates a git repository just as though you’d created it with git init in the current working directory.

If you get an error that looks like this:

AuthReply: cvs [pserver aborted]: descramble: unknown scrambling method

then you’ve most likely specified the CVS document root incorrectly. Most notably, git cvsimport does not understand a CVS document root wherein the password is specified in the document root URL itself. So, for example, git cvsimport -d :pserver:password:username@cvs.server.name:/path/to/CVS/root code/to/checkout will not work. Omitting the password and the separating colon from the URL should fix it.

First things first. Once you have an idea of what work you want to do, you should give yourself a space to do this work without disturbing anybody else’s work. Do this by making a new, personal branch. Unlike Subversion and some other centralized version control systems, git makes it possible to do make all kinds of changes to your repository, including making branches, and even save those changes without having to republish everything back to the central repository server at each step. In other words, if you thought Subversion branches were “cheap,” you’ll love git’s branches.

Also unlike Subversion, which stores its branches in completely separate pathnames, git keeps all branches in the same filesystem tree separated only with metadata (in .git/refs/remotes for remote branches and in .git/refs/heads for local branches, to be a bit more precise), so you don’t have to create lots of different directories for all your branches (unless you want to). With no branches defined, you’re working in the “master branch,” or “the trunk” by default.

git branch
* master

By saying git branch you ask git to print a list of all (local) branches. The one with the asterisk marks the current branch, the one you’re using at the moment you run the command. Since there are no branches, you’re currently in the “master” branch. But we don’t want to make changes here, we want to make changes in our own private branch, so we’ll make a new one.

git checkout -b new_branch_name

This will create a new branch with the name new_branch_name and immediately switch to it. Notice that you’ve made absolutely no changes to the filesystem itself; only the git metadata has been altered. Saying git branch again will show you the change:

git branch
  master
* new_branch_name

Also note that since we haven’t comitted any changes, we don’t need a commit message (or “log message”) for creating this branch. We’ll add one later, when we need it. Now go ahead and write some code in your new branch. At any time, you can create a new branch in the same way you added the first. Each new branch is created at the HEAD (“latest”) revision of whatever branch you’re currently working in.

git checkout -b another_branch
Switched to a new branch "another_branch"
git branch
  master
  new_branch_name
* another_branch

To switch back to any other branch, simply git checkout that branch again:

git checkout master
Switched to branch "master"

If you want to delete a branch you don’t like, that’s easy too:

git branch -d another_branch
Deleted branch another_branch.

Keep in mind that throughout all of these branch creation and deletion actions, the only thing that’s being altered is the git metadata. That’s why it’s so cheap to create new branches. If you ever have a new idea you’re working on, it’s recommended that you create a branch for it, even if that branch is so short-lived it never gets published.

So you have a new local branch, and you’ve been working as you normally do for a few minutes, creating files, editting them, and so on. Running git status now will ask git to show you the changes you’ve made to your filesystem. If you’ve created any files in new directories that git doesn’t know about, it will simply report that directory. If you’ve made new files in directories git does know about, it will list all those files explicitly.

git status
# On branch cartoon_contests
# Untracked files:
#   (use "git add ..." to include in what will be committed)
#
#	sites/default/modules/cartoon_contests/
#	sites/default/modules/factiva/.factiva.module.swp
#	sites/default/modules/testfile
#	sites/default/settings.php
nothing added to commit but untracked files present (use "git add" to track)

In the above sample output, I’m working on a new Drupal module for a web site. I’ve created a new directory, sites/default/modules/cartoon_contests/ (note the trailing slash), and I have several untracked files. One is my vim swap file for a different module, .factiva.module.swp, one’s an unimportant testfile, and the last is the Drupal configuration file, settings.php.

The only thing I want to commit is the new cartoon_contests directory, and all the files within it. Like Subversion, I have to tell git that I want to track this directory, which is done simply by saying git add cartoon_contests. Unlike Subversion, future invocations of git status let me see everything that git is going to go in the next commit.

git status
# On branch cartoon_contests
# Changes to be committed:
#   (use "git reset HEAD ..." to unstage)
#
#	new file:   sites/default/modules/cartoon_contests/cartoon_contests.info
#	new file:   sites/default/modules/cartoon_contests/cartoon_contests.module
#
# Untracked files:
#   (use "git add ..." to include in what will be committed)
#
#	sites/default/modules/factiva/.factiva.module.swp
#	sites/default/modules/testfile
#	sites/default/settings.php

The “Changes to be committed” section of the output is called the staging area, or the index. In this way, I can prepare all the changes I want to commit before I do so, making sure they’re perfect before I actually commit them to the git repository. At any time before I commit, I can make additional modifications, such as git adding more files or directories, git reseting to unstage all (or some) of my changes, etc. git help reset also has a number of handy explanations with examples for different things you might need to do at this point.

If you were using svn:ignore, equivalent functionality exists in git. Simply append file glob patterns, one per line, to the $GIT_DIR/info/exclude file in your git repository. Like so:

echo -e "*.swp\nsites/default/settings.php" >> .git/info/exclude ; git status
# On branch cartoon_contests
# Changes to be committed:
#   (use "git reset HEAD ..." to unstage)
#
#	new file:   sites/default/modules/cartoon_contests/cartoon_contests.info
#	new file:   sites/default/modules/cartoon_contests/cartoon_contests.module
#

If you’ve got many of these, you can use git svn show-ignore >> .git/info/exclude to search through your old Subversion repository and look for any and all ignores, automatically adding them to git’s exclude list. (Checkout Tsuna’s blog entry on learning git for more tips like this.)

Finally, after you’ve done some of your work and you’ve finished staging your changes, you’re ready to commit them to the repository. On the other hand, if you hate what you’ve done and want to undo it all, you can say git reset --hard HEAD to throw away all your local changes. To throw away changes to just a single file, just checkout that file again by saying git checkout filename. This is the equivalent of Subversions svn revert filename command.

Before you actually make your first commit, however, you should properly introduce yourself to git. You don’t have to do this because git will try to figure out who you are by itself (details explained in detail here), but you probably should at least create global defaults for your user (which git will store in ~/.gitconfig). If you want to, you can also create per-repository defaults (which git will store in .git/config), or even system-wide defaults for all users of this computer (which git will store in /etc/gitconfig). To do so, say this:

git config --global user.name "Your Full Name"
git config --global user.email "you@example.com"

This will create the file ~/.gitconfig if it doesn’t alreay exist and will write your name into it. You can also just edit the file directly yourself instead of using git config commands.

Like Subversion, committing will create a saved, fixed point-in-time that marks the changes you have made to your files. Like branches, commits are also very cheap in git, so go ahead and commit at any time you like. Remember to stage your files (by git adding them), and then just git commit.

git add filepattern
git commit -m "My very frst git commit!"
Created commit ef483c1: My very frst git commit!
 5 files changed, 58 insertions(+), 12 deletions(-)

If you forget to specify a commit log message (-m "log message") on the command line, or if you want to enter a multi-line commit log, git will prompt you for it in your favorite $EDITOR. You can view a history, including all the log messages, for your project with git log. You can even the view logs in a number of pretty formats. Check git help log for more information.

If you want to change anything about the commit you just made, such as the author, you can just run git commit again with the --amend flag added to the command. Notice the typo in the commit log? Fixing it is really easy:

git commit --amend -m "My very first git commit!"
Created commit 88602f6: My very first git commit!
 5 files changed, 58 insertions(+), 12 deletions(-)

Finally, with your new code commited to your local git branch, it’s time to share that code with your colleagues who are (for some inexplicable reason) still using Subversion. This is also extremely simple. Just say git svn dcommit. (That’s right, dcommit, not just commit. I don’t know why….)

Not sharing your changes via Subversion, but with a patch instead? git diff -p will generate patches for you. See git help diff-files and look for the “GENERATING PATCHES WITH -P” section.

If you found this helpful, you may also enjoy an alternative tour for beginners from Carl Worth.

Using Mac OS X, first make sure you have the git core installed as well as the git-svn components. Unlike other version control systems, git is not one monolithic program but a collection of smaller utilities, and the Subversion conduits are a subset of these utilities. The git-Subversion utilities themselves depend on having the Perl::Subversion bindings installed for Perl 5.

For the lazy MacPorts user, simply run:

sudo port install git-core +svn

This will install the git-core, the git-svn packages, as well as all the documentation for git and any required dependencies you don’t already have. Note that the documentation (man pages) is installed in /opt/local/man, which may not be in your default $MANPATH, so be sure to add that directory if man git returns a “No manual entry for git” error.

Alternatively, if you don’t want to use MacPorts, you can download a pre-compiled Mac OS X binary that includes git, the git docs, and the git-svn package from the git web site that comes complete with a standard GUI installation procedure.

Next, initialize a new git repository that tracks a remote Subversion one. This allows you to work privately using git, but to collaborate with other people who are using Subversion. Running

git svn init svn://my.svn.server/path/to/svn/repo workingcopy

will give you an empty git working copy named workingcopy configured to use the remote Subversion repository at svn://my.svn.server/path/to/svn/repo. This step is analogous to the need to run svnadmin create repo_db, to initialize a new repository database (where all the file versioning information will be stored). Unlike Subversion, git’s distributed database means that the working copy itself is also the location of the repository database, so there’s no need to deal with two filesystem paths anymore.

Next, change directory to your working copy, and run

cd workingcopy
git svn fetch

to populate your new, empty git working copy (and repository) with all the files from the remote Subversion repository.

Now that you have filled your git repository with a lot of data, if you want to, you can now also save a significant chunk of disk space by repacking that data into a more single, efficient, native packed git archive format (a .pack file). The git-repack -a command is used to do this, and its manual page says:

Instead of incrementally packing the unpacked objects, pack everything referenced into a single pack. Especially useful when packing a repository that is used for private development and there is no need to worry about people fetching via dumb protocols from it.

According to some sources, this can turn a 353MB Subversion repository into 31MB of git pack. Say:

git repack -a -d -f
Generating pack...
Counting objects: 284
Done counting 4475 objects.
Deltifying 4475 objects...
 100% (4475/4475) done
Writing 4475 objects...
 100% (4475/4475) done
Total 4475 (delta 1876), reused 0 (delta 0)
Pack pack-a115c320ff9c9968248bd250bdfea3110d0f0c1a created.
Removing unused objects 100%...
Done.

to perform the compression. For even greater savings, increase the compression by stipulating a high --window option, such as git repack -a -d -f --window=100. (--window defaults to 10—the higher the window, the harder git-repack will try to compress your stuff.) This turned a 36MB Subversion repository in one of my projects to a 20MB git pack. As always, your mileage may vary.

Congratulations, you have transformed your old Subversion repository to a git repository. All that’s left to do now is to get to the coding. Perhaps start by making a local (cheaper than cheap!) git branch….

That was easy, right? Most things in git are, in fact, that easy. If you’ve never used other version control systems before, be grateful. If you have, you can breathe a sigh of relief. Still not convinced? Don’t take my word for it…ask Linus Torvalds (or Randall Schwartz).