Category: Web Standards

WordPress NYC: Enterprise Features for Small Businesses Running WordPress

Earlier this week, the WordPress NYC Meetup group hosted me at their space inside the Microsoft Technology Center. I was there to present some of my recent work on “Enterprise Features for Small Businesses Running WordPress.” I had a lot of fun and really appreciated the opportunity to showcase three projects I’ve been working on recently.

You can find an archive of all recorded sessions that the organizers, Steve and Scott, of the WordPress NYC Meetup have produced at the “WPNYC TV” page on their website. Below, you can find my own presentation from their latest evening, along with a transcript and links to the original presentation materials. This includes my slide deck, presenter notes, and presentation runbook.

>> SCOTT BECKER: Maymay and I just met tonight, but I find what he does fascinating. So, instead of giving you, like, some spiel that, y’know, we’ve written up, I’m just going to read a bit about what we had here at Meetup so you know a little bit about what Maymay’s gonna talk to you about. I find it fascinating.

He’s a Free Software developer and technology consultant who—get this—works without money. Anybody else say that?

>> AUDIENCE MEMBERS: Sometimes.

[laughter]

>> MAYMAY: Good answer!

>> SCOTT BECKER: Instead of owning a home, Maymay lives on the road, traveling wherever I guess he’s needed and wanted, working to help secure and scale small businesses, non-profits, and community groups. He’s on the road and he helps people take advantage of enterprise features through easy to use and easy to understand Free and Open Source Software.

So, with that being said, maymay.

>> MAYMAY: Yeah! Thank you. Thank you so much, Scott and Steve, who I know is not here. And to Microsoft, for the space of course. So, you just introduced a little bit about me. I kinda just want to spend one moment say a little bit more about myself.

People call me “maymay,” that’s the name I prefer. This is a screenshot of my homepage at maymay.net. It’s spelled like the month of May, but twice. Um, I get DDoS’ed occasionally, so if my site’s not up right now, don’t worry about it. It’ll come back in a sec. Apologies if that’s happening, but go there to learn a little bit more about me, and about the work that I do.

In the meantime, I want to talk a bit about what we’re gonna do here. So, I talked a little bit about myself. I won’t bore you more with that. Next we’re gonna quickly spin up a new WordPress Multisite instance, so that we can show some of the demos that I want to show you. I’ll be showing you three WordPress plugins that I wrote that I think you might want to know about. That may be why you’re here. And finally, if there’s time, we can do some Q&A. Hopefully I’ll have some answers to that.

So, all right, let me go ahead and spin up a new WordPress Multisite instance. And, for this, I’m actually gonna go out of the slides, and I’m gonna go to my little demo here. Now, what I am going to do is go to this website that doesn’t yet exist, just to prove that it’s not actually there: W P N Y C dot DEMO. This website doesn’t exist, no one can get to it. It doesn’t yet exist. So we’re going to go ahead and make it!

Can y’all see that, is that big enough? That text is big? Okay.

I’m gonna be using a couple tools that I’m gonna explain in just a second. The one that I want to start with is VV. And, this is gonna basically automate creating an entire new WordPress Multisite install on my machine, and make the site available at WPNYC.DEMO.

So, I’m gonna tell VV to, hey, please create for me a domain called WPNYC.DEMO. And I want the name of this site to be WPNYCDEMO. Is that right? That’s right. And I want it to be a multisite install with a subdomain scheme. You know how WordPress can do subdirectories and subdomains? I want subdomains. I want the admin username to be admin, I want the password to be password, which I know is not a good password, but this is a demo.

>> AUDIENCE MEMBER: Password with five asterisks!

>> MAYMAY: Yeah, and an at-sign!

>> AUDIENCE MEMBER: What is this software you’re using?

>> MAYMAY: I’ll talk about the software in just a sec, I just want to kick this off.

The admin email address is gonna be admin@wpnyc.demo. I want also to remove these defaults. Y’know how sometimes when you install a new WordPress site it installs plugins like Akismet and the Hello Dolly plugin and a bunch of different themes that you almost never use? I wanna remove all those, so I don’t actually want those to be part of the final build. And I’m also going to use the debug flag here because I want to set the WP_DEBUG constant in the wp-config file. This will help me show you some of the output from some of the plugins that we’re gonna demo. Just so that we make sure that’s there.

All right, so I’m gonna go ahead and create that. And VV’s gonna ask me whether or not I want to create a site with a blueprint. I don’t. Whether I want to install a specific version of WordPress. I don’t. I’ll just use the latest version, rather. I’m not going to use any sample content so it’s gonna be totally blank, no users, no nothing. We’re not gonna import any database files. We’re not going to add sample content to any of this. And we’re gonna go ahead and start that off.

Now, VV is gonna go ahead and build me a new server and it’s going to make that site available over on the left-hand side there. And while that’s happening, we can switch back to our presentation and we can talk a little bit about the demos that I’m going to be showing you.

So, very briefly. If you don’t already know about the tools there—

>> AUDIENCE MEMBER: Where’s this site going to be set up?

>> MAYMAY: It’s setting up right on my computer here, in a development environment. And it’s going to be using these things: Virtual Box, which is a virtual machine hypervisor, a type 2 virtual machine hypervisor. That means that I’m gonna have a totally new computer, a Linux server, on my Mac here. That, is being configured using Vagrant, which is a virtual machine hypervisor automation tool. So with Vagrant commands I can tell Virtual Box how to set up that machine; what network interfaces I want, what kind of operating system I want, what kind of ports to use, all that kind of stuff. I’m also using VVV, which is the Varying Vagrant Vagrants project. This is a project originally started by the 10Up company, which is a WordPress development shop. It’s a Vagrant config specific for WordPress development. So, I’ll be using that. And, last one is Variable VV, which is the one that I used that you saw and this is the command that I was using to tell VVV how I wanted it to configure that WordPress setup.

So, all these tools are Free Software, open source. You can grab them on GitHub or these project pages. Variable VV is written by Brad Parbs, who’s an excellent developer and this tool is probably the easiest way to set up a WordPress site I’ve ever seen. I’ve contributed a number of features to it. It’s really, really nice.

Using these kinds of tools makes development a lot easier, a lot more robust, a lot more reliable, ’cause it’s all automated. It takes out human error, and it’s much faster, of course.

So, let’s—

>> AUDIENCE MEMBER: Quick question?

>> MAYMAY: Yeah?

>> AUDIENCE MEMBER: Are you familiar with Local by Flywheel? Is this similar, or are there major differences?

>> MAYMAY: I am only passingly familiar with Local by Flywheel, but if I understand correctly, they’re basically equivalent tools. Right? It’s kind of—again, I’m actually not that familiar with Local by Flywheel. I’ve heard of it. My understanding is that it sets up a development environment for WordPress. Some of you may have heard about XAMP, right? That old thing.

[laughter]

That still exists. It’s kind of like a packaged server in a box. Like an application box. This is using—uh, this is the same effect, but we have virtual machines to do it with instead of putting, like, an Apache server on your laptop. That kind of thing.

All right, so, while that’s all building, we’re here to talk about Enterprise, right? So, I’m gonna assume you all know what WordPress is, and I’m gonna assume you all know what a small businesses are, and what features are, but we’re here to talk about “Enterprise Features for Small Businesses.” So what does, “Enterprise” mean? And some of you may think you already know what this is, and that’s great. Obviously, I’m not here to tell you what to think. That is, or may be, your employer’s job. So instead, I wanna make sure we’re all on the same page by letting you know what I mean when I say “enterprise.”

So what I mean when I say “enterprise” is important capabilities for secure and private collaboration, which utilize multiple tools simultaneously, typically sold to larger corporations that have a lot of money. Right? So, in other words we’re talking about anything that has to do with process or workflow automation, anything that has to do with objectives that touch multiple disciplines at once. Tools, for example, that interoperate between multiple vendors, typically to avoid vendor lock-in so that you don’t have to be beholden to a Facebook or an Amazon or a Google for the rest of your business’s life. Any capability that’s, perhaps, perceived to be super advanced or maybe even unnecessary for small groups, like those zero-to-one employers shops, or sole proprietorships, small businesses of that kind, particularly when they’re security and privacy related. Because those are the kinds of things almost always sold as the “pro” features in add-ons and upgrades that are unavailable to people with not huge budgets.

So, in short, any kind of system or tool that supports truly resilient autonomy. Something you can do yourself. Not this B2B stuff. Right?

So, with that said, I see my role as a Free Software developer to make it more possible for more people to independently access more of those capabilities without needing to have money and without needing to engage in any other form of abusive or coercive relationships in order to do so. I think that’s especially important to do in service to and in solidarity with the specific people whose lives are made dramatically worse by capitalist efforts to do the contrary.

All right, so let’s see where we are with the build of this new website and how far we’ve gotten on creating the server. Okay, there we go. It says here the server URL is wpnyc.demo. Let’s take a look and see if we actually have this available.

There we go, we got a new website up. So this is a pretty standard WordPress site. It’s all empty here, nothing fancy about this at all. We can go ahead and try to log in. And we’ll use our admin and password. And you can see that we have a Multisite install, so we have a Network Admin. We’ve got a users database. It’s all empty. So it’s just a standard, brand-new, WordPress site that we’ve created there.

>> AUDIENCE MEMBER: So, is this replacing what everyone has to do with their hosting company? Making databases, and—

>> MAYMAY: No, this is what your hosting company uses!

>> AUDIENCE MEMBER: Right, okay.

>> MAYMAY: You’re not going to be able to go to this website from, for example, the Starbucks down the street because it’s on my computer. It’s physically here. However, if you were a hosting company and you, for example, didn’t want to go and create a database and install WordPress every time you get a customer request, you’d probably use a tool like this. So, these tools are Free Software, they’re open source, you can do that if you wanted to. They don’t tell you that, but you can.

>> AUDIENCE MEMBER: It sets things up in seconds!

>> MAYMAY: It does. It doesn’t take very long at all. All right! So, we got our website, so we spun up a new WordPress Multisite instance.

>> AUDIENCE MEMBER: Question?

>> MAYMAY: Yeah?

>> AUDIENCE MEMBER: What does Multisite mean?

>> MAYMAY: Multisite means you can have multiple domains, multiple websites, that are running on one WordPress database. You have one database but you have, for example, blog.wpnyc.demo, and test.wpnyc.demo, and maybe even othersite.com, all running on one WordPress installation.

>> AUDIENCE MEMBER: So you can back the whole thing up?

>> MAYMAY: Yeah! It’s all one database, so whenever you backup that database, you backup the whole thing, the whole network.

>> AUDIENCE MEMBER: So you’re managing it from one WordPress site?

>> MAYMAY: Yeah. If Multisite is new to you, definitely check out the WordPress codex, which has a Multisite page. It describes it right there and also tells you how to set it up manually, in case you want to do that. It is good to do that manually at least once or twice so you know what these tools are doing. But once you do know how to set it up manually, using these tools obviously makes the job a lot faster and a lot less error prone.

All right, so we’ve created our new WordPress multisite instance. Let’s move on now. We’ll learn about the Subresource Integrity Manager for WordPress.

So, first, how many of you may already have heard about Subresource Integrity? No? Okay. That’s cool. I really like the Mozilla Developer Network’s definition. It’s pretty clear.

“Subresource Integrity is a security feature that enables browsers to verify that files they fetch (for example, from a CDN [a Content Distribution Network]) are delivered without unexpected manipulation. It works by allowing you to provide a cryptographic hash that a fetched file must match.”

So, what does that mean? For example, let’s say some attacker—or you—want to run cryptocurrency mining JavaScript on hundreds of thousands of users’s Web browsers. You don’t have to actually attack thousands of users’s websites, thousands of websites on the Internet, in order to do that. You could compromise the one website that all those other websites are loading files from. For example, like a [Facebook tracking] pixel, right? Like some sort of web bug. If you can change that and everyone is pulling that site from you, well, there you go. You’re now loading your JavaScript code on multiple websites. So, for example, if I wanted to make the users of USCourts.gov mine Monero or BitCoin for me, then I wouldn’t necessarily have to attack USCourts.gov, I could attack TextHelp.com, because TextHelp.com is serving files for the other sites, USCourts.gov and ICO.org.uk.

Now this isn’t theoretical, either, this happened just last week, with these exact websites—USCourts.gov included—and it’s exactly the sort of scenario that Subresource Integrity is designed to mitigate. So, let’s see how you can prevent this attack against your site’s users for free using the Subresource Integrity Manager for WordPress.

Let’s go back to our demo here. And we’ll switch back to Firefox.

>> AUDIENCE MEMBER: Is this similar to what SSL does?

>> MAYMAY: No. No, SSL would not protect against this because you’re still getting the content and not verifying that what you’re getting is what you expected to get. It just means that no one has listened to what you’re getting on the way. I’ll show you what I mean in just a sec. This will be a lot clearer if you see this in action than if you just see some slides without a demonstration about it.

All right, so the first thing we’re gonna do is, obviously, is we’re gonna go to our Network Admin screen. We’re gonna go to Plugins. We don’t have any. And we need to get one because we need to get the Subresource Integrity Manager, called WP-SRI. I think you can also search for “Subresource Integrity.” Regardless, here it is. We’re gonna go ahead and install this. And there it is.

Now, when you have a Multisite install you can activate plugins for an entire Network all at once or, what we’re gonna do, is we’re going to go to the site itself, our site’s Dashboard—it’s hard to do this backwards. Go to our Plugins, and we’re gonna go ahead and activate the Subresource Integrity Manager for WordPress. That’s it, you’re now protected. By which I mean, if you go to your Tools page, you’ll see a new item called Subresource Integrity Manager, and you see here all the listing of resources that your site’s requesting. Every single one, including the ones on your own site because we set that debug flag—I wanted to make sure there was some content here. Over on the left you see the URL column. That’s the source address of the file you’re loading. So these are JavaScript and CSS styles, etcetera. And over on the right, you’ll see these hashes. And this is the cryptographic hash, this is the one-way mathematical function that proves that the content that’s being served to your site is the content that you expected to get when you first loaded that resource.

So, what do I mean by that? Let’s go to the site itself, and I’m going to go ahead and open up the View Source thing here—and let’s make that a little bit bigger so you can see as well—and let’s just have a look at one of these link elements. Actually, let’s look for stylesheet—style sheet—there we go. So this probably looks familiar to any of you who have seen CSS before, right? Link rel stylesheet. Here’s an href, a reference to the stylesheet itself, but over on the right, you can see crossorigin anonymous and integrity equals SHA-256 dash and then this hash. This hash is the metadata of what was expected to be in that stylesheet. So if some attacker modifies it, it’s not going to load in your browser if you’re using these, if you’re expecting to get this particular hash value.

What do I mean by that? Let’s go a little bit deeper into that. So, let’s close—close? What happened there. There. Let’s close out of this. And I want to create now an example attack on this website to show you how this is actually enforced.

So the very first thing we’re gonna do, I’m going to go to my command line here and I’m going to create a new JavaScript file. This is gonna be going to a CDN that I set up ahead of time, here. CDN dot demo. The important thing about this site is that, again, it’s just a WordPress site, it’s just a different domain. The point being that we have two different domains. One requesting JavaScript sources from another. This is exactly the same thing as you would, for example, do when you do an embed from YouTube and you say, “Hey, YouTube, please load some content on my site.” I’m gonna do the same thing, but it’s gonna be very, very simple. So we’re going to echo some JavaScript, how about alert—whoops, this is hard to do backwards—alert, how about ‘Hello, WP NYC Meetup.’ Right? And we’re gonna go ahead and put this into the root of my CDN demo at htdocs and we can call that test.js for example. And I want that to be output there, whoops! Where did I mess that up? Echo, alert, yup. That’s fine. Uh, that’s not fine? Oh, there’s some other, I can’t see [the full screen on the project] or I mistyped some stuff? I can’t actually see all that. There we go. Nope. It looks like I’m not able to see what is there. Echo alert ‘Hello, WP NYC Meetup.’ Close. Let’s not do this full-screen, because I don’t see what’s over there. Oh! I have another—

>> AUDIENCE MEMBER: Can you explain what you’re doing now?

>> MAYMAY: Yeah! So this is gonna put this text, “alert(‘Hello, WP NYC Meetup’)” into this file. There we go. So if I look at this file, let’s say cat that file out, you’ll see that now that file has “alert(‘Hello, WP NYC Meetup’)” and if we go to our CDN dot demo slash test dot js, we’ll have that file being served in the Web browser.

The next thing I need to do of course is I need to load this JavaScript into the theme that we’re using. So I’m going to go ahead and use Vim to edit wpnycdemo, htdocs, wp content—

>> AUDIENCE MEMBER: And what’s Vim?

>> MAYMAY: Vim is a text editor, like TextEdit, but in a command line. So, like, Notepad or something. WP content, themes, twenty seventeen, because that’s the theme we’re using, and header.php. Right, so this is gonna look pretty familiar to anyone who’s edited a WordPress theme before. You can see here the wp_head hook, function there. And we’re just gonna add another one and it is going to be wp_enqueue_script, and this is gonna be WP-SRI demo, and we’re gonna load CDN.demo/test.js. All right, now our website—we put that embed code, basically, into our website. So now, when we load the WP demo site, we should get an alert that says, hey, “Hello, WP NYC Meetup.” There we go. So we now have that alert. We’re running that JavaScript code.

Here’s the thing. What if I then change that JavaScript code? Well, without Subresource Integrity, if I change that JavaScript code, it’s still gonna run, which means that if some attacker then changes that source code on some other server that you don’t control, that you have no insight into, they’re now running their code on your site. With Subresource Integrity, however, if I change that, so let’s do that, let’s vim this again. This time, let’s vim the CDN site, htdocs, test.js, let’s make something much more malicious happen here. Maybe something like a cross-site scripting attack. With a cross-site scripting attack if I reload this now, if I wasn’t using Subresource Integrity—we’re going to go into Subresource Integrity Manager to see that we have a hash for it—it’s not going to load.

Okay. Why didn’t it load? It didn’t load because, if we look at the Inspector, and go to our Console, you can see, “None of the SHA 256 hashes in the integrity attribute match the content of the subresource.” In other words, the browser says, “Hey, I saw this file, but it doesn’t match what you told me it should contain. And because it doesn’t match, I’m not gonna run it.” Therefore, your users also will not be subjected to that JavaScript. If I—

>> AUDIENCE MEMBER: What will they see? Will they see an error?

>> MAYMAY: No. It just won’t run. So, for example, if you had a personalized resource—Google Fonts does this, where it sends personalized content to each individual user but it’s the same URL, it’s always google.com/fonts/something—that is gonna break this. And so hopefully Google will fix that and will give, like, individual URLs to people. Until then, you have this little Exclude button which just takes out that integrity attribute in case you run into a problem where you’re loading one resource for multiple users but the content of that one resource is different for each user. In that case the integrity attribute won’t be printed, and if I reload this page, we should now have an XSS attack happening. Does that make sense?

>> AUDIENCE MEMBER: Yeah.

>> MAYMAY: All right, so that happens, again, because we did not include the integrity attribute in our test.js script element. If we again go back to Subresource Integrity Manager and re-include it, then when we reload, we will not get the XSS popup because the integrity attribute will be printed and we’ll see in fact the SHA 256 hash was not matching what we expected it to be.

So, let’s return this to the original ‘Hello, WP NYC Meetup’, and we’ll save that. And hopefully, with this, because again, it matches, it’s not excluded, but it does match the content that we’re expecting it to be, we visit the site and we get this back.

So that is Subresource Integrity Manager for WordPress.

>> AUDIENCE MEMBER: Does that add any weight to the site?

>> MAYMAY: It will increase your HTML page sizes, but hopefully you’re using, y’know, HTTP2 for good compression, and it’s not really adding a lot in comparison to the kind of attack that this—the kind of vulnerability that this is for your users is pretty serious. So, this is basically considered a really good thing, it’s been widely developed and deployed.

Some pro-tips for using this. SRIHash.org is a great site to go to if you want one of these one-off. If you have a JavaScript file that you want to insert yourself but you want to include the integrity attribute, go to SRIHash.org, plug the URL into the form there. Hit Hash. It’ll give you the exact code you want to use for that.

>> AUDIENCE MEMBER: That looks like a good password generator.

[laughter]

>> MAYMAY: You can also further harden your site by using the CSP HTTP headers. These are content security policies which tell your visitor’s browsers not to load anything that doesn’t include an integrity attribute if you use the require-sri-for and you can say either script or style, or both. And finally of course, because the free and open Internet is a platform agnostic technology by design you don’t have to be using WordPress to be using this. You can, for instance if you’re using Ruby on Rails, use the sprockets-rails gem. Just use your javascript include tag and add a new parameter there, integrity equals true, and sprockets-rails will take care of it. Similarly, if you’re using any of the NodeJS tools, pick up the SSRI package over on NPM. And again, it’s ssri.fromData or dot fromUrl or something. Give it an algorithm that you wanna use, and the toString method will give you the integrity attribute, what you actually want to print out.

Okay, so that was Subresource Integrity for WordPress.

Yeah?

>> AUDIENCE MEMBER: What happens if I want to change the plugin or the script or whatever?

>> MAYMAY: Yeah!

>> AUDIENCE MEMBER: Do I have to run something again so I can update the hash tags or…?

>> MAYMAY: The easiest thing to do is to go back to your tool, so let’s go back to WPNYC Demo, we’ll go to your tool, and you find the resource you want. So, let’s say, test.js, here. And delete it. Now, the next time it loads, Subresource Integrity Manager will say, “Oh, I don’t know this. I’ll go ahead and fetch it, re-hash it,” and that way basically you’re forgetting the old hash and putting in the new one. So that’s that.

>> AUDIENCE MEMBER: Can you give us an example of how to use this on a site we might already be familiar with?

>> MAYMAY: This is already used on many sites you’re probably familiar with! And it’s used specifically to make sure that there’s no unexpected manipulation from the CDN side. So, for instance, a long time ago—well, not a long time ago but, like, two years ago—there was one of these JavaScript DDoS attacks that happened by Baidu, which was one of the Chinese analytics firms. The Great Firewall of China, evidently, decided to change anyone who was loading Baidu analytics JavaScript that it was sending to DDoS GitHub. As in, to try to get every browser on the planet who was loading Baidu analytics to send a bunch of requests to GitHub.com to take them down. And it worked because, at the time, there was nothing like this. So that, among other reasons is why SRI features became a W3C standard and is now deployed in all these different frameworks.

So, for WordPress, you can use the Subresource Integrity Manager until we get this into core. It looks like it will be into core at some point, but I don’t want to promise about when or how, because I don’t know.

[laughs]

>> AUDIENCE MEMBER: [quiet speaking]

>> MAYMAY: Oh, you just have to hit “exclude.”

>> AUDIENCE MEMBER: [quiet speaking]

>> MAYMAY: Well, I mean, yes. For Google Fonts, or any resource that has the same URL but whose content is different, right? Because, if the content is different, it’s not going to match the hash for all users, it will only match the hash for one user. Probably you, because you’re the first one who has requested it. So you want to exclude those until sites like Google and other CDN providers start making unique URLs per user. All this personalization that happens on just the content and not the URL needs to go away for this to work.

>> AUDIENCE MEMBER: How do I know if I need to exclude a script like that?

>> MAYMAY: See if your site still works.

[laughter]

>> AUDIENCE MEMBER: Oh, just load it again?

>> MAYMAY: Yeah. All right, so, we’re gonna move on to GPG and OpenPGP signing and encryption for WordPress. So, first of all, how many of you are familiar already with GPG or OpenPGP or signing and encryption and technologies? One, two, three, four hands in the back. Okay, cool. Great! Any of you wanna shout out what that is? No? All right.

Well basically, it’s secured email, is the answer to that. In short, GPG or OpenPGP—they’re kind of interchangeable terms—means secure email. But here we have to be pretty careful about what we’re talking about when we say “secured.” What does secured really mean? In a very brief nutshell, ’cause I don’t have that much time, when infosec pros talk about security they’re usually speaking about something that’s known as the CIA triad. It’s called a triad because it has, of course, three parts, and these are confidentiality for the “C,” integrity for the “I,” and availability for the “A.” Now, for the purposes of this presentation we’re only going to concern ourselves with the first two pieces of this triad.

Another common word for confidentiality—you’ll often hear this a lot—is privacy. Much more often used word. And similarly, very similarly to that, another word that’s used for integrity is “authenticity.” So, in the next demo, when I talk about GPG encryption, what I want to be talking about it ensuring privacy: the ability for your website to send a message that only its intended recipient can read. And when I talk about GPG signing, what I’m talking about is ensuring authenticity: the ability for the recipient of that message to verify that it the message was actually sent by your server, not some other imposter, and that the message that they got was unmodified in transit. It’s the actual message that you sent. Very much like the Subresource Integrity thing where you’re hashing stuff. All right, so now let’s see how you can accomplish this with the PGP encrypted emails plugin that I wrote.

So, we’re gonna go back to our demo, and—whoop, there’s my, there it is. And, I want to go to my WordPress site. And, as before, we’re gonna create a new plugin by going to Network Admin, Plugins, Add New, and we’re gonna search for WP PGP Encrypted Emails. That’s the full name of it, but you can also probably search “PGP encrypted emails.” Anyway, we’re gonna hit “Install Now” and there it goes. And we can Network Activate this or we can, again, just go to a site, go to the demo site, go to our plugins, and activate—whoops, no, I don’t want to, I want to do this one—plugins, and activate that. All right, that’s it.

Now, very first thing you’ll notice is that we have one of these admin notices up at the top. It says hold on, you’re not done yet. You’ve got to create or generate an OpenPGP key pair for the website to sign outgoing emails with. In other words, you can’t just send an email. You have to actually stamp that email with the identity of the website, cryptographically. So that’s what we’re gonna do when we generate PGP signing key pair. That’s it, that’s all you gotta do. And this will take you to the Settings screen, with a new item here under Email Encryption.

This email encryption settings will have a number of options. The important one for here is the PGP signing key pair. This is a low-trust, single-purpose key (identity), for the website that you need to distribute any user who wants to make sure that when they’re getting emails from you, it’s coming from the right place. There’s a theme function that you can add to your theme that makes this button, so you don’t have to worry about the code itself, or you can go to or tell your users to go the profile that they have and click on “Download public key” at the very bottom under their personal encryption settings. We’re gonna go ahead and do that. I’m gonna click “Download public key,” and it’s gonna give me this file. I’m just gonna save that, for the moment.

But before we do anything with that, I want to show you what an unsigned and unencrypted email looks like. Regular old email, nothing fancy. This is what you’re doing, probably, right now. And to show you that, I’m gonna go to wpnyc.demo, and I’m gonna go to [port] 1080 here. This is Mailcatcher. This is another one of those development tools that was installed when I did the VV build and what this does is it kind of intercepts any outgoing email from that website and shows it to me in this interface so I can debug it.

You can see that we already have an email in here. This one is the email that was sent when the WordPress site was kicked off and built. Sometimes you’ll see this in the One-click Installers. Y’know, you’ll get an email saying, “Hey, your WordPress site is ready.” That’s what this is. So, this is the source code of that email. You can see the headers up here, and the body down here. And there’s nothing fancy about this, nothing special, nothing cryptographic, no hashes, none of these security features. It’s all just plain text. This is like sending a postcard through the post. Anytime you send a postcard, anyone viewing or handling that postcard can read the contents. That’s what all email, all text messages, all unencrypted HTTP—not HTTPS—traffic is. So we’re now going to add the equivalent of a digital envelope to protect the contents of this message and a digital stamp that says, much like those Game of Thrones, y’know, wax stamps. This definitely came from Jamie Lanister or whoever. We’re gonna add that to our emails. I’m gonna show you how to do that.

Firstly, we have this key that we downloaded. So let’s go ahead and take a look at my Downloads folder. And, or actually, take a look at this here. “To authenticate the emails, download the PGP public key and import it to an OpenPGP-compatible client.” This links to PRISM-Break, which is a fantastic website. If you don’t know about it, check it out. PRISM dash Break dot org. And it lists here all the software that you can use PGP or GPG with. So there’s a vast ecosystem of this. It’s available on Windows, Linux, Android phones, iOS devices, basically any computing device that you have can do this for free already, with either one of these apps if it’s not already built-in. Many Linuxes have this built-in, for example.

So, I’m going to be using GPGTools, or MacGPG for this. That is at GPGTools.org. If you’re on a Mac today and you wanna try this out, go grab this. It’s the best tool for the job I’ve seen. All right, so we’re gonna go ahead and open up this email, I’m sorry, this key, and we’re gonna open it with an application that was installed with the GPGTools package that I installed earlier called GPG Keychain, and we’re gonna import that key, and there we go. Now we have this key.

Now what this means is that we are aware of a cryptographic identify for the website wpnyc.demo. I can now authenticate any emails that are sent there. So let’s go ahead and get an email from there. You could trigger an email by purchasing an order or making a new user account, or you can use the handy “Send me a test email” button, which is what I’m going to do. And when I click this I want you to take a look at the Mailcatcher tab up here, and I want you to take a look at this. This is going to go from 1 to 2. Okay? Ready?

Send me a test email. There it is, now there’s 2. There’s the test email. And now, take a look at how the email is different. We have this “begin PGP signed message” text on top and on the bottom. This is what’s known as a clear signed message. This is saying that this is the, effectively, the integrity attribute, or that metadata for the contents of that message, just like the SRI stuff. It functions very similarly.

If you were using an email client, this would automatically authenticate—

>> AUDIENCE MEMBER: You’d use both?

>> MAYMAY: Use both what?

>> AUDIENCE MEMBER: Use both the GPG and the SRI?

>> MAYMAY: Yeah, they’re separate plugins, they’re separate technologies, but they use the same what are known as cryptographic primitives, which is to say, they use the same mathematics under the hood.

>> AUDIENCE MEMBER: So you’d use one or the other?

>> MAYMAY: No, you can use both, because they do different things. In fact, I would recommend that you use all the plugins I’m gonna demo. [laughs] ‘Cause, they’re all free and they run on any WordPress site of any size. All right.

So, we’re gonna grab this PGP signature here, the contents of this email, and again if you were using an email client like Apple Mail or Microsoft Outlook or something, you wouldn’t have to do this copying-and-pasting, but I’m using a debugging tool so I am. I’m just going to go ahead and open up a TextEdit window and I’m gonna paste this into a new file and we’re gonna say test email, and I’m gonna save it on my desktop—and, sure, you can use the email extension. All right, and now, to verify that this is in fact the—I don’t need this anymore—the message that came from the site, I’m gonna right-click, I’m gonna go to services, and “Verify Signature of File.” See that, there? “Verify Signature of File.”

All right, click that, and we get a verification result: Signed by wordpress@wpnyc.demo. This means, yes, this email came from the site that you think it did and it was unmodified. No one changed the message between the time that the site sent it and the time that you received it. So, to prove that, we can open up this test email again. TextEdit—whoops, come on. Why is it not dragging this over? Drag. Seriously? For real now? All right, let’s do it this way. Other, and we want to use TextEdit, which is here. All right, so now let’s change this email in some way. We’ll just delete some text. We’ll re-save it.

Let’s try to verify again. Now we should get a failed message. This should not verify, because the message was changed. This is really important for things like, for example, security announcements. Apple, Inc, like, the company that makes this computer. They do this, exactly this process for when they send emails to their security announce list because it would be pretty bad if they sent a security announcement to say, “Hey, there’s a new patch available,” and that was actually a fraudulent email. Web hosting providers, DreamHost sends this with their billing emails, with their receipts. Now you can do the same with this plugin. All right, so that is a test email.

So that was signing. Do we have time? Do you think we should do encryption? Do we have time for this?

>> AUDIENCE: Yes!

>> MAYMAY: Yes? Okay. [laughs] Is there a question, there?

>> AUDIENCE MEMBER: [quiet speaking]

>> MAYMAY: I can’t quite hear you. Are you asking if it’s possible, if this works with an external service?

>> AUDIENCE MEMBER: Yes. If you don’t want to use your WordPress server to send the emails.

>> MAYMAY: Yeah. It, um, you will have to do a little bit more work to use an external service because you’ll need to send them pre-signed messages, right? So, for example, a lot of these services will insert things like, you know, “Hello, name,” or, y’know, “who lives at such and such, your account number is such and this.” If you sign the message before they do that, none of these verifications are gonna work, because they’re going to be changing the message on your behalf. On the other hand, there’s many free software plugins for WordPress that can function similarly to MailChimp, for instance, and this plugin is compatible with all of them that I’ve tested, which at this point has been hundreds. So, you could do that, it might take you a little bit more time to send those emails but at least that way you’re actually doing the work on your site, yourself, and not farming that out to a third party that may or may not—and probably is—mining you for data. So, but, y’know, obviously up to you?

All right. What time is it? It is, 8:30. I have until 9, I got a lot of content. You sure you want to see encryption?

>> AUDIENCE: Yeah!

>> MAYMAY: Yeah, y’all are good for that?

>> SCOTT BECKER: Quarter to nine!

>> MAYMAY: Quarter to nine! All right, I don’t have a lot of time, then. Here’s what I’m gonna do—

>> SCOTT BECKER: Well, ten to.

>> MAYMAY: All right. Well, let’s, I’m going to go through this a little bit quicker. So what I’m going to do is I’m just going to create a new key here. This doesn’t really matter. To do encryption we have to do the reverse process. Rather than getting the key from the website, I have to give the website my identity. But to have an identity, I need to make one. So that’s what this is, that’s what the “New” thingy is here. So we’ll do that. Some email dot invalid, none of this matters. The password is password. And password here. And, under advanced options, what’s there? Oh yeah. So I’m just gonna say “this is a test do not use this key.” We’re gonna generate that key. We’re gonna continue with a simple password. And again, all the GPG tools will do this, whether you’re using Windows or Linux, I just happen to be using a Mac. I don’t want to upload the key to the key server, again, because it’s a test.

And what we’ve done is created a new key pair. What’s known as basically a digital lock and a key that opens that digital lock. So if I hit export here, I get a file on this desktop. Let’s go ahead right there, and desktop, and there is my file which just like the other email, this can be opened. Because a key, an identity, a cryptographic hash, is just text, we can open it with TextEdit and you can take a look at what a key looks like. It’s just a really big number, really.

So if I copy this into my—because I’m the admin here, I’m gonna copy it into the Settings, Email Encryption, and Admin Email PGP Public Key textbox here, and hit save. Now, that’s all I gotta do. Now if I get another message, let’s trigger that again. Send me another test email. It’ll go from 2 to 3. And now this was the signed message, now we’ll have an encrypted message. So now we just have this “begin PGP message” block and this is the content that’s gonna be stored, for example, by Google or by Microsoft Live, right? When you are actually using a GMail account, they’re reading all your email. Well that’s because it’s not encrypted. If you use this, and you give the sites that are letting you send encrypted messages your cryptographic identity, Google can no longer read your email because to them it looks like this.

So the question is, how does it look like to you? Well, we’ll go back here, we’ll go to TextEdit, and we’ll just paste this message in again. And, again, on an actual email client this is much, much simpler. I’m just gonna right-click, go to Services, and I’m gonna decrypt selection to new window. And what we should see here is I’m being asked for my password for this identity, which I put before. Right, and say password. Oh, it opened it up over here, so I’m just gonna move this over. Right. This is a test message from wpnycdemo. It’s still signed, right, because the site has a signing key, and it’s decrypted, because I have the matching key to the lock that I gave the website. So that is OpenPGP signing and encryption for WordPress.

Quick pro-tips for making even more use of this plugin. Number one, importantly for small businesses you should know that the WP PGP Encrypted Emails plugin features zero-configuration, out-of-the-box support with WooCommerce. So if you have a WooCommerce store, right, as long as your chosen theme supports the WooCommerce account pages, then all you have to do is install this and your customers will get an out of the way form that looks exactly like this that they can use to opt-in to signed emails or even encrypted emails if they go ahead through the process of making their own identity and uploading a key and giving it to your store. This is an example of what a signed email might look like in Apple Mail. Instead of the Mailcatcher interface it would just look like this. It would say “Security: Signed” and that’s how they would know, yes, this actually came from your store. So this is really important for secured email receipts, for private transfers of communications between, like, tech support. Anything that you basically don’t want other people reading. This is all from a blog post out of New York City called Flora Posidonia: FloraPosidonia.xyz. Check that out at some point if you wanna see this in practice in the wild.

For developers, WP PGP Encrypted Emails features a general-purpose API to cryptographic operations using familiar WordPress plugin hooks. So what I mean by that is that the plugin uses the same hooks that it makes available for other plugins for itself. And that means with as few as about four lines of PHP you actually, as a developer, can build PGP or S/MIME encryption into your own plugins and themes. So you can see here we’re getting the user object, we’re applying the wp_openpgp_user_key filter to that user object to get the key itself, and then we’re using openpgp_encrypt with the message and the public key to get an encrypted message. You can now, using PHP, send this variable over the Internet or in an email or anywhere you want, and it’s that PGP message block instead of the plain text content.

Okay, so that was GPG and OpenPGP signing and encryption for WordPress. I got ten minutes, so I’m going to hold questions. And at this point we’re going to go on to centralized authentication service using OpenLDAP for WordPress.

Now, as before, centralized authentication services with LDAP, anyone use this already? Sound familiar to anybody? No? All right, let’s start with LDAP. So that stands for the Lightweight Directory Access Protocol. It is an open vendor-neutral industry standard application protocol for accessing and maintaining distributed information services. So what that means, for our purposes at the moment, is that an LDAP database, which is called a Directory Information Tree or a DIT, can store user account login details like usernames and passwords and email addresses and phone numbers and this kind of thing, in an application-independent way so that any app that can speak LDAP can actually use the LDAP store as its user database.

Fun fact: LDAP was written by Tim Howes. He was the CTO and founder of a company called Opsware that I worked at for a while and that’s now HP Server Automation suite, HPSA, for those of you actually working in Enterprise.

Let’s take a step back for a moment, though, and talk about what this might look like—a website’s system might look like—without LDAP at all. You have a website. It’s running WordPress. We’ll call it YourSite.com and one of your users, we’ll call them alice, logs into the site. So to successfully log in to that site, WordPress first checks its WP users table for an entry that matches Alice’s account credentials. If those credentials exist and they match the ones submitted by the user, then Alice is successfully logged in, everything is okay, you get the Dashboard, or you get the homepage, it all looks fine. In this setup the user’s account information is stored by WordPress, for WordPress, and is only available to WordPress so we call that application-specific data.

Now let’s imagine that you want to add another app to your network. Maybe you have an intranet, right, and you want to add Nextcloud. This is kind of a Google Docs replacement. You could, and most organizations that I’ve seen typically do just tell Alice that, y’know, they now have two user accounts. They have a WordPress account, right? And they have a completely separate account for Nextcloud. In my experience, this causes a lot of problems. Among other issues, it means that users now must manage two user accounts: two passwords, two user profiles independently. Most users will probably choose the same password and the same profile information on both systems, but once they change their password on one system, the other system isn’t informed, and that leads to confusion, not to mention a lot of help desk tickets.

So this is a classic problem that LDAP is designed to solve. Now, with an LDAP server, you can store account details in a way where you can provide a Centralized Authentication Service, also called a CAS, for any LDAP-capable application that you choose to add to your network. So now, regardless of which app server Alice logs into, their account credentials are always the same. And when they change their password on the one side, say WordPress, they can immediately use their new password to log in to Nextcloud because that authentication check is happening in one central place, which is that LDAP server at the top.

So, let’s see how you can configure this using—whoops, that was too fast—using, WordPress. We’re going to go back to our WordPress demo and I’m going to try to run through this a little bit quickly, I apologize because I’m running a little short on time. But what I have here is another install of Nextcloud. This is a brand-new Nextcloud instance that’s running on the same machine as the other one. And I’m going to go ahead and create a new admin account. If Nextcloud is familiar to any of you this will look pretty familiar because it’s basically just as I did before. It’s a completely blank Nextcloud instance, without anything pre-loaded. So no users, no files, no nothing.

So this is Nextcloud for those of you who haven’t seen it. It looks a lot like Google Docs. Y’know, you can upload files, you can download files, you can open up text files. You can share photos, that kind of thing. You can take a look at the users database here. There’s just one user, the admin, there’s nothing else. And that is just blank Nextcloud. Now we also need, of course, is the wpnyc.demo—oh, yeah, hello—we need the plugin for WP-LDAP, which is…. There are a lot of LDAP plugins, but mine is the one called just WP-LDAP, by me, here. It’s pretty small and pretty new, because it’s not very well-known. But I’m going to go ahead and install it. And what this is going to let me do, when I go to Network Activate it, I will have a new option under my Network Settings called LDAP Settings.

Unfortunately, I need to be using HTTPS to manage it because this is kinda sensitive. So we’re gonna go ahead and do that like this. Password. There we go, LDAP settings. Now, I’m not going to show this because I don’t have time, but I also have a LDAP server running on port 389 on that same machine. Installing an LDAP server, if you’re a server admin, is usually as simple as “sudo apt install slapd” or the standalone LDAP daemon. For the time being, I’m gonna have to skip that.

I set it up so that it has a Bind DN, which is basically like the user account that you’re using to admin the site. This is basically the same as the MySQL user. Y’know how you have a MySQL database and your WordPress website needs to know what the login credentials for the database are? Same exact procedure. In this case, it’s not a MySQL database, it’s an LDAP database. So the syntax is a little bit different. But in general, it looks something like this. DC equals WPNYC, and DC stands for “domain component,” so this is the dots, right? Instead of dot com or dot demo, I’m using DC equals and “CN” is “common name.”

Let’s go ahead and actually double-check that that is correct because I don’t want it to not be. Okay, so we logged in over there, and I’m just going to copy and paste this from my notes to make double-sure that I have this right. So, I’m using an LDAP search tool on a command line, asking for the host, which is localhost in this case, using external authentication meaning the OS itself, to ask for the config common name, and I want the OLC or online configuration root DN. The root DN is basically superuser. You know how WordPress has “Super Admin”? This is Super Admin for LDAP. And sure enough it’s cn admin, dc wpnyc and demo. So that’s right. And then the base is just the end here, the same. WPNYC Demo. You can change this if you want, but it is effectively the same. So, for example, if you wanted to do, a different directory tree you could do OU equals people and this is like, basically choosing the table. Where in the database do you actually want to store what we’re about to put? In our case, ’cause this is a simple demo, we’ll just put it at the root over there. And that’s it. We save that change.

Now your WordPress can talk to LDAP. What does that mean? It means if we create a new user here—let’s say we’re going to make a new user on our Network, and we’ll call it test LDAP and it’s gonna be testldap@wpnyc.demo. We’re gonna add that user. Hopefully, if I got that right—no, I don’t want notifications—if we configure now Nextcloud to add LDAP integration, too. Now, Nextcloud ships with LDAP integration so we don’t even have to write a plugin for this. We’re gonna go to the Apps page on Nextcloud, enable the LDAP user and group backend plugin. We’ll now go to the admin screen for LDAP. We’ll go to LDAP and AD integration, and we’re gonna give it the same details that we gave WordPress. So, 127.0.0.1. Nextcloud has some nice JavaScript that can detect the port. The user DN was cn equals admin, domain component wpnyc, domain component demo, I believe. And the password was password. Let’s see if that, yup. And detect base DN. There we go. And test base DN. We look good.

So we’re gonna hit “Continue.” And now you can see we’ve got three entries available. These are inetOrgPersons. A user account in LDAP is an inetOrgPerson, an Internet organization person. We can verify this, we found 1 user. We’re gonna continue. And here you can say how do you want them to login? Username only, or username and email? We can go with either. This is basically, y’know, log in with your email, log in with your username. Just like WordPress. And if we do testldap here. Yeah, “User found and settings verified.” Continue on, and we’re good. So now, when we go to the users screen, you’ll see another one. And there you go. Now we can log in with this user.

So for instance, let’s say—actually, I can’t log in with this user, because I don’t know their password.

[laughs]

But, if we edit their password and set it, let’s set it to something simple like password. Yeah, confirm that password. We’ll update that. We’ll log out of Nextcloud as the admin, and we’ll log back in as the testldap user. And there we go. We didn’t have to make a user account on Nextcloud because we made on WordPress, and now, no matter how many apps you add to your intranet or your site, you now have one authentication store for all of your users. This is really useful for, for example, employees inside of a company. Taking out one, deleting them from the LDAP store will remove their access to all your apps. It’s also portable so you can transfer it from, y’know, one app to another, as long as the apps you’re using can speak LDAP. And, with WP-LDAP, WordPress can. So that was WP-LDAP on WordPress.

Pro-tips on this: it’s built for Multi-network, not just Multi-site installs. So if you’re not familiar with WordPress Multisite, read about that. Once you’ve read about that, check out the article, again on the WordPres Codex on Multi-network. This is a Network of WordPress Networks, and WP-LDAP works with that, too. You can set different servers, LDAP servers, for different networks so you can do things like network segmentation or perhaps round-robin load balancing, it’s kind of up to you. It’s also already aware of the WP PGP Encrypted Emails plugin, so if you use both of those and your users supply an S/MIME certificate, that will get sent over to LDAP and that will allow you to do transparent email encryption for things that are configured for that, such as iPhones in a BYOD or Bring-Your-Own-Device environment. Microsoft Outlook supports this out of the box as well. All of these features are standard protocols that are for free that you can install on WordPress sites of any size that you never have to pay for, if you don’t want to. You could. I wouldn’t. All of this, of course, is RFC 2798 compliant so any consumer that speaks LDAP—Apple Contacts, iOS, Mozilla Thunderbird Address Book—you can get a people directory and actually have, like, email autocomplete lookups for all of your employees or mailing list subscribers or anything like that.

All right, that was Centralized Authentication Service for LDAP. I don’t know if we have time for Q&A?

>> SCOTT BECKER: We can do a five minute Q&A.

>> MAYMAY: Very short Q&A. I realize that was a lot of information very quickly as well. Yeah.

>> AUDIENCE MEMBER: I run an NGO, a non-profit organization, and we have an account for Google Apps, because they gave us for free. Can I use LDAP instead of Google?

>> MAYMAY: Oh, yeah. Google uses LDAP behind the scenes. So, this is what they’re using. Right, like, there’s not a difference in the technology between Google, Facebook, and this. It’s just a matter of whether or not they put the branding and the sheen on it to make sure that you feel like you’re using their thing, as opposed to the standard thing.

>> AUDIENCE MEMBER: Can you use PGP encryption with Google?

>> MAYMAY: Oh, yeah! I do all the time. Yeah.

>> AUDIENCE MEMBER: Does this address the problem where WordPress sends out emails and it looks like spam? Like, in GMail, it goes to the spam folder, but with Sendgrid and some of these other guys it doesn’t? Does that fix that?

>> MAYMAY: Sadly, no. So the question was if these plugins fix email looking like spam from WordPress. And unfortunately, they don’t. They don’t because of a number of reasons, which we can talk about maybe later if you want, but the short answer is no. The longer answer is they might even make you look a little more suspicious only because they would rather you use their thing.

All right, so, any last question about that? All right.

In case it wasn’t clear, all these plugins are freely available on the WordPress plugin repository today. Here are their permalinks. I’m gonna put up these slides somewhere so you can take a look at them on your time and hopefully we’ve got the recording at some point as well.

Again, my name is maymay. My homepage is maymay.net. It might be down. If it is, try again in a little bit. Again, I get DDoS’ed a lot. At maymay.net the very top link is “Download my digital business card.” Click it to download and import my vCard into your contact app, and that’s all I got. Thank you so much for your time and attention, everyone.

My 2009 essay kinda-sorta about an Anarchist “Internet of Things”

I wrote an essay in 2009 about the Internet of Things, before people were calling it “the Internet of Things.” When I re-read it this afternoon, in 2017, I noticed something rather queer. It wasn’t actually about the Internet of Things at all. It was actually a personal manifesto advocating Anarchism, and condemning techno-capitalist fascism.

Yes, really.

In 2009, despite having barely turned 25 years old, I had already been working as a professional web developer for a little over a decade. (That arithmetic is correct, I assure you.) At the time, I had some embarrassingly naïve ideas about Silicon Valley, capitalism, and neoliberalism. I also had no idea that less than two years later, I’d be homeless and sleeping in Occupy encampments, and that I’d remain (mostly) happily houseless and jobless for the next six years, up to and including the time of this writing.

The story of my life during those two years is a story worth telling…someday. Today, though, I want to remind myself of who I was before. I was a different person when 2009 began in some very important ways. I was so different that by the time it ended I began referring to my prior experiences as “my past life,” and I’ve used the same turn of phrase ever since. But I was also not so different that, looking back on myself with older eyes, I can clearly see the seeds of my anti-capitalist convictions had already begun to germinate and root themselves somewhere inside me.

Among the many other things that I was in my past life, I was an author. I’ve always loved the art of the written word. My affinity for the creativity I saw in and the pleasure I derived from written scripts drew me to my appreciation for computer programming. That is its own story, as well, but the climax of that trajectory—at least by 2009—is that I was employed as a technical writer. I blogged on a freelance basis for an online Web development magazine about Web development. I had already co-authored and published significant portions of my first technical book. And, in 2009, I had just completed co-authoring a second.

That second book was called, plainly enough, Advanced CSS, and was about the front-end Web development topic more formally known as Cascading Style Sheets. But that’s not interesting. At least, no more interesting than any other fleeting excitement over a given technical detail. What’s arguably most revealing about that book is the essay I contributed, which for all intents and purposes is the book’s opening.

My essay follows in its entirety:

User agents: our eyes and ears in cyberspace

A user agent is nothing more than some entity that acts on behalf of users themselves.1 What this means is that it’s important to understand these users as well as their user agents. User agents are the tools we use to interact with the wealth of possibilities that exists on the Internet. They are like extensions of ourselves. Indeed, they are (increasingly literally) our eyes and ears in cyberspace.

Understanding users and their agents

Web developers are already familiar with many common user agents: web browsers! We’re even notorious for sometimes bemoaning the sheer number of them that already exist. Maybe we need to reexamine why we do that.

There are many different kinds of users out there, each with potentially radically different needs. Therefore, to understand why there are so many user agents in existence we need to understand what the needs of all these different users are. This isn’t merely a theoretical exercise, either. The fact is that figuring out a user’s needs helps us to present our content to that user in the best possible way.

Presenting content to users and, by extension, their user agents appropriately goes beyond the typical accessibility argument that asserts the importance of making your content available to everyone (though we’ll certainly be making that argument, too). The principles behind understanding a user’s needs are much more important than that.

You’ll recall that the Web poses two fundamental challenges. One challenge is that any given piece of content, a single document, needs to be presented in multiple ways. This is the problem that CSS was designed to solve. The other challenge is the inverse: many different kinds of content need to be made available, each kind requiring a similar presentation. This is what XML (and its own accompanying “style sheet” language, XSLT) was designed to solve. Therefore, combining the powerful capabilities of CSS and XML is the path we should take to understanding, technically, how to solve this problem and present content to users and their user agents.

Since a specific user agent is just a tool for a specific user, the form the user agent takes depends on what the needs of the user are. In formal use case semantics, these users are called actors, and we can describe their needs by determining the steps they must take to accomplish some goal. Similarly, in each use case, a certain tool or tools used to accomplish these goals defines what the user agent is in that particular scenario.2

A simple example of this is that when Joe goes online to read the latest technology news from Slashdot, he uses a web browser to do this. Joe (our actor) is the user, his web browser (whichever one he chooses to use) is the user agent, and reading the latest technology news is the goal. That’s a very traditional interaction, and in such a scenario we can make some pretty safe assumptions about how Joe, being a human and all, reads news.

Now let’s envision a more outlandish scenario to challenge our understanding of the principle. Joe needs to go shopping to refill his refrigerator and he prefers to buy the items he needs with the least amount of required driving due to rising gas prices. This is why he owns the (fictional) Frigerator2000, a network-capable refrigerator that keeps tabs on the inventory levels of nearby grocery stores and supermarkets and helps Joe plan his route. This helps him avoid driving to a store where he won’t be able to purchase the items he needs.

If this sounds too much like science fiction to you, think again. This is a different application of the same principle used by feed readers, only instead of aggregating news articles from web sites we’re aggregating inventory levels from grocery stores. All that would be required to make this a reality is an XML format for describing a store’s inventory levels, a bit of embedded software, a network interface card on a refrigerator, and some tech-savvy grocery stores to publish such content on the Internet.

In this scenario, however, our user agent is radically different from the traditional web browser. It’s a refrigerator! Of course, there aren’t (yet) any such user agents out crawling the Web today, but there are a lot of user agents that aren’t web browsers doing exactly that.

Search engines like Google, Yahoo!, and Ask.com are probably the most famous examples of users that aren’t people. These companies all have automated programs, called spiders, which “crawl” the Web indexing all the content they can find. Unlike humans and very much like our hypothetical refrigerator-based user agent, these spiders can’t look at content with their eyes or listen to audio with their ears, so their needs are very different from someone like Joe’s.

There are still other systems of various sorts that exist to let us interact with web sites and these, too, can be considered user agents. For example, many web sites provide an API that exposes some functionality as web services. Microsoft Word 2008 is an example of a desktop application that you can use to create blog posts in blogging software such as WordPress and MovableType because both of these blogging tools support the MetaWeblog API, an XML-RPC3 specification. In this case, Microsoft Word can be considered a user agent.

As mentioned earlier, the many incarnations of news readers that exist are another form of user agent. Many web browsers and email applications, such as Mozilla Thunderbird and Apple Mail, do this, too.4 Feed readers provide a particularly interesting way to examine the concept of user agents because there are many popular feed reading web sites today, such as Bloglines.com and Google Reader. If Joe opens his web browser and logs into his account at Bloglines, then Joe’s web browser is the user agent and Joe is the user. However, when Joe reads the news feeds he’s subscribed to in Bloglines, the Bloglines server goes to fetch the RSS- or Atom-formatted feed from the sourced site. What this means is that from the point of view of the sourced site, Bloglines.com is the user, and the Bloglines server process is the user agent.

Coming to this realization means that, as developers, we can understand user agents as an abstraction for a particular actor’s goals as well as their capabilities. This is, of course, an intentionally vague definition because it’s technically impossible for you, as the developer, to predict the features or capabilities present in any particular user agent. This is a challenge we’ll be talking about a lot in the remainder of this book because it is one of the defining characteristics of the Web as a publishing medium.

Rather than this lack of clairvoyance being a problem, however, the constraint of not knowing who or what will be accessing our published content is actually a good thing. It turns out that well-designed markup is also markup that is blissfully ignorant of its user, because it is solely focused on describing itself. You might even call it narcissistic.

Why giving the user control is not giving up

Talking about self-describing markup is just another way of talking about semantic markup. In this paradigm, the content in the fetched document is strictly segregated from its ultimate presentation. Nevertheless, the content must eventually be presented to the user somehow. If information for how to do this isn’t provided by the markup, then where is it, and who decides what it is?

At first you’ll no doubt be tempted to say that this information is in the document’s style sheet and that it is the document’s developer who decides what that is. As you’ll examine in detail in the next chapter, this answer is only mostly correct. In every case, it is ultimately the user agent that determines what styles (in which style sheets) get applied to the markup it fetches. Furthermore, many user agents (especially modern web browsers) allow the users themselves to further modify the style rules that get applied to content. In the end, you can only influence—not control—the final presentation.

Though surprising to some, this model actually makes perfect sense. Allowing the users ultimate control of the content’s presentation helps to ensure that you meet every possible need of each user. By using CSS, content authors, publishers, and developers—that is, you—can provide author style sheets that easily accommodate, say, 80 percent of the needs of 90 percent of the users. Even in the most optimistic scenario, edge cases that you may not ever be aware of will still escape you no matter how hard you try to accommodate everyone’s every need.5 Moreover, even if you had those kinds of unlimited resources, you may not know how best to improve the situation for that user. Given this, who better to determine the presentation of a given XML document that needs to be presented in some very specific way than the users with that very specific need themselves?

A common real-life example of this situation might occur if Joe were colorblind. If he were and he wanted to visit some news site where the links in the article pullouts were too similar a color to the pullout’s background, he might not realize that those elements are actually links. Thankfully, because Joe’s browser allows him to set up a web site with his own user style sheet, he can change the color of these links to something that he can see more easily. If CSS were not designed with this in mind, it would be impossible for Joe to personalize the presentation of this news site so that it would be optimal for him.

To many designers coming from traditional industries such as print design, the fact that users can change the presentation of their content is an alarming concept. Nevertheless, this isn’t just the way the Web was made to work; this is the only way it could have worked. Philosophically, the Web is a technology that puts control into the hands of users. Therefore, our charge as web designers is to judge different people’s needs to be of equal importance, and we can’t do this if we treat every user exactly the same way.6

  1. This is purposefully a broad definition because we’re not just talking about web pages here, but rather all kinds of technology. The principles are universal. There are, however, more exacting definitions available. For instance, the W3C begins the HTML 4 specification with some formal definitions, including what a “user agent” is. See http://www.w3.org/TR/REC-html40/conform.html. []
  2. In real use cases, technical jargon and specific tools like a web browser are omitted because such use cases are used to define a system’s requirements, not its implementation. Nevertheless, the notion of an actor and an actor’s goals are helpful in understanding the mysterious “user” and this user’s software. []
  3. XML-RPC is a term referring to the use of XML files describing method calls and data transmitted over HTTP, typically used by automated systems. It is thus a great example of a technology that takes advantage of XML’s data serialization capabilities, and is often thought of as a precursor to today’s Ajax techniques. []
  4. It was in fact the much older email technology from which the term user agent originated; an email client program is more technically called a mail user agent (MUA). []
  5. As it happens, this is the same argument open source software proponents make about why such open source software often succeeds in meeting the needs of more users than closed source, proprietary systems controlled solely by a single company with (by definition) relatively limited resources. []
  6. This philosophy is embodied in the formal study of ethics, which is a compelling topic for us as CSS developers, considering the vastness of the implications we describe here. []

Introducing “Subresource Integrity (SRI) Manager” for WordPress

Subresource Integrity (SRI) is a proposed W3C standard that Web developers can make use of to protect their websites from being used in JavaScript-based DDoS attacks, such as the one recently suffered by GitHub.com. It’s not yet widely supported by browsers but is harmless to add to your pages for browsers that don’t support it.

Using SRI declarations is a simple matter of adding an integrity attribute to the HTML tag that points to an external resource (like a <script> tag) that contains a hash specifier (like sha256) and the proper hash of the content you expect to load. This indicates to the browser that the resource to which the element points should have the given hash. If the content that the browser receives from the remote server does not produce the specified hash, the browser ignores (refuses to load) the resource, and fires an error event at the element, instead:

<script src="//example.com/example-library.js" integrity="sha256-ab3c54ef..998756"></script>

I wanted to make use of this immediately, but I didn’t want to manually hash all the libraries that my website calls to. These include stylesheets from content distribution networks like Bootstrap and JavaScript libraries such as jQuery DataTables. So I wrote a WordPress plugin that keeps track of the hashes for me.

With the Subresource Integrity (SRI) Manager plugin for WordPress (WP-SRI), each time a stylesheet or JavaScript is enqueued by a WordPress plugin or theme (using wp_enqueue_style() or wp_enqueue_script()), WP-SRI makes a note of the requested resource URL and adds it to a list of known resources. If WP-SRI has not seen the resource before, it grabs the resource content itself, produces a hash, and saves that alongside the resource URL in WordPress’s wp_options table. When HTML is printed to the screen, it adds the integrity attribute and the associated hash automatically.

Using this plugin can dramatically reduce the likelihood that visitors to your site will be strong-armed into participating in an HTTP DDoS attack. In future versions of this plugin, I also hope to provide an easy-to-use interface for site administrators so that they can maintain a customized list of resource hashes, and to trigger on-demand integrity checks of these resources.

Please submit feature requests and/or patches to the issue tracker on GitHub, and post support requests to the WordPress plugin support forum. Thanks. :)

My SVG is Bigger than Your Flash – Presentation Notes

A few weeks ago, right before the Christmas holidays, I gave a brief internal presentation about SVG, an XML-based web standard for interactive images, to my coworkers. Since I was told that the presentation was helpful, I thought I’d share my notes. Those notes follow:

Introduction

First, credit where credit is due.

I learned almost everything I know about SVG from an acquaintance I met in Australia named Dmitry Baranovskiy. He is a superb client-side web developer who I helped add documentation for a brief time for one of his SVG-related projects called RaphaëlJS. I’ll show you that in a little bit.

Okay, so what is SVG?

What is SVG?

In a sentence, according to Adobe, the originators of the technology:

Scalable Vector Graphics (SVG) is a text-based graphics language that describes images with vector shapes, text, and embedded raster graphics.

What does that mean?

But wait, there’s ”more”.

Why use SVG?

  • It’s text-based, which means it’s highly searchable. Remember SEO in the heyday of the “Flash website”? Now Flash has all kinds of meta attributes to provide extra SEO oomph, but if we had just used SVG in the first place, we wouldn’t have had this problem.
  • Mobile support: SVG comes in a few flavors that are all interoperable with each other in order to define different levels of support. Whereas Flash is monolithic, SVG comes in profiles that support low-end devices with reasonable quality while still providing high-fidelity content. Just like the Web was designed to do.
  • Easy to learn: there isn’t much new here because it uses everything you already know about XML, CSS, and JavaScript to provide every feature it has. That’s right, you can use plain old HTML for your page content, add some CSS to style it and the SVG, and then JavaScript to script it. “The trifecta” just added another dimension.
  • And, if you care about such things, SVG has been an open standard well before Flash was.

Browser support

  • Opera has full support and has had some amount of support since Opera 8.
  • Gecko has had increasing support since 2005.
  • WebKit has had increasing support since 2006.
  • Internet Explorer (still) has no native support. We hate you, Internet Explorer. (That said, however, Microsoft has just recently joined the W3C’s SVG Working group, so there may yet be some hope.)

What can we do about IE today?

Here is the breakdown of options we have to deal with IE’s frustrating lack of support.

  • Rely on HTTP content negotiation to serve SVG to supporting browsers, and raster images to Internet Explorer. Works great, except now we’re still using raster images, so basically it feels like 1995 again. It’s also what Wikipedia does.
  • Use plugins. Adobe has had one forever called the Adobe SVG Viewer. It’s pretty good…but it’s a plugin. I’m not sure how Flash got away with also being a plugin, but whatever.
  • Relent and use Flash on the client-side, but revel in delicious SVG pureness for our source files: Using SVG Path Data in Flash.
  • Use a cross-browser API. Turns out, there are quite a few:
    • SVG in The Dojo Toolkit – Early and solid, but it uses Dojo (and we don’t).
    • svgweb – Google’s experimental library. It’s got the biggest hyperight now.
    • RaphaëlJS, written by my acquaintance Dmitry Baranovskiy and, of course, my favorite cross-browser SVG toolkit. Since it’s stand-alone, you can actually mix-and-match with your favorite JavaScript framework. In other words, you take your pick of jQuery, YUI, Prototype, or whatever, and layer RaphaëlJS on top for all your SVG-related work. Now that’s creamy.

How do SVG-writing JS frameworks work?

  1. Provides a cross-browser JavaScript API (think jQuery) for writing SVG commands.
  2. Writes to the DOM based on your Raphaël commands:
    • Writes SVG to supporting browsers.
    • Writes VML1 to Internet Explorer.
  3. Makes you coffee.

Hype it up

1 Minute Intro to (Google’s) SVG Web + Demos + rock and roll (because why not?)

Sources and additional resources

  1. VML is Microsoft’s proprietary, non-standard Vector Markup Language. It’s basically the exact same thing as SVG, but only works in Internet Explorer. Because whereas Apple only thinks differently, Microsoft has to do it differently. We hate you, Microsoft. []

Why CSS needs delegation capabilities and not “variables”

It’s been too long since I joined the fun, if amazingly heated, debates over the direction that Web standards are moving in. Recently, given the “free” time to do so, I decided to dive head first into what is (sadly) an almost 14 year old debate. The result is this blog post, which is mostly a response to Bert Bos’s essay Why “variables” in CSS are harmful and Matt Wilcox’s opposing response to that essay, Why CSS needs to borrow from programming languages. Their articles are each worthy of a read, possibly before this one.

Here’s the summary of my argument.

Adding many “programmatic” features to the CSS language such as variables, macros, or flow control is a mistake. However, CSS’s failure to simply encode visual relationships (instead of merely typographic properties)—a severe deficiency in the core language itself—requires the addition of delegation features. With the additional capability to reference an arbitrary element’s computed value regardless of its hierarchical context, CSS will be more accessible to both amateur and professional web designers, more capable, and will more forcefully promote the semantic Web and its ideals.

In this corner: CSS variables are harmful

Bert does a great job of summarizing the conclusion of his argument himself. In his essay, Bert says:

Adding any form of macros or additional scopes and indirections, including symbolic constants, is not just redundant, but changes CSS in ways that make it unsuitable for its intended audience. Given that there is currently no alternative to CSS, these things must not be added.

As we all know, one of the wonderful things about CSS is that the core language itself is remarkably simple. (What’s not simple is the spectacular way browser manufacturers have destroyed everyone’s hope that implementing CSS-based designs in the real world will ever be easy, but that’s a whole different can of worms.) Fundamentally, CSS’s syntax can be explained with a mere three major components: property/value pairs, declaration blocks, and rule sets.

What this means is that CSS as a language is stupidly easy to learn. I think everyone would agree that it’s certainly easier to learn than, say, JavaScript or XSL. Now, that’s important because, without putting too fine a point on it, Bert mentions multiple times that CSS’s “intended audience” are the diverse and likely relatively technically ignorant content authors that are responsible for the overwhelming majority of web pages on the public Internet today.

He makes the very good point that The value of the semantic Web isn’t defined by how well structured the best documents are, but by how well structured the vast majority of documents are. In other words, CSS needs to remain instantly useable and reusable to these untrained, amateur web content publishers for the benefits of self-describing documents (i.e., the semantic Web) to see mass adoption.

To wit:

reusing other people’s style sheets is more difficult if those style sheets contain user-defined names. Class names are an example. Their names may suggest why the author created them (assuming they are in a language you understand), but typically you will have to look at the document to see where they occur and why. Symbolic constants make that problem worse.

And, later:

For many people, style sheets with constants will thus simply not be usable. It is too difficult to look in two places at once, the place where a value is used and the place where it is defined, if you don’t know why the rule is split in this way. Many people are confused by indirection anyway and adding an extra one, in addition to the element and class names, has the same effect as obfuscating the style sheet.

Whether or not you believe Bert Bos is underestimating the average web designer, it’s pretty clear that these are really good points. Nobody wants CSS to be obfuscated, hard to learn, or hard to reuse. That’d just be crazy talk.

In the other corner: CSS variables are a real-world requirement

The more features you add to an application, a programming language, or indeed any software, the more difficult it becomes to grok it. As the Python people would say, the larger a language gets the more difficult it is to hold all of it in your head. Nevertheless, adding “features” is sometimes the only way to add capabilities, and I don’t think anyone in their right mind would argue that, once written, software should never change. (That’d just be crazy talk, too.)

In his opposing arguments, Matt Wilcox recognizes this when he says, Yes, the syntax should be simple, but the capabilities of CSS should not. What he’s alluding to without verbalizing it is the balance between adding necessary capabilities without unnecessarily growing the “size of the language.”

However, Matt says that modern web design methodologies (e.g., separation of concerns between structure, presentation, and behavior) dictate that CSS needs more capabilities than it currently has:

CSS lacks capabilities to allow truly flexible design, requiring layer upon layer of ‘tricks’ to accomplish certain objectives, requiring content to be structured ‘just so’ to achieve a display objective, or in the case of some designs proving instead to be completely incapable.

[…]

CSS’s positioning is a cludge. It’s a cludge because you can only position relative to the last positioned parent container. Well, that limitation in itself dictates that all positioning relies upon how the content is structured. And that means the presentation and the content are not truly separable.

To align CSS’s capabilities with the requirements of real-world web design objectives, he says, CSS needs to be capable of describing relationships between semantically and structurally arbitrary but visually related elements.

Visual design is fundamentally about relationships between elements. For all of the artistic flourishes and creativity, it’s about relationships. ‘That yellow’ only grabs your attention because of its contrasting relationship with ‘that blue’. ‘This heading’ only works as a heading because of it’s exaggerated relationship to the size of the body text. […] CSS has no clue about relationships, period. And that’s why CSS as it stands right now, is not good enough. That’s why CSS without variables (true variables), without basic logic, without maths, can never be as flexible as we need it to be.

This is what web designers have been complaining about for (what feels like hundreds of) years. The fact that CSS has no capability to describe presentational relationships between elements in addition to directly describing an individual element’s presentational properties is a gaping hole that sorely degrades its ability to be a media-agnostic styling language. Every single web designer I’ve worked with has gasped at this omission, and though at first I didn’t understand why, the more I understood the principles behind graphic design the more I came to realize how fundamentally problematic this omission really is.

Adding delegation makes CSS easier for designers

As Matt eloquently stated, design is all about relationships. Good web designers create designs by constructing visual elements that have strong, often exacting relationships with other visual elements. There are many names and examples for this: visual language, visual hierarchy, the golden ratio, the grid, visual balance, the typographer’s scale, and so on.

What happens when the designer tries to define a relationship between elements? “How do I say that the whitespace between element A and element B should always be the same? How do I define element A’s height as half of element B’s?” These definitions, which are natural and necessary to the way designers work in both their mind and their mediums, are impossible to encode in CSS.

The closest you can get is declaring the same values to each element’s properties, not describing the relationship itself. This suffices only so long as these values are known ahead of time and are the same as one another, which severely limits the design possibilities we are capable of (without resorting to what Matt calls “tricks”). That’s why achieving simple visual effects are actually very complex and so, sadly, that’s where you’ll find the majority of indirection and obfuscation in CSS today. (I’m looking at you, faux columns.)

So who wins?

Both Bert Bos and Matt Wilcox have made some great points. Bert rightfully wishes to keep CSS lean and simple, even at the expense of some arguably beneficial styling power. Matt, on the other hand, argues that our needs as web designers have evolved faster than the technology to the point where CSS is too limited, fundamentally so.

The truth is, they’re both right. And they’re both wrong. Or rather, they are each taking a position that is too extreme. Bert’s absolutely correct when says that many of these proposed extensions are redundant and harmful, and yet Matt’s also correct that CSS lacks some fundamental capabilities that designers expect to be present.

Bert says that the CSS capabilities everyone’s asking for can be implemented using techniques that don’t rely on CSS whatsoever. These techniques, he says, make things like true CSS variables “redundant.”

There are examples of CSS with constants to satisfy all styles of programming, e.g.: David Walsh (in PHP), Tedd Sperling (in PHP), Digital Web Magazine (in PHP), Eco Consulting (in SSI), and Christian Heilmann (SSI and PHP).

Quite simply, he’s correct in stating that programmatic features need not be added to CSS proper to achieve desired results, but he’s incorrect in his apparent thinking that designers will be able to use these other tools to leverage CSS. Take, for instance, the probably more familiar (though not linked above) notion of using JavaScript to manipulate CSS values.

var x = document.getElementById('SideBar'); // get #SideBar element
var y = document.getElementById('MainColumn'); // get #MainColumn
var z = document.defaultView.getComputedStyle(y, '').getPropertyValue('height'); // get computed height of #MainColumn
x.style.height = ( parseInt(z) / 2 ) + 'px'; // set #SideBar's height 1/2 of #MainColumn's

This is an example of programmatic code that uses variables and expressions. It sets the element with the ID of SideBar to half the pixel height of the element with the ID of MainColumn. It does this by obtaining the MainColumn‘s height (at the time this code runs) and saving it in a variable, then performs some trivial math to half the value and use the result as the pixel height of the SideBar.

Doing this is currently impossible with CSS alone, yet it’s something that clearly belongs with whatever other “presentational” code exists and not in “programmatic” scripts that would otherwise be charged with defining “functionality.” As Matt states, using JavaScript to “script” solutions to CSS’s shortcomings like this is not an acceptable answer.

CSS doesn’t have [basic logic or maths]. Nor is it the job of JavaScript to make up for this lack of abilities. JavaScript is about interaction behaviour, and what we are talking about here is pure display logic. Not interaction logic.

Moreover, the place designers expect to put code like this is, of course, into a CSS style sheet. The way designers expect to put code like this into CSS is by adding delegation features. Requiring designers to learn JavaScript (or any other programming language) to encode such design relationships is nothing short of ridiculous. In what world is that easier for untrained laymen to understand than CSS?

Adding delegation to CSS is worth the effort

One of Bert’s arguments against such additions to CSS is that implementations would become harder to create, and that we’ll (almost certainly) see more bugs.

extending CSS makes implementing more difficult and programs bigger, which leads to fewer implementations and more bugs. That has to be balanced against the usefulness of the extension.

Although I do agree with his statement that an extension’s usefulness has to be balanced against its potential costs, I think something so fundamental to design methodology as delegation greatly overcompensates for the cost of such implementation efforts. Moreover, if I understand Bert correctly and as he also discusses, the majority of implementations that would need to implement such delegation already have relatively complex internal structures to make the implementation effort somewhat easier:

There is no scoping [in proposals that only define global constants]. That means that an implementation needs a symbol table, but no stack. A stack would require a little bit more memory, but mostly it would make implementations more complex. (Although every programmer has, one hopes, learnt to program a symbol table with lexical scope during his training.) Constants in CSS are thus easier than, e.g., XML Namespaces, which are lexically scoped.

It is different for those CSS implementations that provide a CSS Object Model (an API for manipulating a style sheet in memory). Those implementations do need to keep track of scope in some way, because adding or removing a line of the style sheet can make a previously redundant definition become meaningful.

In order to use JavaScript to solve many of the shortcomings of CSS, as huge numbers of professional web developers do routinely, we use the very CSS Object Model whose prior implementation already exists for us to build upon.

CSS delegation doesn’t grow the size of the language

For the sake of argument, let’s simplify our requirement somewhat so that our somewhat contrived example of design intent is to create a relationship between the MainColumn and the SideBar elements such that they are of equal height. This is more informally known as “making columns.”

Here’s what a natural, hypothetical snippet of CSS would look like if the language supported delegation features such that it could encode visual relationships.

#SideBar { height: #MainColumn; }

This code theoretically says almost the exact same thing as the JavaScript shown earlier (save for the division, of course); it takes the computed value of the MainColumn element’s height property and applies that value to the SideBar element’s height property. In other words, “The SideBar’s (element B’s) height is always the same as the MainColumn’s (element A’s).” (Of course, this is a parse error in reality today.)

This extremely trivial example has some remarkably far-reaching implications, and yet there is really nothing radical about its syntax. Making this a reality significantly expands the capabilities of CSS without dramatically increasing the size of the language. This capability would not only beat the pants offCSS tables,” it also potentially obsoletes the arguably misguided efforts of the CSS3 Advanced Layout and Grid Positioning modules, too.

We’ve long since abandoned table layouts because they force us to use presentational markup. That’s still what “CSS tables” force us to do, too. In other words, with display: table, the SideBar needs to be a child of the MainColumn element or, maybe worse and more likely, a child of a semantically meaningless wrapper element.

CSS positioning was introduced with the promise of freeing us from source-order-dependent styling, without which there is no hope of efficiently abstracting presentation away from structure. Moreover, abstracting presentation away from structure is the single most important prerequisite needed to improve document reusability and strengthen the semantic Web. Absolute positioning works, but limitations elsewhere in CSS mean its use is problematic for many designs, so in practice it doesn’t gain widespread adoption.

Here’s a theoretical solution to a two-column and a footer layout using CSS delegation with this semantic HTML:

<body>
    <div id="MainColumn">I'm the main column.</div>
    <div id="SideBar">I'm the right-hand sidebar.</div>
    <div id="Legalese">No one will read me.</div>
</body>

The CSS would look extremely familiar, possibly like this:

#MainColumn { margin: 0 25% 1em 0; float: left; }
#SideBar { width: 25%; min-height: #MainColumn; }

Using the same HTML, the same solution using the CSS3 Advanced Layout module would look something more like this, although to be frank I’m not certain I fully understand this syntax even after staring at it for months:

body {
    display: "a  b"
             ".  ." /1em
             "c  c"
             75% 25%
}
#MainColumn { position: a; }
#SideBar { position: b; }
#Legalese { position: c; }

Not only does there seem to me to be far more indirection in this method than there would be using CSS delegation, there is also an enormous increase to the size of the CSS language: a new (ASCII-art?!) value to the display property whose syntax is clunky at best. A similar story can be said of the CSS3 Grid Positioning module, which does lots more than just add a new (already complex) gr CSS unit.

The upshot is that the Advanced Layout and the Grid Positioning modules are doing some of the right things in many of the wrong ways. Both those modules add unnecessary complexity to CSS without giving designers a natural way to say what they mean. They do more to introduce obfuscation and indirection than simple delegation would, and they aren’t as broadly capable. Both of them try to solve a specific problem instead of dealing with fundamental deficiencies in the toolset designer’s have to work with.

Designers want relationships via delegation, not variables

Adding delegation such as that I’ve just shown is a natural, necessary addition to CSS because it is how designers create visual components—such as grids—in their designs. Variables (and constants, and macros, etc.), which simply reuse and modify pre-defined statements aren’t what designers care about. Adding them will bloat CSS without adding useful functionality.

“Okay,” you may be saying to yourself, “but delegation is itself a kind of variable, isn’t it?” Technically yes, however adding delegation resolves the core deficiency in the CSS language that designers need to use every day. Yes, it’s technically a form of variable, but that’s not how designers think of it. To say that one element’s visual properties is like another makes a variable only by creating a logical and visually appropriate mapping from the first element’s property to the second independent of markup, thereby avoiding indirection in the form of a variable name or other unfamiliar symbol.

Delegation like this doesn’t require the addition of anything other than what already exists in CSS. Class names and ID values are identifiers whose indirection people already have to deal with. Using them for delegation (to reference another element’s style) doesn’t increase the cognitive load any more than using them to reference HTML elements does. Though untested, the cognitive load might actually be even less since the CSS delegation’s references could be in the same (style sheet) file.

Moreover, delegation will increase the likelihood of document reusability by enabling style sheets to be more self-describing, more self-referential, in a similar way as good markup is. It satisfies a very fundamental need that designers have to define graphical relationships between elements. At the same time, it does so in a way that is natural to both their way of thinking and beneficial to the separation of concerns principle on which the “web stack” (the trifecta of HTML, CSS, and JavaScript) is based.

WP-Oomph: Add the Oomph Microformat Overlay to your WordPress blog

I’ve just developed a completely idiotic (by which I mean brain-dead simple) plugin for WordPress that will add the Oomph Microformat Toolkit to all WordPress-generated pages. If you use a WordPress template that encodes your data with valid microformats anywhere on your page, this means when you install the plugin your users will see the Oomph microformat overlay and will be able to instantly export this encoded data.

This page is a live example, so if you’re using a JavaScript-enabled browser you should see a microformat icon on the top-left of the viewport that is pulling data from (at least) my “The bio” section in my sidebar. Go ahead, click it. I’ll wait.

Pretty nifty, isn’t it? Naturally, all of the credit for this functionality belongs to the Oomph team, not me. If you want to learn how to add microformats to your blog, I’d recommend Emily Lewis’s latest series of blog posts, Getting Semantic with Microformats. If you want to learn how to easily add the Oomph microformat overlay to your WordPress blog, read on.

The backstory

After Ask.com’s announcement that they are adding semantic search capabilities to their search engine, there’s little doubt in anyone’s mind that the semantic web is the future’s web. As far as I know, Google has yet to reveal similar initiatives but they are clearly in the know as well. Mark Birbeck, one of the smart folks who devised RDFa, recently gave a Google Tech Talk that made the point that semantics are the next big thing in the Internet search engine game.

However, for semantic web stuff to really take hold, two things need to happen first. I think these things need to look like this:

  1. Developers must create tools, plugins, and other software that makes it possible for the wider community to create compelling, interoperable applications that support semantic encoding. Thankfully, we are already at this point, with toolkits like the Oomph Microformat toolkit coming out of MixLabs.
  2. Armed with these software tools, CMS and other publishing platforms need to adopt semantics as first-class features of their platforms, and build interfaces that end-users can make immediate use of. This is where we still need to go, though some platforms like Drupal have begun to pave the way for this.

Drupal 7 will be fantastic, I’m sure, but we live in the here and now. I saw the Oomph microformat overlay on Emily Lewis’s blog and was more convinced than ever that if everyone—programmers and laymen alike—had easy access to these tools, they’d simply be pounding down the doors to use them. So that’s why I sat down and wrote a completely idiotic plugin for WordPress that makes it completely, utterly, brain-dead simple for anyone with a microformats-enabled WordPress theme to add the overlay to their site.

WP-Oomph: Download the plugin

My request to add the plugin to the WordPress.org Plugin Directory has not yet been completed, so in the mean time I’m hosting the plugin right here. (When/if it’s accepted it’ll end up being The plugin is hosted on that site permanently.)

The latest version is: 0.1.1.

Download the latest version of the WP-Oomph plugin.

Thanks to the Oomph team’s work, the plugin is a ridiculous 1-liner (for now) that uses WordPress’s wp_enqueue_script() function to call both its included jQuery library and the Oomph library itself. And, well, that’s it. I told you it was idiotic, but at least now the whole process of microformat-enabling a WordPress blog is 100% point-and-click.

WP-Oomph: Frequently Asked Questions

I installed and activated the plugin, but nothing is different. How come?

First, view the source of your WordPres-generated page and make sure you see a line similar to the following near the top:

<script type='text/javascript' src='http://visitmix.com/labs/oomph/1.0/Client/oomph.min.js?ver=1.0'></script>

If you see that but there’s still nothing different about your page, then you probably don’t have any (valid) microformats. You might consider switching to a WordPress theme with built-in microformat support, or modifying your theme’s code to add some of your own. You can learn more about the support WordPress offers for microformats in the Microformat wiki.

The plugin does let me do X thing that I want to do! Why not?

Most likely because I haven’t taken X thing into account. Sorry, I’m not a psychic (as much as I wish I were). However, you’re encouraged to leave a comment on this post or to contact me elsewhere to request that I add capabilities to the plugin. Better yet, if you’re comfortable doing so, send me a patch.

How web designers can do their own HTML/CSS: Read Foundation Website Creation

Last month, 37signals published a short but sweet post about why web designers should do the HTML/CSS implementations for their own designs. The bottom line is, as we’ve all been saying for a long time now, that the Web is not the same kind of medium as other mediums like print. It is a fundamentally different kind of canvas than most web designers are used to using. As a result, if you as a web designer are not intimately familiar with it, you’re not going to do great work.

designing for the web is a lot less about making something dazzle and a lot more about making it work. The design decisions that matter pertain directly to the constraints of the materials. What form elements to use. What font sizes. What composition. What flow. Those decisions are poorly made at an arm’s length.

I’ve worked with many web designers in the past who only did abstractions and then handed over pictures to be chopped and implemented by “HTML monkeys”. It never really gelled well. The things that got strong attention were all the things that Photoshop did well. Imagery, curvy lines, and the frame. All the around stuff, never the it stuff.

In other words, to do great web design you have to design in the Web, not in some other medium for the Web. I mean, serious magazine firm employs designers who don’t understand how to work with page layout programs like InDesign. Why, then, do so many web design agencies employ designers who don’t know how to work with web technologies, or even how to use programs like Dreamweaver? It doesn’t really make any sense, and it’s no wonder that the resulting implementation is rarely top-notch work.

But if you’re a graphic designer who doesn’t know much about Web technologies, what are you to do? Well, as a first step, I think you should pick up my new book, Foundation Website Creation. It’s available from all good booksellers (and probably some crappy ones) as of today. The book is targeted towards all manner of web professionals, including graphic designers and website producers, who want to learn more about what it takes to actually implement a site.

If I do say so myself, the chapters on XHTML and CSS are exceptionally thorough. The book doesn’t try to turn you into an exceptional programmer. Instead, it will explain the foundational concepts you need to know to understand how XHTML and CSS actually work, and in so doing will enable you to use the tools you already know to solve problems and get things done.

I think this book will be an excellent starting point for lots of designers and other web professionals. However, it is not going to take you from zero to hero—no book can. That’s why I recommend that, after you read Foundation Website Creation and have a solid grasp of what the technology can do for you and how it actually does it, you next take a look at these excellent books:

  • DOM Scripting by Jeremy Keith — if you’re a designer that needs to add a behavioral layer with JavaScript and Ajax to your pages, you need to read this book next.
  • Mastering CSS with Dreamweaver CS3 – if you’re familiar with Dreamweaver and want to keep using it to create standards-based web sites, then I recommend you follow Foundation Website Creation with this book by Stephanie Sullivan and Greg Rewis to take your Dreamweaver skills to the next level.

As always, most of all, have fun. Because if you’re not having fun, you’re not going to make good web sites no matter what you know.

Note: As of this writing, the book listing on Amazon still publishes the wrong author list, which is very frustrating but out of my hands. At least the image of our book’s front cover lists the correct authors.

I’m getting a book published and it’s called Foundation Website Creation

For those who have been wondering what is keeping me so busy these days, the answer is that I’m working on the final stages of a book that is getting published as one of three co-authors. Not only am contributing three chapters (the technical chapters on (X)HTML and CSS, specifically), but I am also technically reviewing the entire book.

My co-authors on the book, called Foundation Web Standards Foundation Website Creation (you can pre-order now) and published by Friends of ED, an Apress company, are Jonathan Lane of Industry Interactive, Inc. and Joe Lewis, who blogs at Sanbeiji.com. I’m not going to say much more until after the book is released in late July.

For the eager, here’s the description of the book posted on the Friends of ED website:

Foundation Website Creation explores the process of constructing a web site from start to finish. There is more to the process than just knowing HTML! Designers and developers must follow a proper process to flush out goals and objectives and determine requirements both prior to, and during project development.

Large Web projects are rarely completed by a single person. Producers, project managers, designers, developers, writers, and editors all play critical parts in a project’s evolution. This book provides an overview of the entire process, and also shows project development from the perspective of these different roles. It introduces the key concepts and duties performed by every member of such a team, and gives you the skills necessary to tackle projects like a professional.

It’s quite exciting getting a book out, and it’s quite a bit more work than I’d have ever originally thought. That being said, it’s extremely rewarding. There’s a lot more work I need to do on it between now and the time it gets released to publishing, so, well…back to work I go.

Now you all know where I’ve been spending my time writing.

Browser Developer Insight and Nightmares

Dave Hyatt, a developer of Apple’s Safari web browser offered some amazingly enlightening insight into the nightmares of browser developers the other day. The only thing I can think to say is: Dave, thanks for trying.

This is a great example of how IE’s inconsistent rendering behavior has long-lasting negative effects on the adoption of CSS not only by web page authors, but web browser authors. As Dave himself says,

So now I really have no choice. This is an example of where the CSS2 standard simply can’t be followed because buggy layout engines have set a bad precedent that the rest of us have no choice but to follow.

It’s a shame that Gecko does not do the right thing in strict mode at least, but I suppose they had no choice in the matter either.

This is reminiscent of back when Opera was beginning to look more and more like Internet Explorer, but at least it’s not a bug this time.