Category: Web Design

WordPress NYC: Enterprise Features for Small Businesses Running WordPress

Earlier this week, the WordPress NYC Meetup group hosted me at their space inside the Microsoft Technology Center. I was there to present some of my recent work on “Enterprise Features for Small Businesses Running WordPress.” I had a lot of fun and really appreciated the opportunity to showcase three projects I’ve been working on recently.

You can find an archive of all recorded sessions that the organizers, Steve and Scott, of the WordPress NYC Meetup have produced at the “WPNYC TV” page on their website. Below, you can find my own presentation from their latest evening, along with a transcript and links to the original presentation materials. This includes my slide deck, presenter notes, and presentation runbook.

>> SCOTT BECKER: Maymay and I just met tonight, but I find what he does fascinating. So, instead of giving you, like, some spiel that, y’know, we’ve written up, I’m just going to read a bit about what we had here at Meetup so you know a little bit about what Maymay’s gonna talk to you about. I find it fascinating.

He’s a Free Software developer and technology consultant who—get this—works without money. Anybody else say that?

>> AUDIENCE MEMBERS: Sometimes.

[laughter]

>> MAYMAY: Good answer!

>> SCOTT BECKER: Instead of owning a home, Maymay lives on the road, traveling wherever I guess he’s needed and wanted, working to help secure and scale small businesses, non-profits, and community groups. He’s on the road and he helps people take advantage of enterprise features through easy to use and easy to understand Free and Open Source Software.

So, with that being said, maymay.

>> MAYMAY: Yeah! Thank you. Thank you so much, Scott and Steve, who I know is not here. And to Microsoft, for the space of course. So, you just introduced a little bit about me. I kinda just want to spend one moment say a little bit more about myself.

People call me “maymay,” that’s the name I prefer. This is a screenshot of my homepage at maymay.net. It’s spelled like the month of May, but twice. Um, I get DDoS’ed occasionally, so if my site’s not up right now, don’t worry about it. It’ll come back in a sec. Apologies if that’s happening, but go there to learn a little bit more about me, and about the work that I do.

In the meantime, I want to talk a bit about what we’re gonna do here. So, I talked a little bit about myself. I won’t bore you more with that. Next we’re gonna quickly spin up a new WordPress Multisite instance, so that we can show some of the demos that I want to show you. I’ll be showing you three WordPress plugins that I wrote that I think you might want to know about. That may be why you’re here. And finally, if there’s time, we can do some Q&A. Hopefully I’ll have some answers to that.

So, all right, let me go ahead and spin up a new WordPress Multisite instance. And, for this, I’m actually gonna go out of the slides, and I’m gonna go to my little demo here. Now, what I am going to do is go to this website that doesn’t yet exist, just to prove that it’s not actually there: W P N Y C dot DEMO. This website doesn’t exist, no one can get to it. It doesn’t yet exist. So we’re going to go ahead and make it!

Can y’all see that, is that big enough? That text is big? Okay.

I’m gonna be using a couple tools that I’m gonna explain in just a second. The one that I want to start with is VV. And, this is gonna basically automate creating an entire new WordPress Multisite install on my machine, and make the site available at WPNYC.DEMO.

So, I’m gonna tell VV to, hey, please create for me a domain called WPNYC.DEMO. And I want the name of this site to be WPNYCDEMO. Is that right? That’s right. And I want it to be a multisite install with a subdomain scheme. You know how WordPress can do subdirectories and subdomains? I want subdomains. I want the admin username to be admin, I want the password to be password, which I know is not a good password, but this is a demo.

>> AUDIENCE MEMBER: Password with five asterisks!

>> MAYMAY: Yeah, and an at-sign!

>> AUDIENCE MEMBER: What is this software you’re using?

>> MAYMAY: I’ll talk about the software in just a sec, I just want to kick this off.

The admin email address is gonna be admin@wpnyc.demo. I want also to remove these defaults. Y’know how sometimes when you install a new WordPress site it installs plugins like Akismet and the Hello Dolly plugin and a bunch of different themes that you almost never use? I wanna remove all those, so I don’t actually want those to be part of the final build. And I’m also going to use the debug flag here because I want to set the WP_DEBUG constant in the wp-config file. This will help me show you some of the output from some of the plugins that we’re gonna demo. Just so that we make sure that’s there.

All right, so I’m gonna go ahead and create that. And VV’s gonna ask me whether or not I want to create a site with a blueprint. I don’t. Whether I want to install a specific version of WordPress. I don’t. I’ll just use the latest version, rather. I’m not going to use any sample content so it’s gonna be totally blank, no users, no nothing. We’re not gonna import any database files. We’re not going to add sample content to any of this. And we’re gonna go ahead and start that off.

Now, VV is gonna go ahead and build me a new server and it’s going to make that site available over on the left-hand side there. And while that’s happening, we can switch back to our presentation and we can talk a little bit about the demos that I’m going to be showing you.

So, very briefly. If you don’t already know about the tools there—

>> AUDIENCE MEMBER: Where’s this site going to be set up?

>> MAYMAY: It’s setting up right on my computer here, in a development environment. And it’s going to be using these things: Virtual Box, which is a virtual machine hypervisor, a type 2 virtual machine hypervisor. That means that I’m gonna have a totally new computer, a Linux server, on my Mac here. That, is being configured using Vagrant, which is a virtual machine hypervisor automation tool. So with Vagrant commands I can tell Virtual Box how to set up that machine; what network interfaces I want, what kind of operating system I want, what kind of ports to use, all that kind of stuff. I’m also using VVV, which is the Varying Vagrant Vagrants project. This is a project originally started by the 10Up company, which is a WordPress development shop. It’s a Vagrant config specific for WordPress development. So, I’ll be using that. And, last one is Variable VV, which is the one that I used that you saw and this is the command that I was using to tell VVV how I wanted it to configure that WordPress setup.

So, all these tools are Free Software, open source. You can grab them on GitHub or these project pages. Variable VV is written by Brad Parbs, who’s an excellent developer and this tool is probably the easiest way to set up a WordPress site I’ve ever seen. I’ve contributed a number of features to it. It’s really, really nice.

Using these kinds of tools makes development a lot easier, a lot more robust, a lot more reliable, ’cause it’s all automated. It takes out human error, and it’s much faster, of course.

So, let’s—

>> AUDIENCE MEMBER: Quick question?

>> MAYMAY: Yeah?

>> AUDIENCE MEMBER: Are you familiar with Local by Flywheel? Is this similar, or are there major differences?

>> MAYMAY: I am only passingly familiar with Local by Flywheel, but if I understand correctly, they’re basically equivalent tools. Right? It’s kind of—again, I’m actually not that familiar with Local by Flywheel. I’ve heard of it. My understanding is that it sets up a development environment for WordPress. Some of you may have heard about XAMP, right? That old thing.

[laughter]

That still exists. It’s kind of like a packaged server in a box. Like an application box. This is using—uh, this is the same effect, but we have virtual machines to do it with instead of putting, like, an Apache server on your laptop. That kind of thing.

All right, so, while that’s all building, we’re here to talk about Enterprise, right? So, I’m gonna assume you all know what WordPress is, and I’m gonna assume you all know what a small businesses are, and what features are, but we’re here to talk about “Enterprise Features for Small Businesses.” So what does, “Enterprise” mean? And some of you may think you already know what this is, and that’s great. Obviously, I’m not here to tell you what to think. That is, or may be, your employer’s job. So instead, I wanna make sure we’re all on the same page by letting you know what I mean when I say “enterprise.”

So what I mean when I say “enterprise” is important capabilities for secure and private collaboration, which utilize multiple tools simultaneously, typically sold to larger corporations that have a lot of money. Right? So, in other words we’re talking about anything that has to do with process or workflow automation, anything that has to do with objectives that touch multiple disciplines at once. Tools, for example, that interoperate between multiple vendors, typically to avoid vendor lock-in so that you don’t have to be beholden to a Facebook or an Amazon or a Google for the rest of your business’s life. Any capability that’s, perhaps, perceived to be super advanced or maybe even unnecessary for small groups, like those zero-to-one employers shops, or sole proprietorships, small businesses of that kind, particularly when they’re security and privacy related. Because those are the kinds of things almost always sold as the “pro” features in add-ons and upgrades that are unavailable to people with not huge budgets.

So, in short, any kind of system or tool that supports truly resilient autonomy. Something you can do yourself. Not this B2B stuff. Right?

So, with that said, I see my role as a Free Software developer to make it more possible for more people to independently access more of those capabilities without needing to have money and without needing to engage in any other form of abusive or coercive relationships in order to do so. I think that’s especially important to do in service to and in solidarity with the specific people whose lives are made dramatically worse by capitalist efforts to do the contrary.

All right, so let’s see where we are with the build of this new website and how far we’ve gotten on creating the server. Okay, there we go. It says here the server URL is wpnyc.demo. Let’s take a look and see if we actually have this available.

There we go, we got a new website up. So this is a pretty standard WordPress site. It’s all empty here, nothing fancy about this at all. We can go ahead and try to log in. And we’ll use our admin and password. And you can see that we have a Multisite install, so we have a Network Admin. We’ve got a users database. It’s all empty. So it’s just a standard, brand-new, WordPress site that we’ve created there.

>> AUDIENCE MEMBER: So, is this replacing what everyone has to do with their hosting company? Making databases, and—

>> MAYMAY: No, this is what your hosting company uses!

>> AUDIENCE MEMBER: Right, okay.

>> MAYMAY: You’re not going to be able to go to this website from, for example, the Starbucks down the street because it’s on my computer. It’s physically here. However, if you were a hosting company and you, for example, didn’t want to go and create a database and install WordPress every time you get a customer request, you’d probably use a tool like this. So, these tools are Free Software, they’re open source, you can do that if you wanted to. They don’t tell you that, but you can.

>> AUDIENCE MEMBER: It sets things up in seconds!

>> MAYMAY: It does. It doesn’t take very long at all. All right! So, we got our website, so we spun up a new WordPress Multisite instance.

>> AUDIENCE MEMBER: Question?

>> MAYMAY: Yeah?

>> AUDIENCE MEMBER: What does Multisite mean?

>> MAYMAY: Multisite means you can have multiple domains, multiple websites, that are running on one WordPress database. You have one database but you have, for example, blog.wpnyc.demo, and test.wpnyc.demo, and maybe even othersite.com, all running on one WordPress installation.

>> AUDIENCE MEMBER: So you can back the whole thing up?

>> MAYMAY: Yeah! It’s all one database, so whenever you backup that database, you backup the whole thing, the whole network.

>> AUDIENCE MEMBER: So you’re managing it from one WordPress site?

>> MAYMAY: Yeah. If Multisite is new to you, definitely check out the WordPress codex, which has a Multisite page. It describes it right there and also tells you how to set it up manually, in case you want to do that. It is good to do that manually at least once or twice so you know what these tools are doing. But once you do know how to set it up manually, using these tools obviously makes the job a lot faster and a lot less error prone.

All right, so we’ve created our new WordPress multisite instance. Let’s move on now. We’ll learn about the Subresource Integrity Manager for WordPress.

So, first, how many of you may already have heard about Subresource Integrity? No? Okay. That’s cool. I really like the Mozilla Developer Network’s definition. It’s pretty clear.

“Subresource Integrity is a security feature that enables browsers to verify that files they fetch (for example, from a CDN [a Content Distribution Network]) are delivered without unexpected manipulation. It works by allowing you to provide a cryptographic hash that a fetched file must match.”

So, what does that mean? For example, let’s say some attacker—or you—want to run cryptocurrency mining JavaScript on hundreds of thousands of users’s Web browsers. You don’t have to actually attack thousands of users’s websites, thousands of websites on the Internet, in order to do that. You could compromise the one website that all those other websites are loading files from. For example, like a [Facebook tracking] pixel, right? Like some sort of web bug. If you can change that and everyone is pulling that site from you, well, there you go. You’re now loading your JavaScript code on multiple websites. So, for example, if I wanted to make the users of USCourts.gov mine Monero or BitCoin for me, then I wouldn’t necessarily have to attack USCourts.gov, I could attack TextHelp.com, because TextHelp.com is serving files for the other sites, USCourts.gov and ICO.org.uk.

Now this isn’t theoretical, either, this happened just last week, with these exact websites—USCourts.gov included—and it’s exactly the sort of scenario that Subresource Integrity is designed to mitigate. So, let’s see how you can prevent this attack against your site’s users for free using the Subresource Integrity Manager for WordPress.

Let’s go back to our demo here. And we’ll switch back to Firefox.

>> AUDIENCE MEMBER: Is this similar to what SSL does?

>> MAYMAY: No. No, SSL would not protect against this because you’re still getting the content and not verifying that what you’re getting is what you expected to get. It just means that no one has listened to what you’re getting on the way. I’ll show you what I mean in just a sec. This will be a lot clearer if you see this in action than if you just see some slides without a demonstration about it.

All right, so the first thing we’re gonna do is, obviously, is we’re gonna go to our Network Admin screen. We’re gonna go to Plugins. We don’t have any. And we need to get one because we need to get the Subresource Integrity Manager, called WP-SRI. I think you can also search for “Subresource Integrity.” Regardless, here it is. We’re gonna go ahead and install this. And there it is.

Now, when you have a Multisite install you can activate plugins for an entire Network all at once or, what we’re gonna do, is we’re going to go to the site itself, our site’s Dashboard—it’s hard to do this backwards. Go to our Plugins, and we’re gonna go ahead and activate the Subresource Integrity Manager for WordPress. That’s it, you’re now protected. By which I mean, if you go to your Tools page, you’ll see a new item called Subresource Integrity Manager, and you see here all the listing of resources that your site’s requesting. Every single one, including the ones on your own site because we set that debug flag—I wanted to make sure there was some content here. Over on the left you see the URL column. That’s the source address of the file you’re loading. So these are JavaScript and CSS styles, etcetera. And over on the right, you’ll see these hashes. And this is the cryptographic hash, this is the one-way mathematical function that proves that the content that’s being served to your site is the content that you expected to get when you first loaded that resource.

So, what do I mean by that? Let’s go to the site itself, and I’m going to go ahead and open up the View Source thing here—and let’s make that a little bit bigger so you can see as well—and let’s just have a look at one of these link elements. Actually, let’s look for stylesheet—style sheet—there we go. So this probably looks familiar to any of you who have seen CSS before, right? Link rel stylesheet. Here’s an href, a reference to the stylesheet itself, but over on the right, you can see crossorigin anonymous and integrity equals SHA-256 dash and then this hash. This hash is the metadata of what was expected to be in that stylesheet. So if some attacker modifies it, it’s not going to load in your browser if you’re using these, if you’re expecting to get this particular hash value.

What do I mean by that? Let’s go a little bit deeper into that. So, let’s close—close? What happened there. There. Let’s close out of this. And I want to create now an example attack on this website to show you how this is actually enforced.

So the very first thing we’re gonna do, I’m going to go to my command line here and I’m going to create a new JavaScript file. This is gonna be going to a CDN that I set up ahead of time, here. CDN dot demo. The important thing about this site is that, again, it’s just a WordPress site, it’s just a different domain. The point being that we have two different domains. One requesting JavaScript sources from another. This is exactly the same thing as you would, for example, do when you do an embed from YouTube and you say, “Hey, YouTube, please load some content on my site.” I’m gonna do the same thing, but it’s gonna be very, very simple. So we’re going to echo some JavaScript, how about alert—whoops, this is hard to do backwards—alert, how about ‘Hello, WP NYC Meetup.’ Right? And we’re gonna go ahead and put this into the root of my CDN demo at htdocs and we can call that test.js for example. And I want that to be output there, whoops! Where did I mess that up? Echo, alert, yup. That’s fine. Uh, that’s not fine? Oh, there’s some other, I can’t see [the full screen on the project] or I mistyped some stuff? I can’t actually see all that. There we go. Nope. It looks like I’m not able to see what is there. Echo alert ‘Hello, WP NYC Meetup.’ Close. Let’s not do this full-screen, because I don’t see what’s over there. Oh! I have another—

>> AUDIENCE MEMBER: Can you explain what you’re doing now?

>> MAYMAY: Yeah! So this is gonna put this text, “alert(‘Hello, WP NYC Meetup’)” into this file. There we go. So if I look at this file, let’s say cat that file out, you’ll see that now that file has “alert(‘Hello, WP NYC Meetup’)” and if we go to our CDN dot demo slash test dot js, we’ll have that file being served in the Web browser.

The next thing I need to do of course is I need to load this JavaScript into the theme that we’re using. So I’m going to go ahead and use Vim to edit wpnycdemo, htdocs, wp content—

>> AUDIENCE MEMBER: And what’s Vim?

>> MAYMAY: Vim is a text editor, like TextEdit, but in a command line. So, like, Notepad or something. WP content, themes, twenty seventeen, because that’s the theme we’re using, and header.php. Right, so this is gonna look pretty familiar to anyone who’s edited a WordPress theme before. You can see here the wp_head hook, function there. And we’re just gonna add another one and it is going to be wp_enqueue_script, and this is gonna be WP-SRI demo, and we’re gonna load CDN.demo/test.js. All right, now our website—we put that embed code, basically, into our website. So now, when we load the WP demo site, we should get an alert that says, hey, “Hello, WP NYC Meetup.” There we go. So we now have that alert. We’re running that JavaScript code.

Here’s the thing. What if I then change that JavaScript code? Well, without Subresource Integrity, if I change that JavaScript code, it’s still gonna run, which means that if some attacker then changes that source code on some other server that you don’t control, that you have no insight into, they’re now running their code on your site. With Subresource Integrity, however, if I change that, so let’s do that, let’s vim this again. This time, let’s vim the CDN site, htdocs, test.js, let’s make something much more malicious happen here. Maybe something like a cross-site scripting attack. With a cross-site scripting attack if I reload this now, if I wasn’t using Subresource Integrity—we’re going to go into Subresource Integrity Manager to see that we have a hash for it—it’s not going to load.

Okay. Why didn’t it load? It didn’t load because, if we look at the Inspector, and go to our Console, you can see, “None of the SHA 256 hashes in the integrity attribute match the content of the subresource.” In other words, the browser says, “Hey, I saw this file, but it doesn’t match what you told me it should contain. And because it doesn’t match, I’m not gonna run it.” Therefore, your users also will not be subjected to that JavaScript. If I—

>> AUDIENCE MEMBER: What will they see? Will they see an error?

>> MAYMAY: No. It just won’t run. So, for example, if you had a personalized resource—Google Fonts does this, where it sends personalized content to each individual user but it’s the same URL, it’s always google.com/fonts/something—that is gonna break this. And so hopefully Google will fix that and will give, like, individual URLs to people. Until then, you have this little Exclude button which just takes out that integrity attribute in case you run into a problem where you’re loading one resource for multiple users but the content of that one resource is different for each user. In that case the integrity attribute won’t be printed, and if I reload this page, we should now have an XSS attack happening. Does that make sense?

>> AUDIENCE MEMBER: Yeah.

>> MAYMAY: All right, so that happens, again, because we did not include the integrity attribute in our test.js script element. If we again go back to Subresource Integrity Manager and re-include it, then when we reload, we will not get the XSS popup because the integrity attribute will be printed and we’ll see in fact the SHA 256 hash was not matching what we expected it to be.

So, let’s return this to the original ‘Hello, WP NYC Meetup’, and we’ll save that. And hopefully, with this, because again, it matches, it’s not excluded, but it does match the content that we’re expecting it to be, we visit the site and we get this back.

So that is Subresource Integrity Manager for WordPress.

>> AUDIENCE MEMBER: Does that add any weight to the site?

>> MAYMAY: It will increase your HTML page sizes, but hopefully you’re using, y’know, HTTP2 for good compression, and it’s not really adding a lot in comparison to the kind of attack that this—the kind of vulnerability that this is for your users is pretty serious. So, this is basically considered a really good thing, it’s been widely developed and deployed.

Some pro-tips for using this. SRIHash.org is a great site to go to if you want one of these one-off. If you have a JavaScript file that you want to insert yourself but you want to include the integrity attribute, go to SRIHash.org, plug the URL into the form there. Hit Hash. It’ll give you the exact code you want to use for that.

>> AUDIENCE MEMBER: That looks like a good password generator.

[laughter]

>> MAYMAY: You can also further harden your site by using the CSP HTTP headers. These are content security policies which tell your visitor’s browsers not to load anything that doesn’t include an integrity attribute if you use the require-sri-for and you can say either script or style, or both. And finally of course, because the free and open Internet is a platform agnostic technology by design you don’t have to be using WordPress to be using this. You can, for instance if you’re using Ruby on Rails, use the sprockets-rails gem. Just use your javascript include tag and add a new parameter there, integrity equals true, and sprockets-rails will take care of it. Similarly, if you’re using any of the NodeJS tools, pick up the SSRI package over on NPM. And again, it’s ssri.fromData or dot fromUrl or something. Give it an algorithm that you wanna use, and the toString method will give you the integrity attribute, what you actually want to print out.

Okay, so that was Subresource Integrity for WordPress.

Yeah?

>> AUDIENCE MEMBER: What happens if I want to change the plugin or the script or whatever?

>> MAYMAY: Yeah!

>> AUDIENCE MEMBER: Do I have to run something again so I can update the hash tags or…?

>> MAYMAY: The easiest thing to do is to go back to your tool, so let’s go back to WPNYC Demo, we’ll go to your tool, and you find the resource you want. So, let’s say, test.js, here. And delete it. Now, the next time it loads, Subresource Integrity Manager will say, “Oh, I don’t know this. I’ll go ahead and fetch it, re-hash it,” and that way basically you’re forgetting the old hash and putting in the new one. So that’s that.

>> AUDIENCE MEMBER: Can you give us an example of how to use this on a site we might already be familiar with?

>> MAYMAY: This is already used on many sites you’re probably familiar with! And it’s used specifically to make sure that there’s no unexpected manipulation from the CDN side. So, for instance, a long time ago—well, not a long time ago but, like, two years ago—there was one of these JavaScript DDoS attacks that happened by Baidu, which was one of the Chinese analytics firms. The Great Firewall of China, evidently, decided to change anyone who was loading Baidu analytics JavaScript that it was sending to DDoS GitHub. As in, to try to get every browser on the planet who was loading Baidu analytics to send a bunch of requests to GitHub.com to take them down. And it worked because, at the time, there was nothing like this. So that, among other reasons is why SRI features became a W3C standard and is now deployed in all these different frameworks.

So, for WordPress, you can use the Subresource Integrity Manager until we get this into core. It looks like it will be into core at some point, but I don’t want to promise about when or how, because I don’t know.

[laughs]

>> AUDIENCE MEMBER: [quiet speaking]

>> MAYMAY: Oh, you just have to hit “exclude.”

>> AUDIENCE MEMBER: [quiet speaking]

>> MAYMAY: Well, I mean, yes. For Google Fonts, or any resource that has the same URL but whose content is different, right? Because, if the content is different, it’s not going to match the hash for all users, it will only match the hash for one user. Probably you, because you’re the first one who has requested it. So you want to exclude those until sites like Google and other CDN providers start making unique URLs per user. All this personalization that happens on just the content and not the URL needs to go away for this to work.

>> AUDIENCE MEMBER: How do I know if I need to exclude a script like that?

>> MAYMAY: See if your site still works.

[laughter]

>> AUDIENCE MEMBER: Oh, just load it again?

>> MAYMAY: Yeah. All right, so, we’re gonna move on to GPG and OpenPGP signing and encryption for WordPress. So, first of all, how many of you are familiar already with GPG or OpenPGP or signing and encryption and technologies? One, two, three, four hands in the back. Okay, cool. Great! Any of you wanna shout out what that is? No? All right.

Well basically, it’s secured email, is the answer to that. In short, GPG or OpenPGP—they’re kind of interchangeable terms—means secure email. But here we have to be pretty careful about what we’re talking about when we say “secured.” What does secured really mean? In a very brief nutshell, ’cause I don’t have that much time, when infosec pros talk about security they’re usually speaking about something that’s known as the CIA triad. It’s called a triad because it has, of course, three parts, and these are confidentiality for the “C,” integrity for the “I,” and availability for the “A.” Now, for the purposes of this presentation we’re only going to concern ourselves with the first two pieces of this triad.

Another common word for confidentiality—you’ll often hear this a lot—is privacy. Much more often used word. And similarly, very similarly to that, another word that’s used for integrity is “authenticity.” So, in the next demo, when I talk about GPG encryption, what I want to be talking about it ensuring privacy: the ability for your website to send a message that only its intended recipient can read. And when I talk about GPG signing, what I’m talking about is ensuring authenticity: the ability for the recipient of that message to verify that it the message was actually sent by your server, not some other imposter, and that the message that they got was unmodified in transit. It’s the actual message that you sent. Very much like the Subresource Integrity thing where you’re hashing stuff. All right, so now let’s see how you can accomplish this with the PGP encrypted emails plugin that I wrote.

So, we’re gonna go back to our demo, and—whoop, there’s my, there it is. And, I want to go to my WordPress site. And, as before, we’re gonna create a new plugin by going to Network Admin, Plugins, Add New, and we’re gonna search for WP PGP Encrypted Emails. That’s the full name of it, but you can also probably search “PGP encrypted emails.” Anyway, we’re gonna hit “Install Now” and there it goes. And we can Network Activate this or we can, again, just go to a site, go to the demo site, go to our plugins, and activate—whoops, no, I don’t want to, I want to do this one—plugins, and activate that. All right, that’s it.

Now, very first thing you’ll notice is that we have one of these admin notices up at the top. It says hold on, you’re not done yet. You’ve got to create or generate an OpenPGP key pair for the website to sign outgoing emails with. In other words, you can’t just send an email. You have to actually stamp that email with the identity of the website, cryptographically. So that’s what we’re gonna do when we generate PGP signing key pair. That’s it, that’s all you gotta do. And this will take you to the Settings screen, with a new item here under Email Encryption.

This email encryption settings will have a number of options. The important one for here is the PGP signing key pair. This is a low-trust, single-purpose key (identity), for the website that you need to distribute any user who wants to make sure that when they’re getting emails from you, it’s coming from the right place. There’s a theme function that you can add to your theme that makes this button, so you don’t have to worry about the code itself, or you can go to or tell your users to go the profile that they have and click on “Download public key” at the very bottom under their personal encryption settings. We’re gonna go ahead and do that. I’m gonna click “Download public key,” and it’s gonna give me this file. I’m just gonna save that, for the moment.

But before we do anything with that, I want to show you what an unsigned and unencrypted email looks like. Regular old email, nothing fancy. This is what you’re doing, probably, right now. And to show you that, I’m gonna go to wpnyc.demo, and I’m gonna go to [port] 1080 here. This is Mailcatcher. This is another one of those development tools that was installed when I did the VV build and what this does is it kind of intercepts any outgoing email from that website and shows it to me in this interface so I can debug it.

You can see that we already have an email in here. This one is the email that was sent when the WordPress site was kicked off and built. Sometimes you’ll see this in the One-click Installers. Y’know, you’ll get an email saying, “Hey, your WordPress site is ready.” That’s what this is. So, this is the source code of that email. You can see the headers up here, and the body down here. And there’s nothing fancy about this, nothing special, nothing cryptographic, no hashes, none of these security features. It’s all just plain text. This is like sending a postcard through the post. Anytime you send a postcard, anyone viewing or handling that postcard can read the contents. That’s what all email, all text messages, all unencrypted HTTP—not HTTPS—traffic is. So we’re now going to add the equivalent of a digital envelope to protect the contents of this message and a digital stamp that says, much like those Game of Thrones, y’know, wax stamps. This definitely came from Jamie Lanister or whoever. We’re gonna add that to our emails. I’m gonna show you how to do that.

Firstly, we have this key that we downloaded. So let’s go ahead and take a look at my Downloads folder. And, or actually, take a look at this here. “To authenticate the emails, download the PGP public key and import it to an OpenPGP-compatible client.” This links to PRISM-Break, which is a fantastic website. If you don’t know about it, check it out. PRISM dash Break dot org. And it lists here all the software that you can use PGP or GPG with. So there’s a vast ecosystem of this. It’s available on Windows, Linux, Android phones, iOS devices, basically any computing device that you have can do this for free already, with either one of these apps if it’s not already built-in. Many Linuxes have this built-in, for example.

So, I’m going to be using GPGTools, or MacGPG for this. That is at GPGTools.org. If you’re on a Mac today and you wanna try this out, go grab this. It’s the best tool for the job I’ve seen. All right, so we’re gonna go ahead and open up this email, I’m sorry, this key, and we’re gonna open it with an application that was installed with the GPGTools package that I installed earlier called GPG Keychain, and we’re gonna import that key, and there we go. Now we have this key.

Now what this means is that we are aware of a cryptographic identify for the website wpnyc.demo. I can now authenticate any emails that are sent there. So let’s go ahead and get an email from there. You could trigger an email by purchasing an order or making a new user account, or you can use the handy “Send me a test email” button, which is what I’m going to do. And when I click this I want you to take a look at the Mailcatcher tab up here, and I want you to take a look at this. This is going to go from 1 to 2. Okay? Ready?

Send me a test email. There it is, now there’s 2. There’s the test email. And now, take a look at how the email is different. We have this “begin PGP signed message” text on top and on the bottom. This is what’s known as a clear signed message. This is saying that this is the, effectively, the integrity attribute, or that metadata for the contents of that message, just like the SRI stuff. It functions very similarly.

If you were using an email client, this would automatically authenticate—

>> AUDIENCE MEMBER: You’d use both?

>> MAYMAY: Use both what?

>> AUDIENCE MEMBER: Use both the GPG and the SRI?

>> MAYMAY: Yeah, they’re separate plugins, they’re separate technologies, but they use the same what are known as cryptographic primitives, which is to say, they use the same mathematics under the hood.

>> AUDIENCE MEMBER: So you’d use one or the other?

>> MAYMAY: No, you can use both, because they do different things. In fact, I would recommend that you use all the plugins I’m gonna demo. [laughs] ‘Cause, they’re all free and they run on any WordPress site of any size. All right.

So, we’re gonna grab this PGP signature here, the contents of this email, and again if you were using an email client like Apple Mail or Microsoft Outlook or something, you wouldn’t have to do this copying-and-pasting, but I’m using a debugging tool so I am. I’m just going to go ahead and open up a TextEdit window and I’m gonna paste this into a new file and we’re gonna say test email, and I’m gonna save it on my desktop—and, sure, you can use the email extension. All right, and now, to verify that this is in fact the—I don’t need this anymore—the message that came from the site, I’m gonna right-click, I’m gonna go to services, and “Verify Signature of File.” See that, there? “Verify Signature of File.”

All right, click that, and we get a verification result: Signed by wordpress@wpnyc.demo. This means, yes, this email came from the site that you think it did and it was unmodified. No one changed the message between the time that the site sent it and the time that you received it. So, to prove that, we can open up this test email again. TextEdit—whoops, come on. Why is it not dragging this over? Drag. Seriously? For real now? All right, let’s do it this way. Other, and we want to use TextEdit, which is here. All right, so now let’s change this email in some way. We’ll just delete some text. We’ll re-save it.

Let’s try to verify again. Now we should get a failed message. This should not verify, because the message was changed. This is really important for things like, for example, security announcements. Apple, Inc, like, the company that makes this computer. They do this, exactly this process for when they send emails to their security announce list because it would be pretty bad if they sent a security announcement to say, “Hey, there’s a new patch available,” and that was actually a fraudulent email. Web hosting providers, DreamHost sends this with their billing emails, with their receipts. Now you can do the same with this plugin. All right, so that is a test email.

So that was signing. Do we have time? Do you think we should do encryption? Do we have time for this?

>> AUDIENCE: Yes!

>> MAYMAY: Yes? Okay. [laughs] Is there a question, there?

>> AUDIENCE MEMBER: [quiet speaking]

>> MAYMAY: I can’t quite hear you. Are you asking if it’s possible, if this works with an external service?

>> AUDIENCE MEMBER: Yes. If you don’t want to use your WordPress server to send the emails.

>> MAYMAY: Yeah. It, um, you will have to do a little bit more work to use an external service because you’ll need to send them pre-signed messages, right? So, for example, a lot of these services will insert things like, you know, “Hello, name,” or, y’know, “who lives at such and such, your account number is such and this.” If you sign the message before they do that, none of these verifications are gonna work, because they’re going to be changing the message on your behalf. On the other hand, there’s many free software plugins for WordPress that can function similarly to MailChimp, for instance, and this plugin is compatible with all of them that I’ve tested, which at this point has been hundreds. So, you could do that, it might take you a little bit more time to send those emails but at least that way you’re actually doing the work on your site, yourself, and not farming that out to a third party that may or may not—and probably is—mining you for data. So, but, y’know, obviously up to you?

All right. What time is it? It is, 8:30. I have until 9, I got a lot of content. You sure you want to see encryption?

>> AUDIENCE: Yeah!

>> MAYMAY: Yeah, y’all are good for that?

>> SCOTT BECKER: Quarter to nine!

>> MAYMAY: Quarter to nine! All right, I don’t have a lot of time, then. Here’s what I’m gonna do—

>> SCOTT BECKER: Well, ten to.

>> MAYMAY: All right. Well, let’s, I’m going to go through this a little bit quicker. So what I’m going to do is I’m just going to create a new key here. This doesn’t really matter. To do encryption we have to do the reverse process. Rather than getting the key from the website, I have to give the website my identity. But to have an identity, I need to make one. So that’s what this is, that’s what the “New” thingy is here. So we’ll do that. Some email dot invalid, none of this matters. The password is password. And password here. And, under advanced options, what’s there? Oh yeah. So I’m just gonna say “this is a test do not use this key.” We’re gonna generate that key. We’re gonna continue with a simple password. And again, all the GPG tools will do this, whether you’re using Windows or Linux, I just happen to be using a Mac. I don’t want to upload the key to the key server, again, because it’s a test.

And what we’ve done is created a new key pair. What’s known as basically a digital lock and a key that opens that digital lock. So if I hit export here, I get a file on this desktop. Let’s go ahead right there, and desktop, and there is my file which just like the other email, this can be opened. Because a key, an identity, a cryptographic hash, is just text, we can open it with TextEdit and you can take a look at what a key looks like. It’s just a really big number, really.

So if I copy this into my—because I’m the admin here, I’m gonna copy it into the Settings, Email Encryption, and Admin Email PGP Public Key textbox here, and hit save. Now, that’s all I gotta do. Now if I get another message, let’s trigger that again. Send me another test email. It’ll go from 2 to 3. And now this was the signed message, now we’ll have an encrypted message. So now we just have this “begin PGP message” block and this is the content that’s gonna be stored, for example, by Google or by Microsoft Live, right? When you are actually using a GMail account, they’re reading all your email. Well that’s because it’s not encrypted. If you use this, and you give the sites that are letting you send encrypted messages your cryptographic identity, Google can no longer read your email because to them it looks like this.

So the question is, how does it look like to you? Well, we’ll go back here, we’ll go to TextEdit, and we’ll just paste this message in again. And, again, on an actual email client this is much, much simpler. I’m just gonna right-click, go to Services, and I’m gonna decrypt selection to new window. And what we should see here is I’m being asked for my password for this identity, which I put before. Right, and say password. Oh, it opened it up over here, so I’m just gonna move this over. Right. This is a test message from wpnycdemo. It’s still signed, right, because the site has a signing key, and it’s decrypted, because I have the matching key to the lock that I gave the website. So that is OpenPGP signing and encryption for WordPress.

Quick pro-tips for making even more use of this plugin. Number one, importantly for small businesses you should know that the WP PGP Encrypted Emails plugin features zero-configuration, out-of-the-box support with WooCommerce. So if you have a WooCommerce store, right, as long as your chosen theme supports the WooCommerce account pages, then all you have to do is install this and your customers will get an out of the way form that looks exactly like this that they can use to opt-in to signed emails or even encrypted emails if they go ahead through the process of making their own identity and uploading a key and giving it to your store. This is an example of what a signed email might look like in Apple Mail. Instead of the Mailcatcher interface it would just look like this. It would say “Security: Signed” and that’s how they would know, yes, this actually came from your store. So this is really important for secured email receipts, for private transfers of communications between, like, tech support. Anything that you basically don’t want other people reading. This is all from a blog post out of New York City called Flora Posidonia: FloraPosidonia.xyz. Check that out at some point if you wanna see this in practice in the wild.

For developers, WP PGP Encrypted Emails features a general-purpose API to cryptographic operations using familiar WordPress plugin hooks. So what I mean by that is that the plugin uses the same hooks that it makes available for other plugins for itself. And that means with as few as about four lines of PHP you actually, as a developer, can build PGP or S/MIME encryption into your own plugins and themes. So you can see here we’re getting the user object, we’re applying the wp_openpgp_user_key filter to that user object to get the key itself, and then we’re using openpgp_encrypt with the message and the public key to get an encrypted message. You can now, using PHP, send this variable over the Internet or in an email or anywhere you want, and it’s that PGP message block instead of the plain text content.

Okay, so that was GPG and OpenPGP signing and encryption for WordPress. I got ten minutes, so I’m going to hold questions. And at this point we’re going to go on to centralized authentication service using OpenLDAP for WordPress.

Now, as before, centralized authentication services with LDAP, anyone use this already? Sound familiar to anybody? No? All right, let’s start with LDAP. So that stands for the Lightweight Directory Access Protocol. It is an open vendor-neutral industry standard application protocol for accessing and maintaining distributed information services. So what that means, for our purposes at the moment, is that an LDAP database, which is called a Directory Information Tree or a DIT, can store user account login details like usernames and passwords and email addresses and phone numbers and this kind of thing, in an application-independent way so that any app that can speak LDAP can actually use the LDAP store as its user database.

Fun fact: LDAP was written by Tim Howes. He was the CTO and founder of a company called Opsware that I worked at for a while and that’s now HP Server Automation suite, HPSA, for those of you actually working in Enterprise.

Let’s take a step back for a moment, though, and talk about what this might look like—a website’s system might look like—without LDAP at all. You have a website. It’s running WordPress. We’ll call it YourSite.com and one of your users, we’ll call them alice, logs into the site. So to successfully log in to that site, WordPress first checks its WP users table for an entry that matches Alice’s account credentials. If those credentials exist and they match the ones submitted by the user, then Alice is successfully logged in, everything is okay, you get the Dashboard, or you get the homepage, it all looks fine. In this setup the user’s account information is stored by WordPress, for WordPress, and is only available to WordPress so we call that application-specific data.

Now let’s imagine that you want to add another app to your network. Maybe you have an intranet, right, and you want to add Nextcloud. This is kind of a Google Docs replacement. You could, and most organizations that I’ve seen typically do just tell Alice that, y’know, they now have two user accounts. They have a WordPress account, right? And they have a completely separate account for Nextcloud. In my experience, this causes a lot of problems. Among other issues, it means that users now must manage two user accounts: two passwords, two user profiles independently. Most users will probably choose the same password and the same profile information on both systems, but once they change their password on one system, the other system isn’t informed, and that leads to confusion, not to mention a lot of help desk tickets.

So this is a classic problem that LDAP is designed to solve. Now, with an LDAP server, you can store account details in a way where you can provide a Centralized Authentication Service, also called a CAS, for any LDAP-capable application that you choose to add to your network. So now, regardless of which app server Alice logs into, their account credentials are always the same. And when they change their password on the one side, say WordPress, they can immediately use their new password to log in to Nextcloud because that authentication check is happening in one central place, which is that LDAP server at the top.

So, let’s see how you can configure this using—whoops, that was too fast—using, WordPress. We’re going to go back to our WordPress demo and I’m going to try to run through this a little bit quickly, I apologize because I’m running a little short on time. But what I have here is another install of Nextcloud. This is a brand-new Nextcloud instance that’s running on the same machine as the other one. And I’m going to go ahead and create a new admin account. If Nextcloud is familiar to any of you this will look pretty familiar because it’s basically just as I did before. It’s a completely blank Nextcloud instance, without anything pre-loaded. So no users, no files, no nothing.

So this is Nextcloud for those of you who haven’t seen it. It looks a lot like Google Docs. Y’know, you can upload files, you can download files, you can open up text files. You can share photos, that kind of thing. You can take a look at the users database here. There’s just one user, the admin, there’s nothing else. And that is just blank Nextcloud. Now we also need, of course, is the wpnyc.demo—oh, yeah, hello—we need the plugin for WP-LDAP, which is…. There are a lot of LDAP plugins, but mine is the one called just WP-LDAP, by me, here. It’s pretty small and pretty new, because it’s not very well-known. But I’m going to go ahead and install it. And what this is going to let me do, when I go to Network Activate it, I will have a new option under my Network Settings called LDAP Settings.

Unfortunately, I need to be using HTTPS to manage it because this is kinda sensitive. So we’re gonna go ahead and do that like this. Password. There we go, LDAP settings. Now, I’m not going to show this because I don’t have time, but I also have a LDAP server running on port 389 on that same machine. Installing an LDAP server, if you’re a server admin, is usually as simple as “sudo apt install slapd” or the standalone LDAP daemon. For the time being, I’m gonna have to skip that.

I set it up so that it has a Bind DN, which is basically like the user account that you’re using to admin the site. This is basically the same as the MySQL user. Y’know how you have a MySQL database and your WordPress website needs to know what the login credentials for the database are? Same exact procedure. In this case, it’s not a MySQL database, it’s an LDAP database. So the syntax is a little bit different. But in general, it looks something like this. DC equals WPNYC, and DC stands for “domain component,” so this is the dots, right? Instead of dot com or dot demo, I’m using DC equals and “CN” is “common name.”

Let’s go ahead and actually double-check that that is correct because I don’t want it to not be. Okay, so we logged in over there, and I’m just going to copy and paste this from my notes to make double-sure that I have this right. So, I’m using an LDAP search tool on a command line, asking for the host, which is localhost in this case, using external authentication meaning the OS itself, to ask for the config common name, and I want the OLC or online configuration root DN. The root DN is basically superuser. You know how WordPress has “Super Admin”? This is Super Admin for LDAP. And sure enough it’s cn admin, dc wpnyc and demo. So that’s right. And then the base is just the end here, the same. WPNYC Demo. You can change this if you want, but it is effectively the same. So, for example, if you wanted to do, a different directory tree you could do OU equals people and this is like, basically choosing the table. Where in the database do you actually want to store what we’re about to put? In our case, ’cause this is a simple demo, we’ll just put it at the root over there. And that’s it. We save that change.

Now your WordPress can talk to LDAP. What does that mean? It means if we create a new user here—let’s say we’re going to make a new user on our Network, and we’ll call it test LDAP and it’s gonna be testldap@wpnyc.demo. We’re gonna add that user. Hopefully, if I got that right—no, I don’t want notifications—if we configure now Nextcloud to add LDAP integration, too. Now, Nextcloud ships with LDAP integration so we don’t even have to write a plugin for this. We’re gonna go to the Apps page on Nextcloud, enable the LDAP user and group backend plugin. We’ll now go to the admin screen for LDAP. We’ll go to LDAP and AD integration, and we’re gonna give it the same details that we gave WordPress. So, 127.0.0.1. Nextcloud has some nice JavaScript that can detect the port. The user DN was cn equals admin, domain component wpnyc, domain component demo, I believe. And the password was password. Let’s see if that, yup. And detect base DN. There we go. And test base DN. We look good.

So we’re gonna hit “Continue.” And now you can see we’ve got three entries available. These are inetOrgPersons. A user account in LDAP is an inetOrgPerson, an Internet organization person. We can verify this, we found 1 user. We’re gonna continue. And here you can say how do you want them to login? Username only, or username and email? We can go with either. This is basically, y’know, log in with your email, log in with your username. Just like WordPress. And if we do testldap here. Yeah, “User found and settings verified.” Continue on, and we’re good. So now, when we go to the users screen, you’ll see another one. And there you go. Now we can log in with this user.

So for instance, let’s say—actually, I can’t log in with this user, because I don’t know their password.

[laughs]

But, if we edit their password and set it, let’s set it to something simple like password. Yeah, confirm that password. We’ll update that. We’ll log out of Nextcloud as the admin, and we’ll log back in as the testldap user. And there we go. We didn’t have to make a user account on Nextcloud because we made on WordPress, and now, no matter how many apps you add to your intranet or your site, you now have one authentication store for all of your users. This is really useful for, for example, employees inside of a company. Taking out one, deleting them from the LDAP store will remove their access to all your apps. It’s also portable so you can transfer it from, y’know, one app to another, as long as the apps you’re using can speak LDAP. And, with WP-LDAP, WordPress can. So that was WP-LDAP on WordPress.

Pro-tips on this: it’s built for Multi-network, not just Multi-site installs. So if you’re not familiar with WordPress Multisite, read about that. Once you’ve read about that, check out the article, again on the WordPres Codex on Multi-network. This is a Network of WordPress Networks, and WP-LDAP works with that, too. You can set different servers, LDAP servers, for different networks so you can do things like network segmentation or perhaps round-robin load balancing, it’s kind of up to you. It’s also already aware of the WP PGP Encrypted Emails plugin, so if you use both of those and your users supply an S/MIME certificate, that will get sent over to LDAP and that will allow you to do transparent email encryption for things that are configured for that, such as iPhones in a BYOD or Bring-Your-Own-Device environment. Microsoft Outlook supports this out of the box as well. All of these features are standard protocols that are for free that you can install on WordPress sites of any size that you never have to pay for, if you don’t want to. You could. I wouldn’t. All of this, of course, is RFC 2798 compliant so any consumer that speaks LDAP—Apple Contacts, iOS, Mozilla Thunderbird Address Book—you can get a people directory and actually have, like, email autocomplete lookups for all of your employees or mailing list subscribers or anything like that.

All right, that was Centralized Authentication Service for LDAP. I don’t know if we have time for Q&A?

>> SCOTT BECKER: We can do a five minute Q&A.

>> MAYMAY: Very short Q&A. I realize that was a lot of information very quickly as well. Yeah.

>> AUDIENCE MEMBER: I run an NGO, a non-profit organization, and we have an account for Google Apps, because they gave us for free. Can I use LDAP instead of Google?

>> MAYMAY: Oh, yeah. Google uses LDAP behind the scenes. So, this is what they’re using. Right, like, there’s not a difference in the technology between Google, Facebook, and this. It’s just a matter of whether or not they put the branding and the sheen on it to make sure that you feel like you’re using their thing, as opposed to the standard thing.

>> AUDIENCE MEMBER: Can you use PGP encryption with Google?

>> MAYMAY: Oh, yeah! I do all the time. Yeah.

>> AUDIENCE MEMBER: Does this address the problem where WordPress sends out emails and it looks like spam? Like, in GMail, it goes to the spam folder, but with Sendgrid and some of these other guys it doesn’t? Does that fix that?

>> MAYMAY: Sadly, no. So the question was if these plugins fix email looking like spam from WordPress. And unfortunately, they don’t. They don’t because of a number of reasons, which we can talk about maybe later if you want, but the short answer is no. The longer answer is they might even make you look a little more suspicious only because they would rather you use their thing.

All right, so, any last question about that? All right.

In case it wasn’t clear, all these plugins are freely available on the WordPress plugin repository today. Here are their permalinks. I’m gonna put up these slides somewhere so you can take a look at them on your time and hopefully we’ve got the recording at some point as well.

Again, my name is maymay. My homepage is maymay.net. It might be down. If it is, try again in a little bit. Again, I get DDoS’ed a lot. At maymay.net the very top link is “Download my digital business card.” Click it to download and import my vCard into your contact app, and that’s all I got. Thank you so much for your time and attention, everyone.

My 2009 essay kinda-sorta about an Anarchist “Internet of Things”

I wrote an essay in 2009 about the Internet of Things, before people were calling it “the Internet of Things.” When I re-read it this afternoon, in 2017, I noticed something rather queer. It wasn’t actually about the Internet of Things at all. It was actually a personal manifesto advocating Anarchism, and condemning techno-capitalist fascism.

Yes, really.

In 2009, despite having barely turned 25 years old, I had already been working as a professional web developer for a little over a decade. (That arithmetic is correct, I assure you.) At the time, I had some embarrassingly naïve ideas about Silicon Valley, capitalism, and neoliberalism. I also had no idea that less than two years later, I’d be homeless and sleeping in Occupy encampments, and that I’d remain (mostly) happily houseless and jobless for the next six years, up to and including the time of this writing.

The story of my life during those two years is a story worth telling…someday. Today, though, I want to remind myself of who I was before. I was a different person when 2009 began in some very important ways. I was so different that by the time it ended I began referring to my prior experiences as “my past life,” and I’ve used the same turn of phrase ever since. But I was also not so different that, looking back on myself with older eyes, I can clearly see the seeds of my anti-capitalist convictions had already begun to germinate and root themselves somewhere inside me.

Among the many other things that I was in my past life, I was an author. I’ve always loved the art of the written word. My affinity for the creativity I saw in and the pleasure I derived from written scripts drew me to my appreciation for computer programming. That is its own story, as well, but the climax of that trajectory—at least by 2009—is that I was employed as a technical writer. I blogged on a freelance basis for an online Web development magazine about Web development. I had already co-authored and published significant portions of my first technical book. And, in 2009, I had just completed co-authoring a second.

That second book was called, plainly enough, Advanced CSS, and was about the front-end Web development topic more formally known as Cascading Style Sheets. But that’s not interesting. At least, no more interesting than any other fleeting excitement over a given technical detail. What’s arguably most revealing about that book is the essay I contributed, which for all intents and purposes is the book’s opening.

My essay follows in its entirety:

User agents: our eyes and ears in cyberspace

A user agent is nothing more than some entity that acts on behalf of users themselves.1 What this means is that it’s important to understand these users as well as their user agents. User agents are the tools we use to interact with the wealth of possibilities that exists on the Internet. They are like extensions of ourselves. Indeed, they are (increasingly literally) our eyes and ears in cyberspace.

Understanding users and their agents

Web developers are already familiar with many common user agents: web browsers! We’re even notorious for sometimes bemoaning the sheer number of them that already exist. Maybe we need to reexamine why we do that.

There are many different kinds of users out there, each with potentially radically different needs. Therefore, to understand why there are so many user agents in existence we need to understand what the needs of all these different users are. This isn’t merely a theoretical exercise, either. The fact is that figuring out a user’s needs helps us to present our content to that user in the best possible way.

Presenting content to users and, by extension, their user agents appropriately goes beyond the typical accessibility argument that asserts the importance of making your content available to everyone (though we’ll certainly be making that argument, too). The principles behind understanding a user’s needs are much more important than that.

You’ll recall that the Web poses two fundamental challenges. One challenge is that any given piece of content, a single document, needs to be presented in multiple ways. This is the problem that CSS was designed to solve. The other challenge is the inverse: many different kinds of content need to be made available, each kind requiring a similar presentation. This is what XML (and its own accompanying “style sheet” language, XSLT) was designed to solve. Therefore, combining the powerful capabilities of CSS and XML is the path we should take to understanding, technically, how to solve this problem and present content to users and their user agents.

Since a specific user agent is just a tool for a specific user, the form the user agent takes depends on what the needs of the user are. In formal use case semantics, these users are called actors, and we can describe their needs by determining the steps they must take to accomplish some goal. Similarly, in each use case, a certain tool or tools used to accomplish these goals defines what the user agent is in that particular scenario.2

A simple example of this is that when Joe goes online to read the latest technology news from Slashdot, he uses a web browser to do this. Joe (our actor) is the user, his web browser (whichever one he chooses to use) is the user agent, and reading the latest technology news is the goal. That’s a very traditional interaction, and in such a scenario we can make some pretty safe assumptions about how Joe, being a human and all, reads news.

Now let’s envision a more outlandish scenario to challenge our understanding of the principle. Joe needs to go shopping to refill his refrigerator and he prefers to buy the items he needs with the least amount of required driving due to rising gas prices. This is why he owns the (fictional) Frigerator2000, a network-capable refrigerator that keeps tabs on the inventory levels of nearby grocery stores and supermarkets and helps Joe plan his route. This helps him avoid driving to a store where he won’t be able to purchase the items he needs.

If this sounds too much like science fiction to you, think again. This is a different application of the same principle used by feed readers, only instead of aggregating news articles from web sites we’re aggregating inventory levels from grocery stores. All that would be required to make this a reality is an XML format for describing a store’s inventory levels, a bit of embedded software, a network interface card on a refrigerator, and some tech-savvy grocery stores to publish such content on the Internet.

In this scenario, however, our user agent is radically different from the traditional web browser. It’s a refrigerator! Of course, there aren’t (yet) any such user agents out crawling the Web today, but there are a lot of user agents that aren’t web browsers doing exactly that.

Search engines like Google, Yahoo!, and Ask.com are probably the most famous examples of users that aren’t people. These companies all have automated programs, called spiders, which “crawl” the Web indexing all the content they can find. Unlike humans and very much like our hypothetical refrigerator-based user agent, these spiders can’t look at content with their eyes or listen to audio with their ears, so their needs are very different from someone like Joe’s.

There are still other systems of various sorts that exist to let us interact with web sites and these, too, can be considered user agents. For example, many web sites provide an API that exposes some functionality as web services. Microsoft Word 2008 is an example of a desktop application that you can use to create blog posts in blogging software such as WordPress and MovableType because both of these blogging tools support the MetaWeblog API, an XML-RPC3 specification. In this case, Microsoft Word can be considered a user agent.

As mentioned earlier, the many incarnations of news readers that exist are another form of user agent. Many web browsers and email applications, such as Mozilla Thunderbird and Apple Mail, do this, too.4 Feed readers provide a particularly interesting way to examine the concept of user agents because there are many popular feed reading web sites today, such as Bloglines.com and Google Reader. If Joe opens his web browser and logs into his account at Bloglines, then Joe’s web browser is the user agent and Joe is the user. However, when Joe reads the news feeds he’s subscribed to in Bloglines, the Bloglines server goes to fetch the RSS- or Atom-formatted feed from the sourced site. What this means is that from the point of view of the sourced site, Bloglines.com is the user, and the Bloglines server process is the user agent.

Coming to this realization means that, as developers, we can understand user agents as an abstraction for a particular actor’s goals as well as their capabilities. This is, of course, an intentionally vague definition because it’s technically impossible for you, as the developer, to predict the features or capabilities present in any particular user agent. This is a challenge we’ll be talking about a lot in the remainder of this book because it is one of the defining characteristics of the Web as a publishing medium.

Rather than this lack of clairvoyance being a problem, however, the constraint of not knowing who or what will be accessing our published content is actually a good thing. It turns out that well-designed markup is also markup that is blissfully ignorant of its user, because it is solely focused on describing itself. You might even call it narcissistic.

Why giving the user control is not giving up

Talking about self-describing markup is just another way of talking about semantic markup. In this paradigm, the content in the fetched document is strictly segregated from its ultimate presentation. Nevertheless, the content must eventually be presented to the user somehow. If information for how to do this isn’t provided by the markup, then where is it, and who decides what it is?

At first you’ll no doubt be tempted to say that this information is in the document’s style sheet and that it is the document’s developer who decides what that is. As you’ll examine in detail in the next chapter, this answer is only mostly correct. In every case, it is ultimately the user agent that determines what styles (in which style sheets) get applied to the markup it fetches. Furthermore, many user agents (especially modern web browsers) allow the users themselves to further modify the style rules that get applied to content. In the end, you can only influence—not control—the final presentation.

Though surprising to some, this model actually makes perfect sense. Allowing the users ultimate control of the content’s presentation helps to ensure that you meet every possible need of each user. By using CSS, content authors, publishers, and developers—that is, you—can provide author style sheets that easily accommodate, say, 80 percent of the needs of 90 percent of the users. Even in the most optimistic scenario, edge cases that you may not ever be aware of will still escape you no matter how hard you try to accommodate everyone’s every need.5 Moreover, even if you had those kinds of unlimited resources, you may not know how best to improve the situation for that user. Given this, who better to determine the presentation of a given XML document that needs to be presented in some very specific way than the users with that very specific need themselves?

A common real-life example of this situation might occur if Joe were colorblind. If he were and he wanted to visit some news site where the links in the article pullouts were too similar a color to the pullout’s background, he might not realize that those elements are actually links. Thankfully, because Joe’s browser allows him to set up a web site with his own user style sheet, he can change the color of these links to something that he can see more easily. If CSS were not designed with this in mind, it would be impossible for Joe to personalize the presentation of this news site so that it would be optimal for him.

To many designers coming from traditional industries such as print design, the fact that users can change the presentation of their content is an alarming concept. Nevertheless, this isn’t just the way the Web was made to work; this is the only way it could have worked. Philosophically, the Web is a technology that puts control into the hands of users. Therefore, our charge as web designers is to judge different people’s needs to be of equal importance, and we can’t do this if we treat every user exactly the same way.6

  1. This is purposefully a broad definition because we’re not just talking about web pages here, but rather all kinds of technology. The principles are universal. There are, however, more exacting definitions available. For instance, the W3C begins the HTML 4 specification with some formal definitions, including what a “user agent” is. See http://www.w3.org/TR/REC-html40/conform.html. []
  2. In real use cases, technical jargon and specific tools like a web browser are omitted because such use cases are used to define a system’s requirements, not its implementation. Nevertheless, the notion of an actor and an actor’s goals are helpful in understanding the mysterious “user” and this user’s software. []
  3. XML-RPC is a term referring to the use of XML files describing method calls and data transmitted over HTTP, typically used by automated systems. It is thus a great example of a technology that takes advantage of XML’s data serialization capabilities, and is often thought of as a precursor to today’s Ajax techniques. []
  4. It was in fact the much older email technology from which the term user agent originated; an email client program is more technically called a mail user agent (MUA). []
  5. As it happens, this is the same argument open source software proponents make about why such open source software often succeeds in meeting the needs of more users than closed source, proprietary systems controlled solely by a single company with (by definition) relatively limited resources. []
  6. This philosophy is embodied in the formal study of ethics, which is a compelling topic for us as CSS developers, considering the vastness of the implications we describe here. []

Introducing “Subresource Integrity (SRI) Manager” for WordPress

Subresource Integrity (SRI) is a proposed W3C standard that Web developers can make use of to protect their websites from being used in JavaScript-based DDoS attacks, such as the one recently suffered by GitHub.com. It’s not yet widely supported by browsers but is harmless to add to your pages for browsers that don’t support it.

Using SRI declarations is a simple matter of adding an integrity attribute to the HTML tag that points to an external resource (like a <script> tag) that contains a hash specifier (like sha256) and the proper hash of the content you expect to load. This indicates to the browser that the resource to which the element points should have the given hash. If the content that the browser receives from the remote server does not produce the specified hash, the browser ignores (refuses to load) the resource, and fires an error event at the element, instead:

<script src="//example.com/example-library.js" integrity="sha256-ab3c54ef..998756"></script>

I wanted to make use of this immediately, but I didn’t want to manually hash all the libraries that my website calls to. These include stylesheets from content distribution networks like Bootstrap and JavaScript libraries such as jQuery DataTables. So I wrote a WordPress plugin that keeps track of the hashes for me.

With the Subresource Integrity (SRI) Manager plugin for WordPress (WP-SRI), each time a stylesheet or JavaScript is enqueued by a WordPress plugin or theme (using wp_enqueue_style() or wp_enqueue_script()), WP-SRI makes a note of the requested resource URL and adds it to a list of known resources. If WP-SRI has not seen the resource before, it grabs the resource content itself, produces a hash, and saves that alongside the resource URL in WordPress’s wp_options table. When HTML is printed to the screen, it adds the integrity attribute and the associated hash automatically.

Using this plugin can dramatically reduce the likelihood that visitors to your site will be strong-armed into participating in an HTTP DDoS attack. In future versions of this plugin, I also hope to provide an easy-to-use interface for site administrators so that they can maintain a customized list of resource hashes, and to trigger on-demand integrity checks of these resources.

Please submit feature requests and/or patches to the issue tracker on GitHub, and post support requests to the WordPress plugin support forum. Thanks. :)

The mystery of the disappearing horizontal scrollbar

A classic exchange from the WordPress Support Forum for one of my plugins:

Them:

Hi,

When I first installed this plugin, there was an automatic horizontal scrollbar so that users could move to see all of the columns. However, it has now disappeared which means one of the columns is not fully readable.

Can you help?

Thanks.

Me:

Right above the button you clicked to post this question there is a line of text that reads:

Did you include a link to your site, so that others can see the problem?

Given that you didn’t notice this, I am going to suggest that you slow down and think about what was different on your site from when you installed the plugin (and experienced it working as expected) and now (when it’s not). If you still need help after that, I suggest you first think more about the answer to the question quoted above before you post again.

Them:

I apologise, it was an oversight on my behalf, as you have pointed out. Put it down to Friday ‘end of the week’ fuzzy head, if you like.

The pages where we are currently using the plugin are [here and here].

To clarify, if you hover around the rows and columns, it appears you can swipe and move it around, but the visible arrows and scrollbar is not visible. We have a lot of not very IT-literate people who use our website for support so it would be handy to make it like a visible scrollbar to click on again, if possible.

Thanks for your patience.

Me:

A clarifying question: you want a scrollbar to appear but one does not exist even when the browser window is too narrow to fit the whole table?

That is to say, I am confused by the statement “the visible arrows and scrollbar is not visible.” :/

Them:

Yes, we want a scrollbar and the arrows that go either end of said scrollbar to display because the browser window is too narrow to fit the whole table.

We feel that people would prefer to have the arrows on the scrollbar visible to encourage them to click them so that they can see the columns that go off of the screen.

Me:

So, when I go to your pages they both show scroll bars just as you say you want them. :\ I’m afraid whatever you’re seeing is specific to your combination of computer and browser.

It’s very likely that users who browse to your site are seeing the scroll bars show automatically. At least, that’s what happens when I load the site.

The beauty of the Web is that users are able to define their experience so it suits them best. Users whose browsers and operating systems are set to show scroll bars are showing scroll bars. This is good news, because it means you do not need to worry about your site malfunctioning: it is not malfunctioning and, as you say, nothing about it has changed.

How would you design an online social network that was hostile to abusers?

Everyone realizes that the Internet’s public squares have a harassment problem. No one seems to know what to do about it. I argue that’s because they don’t know how to think about online harassment and abuse—or even power, more generally. I argue that I do. But don’t take my word for it. Take my ideas, and implement them yourselves. Then let’s let the results speak for themselves.

“So, maymay,” I can already hear you asking, “how would you design an online social network that was hostile to abusers?” You’re probably asking this because you either don’t know that I’ve written about it before, or you haven’t been able to understand from what I’ve written how to take the lessons from code I’ve deployed in the Predator Alert Tool project and apply it to your own projects. That’s okay. You’re not alone.

Recently, I received an email from a developer asking for advice about this exact issue. They’ve told me they’d be fine with my sharing our conversation here, in the hopes that it gets other developers thinking about what they can do to proactively “protect people from abusers online,” as they put it. Here is our exchange (slightly edited for anonymity and clarity) so far. The email I received went something like this:

Hello! I’m building a new social network and want to be pro-active about protecting people. I wanted to reach out as I have little experience with protecting people from shitty people and abusers online, and the Predator Alert Tools is great. Is there any way I can help contribute to those projects, and/or utilise them somehow with [my project] to help protect people?

Any help you can give would be appreciated.

Thanks,

[Anon Developer]

I wrote back a few days later:

Thanks [for reaching out, Anon Developer].

Yes.

You can contribute to any of the PATs in any way you like. Here’s a short “how to help” page for the project. It talks mostly about Predator Alert Tool for Facebook but it’s relevant to all the tools.

Well, there are a number of themes that run through the entire suite of tools, and those are the only things I can talk about without knowing more about [your specific project]. So for now, let me just point your attention to these two blog posts about the tools.

First, “More on ‘The Match Percentage Fallacy’, or The Influence of Rolequeerness on the Predator Alert Tool project.” This post explicitly uses the language of game theory to talk about protecting people from online predation. An excerpt:

Predator Alert Tool for OkCupid highlights the signals players send when they answer OkCupid’s Match Questions to other players in order to de-silo as much information as possible, thereby hoping to expand the set of possible moves a given player (user of PAT-OKC) is aware of and enabling them to analyze the given situation (the decision tree of their “turn”) with the information they received through the tool. This is a fundamentally different approach than the one OkCupid’s “Match Percentage” interface provides, and this is no coincidence.

The “Match Percentage” interface is designed to account for “the best possible outcome” for OkCupid itself, not the best outcome for the OkCupid user. This makes sense when you realize that OkCupid is a company, and they have their own incentives and have defined the win conditions of this complex game very differently than their users (we) have.

In other words, the single most obvious problem with online “dating” sites (a category which include “social networking sites,” obviously) is that they are designed from the ground-up to focus on filtering data out as opposed to considering related data important. This is precisely the environment in which serial rapists are most protected. If you are serious about building a social networking site that is proactive about maintaining an environment hostile to these kinds of abuses, you need to focus on identifying and surfacing information about signals between users that are negative as well as positive. Again: rather than burying those signals, you need to surface them. Use OkCupid’s “Match Percentage” interface as a perfect example of what not to do.

If that’s curious to you and, again, if you’re interested in pursuing this line of questioning further, write back and tell me more about [your project], and yourself, and so on. Let’s have a conversation. Predator Alert Tool’s implementations are different depending on the site for which the specific tool was intended not only because the technology of different sites is different, as you know, but also because the culture of each website is different; users interact with the sites differently based on the messaging, context, and approaches different sites take. So Predator Alert Tool also needs to integrate with a culture, not just a programming language.

For more on that, see this early post by one of my collaborators, “Rape Culture, meet Internet Culture.” An excerpt:

Probably the most well-known recent pushback against rape culture is the Predditors story, in which some Reddit users discovered and published the identities of others who had been posting sexualized pictures of young women. The Predditors tumblr has since been shut down, but its contents are still available in a GoogleDoc here. Sexual abusers have also been outed via YouTube, Facebook, and Twitter. Blogs provide a public square for arguments about rape culture to rage. Twitter users directly critique the media. I’ve heard rumors of a Tumblr hashtag used by survivors to post the names and addresses of their rapists. The FetLife Alleged Abusers Database Engine (recently rolled into the Predator Alert Tools suite as the “Predator Alert Tool for FetLife”) collects anonymous reports of consent violations in the BDSM community and then flags the FetLife profiles of alleged abusers. And I recently helped beta-test a new tool, The Predator Alert Tool for OkCupid, which highlights self-reported sexually violent opinions and behaviors by OkCupid users.

I don’t think any of these tools, or even all of them together, will put the nail in the coffin of rape culture. Like other kinds of abuse, rape culture adapts to new environments quickly. Activists need to stay on our games in order to keep exposing new forms of it as they appear. We need to keep experimenting, trying new things, and being creative with whatever resources we have available. What I find most powerful about these tools is the ways each seems tailored to the specific culture from which it emerged. Predditors addresses rape culture on Reddit by retaliating against its perpetrators using technological savvy, counter-rhetoric about free speech and privacy, and a “troll the trolls” sort of strategy all suited to Reddit’s particular cultural sensibility. FAADE, on the other hand, capitalizes on a mentality strongly espoused by FetLife users that the BDSM community is like a “small town” in which everyone is connected to everyone else by kinship ties. BDSMers often rely on personal references and a player’s public reputation to assess their safety, thus a database allowing FetLife profiles (the site of a player’s public reputation online) to be tagged with negative references from community members has a powerful impact on the sub-cultural consciousness. What would a similar tool look like for Twitter or Facebook?

So again, the question you’re asking is bigger than an email. I’d be interested in having that bigger conversation with you, if you are serious about having it, too.

Thanks again for reaching out.

Cheers,
-maymay
Maymay.net
Cyberbusking.org

I was pleased by the developer’s response:

Thank you so much for all this information.

I often struggle to digest information like this; I’ll be re-reading these articles a few times to try to understand them more fully.

I would like to have the bigger conversation, but […] I need to watch out I don’t bite off more than I can chew. I regard this topic as highly important and a responsibility I now have.

The use of game theory resonates with me, as I’ve used ideas from my basic understanding of game theory as influence in the structure of [my project] (only very crudely). So if I can expand those ideas in a way which protects people, all the better.

Am I right in my understanding that one core idea is that negative information is intentionally hidden in most places, in order to benefit the company? So (and this is a contrived example) where [my project] might track how many messages a person receives as a positive, it should also track, process, and weight the negative events associated; messages which go unrelieved to, messages reported as abusive etc?

Thanks again,

[Anon Developer]

My response tried to elaborate on “negative” signaling:

Of course. That’s fine. Take your time.

It’s good that you consider this a responsibility you have, because you already had this responsibility, even before you were developing [your project]. ;)

You’re almost right about your understanding.

The bigger point being made here is that, from the perspective of users, [your project] is a hostile, not a friendly. You, as the company, are not a passive facilitator of information. You are in a decidedly dominant position over your users, and this means that you have the capacity to be predatory in relation to them, because when it comes to their interactions with or through [your project], you are obscenely more powerful than they are.

So, yes, you should also track, process, and weight negative events. But you should also not presume to necessarily know what events are negative and what events are positive. The minute you think you can determine what negative signaling is for someone else, you become much more likely to fail to empower that other person. It’s not up to you to determine what’s negative or what’s not. You can, of course, do some things to make this more obvious, and the “report abuse” feature is a start. But the problem with “abuse reports” is that those reports are sent to the entity in the [project] ecosystem that already has the most power: [the project/website/company itself]. That’s a recipe for disaster.

One simple way to tweak this system would be to simply display a tally of all the abuse reports a given profile has received next to their profile. Allow people to click-through on that icon to a list of all abuse reports filed against that profile. Don’t hide it. Don’t make excuses for it. Don’t arbitrate it. Don’t moderate it. In a centralized system such as I understand [your project] to be (I signed up for an account today and had a look around), a moderation system is far more likely to end up as a “benevolent” dictatorship rather than an effective means of anti-abuse behavior. You should not appoint yourself as the police.

For more on this point, see my blog post, “Revisiting why ‘no moderation’ is a feature, not a bug, in Predator Alert Tool for Facebook.” An excerpt:

“Moderation” is a governance tool that may make sense in the context of online communities with a relatively homogenous populace, such as multiplayer video games or topically-oriented forums. But moderation is inherently in conflict with the goal of dissolving authority and dispersing power amongst a heterogenous populace already prone to conflict. There is no system of moderation that is not also a system of social control. And in the context of a project explicitly designed to overcome the iniquities introduced to human experience by traditional mechanisms of social control, adding a traditional mechanism of social control is shortsighted at best and active sabotage at worst.

We realize this is difficult to understand at first. After all, there is currently no physical-world social context wherein we are free from the power of authorities we did not choose and also do not agree with. Everyone has a parent, a teacher, or a boss—even the fucking police. As one PAT collaborator wrote:

We’re all so accustomed to having our spaces monitored and moderated and overseen “for our own safety” that sometimes, when we take the well-being of our communities into our own hands, we appear to be doing more harm than good. That’s only because we’re comparing our efforts to the imaginary “safe” world we’ve been told that we live in, not to the dangerous realities that survivors actually face online and off.

Put another way, from the perspective of a vulnerable populace, namely people who are the targets of rape and physical abuse, a system that erodes the power of central authorities (such as website admins, or the cops) is a move towards safety, not away from it.

In other words, the premise of [your project] is to connect people with different characteristics who want to engage positively. This means you have to provide them with the information both to find people they like and to avoid people they don’t like. You can’t do this effectively if you only surface positive signals while hiding negative ones. And to effectively surface negative signals, you have to re-examine your assumptions about what “negative” means because, if you don’t, especially in the context of a diverse user base, you’re going to get it wrong for at least some users. When you get it wrong for them, you create an environment in which it is particularly easy to predate on that specific subsection of your user base.

That’s why most dating sites are a breeding ground for predatory users. Most dating sites are, after all, programmed by men.

Again, feel free to email me whenever you’re ready for another round. This is basically what I do for “a living.” :P I would strongly encourage you to read the posts tagged with “Predator Alert Tool” on the archives of my various blogs, of course.

My hope in sharing this is to encourage other people to think more critically and creatively about what structural changes are necessary to facilitate anti-abuse action. Recent attempts by Twitter and WAM have been decidedly stupid. And I don’t say that lightly. These are some exceptionally talented people in a number of fields ranging from gender advocacy to technology. And yet most acts I see being taken—”moderation superpowers” to use the most recent buzzword—is downright counterproductive. Obviously.

It’s time we stopped believing that authority or authorities in public spheres are a solution. The longer we wait to face the fact that power corrupts, the more abuse we’ll bring down on ourselves, our communities, and our peers. Heed this warning: do not police.

Easy template injection in JavaScript for userscript authors, plugin devs, and other people who want to fuck with Web page content

The Predator Alert Tool for Twitter is coming along nicely, but it's frustratingly slow going. It's extra frustrating for me because ever since telling corporate America and its project managers to go kill themselves, I've grown accustomed to an utterly absurd speed of project development. I know I've only been writing and rewriting code for just under two weeks (I think—I honestly don't even know or care what day it is), but still.

I think it also feels extra slow is because I'm learning a lot of new stuff along the way. That in and of itself is great, but I'm notoriously impatient. Which I totally consider a virtue because fuck waiting. Still, all this relearning and slow going has given me the opportunity to refine a few techniques I used in previous Predator Alert Tool scripts.

Here's one technique I think is especially nifty: template injection.

Read more

Cross-post: Edenfantasys’s unethical technology is a self-referential black hole

This entry was originally published at my other blog. I’m cross-posting it here in order to make sure it gets copied to more servers, as some people have suggested I’ll face a cease and desist order for publishing it in the first place. Please help distribute this important information by freely copying and republishing this post under the conditions of my CC-BY-NC-ND license: provide me with attribution and a (real) back link, and you are free to republish an unaltered version of this post wherever you like. Thanks.

A few nights ago, I received an email from Editor of EdenFantasys’s SexIs Magazine, Judy Cole, asking me to modify this Kink On Tap brief I published that cites Lorna D. Keach’s writing. Judy asked me to “provide attribution and a link back to” SexIs Magazine. An ordinary enough request soon proved extraordinarily unethical when I discovered that EdenFantasys has invested a staggering amount of time and money to develop and implement a technology platform that actively denies others the courtesy of link reciprocity, a courtesy on which the ethical Internet is based.

While what they’re doing may not be illegal, EdenFantasys has proven itself to me to be an unethical and unworthy partner, in business or otherwise. Its actions are blatantly hypocritical, as I intend to show in detail in this post. Taking willful and self-serving advantage of those not technically savvy is a form of inexcusable oppression, and none of us should tolerate it from companies who purport to be well-intentioned resources for a community of sex-positive individuals.

For busy or non-technical readers, see the next section, Executive Summary, to quickly understand what EdenFantasys is doing, why it’s unethical, and how it affects you whether you’re a customer, a contributor, or a syndication partner. For the technical reader, the Technical Details section should provide ample evidence in the form of a walkthrough and sample code describing the unethical Search Engine Optimization (SEO) and Search Engine Marketing (SEM) techniques EdenFantasys, aka. Web Merchants, Inc., is engaged in. For anyone who wants to read further, I provide an Editorial section in which I share some thoughts about what you can do to help combat these practices and bring transparency and trust—not the sabotage of trust EdenFantasys enacts—to the market.

EXECUTIVE SUMMARY

Internet sex toy retailer Web Merchants, Inc., which bills itself as the “sex shop you can trust” and does business under the name EdenFantasys, has implemented technology on their websites that actively interferes with contributors’ content, intercepts outgoing links, and alters republished content so that links in the original work are redirected to themselves. Using techniques widely acknowledged as unethical by Internet professionals and that are arguably in violation of major search engines’ policies, EdenFantasys’s publishing platform has effectively outsourced the task of “link farming” (a questionable Search Engine Marketing [SEM] technique) to sites with which they have “an ongoing relationship,” such as AlterNet.org, other large news hubs, and individual bloggers’ blogs.

Articles published on EdenFantasys websites, such as the “community” website SexIs Magazine, contain HTML crafted to look like links, but aren’t. When visited by a typical human user, a program written in JavaScript and included as part of the web pages is automatically downloaded and intercepts clicks on these “link-like” elements, fetching their intended destination from the server and redirecting users there. Due to the careful and deliberate implementation, the browser’s status bar is made to appear as though the link is legitimate, and that a destination is provided as expected.

For non-human visitors, including automated search engine indexing programs such as Googlebot, the “link” remains non-functional, making the article a search engine’s dead-end or “orphan” page whose only functional links are those whose destination is EdenFantasys’s own web presence. This makes EdenFantasys’ website(s) a self-referential black hole that provides no reciprocity for contributors who author content, nor for any website ostensibly “linked” to from article content. At the same time, EdenFantasys editors actively solicit inbound links from individuals and organizations through “link exchanges” and incentive programs such as “awards” and “free” sex toys, as well as syndicating SexIs Magazine content such that the content is programmatically altered in order to create multiple (real) inbound links to EdenFantasys’s websites after republication on their partner’s media channels.

How EdenFantasys’s unethical practices have an impact on you

Regardless of who you are, EdenFantasys’s unethical practices have a negative impact on you and, indeed, on the Internet as a whole.

See for yourself: First, log out of any and all EdenFantasys websites or, preferably, use a different browser, or even a proxy service such as the Tor network for greater anonymity. Due to EdenFantasys’s technology, you cannot trust that what you are seeing on your screen is what someone else will see on theirs. Next, temporarily disable JavaScript (read instructions for your browser) and then try clicking on the links in SexIs Magazine articles. If clicking the intended off-site “links” doesn’t work, you know that your article’s links are being hidden from Google and that your content is being used for shady practices. In contrast, with JavaScript still disabled, navigate to another website (such as this blog), try clicking on the links, and note that the links still work as intended.

Here’s another verifiable example from the EdenFantasys site showing that many other parts of Web Merchants, Inc. pages, not merely SexIs Magazine, are affected as well: With JavaScript disabled, visit the EdenFantasys company page on Aslan Leather (note, for the sake of comparison, the link in this sentence will work, even with JavaScript off). Try clicking on the link in the “Contact Information” section in the lower-right hand column of the page (shown in the screenshot, below). This “link” should take you to the Aslan Leather homepage but in fact it does not. So much for that “link exchange.”

(Click to enlarge.)

  • If you’re an EdenFantasys employee, people will demand answers from you regarding the unethical practices of your (hopefully former) employer. While you are working for EdenFantasys, you’re seriously soiling your reputation in the eyes of ethical Internet professionals. Ignorance is no excuse for the lack of ethics on the programmers’ part, and it’s a shoddy one for everyone else; you should be aware of your company’s business practices because you represent them and they, in turn, represent you.
  • If you’re a partner or contributor (reviewer, affiliate, blogger), while you’re providing EdenFantasys with inbound links or writing articles for them and thereby propping them up higher in search results, EdenFantasys is not returning the favor to you (when they are supposed to be doing so). Moreover, they’re attaching your handle, pseudonym, or real name directly to all of their link farming (i.e., spamming) efforts. They look like they’re linking to you and they look like their content is syndicated fairly, but they’re actually playing dirty. They’re going the extra mile to ensure search engines like Google do not recognize the links in articles you write. They’re trying remarkably hard to make certain that all roads lead to EdenFantasys, but none lead outside of it; no matter what the “link,” search engines see it as stemming from and leading to EdenFantasys. The technically savvy executives of Web Merchants, Inc. are using you without giving you a fair return on your efforts. Moreover, EdenFantasys is doing this in a way that preys upon people’s lack of technical knowledge—potentially your own as well as your readership’s. Do you want to keep doing business with people like that?
  • If you’re a customer, you’re monetarily supporting a company that essentially amounts to a glorified yet subtle spammer. If you hate spam, you should hate the unethical practices that lead to spam’s perpetual reappearance, including the practices of companies like Web Merchants, Inc. EdenFantasys’s unethical practices may not be illegal, but they are unabashedly a hair’s width away from it, just like many spammers’. If you want to keep companies honest and transparent, if you really want a “sex shop you can trust,” this is relevant to you because EdenFantasys is not it. If you want to purchase from a retailer that truly strives to offer a welcoming, trustworthy community for those interested in sex positivity and sexuality, pay close attention and take action. For ideas about what you can do, please see the “What you can do” section, below.
  • If you’ve never heard about EdenFantasys before, but you care about a fair and equal-opportunity Internet, this is relevant to you because what EdenFantasys is doing takes advantage of non-tech-savvy people in order to slant the odds of winning the search engine game in their favor. They could have done this fairly, and I personally believe that they would have succeeded. Their sites are user-friendly, well-designed, and solidly implemented. However, they chose to behave maliciously by not providing credit where credit is due, failing to follow through on agreements with their own community members and contributors, and sneakily utilizing other publishers’ web presences to play a very sad zero-sum game that they need not have entered in the first place. In the Internet I want, nobody takes malicious advantage of those less skilled than they are because their own skill should speak for itself. Isn’t that the Internet and, indeed, the future you want, too?

TECHNICAL DETAILS

What follows is a technical exploration of the way the EdenFantasys technology works. It is my best-effort evaluation of the process in as much detail as I can manage within strict self-imposed time constraints. If any of this information is incorrect, I’d welcome any and all clarifications provided by the EdenFantasys CTO and technical team in an appropriately transparent, public, and ethical manner. (You’re welcome—nay, encouraged—to leave a comment.)

Although I’m unconvinced that EdenFantasys understands this, it is the case that honesty is the best policy—especially on the Internet, where everyone has the power of “View source.”

The “EF Framework” for obfuscating links

Article content written by contributors on SexIs Magazine pages is published after all links are replaced with a <span> element bearing the class of linklike and a unique id attribute value. This apparently happens across any and all content published by Web Merchants, Inc.’s content management system, but I’ll be focusing on Lorna D. Keach’s post entitled SexFeed:Anti-Porn Activists Now Targeting Female Porn Addicts for the sake of example.

These fake links look like this in HTML:

And according to Theresa Flynt, vice president of marketing for Hustler video, <span class="linklike" ID="EFLink_68034_fe64d2">female consumers make up 56% of video sales.</span>

This originally published HTML is what visitors without JavaScript enabled (and what search engine indexers) see when they access the page. Note that the <span> is not a real link, even though it is made to look like one. (See Figure 1; click it to enlarge.)

Figure 1:

In a typical user’s browser, when this page is loaded, a JavaScript program is executed that mutates these “linklike” elements into <a> elements, retaining the “linklike” class and the unique id attribute values. However, no value is provided in the href (link destination) attribute of the <a> element. See Figure 2.

Figure 2:

The JavaScript program is downloaded in two parts from the endpoint at http://cdn3.edenfantasys.com/Scripts/Handler/jsget.ashx. The first part, retrieved in this example by accessing the URI at http://cdn3.edenfantasys.com/Scripts/Handler/jsget.ashx?i=jq132_cnf_jdm12_cks_cm_ujsn_udm_stt_err_jsdm_stul_ael_lls_ganl_jqac_jtv_smg_assf_agrsh&v_14927484.12.0, loads the popular jQuery JavaScript framework as well as custom code called the “EF Framework”.

The EF Framework contains code called the DBLinkHandler, an object that parses the <span> “linklike” elements (called “pseudolinks” in the EF Framework code) and retrieves the real destination. The entirety of the DBLinkHandler object is shown in code listing 1, below. Note the code contains a function called handle that performs the mutation of the <span> “linklike” elements (seen primarily on lines 8 through 16) and, based on the prefix of each elements’ id attribute value, two key functions (BuildUrlForElement and GetUrlByUrlID, whose signatures are on lines 48 and 68, respectively) interact to set up the browser navigation after responding to clicks on the fake links.

var DBLinkHandler = {
    pseudoLinkPrefix: "EFLink_",
    generatedAHrefPrefix: "ArtLink_",
    targetBlankClass: "target_blank",
    jsLinksCssLinkLikeClass: "linklike",
    handle: function () {
        var pseudolinksSpans = $("span[id^='" + DBLinkHandler.pseudoLinkPrefix + "']");
        pseudolinksSpans.each(function () {
            var psLink = $(this);
            var cssClass = $.trim(psLink.attr("class"));
            var target = "";
            var id = psLink.attr("id").replace(DBLinkHandler.pseudoLinkPrefix, DBLinkHandler.generatedAHrefPrefix);
            var href = $("<a></a>").attr({
                id: id,
                href: ""
            }).html(psLink.html());
            if (psLink.hasClass(DBLinkHandler.targetBlankClass)) {
                href.attr({
                    target: "_blank"
                });
                cssClass = $.trim(cssClass.replace(DBLinkHandler.targetBlankClass, ""))
            }
            if (cssClass != "") {
                href.attr({
                    "class": cssClass
                })
            }
            psLink.before(href).remove()
        });
        var pseudolinksAHrefs = $("a[id^='" + DBLinkHandler.generatedAHrefPrefix + "']");
        pseudolinksAHrefs.live("mouseup", function (event) {
            DBLinkHandler.ArtLinkClick(this)
        });
        pseudolinksSpans = $("span[id^='" + DBLinkHandler.pseudoLinkPrefix + "']");
        pseudolinksSpans.live("click", function (event) {
            if (event.button != 0) {
                return
            }
            var psLink = $(this);
            var url = DBLinkHandler.BuildUrlForElement(psLink, DBLinkHandler.pseudoLinkPrefix);
            if (!psLink.hasClass(DBLinkHandler.targetBlankClass)) {
                RedirectTo(url)
            } else {
                OpenNewWindow(url)
            }
        })
    },
    BuildUrlForElement: function (psLink, prefix) {
        var psLink = $(psLink);
        var sufix = psLink.attr("id").toString().substring(prefix.length);
        var id = (sufix.indexOf("_") != -1) ? sufix.substring(0, sufix.indexOf("_")) : sufix;
        var url = DBLinkHandler.GetUrlByUrlID(id);
        if (url == "") {
            url = EF.Constants.Links.Url
        }
        var end = sufix.substring(sufix.indexOf("_") + 1);
        var anchor = "";
        if (end.indexOf("_") != -1) {
            anchor = "#" + end.substring(0, end.lastIndexOf("_"))
        }
        url += anchor;
        return url
    },
    ArtLinkClick: function (psLink) {
        var url = DBLinkHandler.BuildUrlForElement(psLink, DBLinkHandler.generatedAHrefPrefix);
        $(psLink).attr("href", url)
    },
    GetUrlByUrlID: function (UrlID) {
        var url = "";
        UrlRequest = $.ajax({
            type: "POST",
            url: "/LinkLanguage/AjaxLinkHandling.aspx",
            dataType: "json",
            async: false,
            data: {
                urlid: UrlID
            },
            cache: false,
            success: function (data) {
                if (data.status == "Success") {
                    url = data.url;
                    return url
                }
            },
            error: function (xhtmlObj, status, error) {}
        });
        return url
    }
};

Once the mutation is performed and all the content “links” are in the state shown in Figure 2, above, an event listener has been bound to the anchors that captures a click event. This is done using prototypal extension, aka. classic prototypal inheritance, in another part of the code, the live function on line 2,280 of the (de-minimized) jsget.ashx program, as shown in code listing 2, here:

        live: function (G, F) {
            var E = o.event.proxy(F);
            E.guid += this.selector + G;
            o(document).bind(i(G, this.selector), this.selector, E);
            return this
        },

At this point, clicking on one of the “pseudolinks” triggers the EF Framework to call code set up by the GetUrlByUrlID function from within the DBLinkHandler object, initiating an XMLHttpRequest (XHR) connection to the AjaxLinkHandling.aspx server-side application. The request is an HTTP POST containing only one parameter, called urlid, and its value matches a substring from within the id value of the “pseudolinks.” In this example, the id attribute contains a value of EFLink_68034_fe64d2, which means that the unique ID POST’ed to the server is 68034. This is shown in Figure 3, below.

Figure 3:

The response from the server, shown in Figure 4, is also simple. If successful, the intended destination is retrieved by the GetUrlByUrlID object’s success function (on line 79 of Code Listing 1, above) and the user is redirected to that web address, as if the link was a real one all along. The real destination, in this case to CNN.com, is thereby only revealed after the XHR request returns a successful reply.

Figure 4:

All of this obfuscation effectively blinds machines such as the Googlebot who are not JavaScript-capable from seeing and following these links. It deliberately provides no increased Pagerank for the link destination (as a real link would normally do) despite being “linked to” from EdenFantasys’s SexIs Magazine article. While the intended destination in this example link was at CNN.com, it could just as easily have been—and is, in other examples—links to the blogs of EdenFantasys community members and, indeed, everyone else linked to from a SexIs Magazine article or potentially any website operated by Web Merchants, Inc. that makes use of this technology.

The EdenFantasys Outsourced Link-Farm

In addition to creating a self-referential black hole with no gracefully degrading outgoing links, EdenFantasys also actively performs link-stuffing through its syndicated content “relationships,” underhandedly creating an outsourced and distributed link-farm, just like a spammer. The difference is that this spammer (Web Merchants, Inc. aka EdenFantasys) is cleverly crowd-sourcing high-value, high-quality content from its own “community.”

Articles published at SexIs Magazine are syndicated in full to other large hub sites, such as AlterNet.org. Continuing with the above example post by Lorna D. Keach, Anti-Porn Activists Now Targeting Female Porn Addicts, we can see that this content was republished on AlterNet.org shortly after original publication through EdenFantasys’ website on May 3rd at http://www.alternet.org/story/146774/christian_anti-porn_activists_now_targeting_female_. However, a closer look at the HTML code of the republication shows that each and every link contained within the article points to the same destination: the same article published on SexIs Magazine, as shown in Figure 5.

Figure 5:

Naturally, these syndicated links provided to third-party sites by EdenFantasys are real and function as expected to both human visitors and to search engines indexing the content. The result is “natural,” high-value links to the EdenFantasys website from these third-party sites; EdenFantasys doesn’t merely scrounge pagerank from harvesting the sheer number of incoming links, but as each link’s anchor text is different, they are setting themselves up to match more keywords in search engine results, keywords that the original author likely did not intend to direct to them. Offering search engines the implication that EdenFantasys.com contains the content described in the anchor text, when in fact EdenFantasys merely acts as an intermediary to the information, is very shady, to say the least.

In addition to syndication, EdenFantasys employs human editors to do community outreach. These editors follow up with publishers, including individual bloggers (such as myself), and request that any references to published material provide attribution and a link back to us, to use the words of Judy Cole, Editor of SexIs Magazine in an email she sent to me (see below), and presumably many others. EdenFantasys has also been known to request “link exchanges,” and offer incentive programs that encouraged bloggers to add the EdenFantasys website to their blogroll or sidebar in order to help raise both parties search engine ranking, when in fact EdenFantasys is not actually providing reciprocity.

More information about EdenFantasys’s unethical practices, which are not limited to technical subterfuge, can be obtained via AAGBlog.com.

EDITORIAL

It is unsurprising that the distributed, subtle, and carefully crafted way EdenFantasys has managed to crowd-source links has (presumably) remained unpenalized by search engines like Google. It is similarly unsurprising that nontechnical users such as the contributors to SexIs Magazine would be unaware of these deceptive practices, or that they are complicit in promoting them.

This is no mistake on the part of EdenFantasys, nor is it a one-off occurrence. The amount of work necessary to implement the elaborate system I’ve described is also not even remotely feasible for a rogue programmer to accomplish, far less accomplish covertly. No, this is the result of a calculated and decidedly underhanded strategy that originated from the direction of top executives at Web Merchants, Inc. aka EdenFantasys.

It is unfortunate that technically privileged people would be so willing to take advantage of the technically uneducated, particularly under the guise of providing a trusted place for the community which they claim to serve. These practices are exactly the ones that “the sex shop you can trust” should in no way support, far less be actively engaged in. And yet, here is unmistakable evidence that EdenFantasys is doing literally everything it can not only to bolster its own web presence at the cost of others’, but to hide this fact from its understandably non-tech-savvy contributors.

On a personal note, I am angered that I would be contacted by the Editor of SexIs Magazine, and asked to properly “attribute” and provide a link to them when it is precisely that reciprocity which SexIs Magazine would clearly deny me (and everyone else) in return. It was this request originally received over email from Judy Cole, that sparked my investigation outlined above and enabled me to uncover this hypocrisy. The email I received from Judy Cole is republished, in full, here:

From: Judy Cole <luxuryholmes@gmail.com>
Subject: Repost mis-attributed
Date: May 17, 2010 2:42:00 PM PDT
To: kinkontap+viewermail@gmail.com
Cc: Laurel <laurelb@edenfantasys.com>

Hello Emma and maymay,

I am the Editor of the online adult magazine SexIs (http://www.edenfantasys.com/sexis/). You recently picked up and re-posted a story of ours by Lorna Keach that Alternet had already picked up:

http://kinkontap.com/?s=alternet

We were hoping that you might provide attribution and a link back to us, citing us as the original source (as is done on Alternet, with whom we have an ongoing relationship), should you pick up something of ours to re-post in the future.

If you would be interested in having us send you updates on stories that might be of interest, I would be happy to arrange for a member of our editorial staff to do so. (Like your site, by the way. TBK is one of our regular contributors.)

Thanks and Best Regards,

Judy Cole
Editor, SexIs

Judy’s email probably intended to reference the new Kink On Tap briefs that my co-host Emma and I publish, not a search result page on the Kink On Tap website. Specifically, she was talking about this brief: http://KinkOnTap.com/?p=676. I said as much in my reply to Judy:

Hi Judy,

The URL in your email doesn’t actually link to a post. We pick up many stories from AlterNet, as well as a number from SexIs, because we follow both those sources, among others. So, did you mean this following entry?

http://KinkOnTap.com/?p=676

If so, you should know that we write briefs as we find them and provide links to where we found them. We purposefully do not republish or re-post significant portions of stories and we limit our briefs to short summaries in deference to the source. In regards to the brief in question, we do provide attribution to Lorna Keach, and our publication process provides links automatically to, again, the source where we found the article. :) As I’m sure you understand, this is the nature of the Internet. Its distribution capability is remarkable, isn’t it?

Also, while we’d absolutely be thrilled to have you send us updates on stories that might be of interest, we would prefer that you do so in the same way the rest of our community does: by contributing to the community links feed. You can find detailed instructions for the many ways you can do that on our wiki:

http://wiki.kinkontap.com/wiki/Community_links_feed

Congratulations on the continued success of SexIs.

Cheers,
-maymay

At the time when I wrote the email replying to Judy, I was perturbed but could not put my finger on why. Her email upset me because she seemed to be suggesting that our briefs are wholesale “re-posts,” when in fact Emma and I have thoroughly discussed attribution policies and, as mentioned in my reply, settled on a number of practices including a length limit, automated back linking (yes, with real links, go see some Kink On Tap briefs for yourself), and clearly demarcating quotes from the source article in our editorializing to ensure we play fair. Clearly, my somewhat snarky reply betrays my annoyance.

In any event, this exchange prompted me to take a closer look at the Kink On Tap brief I wrote, at the original article, and at the cross-post on AlterNet.org. I never would have imagined that EdenFantasys’s technical subterfuge would be as pervasive as it has proven to be. It’s so deeply embedded in the EdenFantasys publishing platform that I’m willing to give Judy the benefit of the doubt regarding this hypocrisy because she doesn’t seem to understand the difference between a search query and a permalink (something any laymen blogger would grok). This is apparent from her reply to my response:

From: Judy Cole <luxuryholmes@gmail.com>
Subject: Re: Repost mis-attributed
Date: May 18, 2010 4:57:59 AM PDT
[…redundant email headers clipped…]

Funny, the URL in my email opens the same link as the one you sent me when I click on it.

Maybe if you pick up one of our stories in future, you could just say something like “so and so wrote for SexIs.” ?

As it stands, it looks as if Lorna wrote the piece for Alternet. Thanks.

Judy

That is the end of our email exchange, and will be for good, unless and until EdenFantasys changes its ways. I will from this point forward endeavor never to publish links to any web property that I know to be owned by Web Merchants, Inc., including EdenFantasys.com. I will also do my best to avoid citing any and all SexIs Magazine articles from here on out, and I encourage everyone who has an interest in seeing honesty on the Internet to follow my lead here.

As some of my friends are currently contributors to SexIs Magazine, I would like all of you to know that I sincerely hope you immediately sever all ties with any and all Web Merchants, Inc. properties, suppliers, and business partners, especially because you are friends and I think your work is too important to be sullied by such a disreputable company. Similarly, I hope you encourage your friends to do the same. I understand that the economy is rough and that some of you may have business contracts bearing legal penalties for breaking them, but I urge you to nevertheless consider looking at this as a cost-benefit analysis: the sooner you break up with EdenFantasys, the happier everyone on the Internet, including you, will be (and besides, you can loose just as much of your reputation, money, and pagerank while being happy as you can being sad).

What you can do

  • If you are an EdenFantasys reviewer, a SexIs Magazine contributor, or have any other arrangement with Web Merchants, Inc., write to Judy Cole and demand that content you produce for SexIs Magazine adheres to ethical Internet publication standards. Sever business ties with this company immediately upon receipt of any non-response, or any response that does not adequately address every concern raised in this blog post. (Feel free to leave comments on this post with technical questions, and I’ll do my best to help you sort out any l33t answers.)
  • EdenFantasys wants to stack the deck in Google. They do this by misusing your content and harvesting your links. To combat this effort, immediately remove any and all links to EdenFantasys websites and web presences from your websites. Furthermore, do not—I repeat—do not publish new links to EdenFantasys websites, not even in direct reference to this post. Instead, provide enough information, as I have done, so visitors to your blog posts can find their website themselves. In lieu of links to EdenFantasys, link to other bloggers’ posts about this issue. (Such posts will probably be mentioned in the comments section of this post.)
  • Boycott EdenFantasys: the technical prowess their website displays does provide a useful shopping experience for some people. However, that in no way obligates you to purchase from their website. If you enjoy using their interface, use it to get information about products you’re interested in, but then go buy those products elsewhere, perhaps from the manufacturers directly.
  • Watch for “improved” technical subterfuge from Web Merchants, Inc. As a professional web developer, I can identify several things EdenFantasys could do to make their unethical practices even harder to spot, and harder to stop. If you have any technical knowledge at all, even if you’re “just” a savvy blogger, you can keep a close watch on EdenFantasys and, if you notice anything that doesn’t sit well with you, speak up about it like I did. Get a professional programmer to look into things for you if you need help; yes, you can make a difference just by remaining vigilant as long as you share what you know and act honestly, and transparently.

If you have additional ideas or recommendations regarding how more people can help keep sex toy retailers honest, please suggest them in the comments.

Update: To report website spamming or any kind of fraud to Google, use the authenticated Spam Report tool.

Update: Google provides much more information about why the kinds of practices EdenFantasys is engaged in degrade the overall web experience for you and me. Read Cloaking, sneaky Javascript redirects, and doorway pages at the Google Webmaster Tools help site for additional SEO information. Using Google’s terminology, EdenFantasys’s unethical technology is a very skilled mix of social engineering and “sneaky JavaScript redirects.”

My SVG is Bigger than Your Flash – Presentation Notes

A few weeks ago, right before the Christmas holidays, I gave a brief internal presentation about SVG, an XML-based web standard for interactive images, to my coworkers. Since I was told that the presentation was helpful, I thought I’d share my notes. Those notes follow:

Introduction

First, credit where credit is due.

I learned almost everything I know about SVG from an acquaintance I met in Australia named Dmitry Baranovskiy. He is a superb client-side web developer who I helped add documentation for a brief time for one of his SVG-related projects called RaphaëlJS. I’ll show you that in a little bit.

Okay, so what is SVG?

What is SVG?

In a sentence, according to Adobe, the originators of the technology:

Scalable Vector Graphics (SVG) is a text-based graphics language that describes images with vector shapes, text, and embedded raster graphics.

What does that mean?

But wait, there’s ”more”.

Why use SVG?

  • It’s text-based, which means it’s highly searchable. Remember SEO in the heyday of the “Flash website”? Now Flash has all kinds of meta attributes to provide extra SEO oomph, but if we had just used SVG in the first place, we wouldn’t have had this problem.
  • Mobile support: SVG comes in a few flavors that are all interoperable with each other in order to define different levels of support. Whereas Flash is monolithic, SVG comes in profiles that support low-end devices with reasonable quality while still providing high-fidelity content. Just like the Web was designed to do.
  • Easy to learn: there isn’t much new here because it uses everything you already know about XML, CSS, and JavaScript to provide every feature it has. That’s right, you can use plain old HTML for your page content, add some CSS to style it and the SVG, and then JavaScript to script it. “The trifecta” just added another dimension.
  • And, if you care about such things, SVG has been an open standard well before Flash was.

Browser support

  • Opera has full support and has had some amount of support since Opera 8.
  • Gecko has had increasing support since 2005.
  • WebKit has had increasing support since 2006.
  • Internet Explorer (still) has no native support. We hate you, Internet Explorer. (That said, however, Microsoft has just recently joined the W3C’s SVG Working group, so there may yet be some hope.)

What can we do about IE today?

Here is the breakdown of options we have to deal with IE’s frustrating lack of support.

  • Rely on HTTP content negotiation to serve SVG to supporting browsers, and raster images to Internet Explorer. Works great, except now we’re still using raster images, so basically it feels like 1995 again. It’s also what Wikipedia does.
  • Use plugins. Adobe has had one forever called the Adobe SVG Viewer. It’s pretty good…but it’s a plugin. I’m not sure how Flash got away with also being a plugin, but whatever.
  • Relent and use Flash on the client-side, but revel in delicious SVG pureness for our source files: Using SVG Path Data in Flash.
  • Use a cross-browser API. Turns out, there are quite a few:
    • SVG in The Dojo Toolkit – Early and solid, but it uses Dojo (and we don’t).
    • svgweb – Google’s experimental library. It’s got the biggest hyperight now.
    • RaphaëlJS, written by my acquaintance Dmitry Baranovskiy and, of course, my favorite cross-browser SVG toolkit. Since it’s stand-alone, you can actually mix-and-match with your favorite JavaScript framework. In other words, you take your pick of jQuery, YUI, Prototype, or whatever, and layer RaphaëlJS on top for all your SVG-related work. Now that’s creamy.

How do SVG-writing JS frameworks work?

  1. Provides a cross-browser JavaScript API (think jQuery) for writing SVG commands.
  2. Writes to the DOM based on your Raphaël commands:
    • Writes SVG to supporting browsers.
    • Writes VML1 to Internet Explorer.
  3. Makes you coffee.

Hype it up

1 Minute Intro to (Google’s) SVG Web + Demos + rock and roll (because why not?)

Sources and additional resources

  1. VML is Microsoft’s proprietary, non-standard Vector Markup Language. It’s basically the exact same thing as SVG, but only works in Internet Explorer. Because whereas Apple only thinks differently, Microsoft has to do it differently. We hate you, Microsoft. []

Crosspost: My impressions on the new “sex-positive social network” Blackbox Republic

This post was originally published on my other blog, a much more Not Safe For Work site, at maybemaimed.com. However, it turns out that blog is censored in various countries, such as Dubai. Gotta love Internet censorship. Sigh. Anyways, since I think the material there is interesting and technology-relevant, and in order to help people avoid Internet censorship, I’m cross-posting the contents here. Enjoy.


Social media. Internet publishing. Privacy. Three phrases that have seemed to be at tenacious odds with each other in a multitude of subtle and not-so-subtle ways. For people like me, who have progressive views about sexuality, these three things are constantly on our minds. How do we participate in the online revolution without being forced to “come out” about every sex act we enjoy, some of which are still illegal thanks to draconian restrictions on sexual freedom, even (and especially?) in America.

This month, a new social network called Blackbox Republic (BBR) is attempting to tackle this head-on and aims to create a place for, as Marshall Kirkpatrick put it, this particular large and unserved group of people. Although BBR is clearly a business, it’s a business whose creators have laudable intentions for positive social and cultural change. In that respect, and in many others, Blackbox Republic is worth a close look.

I was informed about the venture via Clarisse Thorn many months ago. I got in touch with BBR and signed up for a limited-offer “founder” account—basically a private beta. The founder account gave me free access to the features of the BlackboxRepublic.com website for what would normally be a $25 monthly subscription fee.

So, without further ado, here are my impressions about Blackbox Republic, and how its launch may be just what the Internet needs to get us moving in the right direction with regards to personal privacy, and mainstream awareness of the different needs of different people on the Internet.

Mainstream sex-positivity or a VIP room in cyberspace? Or both?

Over the past few months, Blackbox Republic has been building a marketing arsenal of anticipation and intrigue. Its creators are successful in non-sexuality-focused spheres of influence: Sam Lawrence is the respected former Chief Marketing Officer of Jive Software, Inc., and April Donato, has experience in community management. They also both jive (pun!) well with the sex-positive movement, discussing it at length in the early stages of their marketing efforts after de-cloaking the new company.

In an interview for Social Networking Watch, Sam Lawrence said,

[Sam Lawrence:] The co-founder [April Donato] and myself are part of [the sex-positive] community. Sex positive means that your sexuality is not an issue. You don’t have an issue with other people’s sexuality. You’re open to what other people are interested in and what their boundaries are, and you’re open with your own.

[…]

[Interviewer:] To what extent do you practice a sex-positive lifestyle?

[Sam Lawrence:] From the perspective of sex not being an issue, I think that love is generated by people being open enough about who they are as people to put all of themselves out on the table. As far as putting all of myself on the table, it’s something that I do every single day.

I have an enormous amount of respect for anyone able to so capably present themselves as authentically as Sam does. On the eve of KinkForAll New York City 2, I met Sam and April at one of their “founder meetups” and had the chance to talk to them face-to-face. Our conversation revolved around the importance of steadfastly holding true to one’s own desires and having appropriate places to express those things with appropriate communication tools. I really liked their emphasis on self-identification over labeling throughout our discussion.

I also really appreciated the way that Sam and April spoke about their target audience. Blackbox Republic will welcome everyone, but it’s not designed for everyone, and I think that’s a good thing. David Evans writing at Online Dating Post says,

BBR has room for everyone, but is not for everyone. Definitely catering to non-mainstream folks, it will soon feature a constellation of micro-communities, or groups, called Camps. BBR doesn’t tell people how to organize their camps; we’ll do it ourselves, thankyouverymuch.

So is Blackbox Republic a dating site, or a social network? Well, both, kind of. Part of BBR’s slogan includes, “Dates will happen. Sex will happen. It matters how you get there.” The implication, of course, being that the current suite of tools for finding love or play online—sites like Alt.com, OkCupid, and countless personals boards—focus too strongly on the end result, turning matchmaking into a meat market instead of the natural process of getting to know one another. The focus BBR is placing on each person’s “journey” is an extremely welcome paradigm shift in the online dating world.

Along with the welcome and (IMHO, painfully obviously better) new approach to online dating, however, Blackbox Republic faces some real challenges. For new users, the service costs a minimum of $5 a month to use (and $9 per month for new sign-ups starting in 2010), which gives access to basic features like a personal profile. For $25 a month, members get added features like the ability to list real-world meet-ups, send private messages, and partake in a virtual “gifting” economy (think LiveJournal’s “virtual gifts“).

For that reason, BBR has been called a “members-only club.” There are some legitimate differences of opinion as to whether this is a positive or a negative thing. In a press release over the summer, Blackbox Republic is reported as stating:

Blackbox Republic will be a members-only experience that will unite the sex-positive community and give them a personal, private and secure way to connect online and in person.

Writing for ZDNet, Oliver Marks likens Blackbox Republic’s approach to online dating to the fashionability of owning an Apple computer:

Think of Blackbox Republic as a fashionable online ‘members-only’ club where you might expect to meet people with similar interests to your own, and ideally the person of your dreams. […] Blackbox Republic is arguably an Apple product to Facebook’s Windows look & feel: a much more intimately crafted, fuller featured personal user interface which should appeal to Apple generation sensibilities.

Many pages on Blackbox Republic's website showcase fashionably dressed women.
Many pages on Blackbox Republic's website showcase fashionably dressed women.

Indeed, almost everything about Blackbox Republic’s marketing and design seems to me as though it’s positioning itself as the equivalent of the hip, new, and exclusive nightclub down the street. There are images of super-chic women in short skirts and tight pants all over the Blackbox Republic promotional pages—way more than there are pictures of men. I was (yet again) put-off by this over-prevalence of women in all advertising material.

This isn’t really a criticism of the site, but rather a statement of disappointment that the marketing gurus behind the effort seemed to me to have succumbed to overwhelming cultural pressure to sell their site with old-school sex appeal: women’s sex appeal, of course. How…traditional.

Not only is the Blackbox Republic intro video markedly gender-skewed, but somewhere along the line Sam and April decided to drop the “sex-positive” phraseology from their marketing:

[L]ike most startups, Blackbox decided it needed to change up. Observers were confused by the sex-positive label.

Oh well. I think this just goes to further showcase how much more social change we really need in our culture.

However, while the clubby, cliquey feel is totally my own subjective perception, there are other issues at play here, too. Most notably, as Clarisse Thorn and many others rightfully remind us very often, the sex-positive movement is overwhelmingly white, middle- to upper-class, college-educated, and privileged in a huge number of ways that many people often take for granted. Even without a for-pay social network, not everyone who wants to can participate in the great-sex-for-everyone party atmosphere of many sex-positive niches.

Will creating a “members-only club” of sex-positivity on the Internet really be a positive thing for “the movement”? Well, maybe. Although it has the potential to exclude lower-income people from the experience, who are sadly also often the people with the most pressing need for the kinds of privacy-related tools BBR offers (school teachers spring to mind!), one upside is that Blacbox Republic promises to pledge a portion of membership dues to a charity of the user’s choice.

It’s $25 a month and $5 of those community dues go to charity. One way to think about it is if you’re sex-positive, you can either spend money on expensive coffee every month or upgrade your social life and meet other sex-positive people like you.

Inescapably, the major selling point of any social network is, of course, the network! If your friends aren’t on Twitter, then you’re probably not going to find it useful. The same truth holds for Blackbox Republic: if the users you want to interact with aren’t there, I doubt you’re going to find the experience fruitful. Due to the membership fees and the socioeconomic realities of the sex-positive community, I’m concerned that BBR’s current business model is too exclusive, and as a result it will have a lot of trouble attracting the kind of diverse community its creators seem to be hoping for.

Yet, some others think differently (pun!). For instance, Dennis Howlett welcomes the for-pay model for a social network:

anyone can join provided they’re willing to pay the $25 a month (I like that he has a pay model from the get go. That sorts out the weirdos and hangers on from day one)

I wonder if adopting a free-mium approach might work better. Still, there are real-world limits to business. Everyone needs to make money, and I don’t think Blackbox Republic’s business model is inherently more exclusive than, say, purchasing access to porn. If anything, BBR’s got some real promise to inject much-needed financial awareness to the sexually insensitive corporate infrastructure of our society. Nevertheless, convincing people to join “the Republic” is going to be a hard sell.

Show me the features!

Let’s say you do decide to join. What do you get? Other than the sex-positive mindset, what’s the benefit?

Well, the bulk of the experience is what you’d expect. Profiles (called “personas”), messaging, user search capabilities (called “explore”), and so forth. A Twitter-like “activity stream” dominates the main page where you can post text, picture, or video status updates. Event listings fill the sidebar. (I’m not going to provide internal screenshots in deference to BBR’s strict confidentiality rules.)

While that’s fun, it’s nothing special. What makes Blackbox Republic different is flexibility, and privacy.

Goodbye drop-downs, hello sliders!

An innovative new interface acknowledges (most of) the diversity in human sexual experience and desire.
An innovative new interface acknowledges (most of) the diversity in human sexual experience and desire.

Blackbox Republic’s most visible feature is the way its interface allows you to flexibly self-identify various facets of yourself. Rather than give you static drop-down menus or radio buttons for things like your sexual orientation and relationship status, you’re presented with sliders you can change at will. Perhaps you’re feeling particularly same-sex attracted one day. Just move the “Orientation” slider towards the “Gay” end and away from the “Hetero” end. If that changes tomorrow, just move the slider back. Sho-weet!

BBR offers you 5 different sliders for your profile. In addition to the one for sexual orientation, you also get one for relationship “status” (ranging from attached to unattached, with Facebook’s famous “it’s complicated” neatly in the middle), whether you’re available for more partners or not, how comfortable you are with casual sexual activity, and how eagerly you’re looking to par-tay. I’m instantly reminded of FetLife‘s innovative, if dull-looking, mechanism for specifying multiple relationships. Blackbox Republic gives you similar flexibility as FetLife does but presented in a superb and far more intuitive interface.

All that said, one slider is conspicuously missing: the one for gender. The sliders are a very interesting idea and might just be the most innovative feature of the entire site. It speaks volumes about the sensitive and thoughtful mindset of the developers, and that’s why I’m so disappointed that the interface for self-identifying gender is relegated to the Sex 1.0 days of a single, binary option of “male” or “female.”

What gives? Are polyamorous people more welcome here than those who don’t fit the gender binary? I hope this is simply an omission that will be fixed as the service matures, since I couldn’t find any other reason why gender was absent from the sliders. For extra credit, I hope to see different profile options for “Sex” and “Gender,” two distinct concepts that frequently and incorrectly get used interchangeably. This would make it possible to represent complex gender presentations like additive gender on a social networking interface for the first time ever, and that’d totally be something to write home about!

Privacy and security

The other major selling point of Blackbox Republic is its careful attention to privacy. The entire offering, including its name, is predicated on letting users very carefully segment their information based on their privacy boundaries. I love some of the things BBR has done to enable this, and I can only imagine it’s going to get better from here.

Blackbox Republic’s Web of Trust

There are three levels of privacy, which (as far as I can figure out) map directly to the level of trust other members have gained within the Republic’s community. It works like a web of trust. New users are “un-vouched.” As they begin to interact with others on the site and, hopefully, make some friends, they should receive “vouches”—or votes of trust—from previously-vouched members. As a member, you get to control whether something you do, such as posting a status update, gets sent to the “public,” (i.e., the entire public-facing Internet), to all Blackbox Republic members (i.e, to both vouched and un-vouched members) or only to vouched members.

Additionally, privacy settings allow you to specify whether you want to allow un-vouched members to send you private messages, to follow your updates, to comment on your posts, or to see you in search results.

Unlike Facebook, which has very good privacy controls that almost nobody on Earth is aware of (thus negating the control’s usefulness), Blackbox Republic makes it a point to highlight their privacy controls at just about every sensical turn. Each of the settings I found defaults to the most private setting, not the most public, which is exactly the right move. I gotta say, I found turning off privacy settings instead of having to turn (or leave) them on to be a really empowering feeling.

You’re not a “friend,” you’re an acquaintance!

Moreover, the Blackbox Republic platform makes a native distinction between “friends” (again, like Facebook, or FetLife) and “followers” (like Twitter). When I friend someone, I’m connected to them in a way that I’m not if I just follow someone. I’m not yet certain what the practical distinction between “friending” and “following” are, other than the fact that your view of the people you’re connected with is segmented based on which button you clicked, but I think the distinction is a very appropriate and natural one to embed in the software.

This separation is probably the single most important innovation in the space of social networks as a medium of communication and collaboration that I can point at. I love that I can indicate without ambiguity which people I want to remain in constant communication with and which I simply want to watch from a distance. After all, aren’t at least some of your “friends” on Facebook really just “acquaintances” in reality? I think that for the first time ever in a social network, Blackbox Republic gets this feature right. Now, if only I could figure out what it actually does. :)

What? No on-the-wire encryption?!

With all that being said, there’s still at least one really frightening problem with Blacbox Republic’s careful attention to privacy: as far as I could tell, no part of my session is SSL/TLS encrypted!

Stunningly, for a site that sells privacy, not even Blackbox Republic's login form is on a secure page.
Stunningly, for a site that sells privacy, not even Blackbox Republic's login form is on a secure page.

The entire BlackboxRepublic.com website is served over HTTP, including the login form and—again, as far as I could tell—every page on the inside of the site. This means that it’s trivial for malicious people who don’t even have a Blackbox Republic subscription to intercept, eavesdrop, and modify my interaction with the site. They could watch—and save—private messages between me and one of my friends (or lovers!), for instance.

In Blackbox’s defense, I don’t know of any social network that protects you from this. FetLife is another example of a website that should seriously consider HTTPS-only pages, but as of this writing hasn’t implemented it. Therein lies one of the most frightening oversights in the entire social networking space: regardless of so-called privacy settings, everything you do on the vast majority of social networks, blogs, and other sites on the Internet are the equivalent of passing notes between friends in a classroom. Better hope that big bully who likes to steal your lunch money doesn’t open the note and read it himself while he’s passing along your login details!

The thing is, few other social networking sites place so strong a spotlight on user privacy and security. Since Blackbox Republic seems to be nobly and rightfully holding itself up to a new standard of privacy, I feel justified in pointing out this glaring omission in their service offering. Given everything else they’ve done so well, and how well-aligned the majority of their technical implementation seems to be with their philosophy, this omission came as a big surprise to me.

Until Blackbox Republic only serves HTTPS traffic for all private areas of their site, I can’t make a recommendation in good conscious that it’s the place to be for privacy-conscious people. But again, despite public opinion to the contrary, I’ve never been able to make that claim for FetLife either.

Conclusion

Blackbox Republic is one of the most interesting websites on the Internet today. Its privacy-conscious and sexually open approach to social networking and online dating deserves huge praise. Its technical implementation—although plagued with some glaring oversights for now—is to be seriously respected.

From a social change perspective, I think the site is a mixed bag. Its exclusivity arguably makes the insularity of the sexuality communities an even bigger problem than it already is. On the other hand, the market-value of that very same exclusivity, if steered toward a benevolent purpose, can end up benefiting philanthropic, non-profit, and other sex-positive endeavors that often struggle to find necessary financial support.

Moreover, Blackbox Republic’s internal gifting economy does seem to encourage a sort of altruistic nature among members. How that may or may not translate into increased support for non-commercial activists has yet to be seen. Nay-sayers should remember that this kind of thing simply hasn’t been done before and the net effect could be quite positive.

Having just launched, however, I don’t think Blackbox Republic should be touted as the go-to site for sex-positive people quite yet. Like other social networks, it needs to grow to become truly useful, and its subscription fee business model poses a serious obstacle to many people. I was fortunate to get in with a free “founder” account, but I have mixed feelings about encouraging my friends to join me knowing they—or someone nice enough to “gift” a limited-time subscription to them—will have to pay for the service.

Additionally, its focus on being, well, a black box and its commitment to not allow Google or other search engines to index its internal content simply doesn’t resonate that strongly with me.

Lawrence emphasizes that what members say in Blackbox Republic will stay private. There’s no danger of what they post inside becoming part of their “Google resume,” as he puts it. He says he would resist efforts from search engines to index content the way Facebook and Twitter allow. “The value proposition is this is the first private, large social network out there,” Lawrence says.

Put simply, and noting that I’m probably not the majority case here, I rely on my “Google résumé,” to use Sam’s words, to live the life I want. My lukewarm reaction to this isn’t a criticism of the goal, simply an observation that it turns out I’m not in the ideal target market for Blackbox Republic’s value proposition.

In other words, I think I’m “too out” for this site to be immediately useful to me. The fact that FetLife is not readily available to the public Internet is the single biggest reason why I don’t sign on to that site very often, and so I have the same reason not to spend all that much time behind the curtains of Blackbox Republic.

Nevertheless, many other people do. If you’re among the cross-section of the populace who’d like a sociosexual experience online and would also like to effectively outsource your social reputation management, if you will, but you feel that sites like Facebook just aren’t cutting it, then Blackbox Republic is definitely worth checking out.

If you do check it out, or even if you don’t, I’d love to know what you think in the comments. And if you’re definitely sold, consider signing up via my partner link. Full disclosure: signing up that way earns me a small commission. If you’d rather sign up but not give me a commission for the referral, just register from the front page.

Buy Web Development Books from SitePoint’s 5-for-1 Sale and Donate to Bushfire Relief

For those of you who don’t already know, I’ve been a blogger over at SitePoint for a few months now. Today, I’m even happier to be a participant in the SitePoint community because, for a limited time only, SitePoint is offering the sale of the century: buy 5 SitePoint books for the price of 1. Every last cent of the proceeds from the sale of these books will go towards relief efforts for the recent Victorian bushfires that have claimed over 300 lives and are among the worst fire disasters on record.

The books are full-color PDF downloads, and include some really awesome titles. These are precisely the kinds of books you want as PDFs, too, since you can search through them and always keep them with you while you’re coding and looking for inspiration or a reference (even when you’re without Internet access). I couldn’t help but pounce on this deal, and I’m now the proud owner of the following books, which have all received some pretty great reviews:

In just 3.5 hours, SitePoint has managed to raise over $15,000 AUD, according to employee Kevin Yank on Twitter. And that’s just on this side of the world. All my North hemisphere friends were asleep when this was announced, but not to worry. SitePoint’s sale will last until this Friday, so there’s plenty of time to take advantage of it.

Obviously, I think you should do so. Not only are you getting some really quality content and helping disaster victims at the same time, you’re also sending a loud and clear message that companies whose humanity outshines their accounting are the ones you’re going to support. I’m thrilled to see that SitePoint is one of these human companies, and ever more thrilled to be a part of it.