Everything In Between

The brutally honest, first-person account of Meitar Moscovitz's life.

Archive for the ‘Information & Communication’ Category

XML.com Managing Editor Kurt Cagle sees the future, one that I’ve experienced a decade ago

leave a comment

I subscribe to a number of really great technology newsletters because they interest me. One of these is the XML.com weekly newsletter. XML is a technology that has exploded in the last several years, and its specifically an area that I, as a front-end and semantic web specialist, find exceptionally intriguing. Most intriguing today, however, was Managing Editor of XML.com Kurt Cagle’s article titled Is Telecommuting the Next Wave for Education? in the XML.com newsletter.

Not only was it a great article that highlights a particular XML schema for education-related material produced by the Schools Interoperability Framework Association, it paints the picture of a future I’ve already lived ten years ago. I am heartened to see that, at long last, other people are beginning to see the benefits of technology used for “distance learning.” Frankly, I can’t imagine learning any other way.

I tried to find this article online so I could link to it, but I could not. Instead, I’m reposting the article in full here. If this is available somewhere on the ‘net that I just haven’t found yet, please let me know.

Is Telecommuting the Next Wave for Education? by Kurt Cagle, Managing Editor,xml.com.

In the great analysis game, I have two particularly adept spies – my daughters. Over the years I’ve noticed that both of my daughters tend to be remarkably good barometers about the way that the wind is blowing with regard to youth trends, which in turn tend to be significant because teenagers in particular often tend to be the earliest adopters of new technologies. If something doesn’t resonate with them, no matter how big a marketing budget, it usually doesn’t fly.

One of the things I’ve noticed with my eldest daughter in particular is how many of her friends either are or have been recently “home-schooled”. Now, for many, home-schooling has long been associated with religious organizations, particularly evangelical Christians, who feel that the school system is too secular for raising their kids. However, given that the kids I know (and their parents) are generally not in that community, I was a little puzzled with what was going on.

Instead, what seems to be happening is a phenomenon that I think will have major ramifications for society, and certainly for the tech community. As the Internet was taking off around 2000, a lot of school districts began implementing a program to help those people who were often at significant distances from a school by offering certain classes online – with exercises online, video conferencing and periodic tests. At first, these classes were ones that you would expect to make the migration – science and math courses – but over time, they have extended to cover everything except those classes that require group participation – band, or choir, for example – or need physical facilities, such as wood-working. Physical education requirements could generally be met by agreeing to some form of monitored activity – swimming classes at a recreational center, for instance.

Yet a funny thing happened while setting up these distance learning programs. While remote users became enthusiastic participants in this new wave, the largest group of users have been urban or suburban kids who, for one reason or another, didn’t fit well within the school paradigm. In some cases, the people who took advantage of these courses were students who were involved in focused activities that involved travel – talented musicians who were often on the road, dancers, athletes who were often involved in activities at different schools or other events, or those whose family travelled frequently, who took to the distance courses because they were able to learn better around their other activities. In some cases, the students were people who were going through emotional issues at home – a death or divorce, for instance, though increasingly that has made its way up into those students who just couldn’t face the high-pressure world of middle-school or high-school dynamics.

In other words, the kids and teenagers who were taking advantage of these courses just recognized that it gave them an opportunity to learn in the same way that they are increasingly interacting with the rest of the world – through the computer. Teenagers are hard wired to be more alert in the evening, and typically to be sluggish in the morning, yet school as it is set up right now forces them to be capable of handling complex math and science first thing in the day, when they are generally least reponsive to learning much of anything. Then they are forced to take home dozens of pages of homework that will force them to stay up until late in the evening anyway, meaning that by the time they reach the end of the week, they are physically and mentally exhausted.

Distance Learning lets them combine the homework with the schoolwork, so they can practice new concepts when they’re presented, not after the concepts have become hazy after a full day. It moves them away from the tyranny of the timetable so that if they are having trouble learning a concept, they can spend the time they need to master it, rather than stopping abruptly halfway through because they have to move onto their next class. It also means that if they have mastered a concept, they do not have to sit around bored while others are still trying to figure out something.

It also lets them have access to the rest of the Internet to use as a research tool. While traditional academics may shudder at this notion, as a recent controversy at one university showcased when a student was expelled for setting up a study group on Facebook, the reality is that we’re moving past the point at which we need to keep a vast storehouse of information locked up in within our brains. In an era of information ubiquity, many of the skills that are taught in schools are beginning to seem increasingly quaint, and the teachers that are effective are typically the ones that have managed to incorporate this info-sphere in their own teachings. One social studies teacher of my acquaintance in particular has become quite effective at teaching using PowerPoint, Wikipedia, and other multimedia resources, and he goes out of his way to teach children not the history itself (which they can generally look up) but how to research and analyze that history and take from it any lessons that a given period may have to offer.

Distance Learning programs are also becoming more popular for the same reason that telecommuting is becoming popular – school districts are facing increasing prices for gas and food as a typical family is, but multiplied by several thousand. Many school districts are responding to this by cutting down on the routes that their school buses follow (or in some cases eliminating bus systems entirely) forcing parents to take their kids themselves, often, ironically, increasing the total gas use dramatically and certainly causing headaches for parents who have to integrate their kids into their own commuting schedules. Similarly, school cafeteria programs are being scaled back or eliminated entirely because the cost of the food is becoming painfully high. Add into that aging infrastructures for schools, in a time when the population itself is aging (and hence less likely to fund school initiatives if it affects property taxes), and what you have is a recipe for disintegrating school systems.

Given that, the idea of distance learning as an alternative is one that may be popular at the school board level, as in general, you can generally buy a whole lot of educational training and assistance for what it normally costs to move and feed kids. While it does require some retraining on the part of the teachers, they’re also attracted to it because their job frequently involves trying to keep order over thirty to as many as forty kids, many of whom simply do not want to be there – and the same time-shifting that occurs for the students typically occurs for them as well. Indeed, in many cases what ends up happening is that the teachers pre-record certain segments of their lectures (and are increasingly posting them up on YouTube), then set up one-on-one or group chat sessions with the students.

As for the parents in this equation, I’ve often suspected that one of the reasons for the decline in the quality of contemporary education has been the fact that school has become the place where parents warehouse their kids for the day. Home-schooling does require more parental involvement, but in general it is also far easier for those parents to keep a fairly close eye on what their kids are up to in this day and age when the parents are not at home, at least for kids of a certain age. What’s more, schools are in many ways like hospitals – while their ostensible purpose is education, the chance of their kids learning bad habits is far higher in school settings than they generally are at home – and the kids that are trying to be good aren’t distracted by the ones determined to make trouble.

That’s not to say that there aren’t distractions. My eldest daughter has been known to keep a chat window up talking to friends when she should be concentrating on homework, though in fairness to her, what she’s chatting about often involves that same homework, albeit in a rather disconnected fashion:

Kat: I'm :-( ):-\. Zuko shippeded Kataara. Urgh ..
Neechan: Urrgghhhhh.
Hey, what's I if volts is 10 and ohms = 3 % (can't write the squiggle for
ohms)?
Kat: Um V =IR so, uhm, I is RV ...
Neechan: No, R over V.
Kat: Oh. Right. :-) .... so 3 divided by 10 - .3. Bad Zuko! ...

Is this bad? No – it’s only different from the way that those of us who grew up pre-Internet see the world. Distance Learning does not change the amount of homework (though I suspect that it cuts down pretty dramatically on the makework that tends to deaden interest in a subject rather than reinforcing the concepts), nor does it change the need for accountability. The kids still need to test, and still need to show that they have learned, but I suspect that their retention rates will likely be considerably higher if they can learn in a way that works well for them.

Long term, I think that this will likely end up deconstructing the traditional school system, though this is a process that will take decades to happen completely. It means, for the aggressive learners, that they could in fact complete a formal curriculum in a fraction of the time necessary, though a good teacher can work with those particular students to provide additional areas of study for them to engage with. It means that slower students can learn at their own pace, and can generally be flagged for additional help if they fall sufficiently behind. Money that school districts save in terms of providing physical infrastructure in transportation can be spent on those activities that do promote socialization – music, art, theatre, sports, civic days, and so forth – and monthly activities that bring students together can keep the bond in place of school as community.

Distributed education is part of the larger process of social redistribution that is occurring because of the Internet. Modern education emerged about the same time that the modern corporation emerged – in the 1930s – during an era when the dominant forces at play involved hierarchies, centralization, consolidation and economies of scale. In this decade, the dominant forces are network related – decentralization, the economies of global localities, the disintermediation of authority and the a shift away from the geographical. Just as these forces are resisted at the corporate level despite the obvious benefits (and just as workers in places that can telecommute are increasingly doing so) , these same forces are resisted at the educational level with much the same results – students (and their parents) are taking advantage of any loophole they can to make it available to their kids because it results in a better education for them).

So far, most educational software is boutiqueware, typically Flash-like applications. This won’t necessarily change moving forward (it is hard in general to make educational software that doesn’t have a strong “games-like” component, and for the most part that game approach makes the educational software far more engaging than it would be otherwise), but one thing that will need to happen as the distributed systems move forward is for the emergence of some kind of general framework for the exchange of educational related information. One particularly promising start in this direction is the development of the Schools Interoperability Framework (SIF) (http://www.sifinfo.org/sif-specification.asp), which is an XML standard most recently updated in March 2008. Its mandate is fairly broad – providing XML schemas for describing grade-books, library services, student information, instructional services and so forth – (see Figure 1. SIF Zone Services).Figure 1. SIF Zone Services.

One of the jobs that the XML community needs to do is to reach out to the developers of educational software and insure that they are aware of the SIF standards, in order to provide better interoperability between their core applications and the growing educational educational noosphere, and to reach out to educators and education IT departments (which are all too typically the math teacher in his spare time) to make them cognizant of these same standards and to help implement solutions around these standards. As more students opt to go “virtual” the ability to maintain consistent, and more important interoperable, records becomes ever more paramount.

Long term, the move towards distributed education will shape society in some very profound ways. The kids going to school in such an environment today are more adept at the art of self-education, are usually more capable at analysis and research, and because they managed to avoid the often harsh emotional trauma of dealing with several hundred other kids of the same age daily, usually have more self-confidence than their school-based peers. They’ll likely have little patience for the Tayloresque approach to college education and will continue their lives in a similar manner by educating themselves within the college’s online environment (and will tend to shun those colleges that don’t offer such services) and when they start coming into the workforce in sufficient numbers, they will reshape the way that organizations are set up. On the flip side, I think this is likely to cause a huge amount of cultural friction between this generation and those educated more traditionally in previous generations, because their respective realities will be very different.

So far, the movement is still just a trickle, but watch this space closely – it will become a flood soon enough – and the bricks in the wall will come tumbling down (to paraphrase Pink Floyd).

Written by Meitar

May 3rd, 2008 at 1:24 pm

Deleting GMail messages from an iPhone or iPod touch won’t “Archive”

7 comments

I just paid way too much money for an iPod touch (because I’m a technoslut), and in playing around with the Mail application it comes with ever since Macworld and the “free Software Upgrade”, I noticed a bit of a gotcha while using GMail IMAP accounts.

It seems, that despite syncing up with my iMac’s accounts, the iPod touch (and, presumably, the iPhone as well), treat the “Trash” mailbox specially. Specifically, whereas you can map the “Delete” button in Apple Mail.app to function as though it were the GMail “Archive” button (by following these simple steps), Mail on the iPod touch or iPhone won’t inherit the same behavior. “Delete” on the iPod touch really is an honest-to-goodness delete—forever.

Instead, to Archive GMail messages on an iPod touch or iPhone, you’ll need to tap the “Move” button (the folder icon with a downwards-facing arrow, second from the left on the bottom toolbar) and select the “All Mail” folder. I wonder why the difference in interface choices….

Written by Meitar

February 13th, 2008 at 10:41 pm

Steven Pinker’s ‘The Stuff of Thought’

one comment

This video, which is one of the recent TED Talk videos, is of Steven Pinker’s talk called The Stuff of Thought. This is simply brilliant. So brilliant, in fact, that those who know me well are about to be utterly astounded by what I am going to say:

I now understand the value of indirect communication. And it is immense.

I also understand why I never saw it before: the benefits are reaped solely through language’s social applications, not its analytical ones. See for yourself by watching the video.

An incredible interview with this Harvard professor is available on Google Video.

Written by Meitar

December 28th, 2007 at 4:21 am

Wikipedia showcases the value of simple

one comment

Simplicity is a challenging goal for virtually every task you (or I) may have. Why is it a goal at all? Successfully reducing the presentation of complicated tasks into simple components is a goal because it is typically a required part for the success of the task.

Possibly the best example of this phenomenon in action is Wikipedia, which hosts several different versions of its pages. The version everyone knows about is the supremely academic one, the one Wikipedia presents by default. Here’s an excerpt of one such page’s introduction, the Wikipedia entry for the Standard Model of particle physics.

The Standard Model of particle physics is a theory that describes three of the four known fundamental interactions between the elementary particles that make up all matter. It is a quantum field theory developed between 1970 and 1973 which is consistent with both quantum mechanics and special relativity. To date, almost all experimental tests of the three forces described by the Standard Model have agreed with its predictions. However, the Standard Model falls short of being a complete theory of fundamental interactions, primarily because of its lack of inclusion of gravity, the fourth known fundamental interaction, but also because of the large number of numerical parameters (such as masses and coupling constants) that must be put “by hand” into the theory (rather than being derived from first principles).

Contrast the above with this excerpt for the same page, the Standard Model of particle physics, taken from the simple English version of Wikipedia:

The Standard Model of physics is the best idea to say how fundamental forces and elementary particles work. It uses quantum mechanics and special relativity. In physics there are many different particles and forces, the Standard Model says that all particles and forces are only two different types: fermions and bosons.

Okay, now that’s a lot easier to understand. In this example, the simple English version is a lot shorter, and at first glance that might strike you as its major distinguishing factor. However, if you read closer, you’ll notice many things specific to the language that was used that serve to give the simple English version much more accessibility than the academic one. Some of these things include:

  • Simpler, more familiar vocabulary. Instead of using surgically-precise words that may not be familiar to an uninformed reader, plainer words (and no less accuracy) are used to describe concepts.
  • Dense sentences are broken up into smaller chunks. When accessibility or successful communication is the primary concern, longer sentences that deliver more information in one punch may be counter-productive. Instead, it’s often better to chop up larger concepts and deliver them in smaller-sized chunks that are easier to digest.
  • Specifics are introduced one at a time, and defined at each instance. Possibly the most common error writers (especially technical writers) make is introducing lots of interdependent ideas at once or without proper prior context. Rather than work your way from a complicated idea to a simple conclusion, work instead from a simple foundation to a complicated idea, building vocabulary as you go (see point the first about vocabulary).

Of course, this is always easier said than done, and it is also why simplicity is intuitively understood by lots of people to be a hard thing to create. Presenting things simply is a challenge because it requires more knowledge than simply understanding the thing; it requires understanding the thing and understanding what pieces of the thing your audience does not (yet) understand. The value of simple lies in being able to fill those gaps.

Written by Meitar

November 16th, 2007 at 2:33 pm

The 10 Geekiest Leopard Features I Will Probably Love

one comment

This is already horribly old news, and by old I mean several days ago since that’s about as fast as it takes technology news to grow old, but Apple is releasing Mac OS X 10.5 “Leopard” at the end of this month. Apple is calling this release a “major upgrade,” and indeed Apple has rarely made its users wait so long between operating system releases as they have done between Tiger (Mac OS X 10.4) and Leopard. So, I’m already excited.

But then today I was glossing over Apple’s featured features list and I got even more excited. There are the usual, largely meaningless, fluff updates that are nice for Joe Schmo or his mother, but that power users simply don’t care about, like the new iChat support for animated buddy icons, but the list is also chock-full of really cool, really useful features.

What’s interesting is that a good deal of these features aren’t really new features at all. For instance, if you knew how to manipulate the NetInfo database on your Mac, you could already share any folder via Apple’s “Personal File Sharing” feature. (Here’s a Mac OS X Hints hint explaining how to do it.) In Leopard, however, Apple claims that this functionality is now integrated straight into a folder’s Get Info… window. If it works as smoothly as Apple claims, this is finally going to bring Mac OS X (client) into decent competition with Windows XP Professional in terms of GUI-level power-user features.

However, while all of these features are really cool, here’s a list of the ten geekiest features I will probably absolutely love, for one reason or another.

  • Ruby on Rails, out of the boxThe hot thing in web development right now is Ruby on Rails. Macs have already been the best personal desktop and web development platform because they have built-in support for the Apache web server and a host of other features, but now they will come with a ready-to-roll installation of Ruby on Rails, sporting Mongrel and (better yet) Capistrano! Specifically with the addition of Capistrano, which is terribly undersold as simply a Ruby on Rails deployment platform, these UNIX-y “toolbox” items are bound to make Macs that much more useful right out of the box.
  • Safari’s full history search — As their recent public partnerships with Google have shown, Apple is very clearly invested in search technologies. Spotlight gets a huge number of improvements in Leopard, but none which I think are going to be more useful to more people than this one: spotlight searches on the full text of each web page in your visited history list. That’s just awesome. Also awesome: using spotlight as a calculator and as a dictionary, which also shows just how Google-like Apple is trying to be. (Google also lets you ask it arithmetic questions and a dictionary.)
  • Wikipedia articles in Dictionary.app — I love Wikipedia because it’s one of the fastest ways to get (relatively) reliable information quickly. Now that Dictionary.app has built-in integration with Wikipedia, imagine the possibilities for getting that knowledge instant-gratification craving fixed. Apple has not yet announced this capability, but I can easily envision a scenario where all Cocoa text fields are instantly “wikified” (with text that matches Wikipedia articles highlighted) much in the same way that current Cocoa text fields allow you to right-click on a misspelled word and have it corrected by Dictionary.app.
  • Application-based firewall — In classic Apple fashion, functionality that was previously available via third-party additions is now available from Apple itself. In this case, I have to wonder how well Apple’s updates to its firewall will obviate the need for Little Snitch, which is basically an application-based firewall, too, and a good one at that.
  • Built-in guest log-in account — If you’re as paranoid about security as I am, you’ve already created a special, limited-access user on your system (called Guest or Visitor or whatever) and whenever friends are over, you tell them to use that account instead of your own. Now in Leopard, Apple has gone through the trouble of setting this up for us already. A small change that is going to have a big impact.
  • Scriptable System Preferences & applications — With AppleScript, you can automate the things your computer does with scripts, as long as those things are “scriptable.” In previous versions of Mac OS X, huge gaping holes of what things shipped by Apple were scriptable existed, causing me (personally) some really annoying headaches. AppleScript GUI scripting helped me get around many of those roadblocks, but now it seems Apple is finally filling in some of the most notorious gaps in this functionality with scriptable System Preferences. Yay!
  • Automator workflow variables — Automator brings the power of AppleScript I just mentioned to more people with a completely graphic programming environment. There is no need to open up a text document and write AppleScript code because Automator lets you create a script (called a Workflow in Automator jargon) using your mouse by dragging and dropping actions into the order you want them to be performed. It’s very slick, but until now it’s been very limited. With Leopard, Apple is beefing up Automator so that it includes things like variables, basic programmatic capability that was sorely lacking before. (Also majorly cool: a command-line utility to access Automator!)
  • Finder.app’s path bar — Every serious Mac user knows that the Finder needs a lot of help. Now, it’s getting some. Something the Windows Explorer has had forever (as had every desktop environment for Linux, of course) is a visual cue to show you where in your filesystem tree a given folder is located when you are viewing said folder. Now the Finder gains this capability (though Apple’s description implies that it’s going to be off by default) with what Apple is calling a “Path Bar”. Finally!
  • Cocoa and scripting bridges — Even though no one really seems to know about it, it has long been possible for languages other than AppleScript to do things like send Apple Events to Mac OS X applications. Specifically, Ruby and JavaScript, two of the most well-known web development languages in existence, can already do this with a single ScriptingAddition (OSAX). But now Apple is making this functionality a central feature and fully extending it to their Objective-C (and Cocoa) language and applications such as Xcode and Interface Builder. This means people like me will have a shallower learning curve before we’re able to create full-fledged, native Mac OS X applications. Now that’s exciting!
  • Xcode 3 refactoring — This is something you kind of have to see to believe. I got the opportunity to see it demoed at Apple’s Leopard Tech Talks last year and I was really excited by it. With the new Xcode, Apple’s development IDE, you can do away with find-and-replace searches for things like renaming functions because Xcode understands what parts of your code are what structures and, when you tell it to “change the function named myFunction to myNewFunction,” it’ll only find-and-replace function names instead of every instance of the string “myFunction.” That’s pretty big, and if it were available for more languages, it’s almost enough to make me ditch vim.

So there you have it. Ten features you might not have already known about that are some of the most promising features I can see in Leopard. And I didn’t even get into Wide-Area Bonjour, which could make services like DynDNS or No-IP a thing of the past (and which I still want to learn more about), or the new Terminal application (finally with tabs!), or even the multiple user certificates for S/MIME encrypted email.

Note: One of the least known security features available on Mac OS X is also possibly one of the best, and the simplest. Evidently, all Intel-based Macs are shipped with the XD (aka. NX, aka. DEP) bit turned on—and thankfully there doesn’t seem to be any way for users to turn it off. However, this isn’t a silver bullet and if you want to learn why you should check out this excellent Anandtech article: A Bit About the NX Bit.

The Map Of The Internet, Circa 2007

leave a comment

This map of the Internet is pretty neat. It even gives you a “you are here” signpost when you first arrive.

More information on the map is available from the ISI ANT Census home page.

Written by Meitar

October 9th, 2007 at 5:23 pm

Doing a lot with simple tools—Google Docs as a wiki and web site

leave a comment

A friend and acquaintence of mine wants to build a web site. He wants something simple up quickly, and he needs it to be easy for a large team of non-technical people to update fairly quickly. Impressively, he’s come up with a very novel solution: use a collection of publicly-viewable Google Docs to emulate the feel of a “wiki”-like web site.

This is actually quite a clever and novel use of Google Docs that had never occured to me before. Furthermore, it’s a wonderful example of how flexible tools that impose very little in the way of their own use can allow people to add arbitrary meaning to the things they create and, thereby, create arbitrarily complex relationships among the things they create. Said more simply, leaving the meaning part of things up to computers is a perfect way to box yourself into a useless use case.

Google Docs contains all the functionality required to build a very simple web site. Google’s own accounts provide authentication for arbitrary users. The documents themselves can be shared and viewed publicly. An editor can embed links, styles, and other rich text, media, and formatting elements into the page. And best of all, any arbitrary selection of users can be invited to edit the page.

Does this sound familiar to you? It should. It’s the fundamental building blocks of a wiki. Sure, it doesn’t have any of wikipedia’s fancy features, but it does have all the basics. And with a little ingenuity, you can end up doing quite a lot even through the use of only very simple tools.

Written by Meitar

October 1st, 2007 at 2:53 pm

Use Google Docs to compose and publish blog posts

leave a comment

While this feature has been around for a long time, not many people are aware that you can publish blog posts straight from Google Docs to just about any blog of your choice, including Blogger, WordPress, and others. This is pretty awesome for many reasons, but most important for many people is the incredible rich text, WYSIWYG editor that Google Docs offers.

I’m tempted to give it a shot myself, for many reasons. I think using Google Docs to compose and edit my blog posts may make it easier to:

  • keep copies of my blog posts somewhere other than my blog (just in case),
  • work on my blog posts from anywhere with an Internet connection (and under the security of an HTTPS connection!),
  • version-control all of my writing without changing my workflow the least bit,
  • invite specific users to preview my drafts before they go live and even help me edit my posts so I can catch typos and grammatical errors,
  • easily export my writings to a variety of formats, such as PDF or a ZIP file with the ease of a single menu,
  • and of course, instantly take advantage of whatever wonderful things Google has in their pipeline.

Another blogger wrote an entire how to article showing tips for using Google Docs to blog that is quite a good read, as well.

Written by Meitar

July 27th, 2007 at 1:09 pm

The Simplest Personal Email Spam Solution EVER!

leave a comment

I have the simplest personal email spam solution in the world. I use Apple’s Address Book and, in it, I keep all the email addresses I ever want to get mail from. In Apple’s Mail program, I simply tell it that email from an address in my address book is exempt from being treated as junk mail. Then I set up a Mail rule that says if the sender is not in my address book, the message should be moved to the Junk Mail folder.

Voila. This system is flawless. You will never be able to send me loads of spam that go anywhere but my spam box, and I hardly ever look in there.

Naturally, there is a caveat to using this technique, but I actually consider it to be an advantage. By necessity, this technique, keeps me pro-active about getting people’s contact information when I meet them (and want to talk again). If I don’t get that person’s email address, I’ll probably never see that person’s email unless I’m looking out for it. Nine times out of ten, however, that’s what I want to have happen anyway.

So this solves the problem of unwanted mail. However, what if I want to let people contact me that I don’t know ahead of time or have previous whitelisted? Well, in that case I rely on an out-of-band communication, such as an introduction from a friend, leaving a comment on my blog(s), or some other method such as an instant message to let me know that there is someone who wants to talk to me.

My contact information is so available (in so many places), and many IM services are now equipped with store-and-forward messaging that there really is no reason for email to be the first time I hear from someone. Even better, if I’m contacted over Google Talk (as an example), I automatically have an email address for that person.

Voila. Simplest. Spam. Filter. Ever.

Written by Meitar

June 25th, 2007 at 5:08 pm

A Better Expect Subversion Post-Commit Hook

10 comments

In a previous post I wrote a small expect script to update a remote web server’s deployed code on a new commit to a Subversion repository using Expect and Subversion’s post-commit hooks. That first script was extraordinarily basic, so I’ve been wanting to add some sanity and error checking to it for a while. I finally got around to it today.

This improved version of the post-commit hook does the same thing as the last one (that is, it logs into your web server over SSH with the given user and password, and yes, I’m aware of the scariness of embedding a password in such a way, so you should really set up SSH to use public keys for authentication for this), except now it also produces useful output.

Here’s the same script as before, but improved:

#!/usr/bin/expect -f

#
# AUTHOR: Meitar Moscovitz 
# DATE  : Thu Jun 21 16:32:42 EDT 2007
#

set HOST my.web.server
set USER someuser
set PASS xxx

# the working copy we're going to update
set WC /path/to/working/copy

# the path to the svn executable on the remote web server
set SVNBIN /usr/local/bin/svn

# our network is slow, set a long timeout
set timeout 30

##### DO NOT EDIT PAST THIS LINE! #####

# POST-COMMIT HOOK
#
# The post-commit hook is invoked after a commit.  Subversion runs
# this hook by invoking a program (script, executable, binary, etc.)
# named 'post-commit' (this file) with the
# following ordered arguments:
#
#   [1] REPOS-PATH   (the path to this repository)
#   [2] REV          (the number of the revision just committed)
#
# Note that Subversion does not provide this program with an environment
# of any kind. That means this program lacks a current working directory,
# a home directory, a $PATH, and so on.

set REPOS [lindex $argv 0]
set REV [lindex $argv 1]

# Define error codes
set E_NO_SSH     1 ;# can't find a usable SSH on our system
set E_NO_CONNECT 2 ;# failure to connect to remote server (timed out)
set E_WRONG_PASS 3 ;# password provided does not work
set E_UNKNOWN   25 ;# unexpected failure

# find the SSH binary on our system
if {[file executable /usr/bin/ssh]} {
	set SSHBIN /usr/bin/ssh
} elseif {[file executable /usr/local/bin/ssh]} {
	set SSHBIN /usr/local/bin/ssh
} else {
	send_error "Can't find a usable SSH on this system.\n"
	exit $E_NO_SSH
}

spawn $SSHBIN $USER@$HOST $SVNBIN update $WC

expect {
    "continue connecting (yes/no)? " { send "yes\r"; exp_continue; }
    -nocase "password:" { send "$PASS\r"; }
    timeout {
        send_error "\nWe have timed out after $timeout seconds while trying to connect to $HOST!\n";
        exit $E_NO_CONNECT;
    }
}

expect {
	-nocase "password:" { ;# if we are asked for the password again, then we have provided the wrong password
		send_error "\nCan not log in to $HOST because the password provided for user $USER has been rejected.\n";
		exit $E_WRONG_PASS;
	}
	-re "revision (\[0-9]+)." {
		if {$REV == $expect_out(1,string)} {
			send_user "\nSuccessfully updated $WC on $HOST to revision $REV.\n"
		} else {
			send_user "\nUpdated repository to revision $expect_out(1,string), but svn reports that we are at revision number $REV.\n"
			send_error "CAUTION: Repository updated to revision $expect_out(1,string), but committed revision $REV.\n"
		}
	}
	default {
		send_error "An unexpected error has occured. The process at spawn ID $spawn_id has produced the following output:\n"
		send_error $expect_out(buffer)
		exit $E_UNKNOWN
	}
}

Written by Meitar

June 21st, 2007 at 11:41 pm