Entries For: March 2008
March 11, 2008
Andrew Burkhalter: Sightline Daily Recap
A recap of a developer's favorite features in action on the Sightline Daily Plone-powered website.
Yeah, I know that Jon already let the Sightline Daily cat out of the bag and every day brings ever more impressive and interesting Plone sites, but before I move onto the next project, I'll afford myself a few minutes to get nostalgic. Thanks for humoring me. I figure anything that consumes several hundred hours+ and a few months of your life deserves at least a blog post for posterity. Plus, I'm happy with the result.
So, in no particular order, here are several of the noteworthy pieces that make up Sightline Daily:
- I suppose the first thing anyone notices in a site is the graphic design and, *man*, is Sightline Daily tastefully done. It's clean, skimmable, attractive, and presents a lot of metadata about news clippings in a non-overwhelming way. A few times, I found myself mindlessly clicking around and just absorbing the niceness. Sightline Institute managed the design process with Ryan Swarts and it's a great product. My colleague David Glick implemented the CSS to perfection with some initial bootstrapping and work by ONE/Northwest friend Trey Beck.
- By ONE/Northwest standards, there was a rather large team collaborating on this project. Sightline brings ~10 content editors, bloggers, content creators, and project managers to the table. On the implementation side, we teamed with Web Collective (they focused on the blog features and portlets mostly, while I did the daily news headline-related programming), bringing the number of programmers, templaters, specification writers, and CSSers to 8. On top of the implementation work I was responsible for, I had great time working with my fantastic co-worker, Kelley, helping to manage the biggest team I've worked with to date.
- I believe Jon mentioned the bookmarklet for clipping headlines, but this is fantastic. It's the fastest method I've seen for *manually* pushing content into Plone. Leading up to the implementation, I was a bit nervous about this piece, but equipped with a similar snippet produced for Salesforce.com by a co-worker, Steve, some inspection of the del.icio.us bookmarklet, and some Googling about, this was a cinch. You have to see the screencast below to get a sense for how it works. Money quotes from Sightline Daily editor Kristin Kolb were "You're saving me from carpal tunnel" and "I feel like I just received my Christmas present early" (I first demoed this in early December) upon seeing this. Awesome. I couldn't make that up.
- Auto-generation of related items based on a Plone catalog query for 2 relevant news headlines and 1 blog post. The effort to navigate to and hand pick related items was too time-consuming for the editorial process, so we wrote code that takes into account key terms to use for a query of Plone's "SearchableText" index and creates references to the returned results.
A wizard-esque configlet that logically steps through the entire publishing process from choosing the date for the next "edition" of news clippings through to bulk editing all the way to previewing and publishing for site visitors. The entire process can be done outside the context of the normal Plone user interface. The highlight is definitely the bulk edit screen (seen to the right), which allows for starring news headlines as "top picks", editing of the keywords that should be used for auto-relating items, and drag 'n' drop reordering of stories. Kudos to Jon for brainstorming a solid, implementable specification for this that held pretty closely from mockup to implementation.
- It's been said before, but Zope 3 brings us a fantastic event system. The bookmarklet mentioned and screencasted above uses Javascript to get the current browser location and pre-populates the news headline with a permalink. There's no need to slow the clipping down by forcing the editor to manually type or select the source publication as well (i.e. The New York Times or Washington Post). Rather, site users manage a centralized list of publications (new publications are added organically over time) using ATVocabularyManager. Using each object's permalink and Python's urlparse, we simply index the permalink's netloc for fast retrieval. With this, we can compute and index the publication automatically. This is where Zope 3's event system comes into play. Because ATVocabularyManager fires off custom events upon edit and deletion of vocabulary terms, our headline's source publication stays up-to-date with some super simple event handling code. Thanks to the kind soul that baked this capability into ATVocabularyManager, because I wasn't going to port AT-events back from Archetypes 1.5.x.
- Once launched and under normal load, the site turned out to be unexpectedly fast thanks to Cachefu and some minimal configuration effort.
- The site includes a nice bit of tasteful, interactive Javascript, again mostly implemented by my colleague David Glick. One can quickly expand and collapse the headlines and toggle between the days of the week. This makes the site highly skimmable.
- Every morning, Sightline sends out the top 10 or so headlines via email. In the past, one manual effort involved populating the site, while another involved drafting the daily email for a blast out to subscribers. This was a flat out waste of time and when you mix in how temperamental email clients are about HTML in email, this was a drain on Sightline staff. ONE/Northwest often uses an ASP-based email broadcasting service called WhatCounts for the "email newsletter" part of our projects. This service provides an incredible feature called "Smartget" (and it's near identical siblings "cacheget" and "get" -- which I like to call "Dumbget"). After the morning publishing process is finished on the site, Sightline logs into their WhatCounts instance and are greeted with a 1-line email template that goes out and grabs the raw HTML contents of the following page showing the latest news. Prefer plain text email, yeah, we did that too. No double entry at all. My colleague Sam, did a fantastic job of giving me a plain-old HTML template that was cluttered with the most nasty HTML tables and inline styles needed for a consist look in the wild world that is HTML email compatibility. I don't think in HTML 4 and tables, so this was a huge boost.
- Clean urls for news by topic, news by region, and rss by whatever. I have an unreasonable hatred of nasty urls. Try passing the following out over the phone http://www.domain.com/some/path/page?query_param=@value@&query_param_2=value2&template=some_template.htm. That's not even social. Thanks to Plone, we almost never have to worry about this. Before this project, I had never noticed the "subpath" (update: I intended to write "traverse_subpath" here, not "subpath" -- sorry if that threw anyone off.) parameter that's available in Zope's request. Essentially, you can add whatever directory structure onto the end of a skin-based template and it's right there for your consumption as a Python list in the request's subpath. I do a bit of processing in a browser view and instead of things like ?topic=energy, we have the following clean, friendly urls:
- daily.sightline.org/news/topic/energy
- daily.sightline.org/news/region/washington
- daily.sightline.org/rss/topic/energy/region/washington
- In my opinion, Sightline's best content is their in-depth blog entries that are part of a series. They've done great ones on bicycle neglect and the quest to go an entire year carfree. Brian and Justin of the Web Collective put together a basic "series" content type and extended the blog entry, so that in the context of a blog post, the author can assign an entry to an existing series. This isn't rocket science, but it works really well for Sightline's needs and the graphic design for a series object is one of the best templates on the site.
- Tests, tests, and more tests. I insisted before development started that because this application was so critical to the organization, because it was fairly complex with significant code interaction, and because there was a decent sized team of folks that had no clue what other developers were doing, that we needed to embrace a test driven development approach and push as much presentation logic into easily testable Zope 3 views. I even convinced myself that we had budgeted correctly for it ... oops. Test driven development is paradoxically both time consuming and time saving. At the end of it all, I don't regret the decision to adopt this approach one bit. I doubt few do. At the time of this writing we have 399 unit tests and all of them pass but 1, which I left unfinished to remind me of a feature I couldn't get working within the bulk admin interface and therefore disabled entirely. I wanted to finish that, but it was unnecessary and I needed to move onto other scoped items. The fact that we still have entirely passing tests after site launch for the enabled features signifies that even as we approached the looming deadline, we didn't abandon our agreed upon approach.
So, there you have it. I suppose I could write another post or two covering all the lessons learned and the endless details that went into getting this site launched, but for now, I'll leave things at that. It's best to reflect on what worked well in the end anyways. Furthermore, I think Jon still plans to do a case study and possibly a screencast about the editorial process. That will certainly be interesting and take a closer look at the big picture.
March 10, 2008
David Glick: collective.recipe.omelette: for more navigable eggs
Crack 'em open and mix 'em together...
I've been using zc.buildout for a few months now, and am quite happy with it. However, I got tired of navigating to my egg cache, locating the right package (and version) within a long flat list, and then clicking a few more times to actually get to the contents of the package. And I was looking for a project to practice my buildout recipe fu.
So I created collective.recipe.omelette. It's a simple buildout recipe that takes all your namespace-packaged eggs and merges their contents together into a single directory tree, in its own parts directory, with symlinks to the actual egg contents. The result looks much the lib/python of days gone by. So instead of:
egg-cache/ my.egg.one-1.0-py2.4.egg/ my/ egg/ one/ (contents of first egg) my.egg.two-1.0-py2.4.egg/ my/ egg/ two/ (contents of second egg)
you can now make use of:
parts/ omelette/ my/ egg/ one/ (contents of first egg) two/ (contents of second egg)
with obvious advantages in navigability and greppability.
Collective.recipe.omelette is available from the Cheese Shop. You can add an omelette to your buildout by making the following additions to buildout.cfg (assuming you have a part called instance and you want to generate symlinks for its eggs), and then re-running your buildout:
[buildout]
...
parts =
...
omelette
[omelette]
recipe = collective.recipe.omelette
eggs = ${instance:eggs}
One caveat: Since this makes use of symlinks, it of course won't work on a filesystem that doesn't support symlinks, such as Windows'.