Using screen scraping to expose legacy web pages in RSS

As part of my (almost) daily drive to and from one of my clients I pass through the sub-sea Oslofjord tunnel (Oslofjordtunnelen). Now what has driving got to do with screen scraping and RSS, you say?

Hang on, I’m getting there.

Below is a map extract that shows part of my route. The topmost pin is where I start out, the bottom-most pin is the Oslofjord tunnel. The pin in the middle is where, more often than not, a sign shows up stating that the tunnel is closed for maintenance.  You can imagine my frustration when I’m forced to drive all the way back north to get around the Oslofjord!

Map picture

To avoid that pain I set out to find a feed with traffic status updates and ended up at this page published by the Norwegian Public Roads Administration (NPRA). The page have regularly updated traffic information (all in Norwegian mind you) but to my frustration all as static web pages. No feed in sight!

At last here comes the screen scraping into play. You could write up your own scraper in any modern runtime these days. But being a good/lazy developer I know there are already quite good services out there that makes it a breeze setting up feeds with data scraped off of web pages. And lo and behold, I now burn a feed with the latest traffic updates!

So… basking in the glory of my genius for a couple of days I thought it a good idea to write up this blog post for the greater good of mankind. To make the story a bit shorter, what I discovered while rummaging around the NPRA site is that they indeed have great support for RSS!

Feeling a bit stupid I will now go and redirect my FeedBurner setup…and please let me know if there is a moral to this story.

Good night.


Magnificent game music live!

I am a regular metal head and perhaps because of that I also enjoy classic music. Since I am also a gamer I had to go to Play! A Video Game Symphony in Lillestrøm, Norway last Friday. It was truly an amazing experience to hear some of my favorite game themes performed live by the Norwegian Radio Orchestra and Oslo Philharmonic Choir.

Memories of old came back when pictures from The Last Ninja filled the screens accompanied with the dramatic theme. And the themes from The Elder Scrolls: Morrowind and Oblivion beckoned me back to the lands of Tamriel.

I must also mention the drummer Rony Barrak who totally owned the show with his amazing skills at the Darabouka. With only one drum and a couple of cymbals he performed a drum solo like I never heard before. Respect, man!

Update: Someone filmed the Final Fantasy theme. Watch it at YouTube.




The Abyss and the Bridge

Yesterday I attended a technical conference where the speaker talked about BizTalk 2006. I had brought along a couple of friends of mine, both keen to learn the new and cool stuff but lacking a multi-year IT industry background.

So I got kinda embarrassed when the speaker started talking about ESB and SOA and totally failed to mention what these acronyms stood for. Ok, so most of the audience probably knew what he was talking about but can you really assume that? How often do we put distance between us and “the others” (who more than often is saluted Dear Customer) by throwing “orchestrations” and “service busses” at ’em.

I tell you, I can feel myself at the edge of the abyss when I try to explain my latest project to my wife… God help me learn building bridges