Blog posts tagged "webservices"

Weather over Twitter

January 12th, 2007

Too Close!!!

And while we’re talking about recent hacks, Blaine and I whipped up a Jabber bot using his Jabber::Simple and the Yahoo weather feeds, to provide twice daily weather updates via Twitter.

Jabber is an intriguing platform to build on top of, and the more I play with it the more potential I find. I keep checking in on it every few years (since MetaEvents days), but recently its gotten much more interesting. In part thats Google’s adoption of the standard (and the subsequent enhancement in tools, libraries, and clients), and partially standards bake slowly, but at the core of it I think we’re reaching a point in the evolution of the Web where Internet-scale deployed messaging standards have a lot to offer of us. A protocol for when HTTP fails you.

If you follow these bots, you’ll receive those updates wherever you normally get your Twitters; IM, Phone, RSS, or just on the web. So far, we have bots for the following cities: Boston, Brighton, Chicago, Helsinki, London, Los Angeles, New York, Paris, Portland, San Francisco, Seattle, Singapore, and Vancouver. If you’d like to see another city, just ask and we’ll provide.

Slightly out of date source available at twitter-weather – Google Code

And taking requests for new cities. Probably do a big batch of new ones sometime next week. (not really an automated process)

Photo by bonsaikiptb

Amazon EC2: Still working on the “elastic” part?

August 24th, 2006

I’ve been waiting for an Amazon compute cluster ever since S3 came out, and like Les I tried, and failed, to sign up for EC2 beta as soon as I got the email. What all you freaks were doing up around 5am signing up for webservices I’ll never know.

Nik over at TechCrunch however ran the numbers, and its looking more like what I get from John Companies, and less like the great mapreduce grid in the sky I was hoping for.

Tagged: Uncategorized , , , , , ,

Weather RSS and the Dangers of Screen Scraping

August 24th, 2003

So the Weather RSS service is down right now. The whole thing is driven by screen scraping because the freely available sources of weather info suck (or at least the ones I can find do).

Weather.gov

I have a working screen scraper for Weather.gov but unforunately their URLs are relatively obscure. I have some code to mechanize their search form, but they got a bit jumpy, and temporarily blocked my IP while I was tuning it. So I put them on hold, and went with Wunderground.

Wunderground

Well the Wunderground has US and international weather, so that is a plus, and nicely predictable URLs, which really helps when you’re screen scraping. But they HTML sucks. Its very much circa 1997, but more cluttered. Its not a field I have a lot of expirence with, but I thought my scraper was pretty good, but 2 week later it is broken. Ugh.

Weather.com

Haven’t tried weather.com yet, years ago when I was last playing with weather, they blocked my IP for scraping (and I’m being well behaved I promise!), don’t know if they still do that, my instinct is not, at least not within reason. However their URLs seem totally arbitrary, probably pegged to an internal numbering scheme. If anyone knows differently that would be great. (Also their editorial voice is insipid, the differnece I suppose between people who study weather for a living, and those who sell it for a living, e.g “Sunny” becomes “Plentiful sunshine”.)

More Domain Knowledge and Directories

Looks like I’m going to have to move away form Wunderground, their HTML just isn’t reliable enough. To do that I’ll need to understand the identifiers the NOAA and Weather.com are using. The NOAA identifiers are standard I think, but I havne’t found a good documentation of them. The Weather.com identifiers could probably be fetched by walking their website (or their syndicated Yahoo weather site which has a more directory like structure) The one problem with moving to using the identifiers rather then a search interface is rather then allowing free form entry, one would want to present the users with a deeply nested list of possible choices, which has never worked all that well on the web. I just downloaded WeatherPop (sweet little app btw), who I suspect is also a screen scraper (though I should probalby examine my outgoing traffic before making that claim) to see how it handles this problem, and presenting a drill down list is exactly what it does.

Anyone has other suggestions for data in a workable format, let me know. God knows I hate screen scraping in this day and page (CSS makes it easier, but it feels so backwards)

That and I never quite got conditional gets supported, it was working in Magpie, and at least one of the Windows clients (don’t remember which one) but not in NNW, or most of the other readers.

And last but not least, there is the lovely “Divsion by zero” error I get when trying to view Denver’s weather.

Tagged: Uncategorized , , , ,

Allconsuming Soap

January 22nd, 2003

So I wrote up my own little SOAP client to Allconsuming, which, while not nearly as cool as booktalk, works nicely to maintain my little READING sidebar. (though as you can see in the case of Applying Patterns there are still some aesthetic tweaks to make). Get the script and the template.

By the way, it looks a little different then DJ’s because SOAP::Lite’s autodispatch+ feature breaks Template Toolkit (and is kind of icky anyway)

Tagged: Uncategorized , , , , ,