Netvibes Developers blog

(See bottom of the post for an update)

Netvibes was created as a way to gather all your digital life in one handy place, providing in-house for your most important digital needs: search, e-mail, to-do list, feed-reader, bookmarks & links, weather, calendar…

It quickly turned out that our users needed access to more digital content than we where able to provide widgets for. Hence the launch of the MiniAPI in March 2006, merely 6 months after our initial launch, allowing developers to build Netvibes modules using well-known technologies, such as XHTML, CSS and JavaScript/Ajax. The Universal Widget API (UWA), launched one year later, was the next step in the evolution of this API, offering portability for the major widget platforms.

But that might still not be enough. For starters, there are still a lot of useful and frequently-update data out there that is not available through a feed or an API – and thus, hardly easy to implement into a widget, even through an iFrame. For these data, users are pretty much left to themselves, or to the cleverness of widget developers.

The Automatic Parser widget is one of these extremely clever widgets. Built by Netvibes user Tristan “Twisterss” Groléat, it features a state-of-the-art HTML parser that allows users to pull just about anything out of any web-page, as long as it can be found using the DOM.

The instructions are sparse, and the usage is quick to grok:

1. Enter the URL of the webpage
2. Select the information you are interested in on the preview of the page: one block or many blocks of the same type
3. Check that the widget really selects what you want
4. Confirm your selection
5. The widget displays the information you want, and refreshs it automatically every 20 minutes.

Let’s say you want a widget that always displays the latest “Hawt post” from the WordPress.com home page. Just paste the http://wordpress.com/ URL into the widget:

It loads the page – removing a few styling rules for readability. Scroll to the section you are interested in, and put the cursor over it: it highlights with a red background. Click when the want data is highlighted: the highlighting turns to blue.

Click the “Parse the information” button. If only what you wanted is selected, then click the “Validate” button.

There you go: you know have a full-working UWA widget which displays a regularly-updated portion of an external website. Since the widget takes the external content as-is, you might need to resize your Netvibes page’s column, so as to display it all. You can also change the widget’s title in the preferences.

Get inspired! What kind of widget do you see being built with this nifty new tool? What can you learn from its code?

UPDATE

Tristan has done it again with a new tool based on his Automatic Parser widget: “Build your own parsing UWA widget in 3 minutes“. Simply put, it starts with the Automatic Parser widget, and adds tools to customize and share it easily with the community. Excellent thinking!

Tags: automatic, parser, uwa

This entry was posted on Friday, February 22nd, 2008 at 7:13 pm and is filed under Articles, Showcase. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.