News feed aggregation

The folks at Humanized had a nice idea that seems a perfect use of AJAX. They proposed that a user interface could get rid of the page chunking links you see at the bottom of search results lists and instead, dynamically add more entries to the bottom of the page as the window is scrolled. To prove the concept, they've implemented this idea in a news feed aggregator. They call this automatic expansion of the list of entries "Humanized History".

Scrolling in GoogleReader is essentially the same. There are two panels, one on the left to scroll the entries titles and one on the right for viewing the individual entry selected on the right. Humanized Reader does it all in a more typical blog-format, with a simple reverse chron list. Google Reader is more like a conventional application UI, but has the same page-less model.

The concept is very powerful. Using technology appropriately, let the interface get out of the way when possible so that people get right to using the content. Humanized does this with a design that is simpler and more elegant than the Google Reader. They say the point is to let you spend more time reading and less time thinking about navigation. Put another way, they say, "Don't force the user to ask for more content: just give it to them." Sounds like something Steve Krug would agree with. I like it.

google finance

Google's new Finance site is really quite elegant. The site offers information on North American stocks, mutual funds and public and private companies along with charts, news and fundamental financial data. Different things to watch for here are interactive charts, and the blog and discussion group retrieval. Most of the other tear sheet type information, e.g. news, company profile (description), and finances you'll find on all of the other finance sites as well.

The line/spark line chart scrolling is cool. it automatically scrolls to the news for the period you are browsing in the chart. You can also change the range of dates in the chart by resizing the year widget -- mouse over the years at the top of the chart and a little resizing widget appears. When you drag and resize the date range, the main line graph shrinks or expands to show better detail on that range and the news box on the right refreshes to show only the items in that date range. Very nice, clean and simple use of AJAX.

Don Norman recently attempted a simplicity backlash after a few articles touted Google's simple UI as one of the reasons for it's success. Most of these simplicity articles talk about the spareness of its search interface as opposed to Yahoo's, for instance. Finance people are also saying that Google is not presenting a clear enough strategy and that their tools are all over the place. I might agree with that. They have a lot of applications that never seem to make it out of Beta.

Norman says that the simplicty factor breaks down when you try to do anything outside of searching web corpus. His argument is valid. If you view Google as a suite of tools for retrieving information, there is often a disconnection between the bodies of indexed data. The problem is rooted partly in poor information architecture problem and partly in poor interaction design. Norman is saying, I think, that the site doesn't yet allow the integration of the pieces into one UI, and rather segments it by application (and dare I say, by working group within Google?).

But when they do rich applications like Google Maps and this new Finance site, they DO do it rather simply and elegantly. (The Google News Reader on the other hand, ugh! That thing needs to take a lesson from these Beta apps.) With Maps and Finance their focus and execution on the functionality of simple little interaction widgets, e.g. moving a Google map around with a cursor, changing a data set range with a scrolling widget, is what sets them apart. In the end, our discerete interaction with specific tools is what is simple, and it's why I continue to use them over other sites. I don't care if their products are siloed and perhaps require poking around in the labs or clicking tabs to find them. When I get there, there is very little menu cruft in the way and it lets me get the job done quickly and efficiently.

Suprglu is super cool. You enter your account names for web services you use (e.g. flickr, del.icio.us, digg, etc.) and RSS feeds for your blogs and Suprglu displays it all in one place. Nice and simple, and you get to modify the style sheet as much as you like. My Suprglu.

[via swissmiss]

Aarrr, matey. Here be a tale of a blogging practice that makes ye look like a bilge rat pirate

As a rule, you should periodically check referrer logs. Usually it's good practice because you find out who's linking to your work. But once in a while you'll also find a site that's either copying your content outright without permission or that's embedding links to your media (images, MP3's, etc.) in their site and essentially pirating your bandwidth. This morning I found a site that was embedding links to my images in their page. Avast! The image on the right shows a bit of their page and how I'm replacing images (See the "Revenge" section below to see how this works).

My site publishes complete entries in its RSS feed. Because of that, other people's web-based aggregators are able to republish my content in its entirety. In the best case, a blogger uses a web-based aggregator to watch feeds and post the ones they like, excerpting the entry. In the worst case, they republish your entire entry without attribution. I don't know what this site owner was doing, but I noticed that their blog was basically aggregating other people's posts. There doesn't seem to be any original content. But in my case, they didn't excerpt, they re-copied my entire blog entry verbatim. What pisses me off is it looks like they wrote the article.

I suppose it's partly my fault for putting full entries in my RSS feed rather than excerpts, but I do this so that people can read my blog in their aggregators without having to actually go my site. This is the downside, I suppose. Web-based aggregators will republish whatever they get.

I take me revenge

To play with them a little, I now replace images referenced from another site with a STOP image. I hate to have to do this, because it messes up the images for legitimate aggregators. I suppose you could be really malicious and post a hardcore porn picture in there instead to make thinkgs look even worse. I'm not that malicious.

You can do the same thing if you find that someone is pirating your media. Using altlab's examples for dealing with bandwidth theft, I modified the .htaccess file on myserver to include these lines.

RewriteEngine On
RewriteCond %{HTTP_REFERER}
    !^http://(www\.)?urlgreyhot\.com/ [NC]
RewriteCond %{HTTP_REFERER} !^$
RewriteRule \.(jpe?g|gif|bmp|png)$ img/pirate.png [L]

To use this code on your site, replace the second line with your domain and modify the fourth line to use the path to your stop image.

I thought about this a few minutes. Because I don't want to do this to everyone, I can use the code below as a method to block from that domain only:

RewriteCond %{HTTP_REFERER}
    ^http://(www\.)?badsite\.net/ [NC]
RewriteCond %{HTTP_REFERER} !^$
RewriteRule \.(jpe?g|gif|bmp|png)$ img/pirate.png [L]

To use this code on your site, replace the domain first line with the domain of the bad site and the change the third line to use the path to your stop image. Take that, ye scurvy lubber!

Yo ho! Here be Mr. Krabs bit of advice to ye
This is why you should always look at your referrers! Be smart about RSS aggregation and blogging. If you are going to use an RSS aggregator to feed your blog, be sure to excerpt and ALWAYS link to the original article and attribute the author.

Update
Moments after doing this, they must have seen the replaced image, so they removed the copied entry from their site.

FeedWordPress is an Atom/RSS aggregator for WordPress. It syndicates content from newsfeeds that you select into your WordPress blog; if you syndicate several newsfeeds then you can use WordPress’s posts database and templating engine as the back-end of an aggregation website.

CNet's preview release of Newsburst is quite good. It combines some of the better features of Bloglines and Kinja. It's not perfect yet, however. For one thing, the tabs don't maintain a selected state when you click on them. Also, the "Stream" view would be much better if it allowed you to browse backwards chronologically through the spool of the entries that have been collected. Kinja does that extremely well and has been my news reader of choice because of that one feature. It follows how one type of behavior -- watching news feeds as they arrive. My other common behavior is to read a particular site in my news reader. Bloglines does that well with their multi-framed interface. My client reader, NetNewsWire does an even better job. Kinja doesn't let me do that at all for some reason.

Newsburst seems to address these multiple news reading behaviors, providing an interface that serves all. But does it do it well? Yes and no. I like the features, but the simplicity and speed of Kinja still win me over. Newsburst just feels a little bloated and sluggish right now. Maybe they'll evolve it some more and I will come back and love it. I guess I'm still looking for the perfect reader. For now it looks like I'm still going to pogo between Kinja and NetNewswire.

Update. And the best thing about NewsBurst is that I sent them feedback and I was almost immediately contacted by someone high up in their product development team. They're already working on the issues I mention above. This will be an application to watch.

Server side RSS and Atom news feed aggregator that requires PHP/MySQL.

Cool RSS/RDF/Atom aggregator.

PubSub is an alerting system that can monitor various sources including weblogs, newsgroups and Edgar/SEC filings. You create a search and alerts are delivered as RSS.

I used to use Drupal's built in RSS aggregator on iaslash a lot. I used the "blog it" links to move stuff from the aggregator into the blog. But for my own news watching I've been relying on NetNewsWire Lite on the Mac and recently on SharpReader on the PC. SharpReader has a nice threaded feature that I wish NNW had. I remembered that the paid version of NNW came with a built in weblog editor and note taking tool, so I decided to have a look at it again.

I installed the demo version and tried the built in weblog editor. It uses XML RPC to post entries -- make sure you have the blogapi enabled. You can view recent entries and post new entries to your Drupal site from this desktop application. This is not new, but I haven't tried using it with a Drupal site. NNW makes dashing off quick notes on urlgrehot very easy. The one down side, however, is that the categories function doesn't seem to work with my Drupal site. I don't know why that is, but it would be nice if someone helped NNW figure that one out. (Update: According to Boris Mann, this works in 4.4RC) One thought I had was to create a different user and post entries from NNW using that user login so I could separate blog entries into those formally entered via the web site and those entered from NNW. Since you can't see and apply categories for entries you have to edit via the web site after the fact. Separating the blogs entries might make my process of post-entry categorization go a little more smoothly. I could also change my home page module to show the real blog entries in one column, and the thought wanders (from NNW) in a separate column. Not sure if this will work for me yet.

Another con is that rather than entering your title in a form, you have to wrap your title in a title tag. Not a biggie, but am sure that can be fixed. (Update: Tried using the Other/MetaWeblogAPI after the suggestion was made by Boris and Brent from Rachero -- the Internet is cool like that -- and Title field works.)

Have a look below to see how this works:

This is nice for dashing off thoughts. Adding taxonomy terms later is maybe not such a big deal, but this investigation is driven by my need to keep track of thoughts in an easy to use tool. The real thing that is drawing me to NNW is the built in Notepad, which is an outliner that lets you drag headlines over from the aggregator. This works for me as a quick and easy way to capture the headlines I want to look at later, categorized by topic. When I read the full post, I can enter a few notes in the Notepad, then perhaps blog it in the Weblog Editor when I have some fully formed ideas to write about. This might be ideal. I know it takes my link capturing away from the link log on urlgreyhot, but I'm not sure that's a bad idea.

Another thought I had was that NNW could add threading to show which entries referred to each other. SharpReader for Windows does this very nicely, which is why I use that on my work PC. [See this screenshot of SharpReaders' threading in action.] Then I could drag an entire thread to my notepad for later reading and note taking. What would be really fun is if I could then take a bunch of headlines and their attached threads and drag it into OmniGraffle to play at showing relationships between posts. This is the kind of thing Anacubis would excel at. If their tool allowed you to view a set of RSS headlines and relate them (by the reffering URLs in the entries) then you could visualize the connections and perhaps arrange them by dimensions such as date of entry. Would be an interesting experiment probably to get something approaching HP Labs' Blog Epidemic Analyzer.

Anyway, current scenario I have to work with seems serviceable for now. One thing that would be really nice, and probably not hard to achieve, is if this could be better integrated with OmniOutliner. The advantage of using OO is that it allows you to use multiple columns in your outline (sort of like a spreadsheet) and you can enter lengthy notes per outline entry. Right now I can drag headlines to OO and it creates a new item with only the URL of the headline. This is not ideal because it again forces me to take the extra step of typing. Using the NNW notepad lets me organize and arrange the recent items I want to look at, then view and read them later. But at some point, I will want to annotate what I view. Perhaps what it needs is an export function that will work with OO. Or, I suppose, I could try to automate some of this with an AppleScript -- I dread having to learn another scripting language, though. (Update: Brent from Ranchero tells me that the notepad is very well documented and suggested I make a feature request of OmniGroup for the drag and drop functionality. I'm going to play at it a bit more before I do, but I think that's a great idea.)

Upon writing and thinking about this, I think a well-integrated NNW and OO environment would be a wonderful way to do personal information management. I don't see why this can't work better for me than a wiki. Well, except that wikis still have the advantage of being wickedly flexible and easy to use if you give it enough time. OK. I will have to ruminate on this for a while.

Later that day...

Upon further investigation I find that the notepad is actually an OPML file saved in ~/Library/Application Support/NetNewsWire/Notepad/Notepad.opml. When you
import the OPML file into OmniOutliner, it gives you a multi-column formmated outline, retaining the folders (main outline branches) you created in NNW. Very cool. There are separate fields for the feed title, site URL, RSS item URL, RSS item description, rss item title, and notes. Not sure how to enter notes in NNW, though. The output looks like this:

This is really excellent. XML makes integration with OO a no-brain activity. Just import and go. Synchronization would be hard if you modified the OO file, though. What would be really excellent is if you could modify the OPML file via OO and the changes got reflected live in NNW. Also nice would be the ability to create multiple notepad (OPML) files in NNW. All in all I'm really pleased with what you can do without much effort.

[P.S. this entry was created in NNW, and post-categorized and post-edited in Drupal.]