| XML
About
Home
RSS
OPML
XML-RPC
SOAP
|
|
 |
|
Formats for Blog Browsers
Previous topic
|
Next topic
|
|
Formats for Blog Browsers
started 11/25/2002; 5:23:46 AM - last post 5/29/2006; 1:06:20 PM
|
|
Dave Winer - Formats for Blog Browsers 
11/25/2002; 5:23:46 AM (reads: 104382, responses: 31)
|
|
Listen up. I'd like to tell you a story about how I tripped over what may turn out to be a very interesting common feature of weblog software, otherwise known as a "standard." 
The trail began with a new feature in development for Radio. I wanted to add a facility that would automatically back up all your weblog posts and settings/preferences every night at a time of your choosing. At first I thought I'd just use a binary format, I was tired of doing battle on the mail lists over XML-based formats, and just wanted to get the job done. But it bothered me that the data would require another translation step to get it into another application. So I changed my mind and went with an XML format. As I started to code it a lightning bolt hit me. "I bet RSS 2.0 could do this," I said out loud. And now that the code works, the answer is clear. It can.
Why RSS 2.0 is a perfect fit for weblog archives 
There are two reasons why RSS 2.0 works in this application: It's fuller than 0.91 and 0.92. Most of the attributes of a weblog post are already defined in the core of 2.0. There's no need to invent much that's new, because most of it has already been invented. And where the 2.0 spec is incomplete, because it is extensible through namespaces, it's easy to add fields that are not defined in RSS 2.0. This was necessary in writing the code for Radio, and it may well be necessary for other blogging tools. They may define fields that we don't understand. As there is a new namespace for Radio weblog posts, there may be one for Blogger, Movable Type, etc. So, there was no reason to implement a new format. RSS 2.0 is a perfect fit for backing up weblog posts.
The guinea pig site 
So I finished the code, and tested it on Dave's Handsome Radio Blog (or DHRB), a weblog that goes back to Y2K.
It's where I test new features like this one before we release them to Radio users.
This is where the archives are at:
http://radio.weblogs.com/0001015/backups/posts/
One folder for each year. One RSS 2.0 file for each month.
Then another lightning bolt hit me.
The second lightning bolt 
What if someone made a browser that worked with this format?
Let's call them Blog Browsers, apps specially designed for reading weblogs.
And of course I knew just who to ask about that: Brent Simmons.
Brent and I have worked together closely for quite a few years starting in 1996 on the 24 Hours of Democracy project and through early versions of Frontier's content management system, the website framework, mainResponder, Manila, the outliner, Radio, etc etc. Now Brent is working on his own stuff, in exactly the right area for this, on a program called NetNewsWire, that makes it easy to browse RSS content on Apple's Mac OS X. It's a very popular program, deservedly so, it's got the fine Simmons Seal, or the Eye of Brent, or whatever you want to call it. It feels good.
I sent him an email. Hey Brent, look at these files. They're an archive for a weblog. Can you do a browser for it? (I also telegraphed the idea on Scripting News at roughly this time.)
He can, it turns out. Yesterday he sent me a link to a screen shot and a download for people to play with while he's on vacation for the Thanksgiving holiday. It looks beautiful. It's a sweet idea. Everyone who does a wizzy browser for RSS (and there are a few of them) should explore this. Now that I've explained the idea (hopefully) I'll spend the rest of this doc pointing to various resources and explaining how they work.
The gritty details 
Here's a folder that contains all the posts from DHRB.
The format is RSS 2.0, with several Radio-only elements coming from a namespace that's explained here.
The files validate. Here's an example.
If you want to find out all the months for a weblog with urls to their archive, check out files.xml. There's no guarantee at this early date that the format of this file won't change. But for now, if you want to get started implementing, this format will do. If you have suggestions or comments, post them in the discussion group here. Brent is on vacation this week, so don't expect much help from him. I may have to go to NY later in the week. Such is life at the end of November in the US.
Here's a screen shot of Brent's app. It's so pretty!
Brent's app 
Download it here. Mac OS X only.
Caveats from Brent. "It's just a proof-of-concept at this stage. Lots of UI glitches etc. It has been tested with the archive for Dave's Handsome Radio Blog only (no other weblog archives)."
How to use it. "It comes with the URL of the Dave's Handsome Radio Blog archive already filled in. Click the Open button. It downloads the files, then populates the outline. It displays HTML when you change the outline selection."
Radio code 
It's not ready for testing yet. Soooooon. 
Comments? 
Professional and respectful comments are requested here.
Warning: We delete comments that are personal and/or paranoid. If you want to flame us, pick any one of a number of Syndication mail lists. Many of them seem to accept flames about UserLand.

|
|
Jeremiah Rogers - Re: Formats for Blog Browsers 
11/25/2002; 11:28:50 AM (reads: 33482, responses: 0)
|
|
|
Wondering if the files.xml follows some standardized Userland file scheme, or if it's off the cuff. I recall seeing something like it elsewhere but I want to verify.
Also, if someone were to use existing RDF technologies or develop an new RDF-based system for this (files.xml), would Userland be able to support it? If I recall correctly Radio supports RSS 1.0, but I'm not sure.
I've been hoping to work on something to tie all sorts of RSS feeds together, and recently had been thinking that past data (not just the most recent stuff) should be archived in RSS as well as HTML. I wasn't sure if anyone was already doing this. Looks like you are.

|
|
Jeremiah Rogers - Re: Formats for Blog Browsers 
11/25/2002; 11:40:45 AM (reads: 33077, responses: 0)
|
|
|
Second idea Dave (before the school admin catches me for using the network improperly). Perhaps we could merge this with instant outlining software to create one single system? I think that's when it would really get interesting.
-Jeremiah

|
|
Mark J. Gardner - The third lightning bolt 
11/25/2002; 11:55:33 AM (reads: 33368, responses: 0)
|
|
|
|
Dethe Elza - Manila support? 
11/25/2002; 1:57:23 PM (reads: 33057, responses: 1)
|
|
|
Hi Dave. Will this be supported in Manila at some point?
--Dethe

|
|
Phillip Pearson - Re: Formats for Blog Browsers 
11/25/2002; 2:06:20 PM (reads: 32236, responses: 11)
|
|
|
|
Mark J. Gardner - Re: Formats for Blog Browsers 
11/25/2002; 2:11:20 PM (reads: 34647, responses: 6)
|
|
|
Oh wow, that's flattering. "MJG's renderer."
Truth is, it's just XSLT with a few lines of PHP glue code.
I was wondering what kind of file format to use for my archive file list. Is files.xml worthwhile for this purpose, and/or is there other prior art I should look at?

|
|
Dave Winer - Re: Manila support? 
11/25/2002; 2:11:39 PM (reads: 33825, responses: 0)
|
|
|
I don't know. It's still just the beginning of a bootstrap. If it catches on my guess is that yes of course it will be supported by Manila.

|
|
Dave Winer - Re: Formats for Blog Browsers 
11/25/2002; 2:13:46 PM (reads: 35702, responses: 5)
|
|
|
I was wondering what kind of file format to use for my archive file list. Is files.xml worthwhile for this purpose, and/or is there other prior art I should look at?
Mark, I just went for the most obvious flat format. The only prior art I could find was changes.xml for weblogs.com. We had directory.opml as part of upstreaming, and there's still some support for it in Radio Community Server, but that was one dimension too complex for this app. It's a hiearchy, I didn't feel this one needed to be hierarchic.

|
|
Mark J. Gardner - Re: Formats for Blog Browsers 
11/25/2002; 2:26:44 PM (reads: 37841, responses: 4)
|
|
|
I'd like my setup to work with others' data as well, so if files.xml is something that's going to see further adoption, I'll just switch from my homegrown format to that.
I wonder if there's any way for an RSS file to refer to its files.xml file -- that way, given an RSS feed, a blog browser could find the files.xml, and from there find all other archived posts from that feed.
Sounds like another namespace add-on for RSS 2.0.

|
|
Phillip Pearson - Re: Formats for Blog Browsers 
11/25/2002; 2:27:48 PM (reads: 40355, responses: 0)
|
|
|
Something to go in blogChannel, perhaps?

|
|
Jeremiah Rogers - Re: Formats for Blog Browsers 
11/25/2002; 2:54:24 PM (reads: 31992, responses: 3)
|
|
|
Philip -
Your file is not working with Brent's software and neither are either of mine (http://www.kingprimate.com/weblog/files.xml or http://www.kingprimate.com/weblog/files2.xml). Both of them now show a year date, with a drop-down outline menu, and then a bunch of blank outline entries (they are selectable but not viewable). I've been modifying them trying to get them to work and here's my hyptothesis:
I think Brent did a quick code job on his thing and assumed that the blogs would always be under posts/ and then the year.
Looking forward to his return from vacation.

|
|
Dave Winer - Re: Formats for Blog Browsers 
11/25/2002; 3:00:26 PM (reads: 39494, responses: 0)
|
|
|
I wonder if there's any way for an RSS file to refer to its files.xml file -- that way, given an RSS feed, a blog browser could find the files.xml, and from there find all other archived posts from that feed.
Good idea. I'll give that some thought.

|
|
Phillip Pearson - Re: Formats for Blog Browsers 
11/25/2002; 3:31:14 PM (reads: 34431, responses: 2)
|
|
|
Interesting. I've put them in the same path ... maybe I messed something else up.
Update: I'm guessing that the problem is that I wasn't putting pubDate elements in the RSS. Can you try it again now?

|
|
Bruce Loebrich - Why not use RSS for files.xml? 
11/25/2002; 4:52:40 PM (reads: 30323, responses: 2)
|
|
|
Since a blog browser would only need to understand one format instead of two, it seems like there would be an advantage to using RSS for the files.xml file as well. RSS has all the elements necessary to duplicate the functionality contained in the existing format used for files.xml.
Jeremiah Rogers mentions somehow merging RSS archives with outlining. Nested RSS files could give an outline like structure to archives by having different levels for the achive as a whole, its years, months, days or even posts. I've been playing with a primitive version of this concept to create OPML archives of my website.
Mark J. Gardner's third lighting bolt is right on target. With modern browsers, there is no need for standalone HTML pages if an XSLT stylesheet declaration is included in each archive file.

|
|
Dave Winer - Re: Why not use RSS for files.xml? 
11/25/2002; 5:01:46 PM (reads: 32144, responses: 0)
|
|
|
Sometimes creeping elegance gets the better of oneself. I liked files.xml for its simplicity. I've never regretted that changes.xml was a flat ultra-simple format.

|
|
Chris Brody - Re: Formats for Blog Browsers 
11/25/2002; 5:44:57 PM (reads: 29568, responses: 2)
|
|
|
I was tired of doing battle on the mail lists over XML-based formats
Don't give up, you're both right! I'm no expert but I believe that RDF has a LOT to offer in future - don't write it off just because it's "too complex" for the relatively simple applications you're proposing.
Will RSS 3.0 be the answer? Proprietary formats have their place, but not in creating potential building blocks for the web. I don't see hordes of RSS 0.x & 2.x supporters arguing the ground for you where it matters, Dave. Perhaps there are some non-partisan folks at w3c to arbitrate?
For those interested in what RDF has to offer there's a primer at W3C and a good chapter in Wrox Press' Professional XML.

|
|
Jeremiah Rogers - Re: Formats for Blog Browsers 
11/25/2002; 6:52:42 PM (reads: 36899, responses: 1)
|
|
|
Nope. I thought it was the same error and it still didn't work for me either. Getting blank lists still. Argh!!!

|
|
Mark Gardner - Re: Why not use RSS for files.xml? 
11/25/2002; 7:38:06 PM (reads: 32099, responses: 0)
|
|
|
I don't think it's a big problem to have a different format for what seems like a different purpose. And if you want to reduce the complexity on the client via XSLT, just make the stylesheet handle both formats. That's what I did with RSS and my homegrown archive filelist format -- it just keys into the root element.

|
|
Timothy Appnel - Re: Formats for Blog Browsers 
11/25/2002; 7:53:31 PM (reads: 39496, responses: 1)
|
|
|
No need to create an RSS module. The annotate module has a single reference tag that was designed for such purposes.

|
|
Joe Friend - Re: Great export/import format 
11/25/2002; 8:06:24 PM (reads: 30973, responses: 4)
|
|
|
Seems like this would be perfect as an exchange format for moving your content from one blogging software to another. No?

|
|
Mark Gardner - mod_annotation vs. <comments> 
11/25/2002; 8:31:00 PM (reads: 42100, responses: 0)
|
|
|
That seems to encompass the functionality of the RSS 2.0 <comments> element -- it's a bit less specific, though.

|
|
Dave Winer - Re: Great export/import format 
11/25/2002; 11:34:25 PM (reads: 32181, responses: 3)
|
|
|
Yes yes yes.

|
|
Joe Friend - Re: Great export/import format 
11/25/2002; 11:47:38 PM (reads: 34791, responses: 2)
|
|
|
Dave,
Good developers (like you) realize that people sometimes have to leave their products (like Radio) behind for other products, needless to say it is still surprising to see you excited about the idea of using Formats for Blog Browsers as a transitional format.
I have a love/hate relationship with Radio. It isn't always as stable as I want. Also, I'm developing some other projects in pMachine, so I'd like to focus developing my knowledge of the pMachine backend. Hence moving from Radio to pMachine for my personal blog. I can't wait for this feature because it will make moving my content to pMachines much easier. Now I just need to find someone to write the PHP to import the entries into pMachine.
All that said, I'll still be actively watching the progress of Radio! Keep up the good work Dave!
Joe

|
|
Dave Winer - Re: Great export/import format 
11/26/2002; 4:57:24 AM (reads: 37138, responses: 1)
|
|
|
It goes the other way too Joe.
Sometimes people want to move their data into Radio. 

|
|
Phillip Pearson - Re: Formats for Blog Browsers 
11/27/2002; 2:16:07 PM (reads: 39139, responses: 0)
|
|
|
Odd. It's working for Robert Barksdale. Can you try pycs.net/devlog/backups/files.xml?

|
|
Joe Friend - Re: Great export/import format 
11/28/2002; 7:26:50 AM (reads: 39289, responses: 0)
|
|
|
Of course, didn't mean to infere otherwise.
BTW, can I have a bite of that cheese cake?

|
|
Chris Brody - 
11/29/2002; 12:07:57 AM (reads: 31667, responses: 0)
|
|
|
|
Chris Carlin - Mozilla 
11/29/2002; 8:43:49 PM (reads: 30316, responses: 0)
|
|
|
I have no idea what this forum is focused on, I just saw the post above and thought you might be interested...
Mozilla (I believe) includes native RSS support of some sort. Check out Forumzilla to see an old Mozilla app that was meant to read RSS feeds. Unfortunately the developer's house burned down and he stopped development for a [long] while. The client only works in ancient versions of Mozilla.
He says he's going to continue work on it soon, though...

|
|
Mark J. Gardner - Yahoo group 
12/18/2002; 1:05:03 PM (reads: 32889, responses: 0)
|
|
|
I've started a Yahoo group so folks can discuss this without having to jump around different individual weblogs. Feel free to stop by.

|
|
Tim - Re: Formats for Blog Browsers 
5/29/2006; 1:06:20 PM (reads: 27594, responses: 0)
|
|
|
For those of you that can have a blogspot blog (blogger) or a Movable Type, or Typepad blog, then there is a great tool out there which will back up 'everything'.
Just grab yourself a blog at http://wordpress.com/ and their "import" tool will perfectly grab anything from Movable Type, Typepad, or Blogger. It even imports comments from a Blogger accounts! After scouring the Internet for a solution over the last three days I 'finally' found it and it works! I'm so pumped :-)
... Tim

|
|
|
|