None

New Technorati API calls, part I

I've been working three new Technorati API calls.  The first iscalled getinfo.  It tells you things that Technorati knows about a user.  In the simplest case you can use  getinfo to find out information that a blogger wants to make known about himself, along with some information that Technorati has calculated and verified about that person.

http://api.technorati.com/getinfo?username=dsifry (experts, see expert note below)

This will return an XML structure with information about the user.  Here's an example of the XML returned:

<?xml version="1.0" encoding="iso-8859-1"?>
<!-- generator="Technorati API version 0.9 /getinfo" -->
<!DOCTYPE tapi PUBLIC "-//Sifry Consulting//DTD TAPI 0.01//EN" "http://api.technorati.com/dtd/tapi-001.xml">
<tapi version="0.9">
<document>
<result>
<username>dsifry</username>
<thumbnailpicture>http://www.technorati.com/progimages/photo.jpg?uid=1&amp;mood=default</thumbnailpicture>
<item>
  <weblog>
    <name>Sifry's Alerts</name>
    <url>http://www.sifry.com/alerts</url>
    <rssurl>http://www.sifry.com/alerts/index.rdf</rssurl>
    <inboundblogs>150</inboundblogs>
    <inboundlinks>184</inboundlinks>
    <lastupdate></lastupdate>
    <foafurl>http://www.sifry.com/alerts/foaf.rdf</foafurl>
  </weblog>
</item>
</result>
</document>
</tapi>


The document is broken up into two sections: The first part describes some information that the user (dsifry) wants to allow people to know about himself.  This information is available in HTML form at:
http://www.technorati.com/profile/dsifry
In this case, all the personal information that I want you to know about me is my username, and a link to a picture of myself, and the blogs I author.  The second part of the document is a listing of the weblogs that the user has successfully claimed and the information that Technorati knows about these weblogs. This is the default, by the way.  You'll see lots of information in there, like the Name and URL of each blog, its RSS feed (if it exists), number of bloggers who are currently linking to it, when it was last updated, and some other optional data, like a link to a FOAF file, or GeoURL. 

More to come, including information on the validate API call, which is built to make it easy to validate blog and comment postings...

(Expert note: This is a RESTful call, as with all the Technorati API calls, you can use GET or POST to the http://api.technorati.com/getinfo URI)

On Business Models

DonPark and Tim Oren are engaged in an interesting discussion of business models in the wifi world. Don kindly makes a suggestion to have Access Point hardware vendors subsidise the price of the wifi Access Point (AP) by bundling it with a services oriented business model.  It's like a blast of deja vu to 2001, back when Sputnik was getting started, and our original business model. 

Don lays out the basic ideas behind Sputnik's original model pretty succinctly:

  1. Bob, a store owner, buys Sputnik at 1/4 of the price, plugs it in at his store, and use the installation software to register the AP with Sputnik Network.

    • The AP is configured so that only Sputnik Network members can use it. 
    • Administration, security, and account management is all handled by Sputnik Network.
  2. James, a Wi-Fi user, subscribes to World-wide Sputnik Network service for $10 per month, enabling him to use any Sputnik Network AP around the world.

    • Sputnik client software running on his laptop automatically handles authentication with each AP.
  3. AP usage is metered so Bob might receive a check each month if his AP gets a lot of traffic.

In late 2001, Sputnik released its Sputnik Community Gateway to the world, which would turn any old PC with a wireless card into an AP that authenticated users onto the Sputnik Network, a centralized authentication service.  Lots of people downloaded the code, used the gateway, and people joined the Sputnik Network.  But we decided that we were pursuing the wrong business model, and changed our plans. Here's why:

  1. Revenue split.  Each subscriber is paying $10 a month in Don's example.  Some of that money is going to go to Bob, the store owner who has installed the subsidised AP.  The revenue split needs to be compelling enough to make it interesting for Bob.  But there are other folks in the mix here too - like the ISP, see below, the VAR or SI who installed the AP, the location owner, and possibly the roaming agreement provider (like iPass or Boingo) .  So now that $10/month is split with even more people.  That's a lot of ways to split $10, so your service margins get pretty thin, even as it is. Now add in the fact that James is calling customer support because he's getting unreliable service (see Customer Service Headaches below), and it becomes nearly impossible to make money.
  2. Legal issues.  Most residential broadband connections come with a pretty strict Terms Of Service and Acceptable Use Policy which prohibit the sharing of broadband connections.  Of those that do allow sharing, most only allow for sharing within a single household, not reselling of service.  One way around this is to cut the ISP into the revenue split, which would hopefully provide an incentive to them, and cover their costs of extra data traffic passing over their backbone as well as (potentially) James as a lost customer - why should he buy a new DSL or Cable modem if he can surf on his neighbor's connection?
  3. Customer Service headaches.  If we presuppose that our aforementioned wi-fi user, James, is paying a monthly subscription fee, then he's going to demand some kind of service level, otherwise he's going to feel like he's wasting his money.  The problem is that Sputnik has no control over how James is getting his access - for example, what if his neighbor, who is providing him with wifi access decides to move?  Or if he unplugs his AP when going on vacation?  James doesn't know about this, and the Network provider has no control over James' neighbor - we can't go over to his house and turn his AP back on.  James' percieved value of the service drops precipitously, and he gets puzzled, or even angry.  Then the support calls begin - Since he was getting service just fine the day before, he is going to try to figure out why the service isn't working now.  Now James starts calling the Sputnik call center, trying to diagnose the problem with "his computer".
  4. The rise of "free" networks.  A significant number of businesses are giving wifi away for free - as an incentive to get butts in seats, who then order coffee, or happy meals, or whatever.  Other businesses are using an advertising-based support model - watch an ad when you log in, and you get free access for the day.  Others are using wifi as a customer affinity program, or CRM system - why go to the cashier when you can order your food and drinks while at your seat - "oh, and can we sell you a new CD with that, sir?"  Some businesses just want to get a better idea of customer demographic.  The point is that a traditional for-fee network isn't necessarily the right business model for all occastions or for all locations.  

Believe me, we looked long and hard at the business model and tried to find ways to make it work.  What we realized is that wifi is not a one-size fits all service model.  Sometimes a per-minute or per-day for fee network is the way to go.  Sometimes it isn't.  What we realized, is that by creating an architecture that supported Don's idea allowed us to let our customers figure out the business model that was right for them.  In addition, the further commoditization of wifi hardware means that it is going to become more and more ubiquitous - so the number of potential wifi customers will increase, but hardware profit margins will decrease.  So we embarked on a new strategy:

  1. Give AP manufacturers something to differentiate themselves.  We shrunk our codesize so that it now fits onto the standard flash sizes of inexpensive APs.  Our core code only takes about 150KB of space, which means that there is no need to change hardware designs or increase hardware costs.  At the same time, optional components allow for manufacturers to add additional value through combinations of hardware of software, like VPN accelerators, group policy, bandwidth shaping and throttling, and more.  Licensing is very affordable, and it allows the AP manufacturers to increase their margins by selling differentiated products.
  2. Make money as a software company.  Sputnik's business model is based on selling the management system that lets you control and manage all of those inexpensive APs in a centralized manner.
  3. Let our customers decide on the right business model for them.  We built the Sputnik Central Control system using a set of open interfaces - so that our customers could use different billing systems, settlement systems, and authentication systems.  Because their capital expenditure is reduced by using inexpensive APs, and their operational expense is reduced by using the Sputnik Central Control management system, our customers are free to deply wifi in interesting ways, and to experiment with different service business models.  At $895 for Sputnik Central Control, it is also 1/4 to 1/10th the price of competitive systems.
  4. Be backwards-compatable.  Our products don't use any proprietary new radio encoding method, or even require special client software - all you need at a minimum is an SSL-capable web browser.  That means that all the major operating systems, all the major handhelds are immediately able to authenticate to a Sputnik-powered network.  Of course, client software can make things easier and more functional, but it is not a requirement for the system.  And IT directors can rest easy knowing that they don't have to add a single new piece of software to their standard builds.
  5. Be forwards-compatable.  Wifi is constantly changing - new speeds, new radios, new encryption methods are coming out all the time.  There's a lot of innovation going on in the space.  This is good and bad - you don't want to get locked in to buying a system that will be incompatable with tomorrow's standard. There is one body that everybody looks towards: the IEEE.  802.11 is the name of the IEEE working group that covers this whole area - and all the vendors work on and respect the standards coming out of the working group.  For example, when WEP was broken, the IEEE embarked on a new standard for encryption, based on AES, which is being hammered out by the 802.11i task group.  It's not ready yet.  When it is, Sputnik products will support it.  Until then, we're not getting into the crypto debate or muddying the waters with some proprietary crypto scheme.  A proprietary scheme ends up locking in a subset of customers, but it also ends up fragmenting the market, hurting everybody, especially the hapless souls who are now locked in.

I still love the "change the world" aspect of Don's idea, and ideas like his that build on network effects can certainly create economies of scale and competitive advantage.  In fact, I want to encourage Don to go out and build it and turn it into a gold mine.  Along the way, we're happy to sell him the picks and shovels he'll need to mine that gold.

Technorati Downtime

I just got off the phone with my colo provider who seems to be having a major unplanned outage. The long and short of it is that Technorati is down right now, and I don't have a firm ETA on when things will be fixed.Hopefully soon. UPDATE, 1 minute after posting this entry: Amazing, things are back up and running. I got a call right after posting this entry, and the systems appear to be back up and running. Hooray!

Technorati Tutorial, Part 1

Lilia Efimova at Mathemagenic asked an interesting question about Technorati on her weblog today, and I popped by (thanks to my watchlist) and answered her questions. Given the interest, I thought I'd republish my response here, along with a few elaborations.

Lilia asked,

Does anyone knows how Technorati works? Do they process blog homepages only? Or only items in RSS feeds? Or only things "not older than ..."?

I wonder because I usually observe some fluctuations in numbers of inbound blogs and inbould links. E.g. yesterday I had 100+ inbound blogs and today it's 80+. It would be interesting to know why these things change. I tried Technorati site and weblog of David Sifry with no luck.

I guess this is a quite typical question that user has about systems that digest information: what are the criteria that are used?

Some basics about Technorati

1) We spider weblogs, and correlate each weblog's outbound links to any page on your blog/site
2) Technorati works on any URL - not just URLs for weblogs. For example, you can see what people are saying about an interesting article or favorite company, and get an instant read on the conversations going on around that article or site.
3) The simplest way get your weblog included in the Technorati index is to ping us whenever you update your weblog. That puts you in the high-priority queue for indexing. You can save the page as a bookmark, or you can program your weblog software to do it automatically.
4) To calculate the inbound blog list, we use the outbound links from the blog homepage, not from the archives
5) We do process RSS feeds an other metadata, but that doesn't affect your inbound blog stats. As long as you produce HTML, you're OK.
6) Nightly, we go through the database and re-calculate the number of inbound blogs and links to every weblog we track, which helps us double-check our work and also allows us to create the interesting newcomers list, the interesting recent blogs list, etc.

We strive to be accurate all the time. Sometimes things slip through. For example, one of the reasons why your inbound blog count may be smaller today is because we were doing maintenance of the database last night to remove duplicate blogs - for example, Radio Userland has an obnoxious habit of sending pings to www.weblogs.com for each weblog "category" if you use multiple categories on your blog. Same information, same author, just link spam, basically. So, last night we cleaned out a bunch of that stuff. If you were linked from a bunch of people's blog categories, then you lost those inbound blogs. Then again, so did everyone else. :-)

The last thing to remember is that while we strive for accuracy and completeness, we still do have bugs and have to fix things. If you notice something strange, please don't hesitate to send us feedback (feedback@technorati.com) and let us know.

Looking for dietbloggers?

I got pinged today by a reporter looking for people who are using blogs in innovative ways to help them diet. I know that I was inspired by folks like Cory, Glenn and Doc when I got started doing Atkins, but I wouldn't say that I was using my blog to help motivate me or keep track of progress or anything. It does seem that lots of other people are using blogs to talk about their diets, however. Searching through Technorati, I found 592 weblogs that mention "Weight Watchers" and 740 that mention "Atkins", for example. Seems natural, given that most of us feel comfortable to talk about our lives, and that's a part of our lives.

I remember seeing some group weblogs where people were reporting regular diet progress, and egging each other on. I know I saw some folks start a public bet with each other - $100 each in the kitty? - to see who could reach their goals. It is late though, and I can't seem to find the sites (yes, insert irony here). Can anyone send some pointers or links? Is the blogosphere seeing a new trend of dietbloggers? Got any great diet blogs or diet stories? Post 'em in the comments section, or drop me an email. BTW, I've dropped another 10 pounds since the last time I blogged about it. Feeling great.

RSS as Infrastructure

With the announcement yesterday of the assignment of the RSS 2.0 specification to Harvard University, along with a Creative Commons license and a new 3 person steering committee, RSS 2.0 has become more firmly cemented as an infrastructural element in the web publishing world.  This is a good thing.  It will help wary organizations to feel more comfortable using a syndication standard with the assurances that it is not going to be changed on a whim or hijacked by someone with a hidden agenda.  RSS 2.0 isn't perfect, and that's one of its best qualities.  It was designed with a "worse is better" mentality, what I like to call POGE - the Principle of Good Enough.  That means it is simple, easy to understand and to code.  It means that it doesn't have a lot of bells and whistles, and it isn't a format for all things or all purposes.  It has a history, which means it has some bumps and warts, but IMHO, it does a pretty good job of doing what it sets out to do: Be a format for the syndication of published content. 

This is not a knock on other efforts that attempt to achieve other goals.  My perspective is to use the best tool for the job at hand, and it is OK for different people to have different opinions on what that is.

Kudos to Dave Winer, the folks at Berkman, and the Advisory Board for taking this positive step.

New Technorati Pinger is active: Hot 'n fresh weblog indexing

Over the past months, I've had a lot of people send me email or leave comments asking how they can get their weblog indexed by Technorati.  This weekend, I had a few hours of free time and I whipped a new feature with two interfaces:  It is a standard XML-RPC ping server (like www.weblogs.com) that you can add to your weblog software configuration (or that hosting providers can use) that will immediately add your weblog into a special high-priority queue for Technorati's indexing runs.  This means faster, more accurate search data.

To use the pinger, there are two interfaces: an XML-RPC interface at http://rpc.technorati.com/rpc/ping which understands the weblogUpdates.ping and weblogUpdates.extendedPing specifications detailed in the weblogs.com specification and the blo.gs specification.  I don't have a SOAP 1.1 version done yet, but I can add that, if people want it - leave a comment below.

If you're using a modern weblog package like Moveable Type, blosxom, pMachine or many others, simply add http://rpc.technorati.com/rpc/ping to your notifications configuration.

The other way of notifying Technorati that you've updated your weblog is the web interface available at http://www.technorati.com/ping.html  Simply fill in your weblog name and its URL and voila, your weblog will be added into the fast indexing queue.  Once you've done it, you can even bookmark the page so that you can easily notify Technorati whenever you update or add items to your blog.

Of course, if you're currently notifying www.weblogs.com, www.blo.gs, or www.blogrolling.com when your weblog is updated, your weblog will continue to make its way into the Technorati index, albeit a bit more slowly.  Notifying Technorati directly will put your weblog (or weblog hosting site) onto a higher priority track to get included in the index.

These two features are new, so there might be some downtime or fixes during the next few days if I need to fix a bug or two, but the interface and the ping URLs are set in stone, so if you are a blogging tool author you don't need to worry that the service will move to a different location, or that the API will change.

A new wiki page describes how to add the ping service to your favorite weblogging tool's configuration.

As always, feedback is appreciated. 

--

Newsmonster 1.0RC1 ships with Technorati support

Kevin Burton has been a busy guy lately, getting a new release of his uber-aggregator tool Newsmonster out the door.  I've been playing with it and it is really quite slick.  Kevin's built a cross-platform tool because it runs inside of Mozilla.  So, right off, it runs on every platform that Mozilla runs on, including Windows, Mac OS, and Linux.  Kevin adds in multipaned windows, and automatic integration of feeds/pages that don't have RSS, and things start to really cook. 

Then to top it all off, he has integrated the Technorati API into the application, which makes finding the conversations around any article you're reading as simple as pie.  He's posted some screenshots as well.  Here's a view of Technorati threading and here's one of its Technorati Keyword Search integration.  It even has its own reputation system built in. These features are what sets NewsMonster apart for me - being able to quickly get context information on news and events of the day ads a tremendous amount of value to my daily web experience, and by displaying the Technorati Link Cosmos results as threads in NewsMonster makes following the conversation a much richer experience.  Kudos, Kevin!  Now all we need is the ability to post to a weblog via NewsMonster, and I'll be all set.

--

Comments on the RSS Controversy

There's a lingering open wound in the weblog technology community, and it is called RSS.  Last week, Dave Winer and I had a long phone call discussing the issue, and I've been doing some reading and thinking about the controversy.  My perspective on the RSS issue: I think it all comes down to naming.  Dave has said (on numerous occasions) that if the folks who created RSS 1.0 had given it another name, he wouldn't have had a problem with it.  For example, here's Dave's perspective from Scripting News on 8/31/2000:

The proposed syndication format takes a new direction. It should have a new name. Then there's no problem. Further, a new format is not bad or good. If there's a suitable new name, and if it gains traction, we'll support it, as we would any Web syndication format that has content support.
Mark Pilgrim has also written up a great summary of the history of the fork. 

Does the problem really boil down to this fundamental issue?  Last week, I asked Dave on the phone, "So would this whole controvery be solved if RSS 1.0 was renamed to something else?" 

His answer?  "Yes, absolutely." 

Now here's the really good news.  A number of folks who helped write the RSS 1.0 specification, like Aaron Swartz and Sam Ruby, have expressed willingness to drop RSS 1.0 future development, and to rename their future weblog standards development.  Sam's even put out a call for new names.  Here's my suggestion:  Call the new work "MSS 1.0".  MSS would stand for Metadata Site Summary. Make it clear that this is solving a different set of problems than RSS 2.0 solves - I think the wiki already goes a long way to describe the differences, both in scope and in philosophy.  Let's let the confusion end, and bring some healing to the weblog technology world. 

Then we can all move forward together.  And get to what's really important - sophisticated interoperability and new features that users will love.

UPDATE:I've been told that Sam Ruby was not one of the original RSS 1.0 authors, sorry for the mischaracterization.