Tuesday, February 14, 2006

Dijjer: Future of Online Media Distribution?

Josh just pointed me to Dijjer.

It looks fantastic. Here's why I think it's better than BitTorrent, keeping in mind i've only read their FAQ, and haven't done extensive testing of it.

BitTorrent doesn't work through NAT. It actually works, but only in leech-mode. Which means, if you're an average non-techy user behind a broadband router's default-configured NAT, install BitTorrent and download files from other peers, you actually do not contribute any bandwidth back to the rest of the network, because other machines on the Internet don't have a clue on how to reach you. Of course, it's very easy for someone with a little bit of tech savvy to do the necessary port forwarding. But realistically, on a large scale, who ever bothers?
update: ok it would appear the above paragraphs are angering armies of geeks. Take a deep breath as I concede your collective point with the following: Yes, if your BitTorrent client already has an active connection while downloading from a peer, that peer will be able to get data from you, and you're therefore indeed "contributing bandwidth" back, to some extent. I've experienced this first-hand. It's nice. But if I'm behind NAT, and I don't have port forwarding, and you're behind NAT, and you don't have port forwarding either, then our BitTorrent clients won't exchange data directly from each-other. Period. Is this a big deal? Obviously not, as there are enough people out there with properly set-up connectivity to keep BitTorrent a successful and vibrant community ... of mostly geeks who know what they're doing. I'm looking at a different market. I'd like to see a peer-to-peer solution that enables a wider audience to become first-rate bandwidth contributors. With some maturation and polishing, working out a few kinks, it would appear dijjer just might get us there.
It's not impossible to defeat the vast majority of NAT issues in the wild. The STUN protocol, released around 2003, tells us how to do it. Most SIP-enabled client software and devices make use of it, to avoid having to relay RTP packets between peers.

Dijjer claims to be able to defeat most NAT issues. And I wouldn't be surprised if it used many of the techniques outlined in the STUN protocol.

It also looks far easier to deploy than BitTorrent, as most of the work of finding peers seems to be handled by the clients. It may or may not be a good thing, it looks like they use some "seed peers" to establish initial contact. But they've removed the need for the content owner/deployer to create anything "special" on their server. They just need to link to their a file a bit differently. And as their FAQ mentions, the file's presence on the original HTTP server is used for policing its distribution over the dijjer network: If a file lives on your web server, and you decide to link to it via dijjer, then subsequently change your mind, you can just remove it from your server, and nobody on the dijjer network will be able to download it.

On the peer's machine, it appears to expose a lightweight http interface on a high port. When you're downloading a file, the browser actually retrieves it over normal http from localhost, over port 9115. If you've installed dijjer, try this URL: http://localhost:9115/. It seems to be a technique vaguely similar, in principle, to what certain "web accelerators" do. The fact that as a user, I don't have to "switch" to another application to merely download a file, before resuming web browsing back into the web browser, makes the whole experience more usable, less clunky, less nerdy.

It also opens the doors for 3rd-party applications to interface with Dijjer via straight HTTP, without having to write custom components that comply to a different protocol. These folks might one day likely be interested in switching from BitTorrent to Dijjer.

One thing to keep in mind is that dijjer wants to keep running in the background. When double-clicking that .jar in Mac OS X, it's not readily obvious that dijjer is running. You can easily shut it down via http://localhost:9115/, or, on the Mac, a "killall java" from the terminal. I haven't tested Linux and Windows yet. Their FAQ also mentioned that dijjer might pre-emptively start "caching" stuff on your hard drive, stuff you may have never requested, but that's useful to other people on the dijjer network. But again, a peek at http://localhost:9115/ tells you what it's doing, how much space on your hard drive is being used, and your download/upload activity. Before putting-on the tinfoil hats, remember that dijjer is 100% open-source, and verifiably crap-ware free.

In contrast, with BitTorrent, you'll only ever seed files you've previously downloaded, or are in the process of downloading. BitTorrent is very-much more of a "foreground" process.

Going forward, dijjer will likely mature and become more polished. It's already very usable, and very promising.

A few replies to some comments in the digg submission:

I *did say* you *can* make BitTorrent work through NAT to enable other peers to 1) discover you 2) connect to you without your having previously connected to them, *if* and only *if* you do port forwarding, as outlined in the BitTorrent FAQ which i linked to in the article. As i mentioned in my review, it is simple enough for us geeks, but not for your average newbie. Having out-of-the-box NAT traversal support dramatically increases your potential user base.

When you look at peer to peer frameworks, you've gotta distinguish a few aspects of the peer to peer features. Two of which are: 1) file and peer discovery. 2) distributed file transfer.

Some peer to peer frameworks have a highly decentralized file and peer discovery mechanism, and those are nicely suited for illegal sharing of files.

Other peer to peer frameworks, such as BitTorrent and Dijjer do not seek to make the file discovery aspect decentralized, because their designers didn't seek to build a rebel or illegal file sharing network. That's why it's so easy for the RIAA to go after torrent sites today. And Dijjer is even more tightly coupled to the source than BitTorrent is, as it is a feature.

There are people out there who don't use peer to peer technology to share illegal content.

The folks (Nicholas Reville, of downhillbattle.org fame and many more) behind http://getdemocracy.com/ have embedded their own BitTorrent client within their TV player to save bandwidth for the various content providers. And that's already very cool. Integrating Dijjer instead of (or in addition to?) BitTorrent into their client might make their platform even more efficient. One of their developers has mentioned to me they've indeed been looking at dijjer.

The end-game of legal use of peer to peer technologies is a new framework for content distribution. Profound disruptions of today's media. Pretend for a second that something such as dijjer, or something better, gets wide adoption, among the masses. And i'm talking about the real masses, our Moms and Dads, with their home broadband connectivity and plenty of bandwidth to share.

Say you suddenly decide to become your own movie producer and director, and you manage to make movies people are actually interested in watching, and perhaps eventually buying. In a dijjer-enabled world, you could likely serve your movie from any normal web account with a low bandwidth quota, but link to it through a dijjer-ised link.

Unlike with bittorrent, you didn't have to purchase a special hosting account at a BitTorrent provider (prodigem are cool guys), nor did you have to set-up your own BitTorrent server/tracker. You just uploaded the file to your web account, but instead of giving this url to your friends: http://someisp.com/youraccount/mymovie.mov , you gave them this url: http://dijjer.org/get/http://someisp.com/youraccount/mymovie.mov .

I'm interested in peer to peer technologies to replace traditional media. Not to illegally share the crap they shove at my TV. Hence the title of this blog post.


artreality said...
This comment has been removed by a blog administrator.
Achille said...

dont digg to your blog darn it. write it to wikipedia or somewhere else...

Anonymous said...

If this is so great, than why not have BitTorrent implement it?

Anonymous said...

Submitting your own blog to slashdot isn't very nice..

Stiles said...

I saw an early version of dijjer a few months ago. Like you I thought it looked promissing, but what gave me pause was the the always on nature and pre-emptive caching of data. I had visions of this thing just eating up all my bandwidth delivering files to others while I am struggling to keep my web browsing up. I don't know, maybe it wouldn't be a problem. I agree though, seems like a similar, better, bittorrent.

Anonymous said...

I'm not extremely computer savy ... I checked the website and everything but I still don't qute understand how to use it like bittorent to dowload stuff ... any help you can give would be appreciated -- holywarrior7@gamil.com

Anonymous said...
This comment has been removed by a blog administrator.
Stiles said...

Just tried this out again... Seems to be rubbish. I get nothing but tons of errors.. Downloaded one file successfully 5 others failed, And these are the ones from Dijjers website. Although I am glad the one I did get to see had a skateboarding dog in it. Very nice.

Anonymous said...

Note "killall java" may kill more than the dijjer process - recommend using "ps" to identify the process, and kill it by id instead.

Younes Zouhair said...

BitTorrent doesn't work through NAT. It actually works, but only in leech-mode.

No way, I am behind a NAT and bittorent works nicely Uping and Leeching.

Bittorent have nothing to do with NAT in anyway, just be sure to have good port pointing to good PC.

So please change your news

trickykungfu said...

You say that bittorrent does not contribute bandwidth back to the network while seeding if you're behind a NAT, but that's patently untrue. Your client still interfaces with the tracker regularly, you get a list of peers that need pieces, and you initiate connections to them. It of course works better if you have a port forwarded, but as long as a decent percentage of the swarm does, not everyone has to. The only advantage I see is the questionable NAT transversal capability, and it loses many of the other advantages of bittorrent.

Bob/Paul said...

You're dead wrong about the leech-mode only behind a NAT. Read that FAQ entry again, and then try using bittorrent for yourself (I use Azureus, so it might be a tad different than the official client, but I doubt it).

Behind a NAT other clients cannot connect to you directly and say "Give me this piece!" However, you're computer can still connect out and say "Here's this piece you need".

I've frequently run behind a NAT with azureus giving me the Yellow "No Port Forwarding!" icon, and I've still been uploading to the max my internet connection will handle. I've even continued afterwards and seeded until I reached a 1:1 ratio. Normally I am behind a router with portforwarding configured, but not always; I'm not always at home running Bittorrent on my PC.

Not that Dijjer isn't cool, but it's very old news and your analysis of Bittorrent was only half right in this regard.

Anonymous said...

"BitTorrent doesn't work through NAT. It actually works, but only in leech-mode. "

This is incorrect! When a machine behind NAT connects to other machines it is able to upload AND download -- the only difference is other machines are unable initialize a connection to it.

So NAT may affect the ability to find and connect to peers but does not affect the flow of data once connected.

Alexey Melchakov said...

with bittorent behind nat you still contribute traffic to network. but your own download speed may slow down. "uchi matchast!"

Geoff said...

Dijjer definitely has it uses. I like the results I had the few times I've used it. These folks going all postal about BT and NAT are ignorant. Sure, it works just fine if you open your ports. But the majority with firewalls can't or don't know how. Really. If you know how, great. Big claps for you.

Anonymous said...

"Here's why I think it's better than BitTorrent, keeping in mind i've only read their FAQ, and haven't done extensive testing of it."

In the world of analysis, your credibility ends there.

Anonymous said...

dijjer loses funding and digger.org goes unregistered, boom dijjer is useless. this fails. bittorrent forever!

Bob/Paul said...

That's the thing though, Geoff... You don't need to configure NAT and BT works just fine. You just won't be able to connect to people who also are behind a NAT.

Dijjer has this exact same problem. The only way you can get around double NATs (a NAT on each side of the connection) is through a server in the middle, which means you're also using a third parties bandwidth.

Most people don't need to configure NAT and they'll be fine. For a great deal, uPnP will take care of it for them. For the rest, both Dijjer and BT will work /about/ equally well (Dijjer looks like it should handle it better, but it's still not going to work as well.)

The only reason Dijjer wins is because it's easier for the publisher. And, as noted, if Dijjer.org goes down, no one can download new files.

Jordan said...

Actually, most up-to-date Bittorrent clients (uTorrent, Azureus, Bitcomet, etc.) can traverse NAT's through UPnP, which most modern routers support. No port forwarding, no configuring ports, no intervention needed.

Poopy McPoop said...

Sorry, dijjer sounds like the millionth beating of a dead horse. I seriously have seen this particular idea at least fifteen billion times already.

Chris Holland said...

@Bob/Paul: actually, you're wrong on this one. Dijjer claims it *can* do double-NAT traversal. The whole section i wrote about STUN and NAT traversal in the post talks about just that.

It *is* possible to do, and from what I've experienced with a couple of dijjer peers, it actually works.

Since most average non-techy broadband users are behind some sort of NAT, dijjer offers a solution that works out-of-the-box. well ... minus the kinks and whatnot, as i said at the end of the post, it has room to mature. But in principle, dijjer could dramatically increase your distribution network.

Anonymous said...

"BitTorrent doesn't work through NAT. It actually works, but only in leech-mode. "

That statement is entirely false. Where did you even get that idea? It is unfortunate that thousands of people have now read and believed your misunderstanding!

You cannot receive inbound connections if you are behind NAT, but you can (obviously) initiate outbound connections... and (perhaps less obviously) you can both upload and download using solely outbound connections.

Thierry said...

Interesting article.

I wonder why people keep whining about blog articles submitted to slashdot or digg. It's ridiculous when the blog article just links to another web page on a news site, but in this case, it's a genuine article with its own informations, it's as good as any other static web page.

Anonymous said...

Thierry, actually this blog post is a prime example of the low quality of blog information. Wikis are inherently more reliable sources of information; if this had been posted on a wiki instead of a blog then surely one of the first 100 people to read it would have fixed the incorrect statements about bittorrent and NAT before thousands of other people got misinformed. There are now, and probably will be for a long time, people who believe bittorrent doesn't work with NAT because this blog post was linked from major sites today. Sure, the post has been edited and fixed now, but the damage is already done. The lesson here is: If it isn't editable, or if it is editable by someone but the history isn't transparent, then it is not a reliable source of information.

I must say Chris Holland regained a little cred imo with the MS/Darwin thing though. LOL!

Anonymous said...

From other comment: "Actually, most up-to-date Bittorrent clients (uTorrent, Azureus, Bitcomet, etc.) can traverse NAT's through UPnP, which most modern routers support. No port forwarding, no configuring ports, no intervention needed."

NAT Traversal doesn't have anything to do with UPnP - with UPnP you can open port, while NAT traversal can help even if you can't do this (eg. when you have private IP).
Also, uTorrent doesn't support NAT Traversal (bitcomet does).

Anonymous said...
This comment has been removed by a blog administrator.
vladu11 said...

Hello! I`m student at University and I need your help for my bachelor degree. I have read your post for Dijjer application: http://chrisholland.blogspot.com/2006/02/dijjer-future-of-online-media.html

I have installed Dijjer application and it looks fine between 2 peers (the host with http server and the client who is watching something). My problem is why others peers can`t connect? On the events page (http://localhost:9115) is written that any peer is connected.

It`s a NAT problem?

Please help me! I`m waiting for your response.

Thanks a lot!