Home

Advertisement

Natarajan's journal - LiveURLs - my best hack till date [entries|archive|friends|userinfo]
Natarajan

[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

Links
[Links:| Photos Bookmarks Tech ]

LiveURLs - my best hack till date [Apr. 19th, 2006|05:40 pm]
Previous Entry Add to Memories Tell a Friend Next Entry
[Tags|, , , , , , , ]

Have you ever wanted to share a portion of a web page with others?  We've made it easier for you.  Toufeeq and I have been working on this Firefox extension project for sometime and now it is out for everyone to try and put to use.

The extension page at Firefox addons
The extension page at Mozdev
Screenshots


Toufeeq, [info]mahendra and I exchange a lot of information through URLs.  One day, I wanted to point out a certain paragraph on a lengthy page to Toufeeq.  I realized I had very little option of doing such a thing.  I could either mail the link to him or copy-paste the contents in the mail.  There was practically no way of telling him to visit this URL and read this paragraph (unless it had a named anchor though).  We believed there must be way out.  After spending a week or so, we zeroed in on a few ways to implement it.  We were pretty surprised that there has been no implementation that solves this pain yet.  Then we discovered Ahoy! - Everything can be a link with Mozilla's window.getSelection().  Ahoy also aimed at generating links to portions of web pages.  It is a very very beautifully described implementation .  And it was under the GPL :) Thanks a ton, Brian Donovan,  you set our ball rolling.

The most obvious place of putting the information about the selected content was the fragment identifier of the URL.  After all the fragment identifier was meant to locate fragments.  We wrote out a greasemonkey script (based on the Ahoy idea) the same evening we found about Ahoy and by midnight we had the first version of the Firefox extension ready.

The Ahoy idea uses DOM tree offsets to indicate the starting and ending nodes of the selected range.  Though we had a fully functional extension we were'nt very convinced about using DOM offsets to indicate ranges.  It was'nt immune to the smallest changes in the DOM structure.  With literally every page flaunting dynamic ads and client side scripting usage at an all time high, one can never be sure about the same DOM tree being generated for two rendering instances of the same page.

We needed something to reflect the content that was selected and not the position alone.  This was because,  in most cases, people would want to share content, rather than the position of the content.  Marooned without internet access on a weekend at home, we started off with a crude search algorithm which included as part of the fragment identifer the starting word, ending word, number of characters and number of words of the selection.  This worked well enough.  But it still did not reflect the exact content.  This was nagging me for sometime.  Then I recollected a discussion that we had at InfyLUG (The Infosys Linux Users Group) during the initial stages of development.  Raghuveer (a Delivery Manager and an InfyLUG old hand) had suggested we should be using some sort of a checksum to 'denote' content.  And what a brilliant piece of idea that was.  We removed the end word and number of words parameters from the fragment identifier and put in a CRC checksum parameter in their place (thanks to the sample PNG CRC32 implementation).  We now have a method that can find content (arbitrary text ranges) within a page with a very high degree of accuracy.

Now that we had a way to locate content reliably, we started working on reducing the length of the fragment identifier.  We minimized the use of delimiters.  Instead we used a header character indicating the length of data  to follow.  We converted numbers to their hexadecimal notation to save up on crucial character space.  Intially we had used ":" as the delimiter and its url-encoded form (%3A) was consuming valuable space.  Three characters for a delimiter was not a luxury we could afford.  We changed the delimiter to "+" instead and saved valuable character space.  Now the average length of the fragment identifier for the checksum/search based approach is about 18-20 characters and around 26 characters for the DOM based approach, which is'nt very bad at all, considering the things you could do with it.  Please read the technology page for more information.

In between all this, we managed to get an opportunity to present this work at the 15th International World Wide Web conference.  The poster we submitted to this conference has been selected. We got some company press here.

At the end of all of this, I've got this to say
  • FOSS rocks.  Without an awesome open development platform like Firefox, all this just might not have been possible at all.
  • Thanks to all the projects that helped us do what we wanted to do.  The Ahoy project which gave us a headstart.  The awesome Scrapbook project was a great source of information on how to handle node ranges of a DOM tree.  A special thanks to the bookmarklets at www.squarefree.com.  It is a very useful Javascript debugging and development tool.
  • Thanks to the wonderful team at InfyLUG - Raghuveer, Mahendra, Naveen Unni, Sanjay Eknath, Anoop Varghese, Basavesh and everyone who helped make this happen.  The credits page goes out here.
  • Patents in their current form suck.  You start thinking about them and they suck the creative energy in you.
  • Our day would be made when the phrase RTFH (Read the Fine Highlight) becomes as ubiquitous as RTFM ;


The impatient ones please
  1. Install the Firefox extension from here.
  2. Select a portion of a web page and invoke the context menu (i.e. right click menu)
  3. Select either "Copy link to selected content" or "Copy position of selected content". This action copies the selected content as a URL to the clipboard.
  4. Open the LiveURL in a different tab/window of Firefox which has the same version of the LiveURLs extension installed.
  5. The browser should show the previously selected content highlighted. (If it does not work, report the bug with the LiveURL that did not work)
Note:
The sender and receiver of the LiveURL should have the same version of the LiveURL extension installed.  Though fully functional, this is in its early stage of development.  So backward compatibility of LiveURLs might not be supported i.e. there is no guarantee that LiveURLs generated with one version of the extension will work with another version.

This is one hack that I will be proud of, all my life. I hope you find it useful.
LinkReply

Comments:
Page 1 of 3
<<[1] [2] [3] >>
[User Picture]From: [info]mssnlayam
2006-04-20 12:49 am (UTC)

(Link)

I am speechless.
[User Picture]From: [info]mewpsych
2006-04-20 05:09 am (UTC)

(Link)

Awesome! Hearty congratulations!
May you rock more and more!
[User Picture]From: [info]sunson
2006-04-20 06:32 am (UTC)

(Link)

Awesome. This is very very neat and I badly wanted this feature! :) You just did what I exactly had been looking for :)

Thanks!
[User Picture]From: [info]knutties
2006-04-20 10:02 am (UTC)

(Link)

Thanks, Suraj. I too badly needed this. Feels good to have created something that it is useful :)
From: [info]nearfar
2006-04-20 06:34 am (UTC)

(Link)

The client need not have the firefox extension installed. Instead you can send him a link like, http://www.liveurls.com/url/http://slashdot.org/foo/bar?3234...

That page, will show the actual page (slashdot, here) in an iframe within the parent page which contains the javascript that parses the iframe src, highlighting the text, etc..

This should even work with other web browsers. Requiring Firefox-with-a-specific extension would lessen the chances of making your `LiveURLs` popular among users.

Oh, btw, you can earn money through ads in liveurls.com. :)
[User Picture]From: [info]sunson
2006-04-20 06:40 am (UTC)

(Link)

Good idea!
[User Picture]From: [info]sajith
2006-04-20 08:28 am (UTC)

(Link)

Impressive!

[User Picture]From: [info]knutties
2006-04-20 10:07 am (UTC)

(Link)

Thanks Sajith. Get your friends to install it (branding, branding :)). Only then you can use LiveURLs :)
From: [info]nearfar
2006-04-20 09:20 am (UTC)

(Link)

Nat, good for you that, in Infy, you are working on cool stuff, that you really like. :)
[User Picture]From: [info]mewpsych
2006-04-20 11:57 am (UTC)

(Link)

Any possibility in future to enable the users to do more than one selection per page? Something like 'Add' to the URL.
Probably it is too much to ask for i guess.
From: [info]toufeeq
2006-04-20 01:11 pm (UTC)

LiveURL's stacking

(Link)

Wait for 0.2. That's one of the features everyone is demanding.
Raise this as a BUG(feature request) in the Bugzilla page and we will work on it. :)
From: (Anonymous)
2006-04-20 09:24 pm (UTC)

Why two options?

(Link)

Awesome! Congrats Natty.
BTW, What is the need for two options - "Copy link to" & "Copy position of" - funtionality-wise both are the same right - then why show both? - am I missing something? Can you explain when to choose one over the other?

-Janani
[User Picture]From: [info]knutties
2006-04-21 01:15 am (UTC)

Re: Why two options?

(Link)

Thanks a lot Janani :) The options are two different ways of achieving the same result. This version is a development version and so we wanted to make it easy for users to try out both options and get feedback. We will be standardizing on one approach.

Now when to chose one over the other? The "Copy link to content" is an option which is based on the text that is selected and nothing else. So use this when you want to share text portions of web pages with people.

The "Copy position of content" uses the position of the selected portion as the parameters. Say you want to show people one particular HTML table to people without regard for what contents it has, then you use this. But this option is very very sensitive the HTML structure.
[User Picture]From: [info]prashmohan
2006-04-21 12:26 pm (UTC)

another feature add

(Link)


How about giving some mechanism to do discontinuous selections. Often the required information doesn't follow the principle of spatial locality. I might need to highlight 1 paragraph on the top of the page and another on the bottom :p
[User Picture]From: [info]knutties
2006-04-22 06:13 am (UTC)

Re: another feature add

(Link)

Point noted :) Actually that feature is in the pipeline. This is in fact the most requested feature. See [info]mewpsych's query here . [info]adityag was the first requestor of this feature.
[User Picture]From: [info]bhuvanbk
2006-04-26 04:22 am (UTC)

(Link)

Somehow i missed this entry.
Excellent work. I smell the plume on your maiden contribution to open source. Congrats!
[User Picture]From: [info]knutties
2006-04-26 04:24 am (UTC)

(Link)

Thanks a lot Bhuvan. It was a great feeling to do that cvs commit :)
[User Picture]From: [info]bhuvanbk
2006-04-26 04:38 am (UTC)

(Link)

Change point 3 as follows:

Select either "Copy link to selected content" or "Copy position of selected content". This action copies the selected content as a URL to the clipboard.

Frankly, it took sometime for me to understand what you mean by "liveurl" in point 3 & 4.
[User Picture]From: [info]knutties
2006-04-26 05:15 am (UTC)

(Link)

:) Thanks. Changed it. Made a lot of sense.
From: (Anonymous)
2006-08-04 06:08 am (UTC)

Some History, References, and Congratulations

(Link)

Natarajan,

Very nice. I have a server side script called QuIP[1] which is similar. QuIP is descended from PurpleSlurple[2], which I also created. Neither is particularly robust, but other's have extended the idea c.v. Slurpler[3], and further discussion has resulted in some related work c.v. xPunt[4]. After PurpleSlurple and during the development of QuIP I too came across Donovan's Ahoy, but like Purple[5], Ahoy is a tool for the publisher. I wanted something for the consumer, so I created PSAhoy[6].

Look for lots of support and inspiration on the Yak list[7]. I don't know whether you know it but you are in good company pursuing the granular addressability[8] issue. Quotation in part[8.1] has interested Ted Nelson for decades. Purple and the other scripts mentioned above were inspired by the work of Nelson, Doug Engelbart (and others). PurpleWiki[9] might interest you also as it supports transclusion (a related concept), as will, I believe, xPunt.

I agree that functionality like this should be natively built into browsers. I'm aware of one attempt in the past[10] and some recent work[11] to do this. Work by you and others will make this come to pass in the fairly near future I believe. In the mean time I continue to evangelize the idea and try to connect inerested parties and programmers.

Keep up the good work.

Best regards,

Matthew A. Schneider

[1] QuIP: http://sasites.com/quip/
[2] PurpleSlurple: http://purpleslurple.net/
[3] Slurpler: http://www.concept67.net/perl/index.html
[4] xPunt: http://www.concept67.net/blog/?page_id=141
[5] Purple: http://www.eekim.com/software/purple/purple.html
[6] psAhoy sample: http://sasites.com/suse/apache/files/psAhoy.php
[7] Yak: http://collab.blueoxen.net/forums/yak/
[8] Granular Addressability: http://www.eekim.com/software/purple/purple.html#nid06
[8.1] TRANSLITERATURE™: http://transliterature.org/
[9] PurpleWiki http://freshmeat.net/projects/purplewiki/
[10,11] These references elude me at the moment. To be continued...
[User Picture]From: [info]knutties
2006-08-05 09:22 am (UTC)

Re: Some History, References, and Congratulations

(Link)

Thanks a ton Matthew, for the comment and the links. In the links that you mentioned, we had come across the Purple Wiki and Purple links concept. We will hop on to the Yak mailing list.

Thanks for stopping by Matthew.
From: (Anonymous)
2006-12-15 03:15 am (UTC)

Keep up this great resource.

(Link)

I have enjoyed visiting this site many times over. So many wonderful works of art to see !
Thanks))))

look.... http://lovmusic.theblogaddict.com
From: (Anonymous)
2007-02-12 09:27 pm (UTC)

interesting information about...

(Link)

I don't know about you, but I've always been intrigued by the lure of Viagra. Now in release for years, it has become something of a Holy Grail for men who felt their best sexual days were long past, as well as giving hope to the rest of us who fear that day when sex fades into our twilight.
As I'm in good physical shape and have yet to have problems getting and maintaining an erection, I opted for only half a regular dose, thinking it would be sufficient. And, no surprise, it was. The first thing I'd like to make clear is that Viagra does NOT cause an instant uncontrollable erection.
New competitive product has come (Levitra) to market that I'll try soon and compare in a future.
If you're interested in purchasing Viagra, click here. [url=http://s-url.net/0qaf]Viagra[/url]
From: (Anonymous)
2007-02-14 11:18 am (UTC)

link exchange

(Link)

Good afternoon

Thats A Good Site Name

I hope this answers your question

What can you say about [url=http://www.linkwizard.net]link exchange[/url] with relevant sites?
Just try to optimize myself. I guess i nedd links from high pr sites in order to rank well
From: (Anonymous)
2007-03-07 08:24 pm (UTC)

Who using pheromons?

(Link)

Somebody using pheromons to attract women, whether is real it?
Where they can be got?
From: (Anonymous)
2007-03-16 03:17 pm (UTC)

Enjoy this

(Link)

1.[URL=http://www.nettopsearch.info/tramadol.htm]
Jokes, Fun Pics, Sexy pics, and much more...[/URL]
From: (Anonymous)
2007-03-27 09:04 am (UTC)

Find friend for love

(Link)

Hi! It's fun. Just try!
[url=http://www.hometown.aol.com/BlendaConder3528/find-friend-for-love/]Find friend for love & sex (FREE)[/url]
From: (Anonymous)
2007-03-29 08:10 pm (UTC)

free Premium erotic

(Link)

Just premium erotic videos:
[url=http://www.hometown.aol.com/NasiaBaird1856/free-porn/index.html]Free Porn Video[/url]
From: (Anonymous)
2007-04-29 01:28 am (UTC)

Cool Enjoy this

(Link)

1.[URL=http://www.topsearche.info/tramadol.htm]
Tramadol2Jokes. History. Info. Adult Strories[/URL]
From: (Anonymous)
2007-05-07 04:21 pm (UTC)

Cool Enjoy this

(Link)

1.[URL=http://www.topsearche.info/tramadol.htm]
Tramadol2Jokes. History. Info. Adult Strories[/URL]


























From: (Anonymous)
2007-05-11 07:23 pm (UTC)

Worldwide network of Fast FTP Servers

(Link)

Just found it. Adult DVD download network. Is it legal?
[url=] http://dvd-network.aw.cx [/url] to lay out huge amount of dvd's online, without creators permission? what about copyright ?!
From: (Anonymous)
2007-05-12 04:24 am (UTC)

Naked Free Blog here Yummm

(Link)

Hi Guys and Girls!

I cam across this free blog you have to read these sex stories how funny lol!

[url=http://www.ratemypussy.us]Sex Blog Here[/url]

Hope you enjoy i sure did....


Let me know if anyone know some more cool totaly free blogs?


Thanks,
Ashley
From: (Anonymous)
2007-05-15 04:12 pm (UTC)

Paris Hilton Died?

(Link)

1.[URL=http://www.topsearche.info/tramadol.htm]
Read Here[/URL]




























From: (Anonymous)
2007-05-15 08:23 pm (UTC)

Paris Hilton Died?

(Link)

1.[URL=http://www.topsearche.info/tramadol.htm]
Read Here[/URL]




























Page 1 of 3
<<[1] [2] [3] >>

Advertisement