Mapping the Web
(these notes were written around 1996-97)
I want a program that will map the web in a way that lets you fly around, getting an understanding of the relationships between sites. I drew some clickable 2D maps by hand with pencil, scanned them in and added hyperlinks. They give some feeling for what the more linked sites are surrounding my site. With a map like this, a websurfer could jump ahead by several links in any direction without waiting for the intermediate pages to load.
When i revised the map to shows many more links, it started to suffer from information overload.The links get crowded and tangled; it's hard to see the immediate neighborhood for each site.
In the public domain and commercial software worlds, the closest thing i've found to a 3D web mapping program was WebSight from Fourth Planet - "a tool which utilizes advanced visualization technology to effectively monitor and manage access patterns within your web site in a manner that is visually intuitive." Unfortunately, it was under development but not very far along - it's no longer mentioned on their website. It's not clear whether they would support true webs (as opposed to simple heirarchies). One nice approach they took is using small literal images to represent pages, as i did on one of my early page designs.
Also noteworthy was a product called WebMapper from NetCarta. (They got bought by Microsoft and the product got renamed "Site Analyst", and buried in Microsoft's evil "Site Server"). It has a slick "Cyberbolic" 2D view - unfortunately, it's really just a simple top-down hierarchical graph which doesn't show the shape (topology) of the web at all. One useful idea learned from WebMapper is: it may be beneficial to have the map change dynamically depending on which site you're focused on. This could even be extended to a multi-site selection - select a few sites and the map arranges itself to highlight connections between those sites in the clearest possible way.
I've stopped trying to keep track of work in this area, because it's maintained wonderfully by the Atlas of Cyberspaces. Visit it today!
For creating and maintaining 2D maps, i could do it by hand if i had a program for drawing and editing 2D diagrams. These are the options i've found:
Perhaps using 3D might keep things from being tangled visually. I created a proof-of-concept 3D mapping program using World Up - initially, just arranging the objects and links in a 2D plane. Here is what my 3D map looks like on a 2D plane. Since i've only implemented straight lines for links, it's not terribly pretty, but i hope to get around to improving it soon.
Next step: Exploit the 3D medium by raising and lowering sites. This could both de-tangle the connections, and let the user gain an intuitive grasp of the topology of individual sites. This is only useful in a truly realtime 3D system, because from each point of view some information would be obscured - the user needs to move about smoothly to keep the spatial structure freshly imprinted in their head.
Practical consideration: On an unaccelerated P133, i got only 2-3 fps with the example map, so 3D acceleration hardware is gonna be required.
In the future, the program could do the initial
mapping, instead of making the user surf manually and record the links they find, as it
does. This would require learning about some stuff like URLStream and Asynchronous
Monikers. Parsing the HTML should be the easy part since all you need to extract is the