http://tagshadow.com/forum/tags/amazon a quantitative visual SFF book recommendation ... thingy en-US Sun, 01 Aug 2010 00:18:37 +0000 http://bbpress.org/?v=1.0.2 q http://tagshadow.com/forum/search.php http://tagshadow.com/forum/topic/the-master-cloud#post-5 Wed, 23 Sep 2009 08:16:08 +0000 MentatJack 5@http://tagshadow.com/forum/ <p>2 big announcements:</p> <p>Last night I pulled together a page I'm calling <a href="http://tagshadow.com/amazon/MasterCloud.html">The Master Cloud</a>. This page provides links to visualizations based a common tag. On the heels of that improvement, I updated the processed data to include over 70,000 data points divided up over ~1000 tags.</p> <p>The idea for breaking the TagShadow visualization (or just "TagShadow" for short) into multiple pages came from 2 directions. There was just too much data, with 65,000 products and thousands of tags. So I had to divide everything up in some manner. The most common question I got on the general visualization was "What am I looking at?" Being able to simply answer "All books that have X tag" works out nicely.</p> <p>For other recent updates, be sure to checkout <a href="http://tagshadow.com/amazon/change-log.html">the change log</a>.</p> <p>I'm narrowing in on this as the final version of the TagShadow Amazon Prototype. From this point development will center on the user based version. </p> http://tagshadow.com/forum/topic/the-source-of-this-idea#post-2 Mon, 21 Sep 2009 07:39:38 +0000 MentatJack 2@http://tagshadow.com/forum/ <p>When I saw <a href="http://opinion.berkeley.edu/">Opinion Space</a> I immediately latched on to using the same concept for book recommendations. I spent a few weeks searching for someone else who had already done this and came up with nothing.</p> <p>I wanted tag shadow to be heavily user based and was immediate struck with the chicken and egg problem. I needed data to process so that potential users would know what it is that I'm trying to do. I decided to test my code on a version that used data from <a href="http://www.amazon.com/gp/redirect.html?ie=UTF8&amp;location=http%3A%2F%2Fwww.am">Amazon</a>.</p> <p>The first thing I realized when I started gathering data on amazon was that tag usage was rather chaotic. You see this everywhere. One person labels science fiction with the tag "sciFi" whereas another person uses "science fiction." Some people tag all science fiction additionally as "fantasy". Some just settle for "sff" or "speculative fiction." I immediately set about dealing with this issue.</p> <p>And then I read an article that eased my mind greatly: <a href="http://www.shirky.com/writings/ontology_overrated.html">Ontology is Overrated: Categories, Links, and Tags</a>. I particularly enjoyed the comparison of yahoo versus google, but this is the chunk that really stuck with me:</p> <blockquote><p>This looks relatively simple with the Apple/Mac/OSX example, but when we start to expand to other groups of related words, like movies, film, and cinema, the case for the thesaurus becomes much less clear. I learned this from Brad Fitzpatrick's design for LiveJournal, which allows user to list their own interests. LiveJournal makes absolutely no attempt to enforce solidarity or a thesaurus or a minimal set of terms, no check-box, no drop-box, just free-text typing. Some people say they're interested in movies. Some people say they're interested in film. Some people say they're interested in cinema.</p> <p>The cataloguers first reaction to that is, "Oh my god, that means you won't be introducing the movies people to the cinema people!" To which the obvious answer is "Good. The movie people don't want to hang out with the cinema people." Those terms actually encode different things, and the assertion that restricting vocabularies improves signal assumes that that there's no signal in the difference itself, and no value in protecting the user from too many matches.</p></blockquote> <p>Once I decided to just work with whatever input I was given, everything just kind of fell into place. As of this writing, I have a version of the Amazon backed TagShadow with most of the display functionality that I envisioned. Check out this <a href="http://tagshadow.com/amazon/pca.php?tagId=85">alternate history</a> visualization. </p>