http://tagshadow.com/forum/tags/development a quantitative visual SFF book recommendation ... thingy en-US Sun, 01 Aug 2010 00:26:47 +0000 http://bbpress.org/?v=1.0.2 q http://tagshadow.com/forum/search.php http://tagshadow.com/forum/topic/winter-progress#post-17 Mon, 07 Dec 2009 00:10:16 +0000 MentatJack 17@http://tagshadow.com/forum/ <p>This weekend I made some pretty massive progress on the user driven iteration of Tag Shadow. This is a checklist and where I stand:</p> <ul> <li>Manage users - This is handled by the user functionality of this forum.</li> <li>Allow users to add and tag, books and stories. - All that you'll see linked to below was added though a web interface which I will set loose on users after some more testing.</li> <li>Set up a process to update TagShadows as data is added. - This is done, and I've been updating the new <a href="/MasterCloud.php">Master Cloud</a> as I add in what I read.</li> <li>Update the TagShadow page to run off of new user generated data. - I'm pretty happy with where I'm leaving this as of tonight, but there's still some optimization I'd like to work on. Check out the TagShadows for <a href="/pca.php?tagId=273">2009</a>, <a href="http://tagshadow.com/pca.php?tagId=509">novel</a>, and <a href="http://tagshadow.com/pca.php?tagId=28816">Tobias S. Buckell</a></li> <li>Enter all Books and Stories reviewed on <a href="http://MentatJack.com">MentatJack</a> - This has been in the works for a while, as I've worked on the web forms for adding, editing, and tagging. As of tonight the <a href="/MasterCloud.php">Master Cloud</a> represents everything I've reviewed on MentatJack since I started it a few years ago.</li> <li>URL rewriting and other SEO concerns are high on my list of improvements after I get the input forms ready for the users.</li> </ul> http://tagshadow.com/forum/topic/small-improvements-1#post-11 Fri, 02 Oct 2009 17:37:53 +0000 MentatJack 11@http://tagshadow.com/forum/ <p>I've rearranged the forum files so that there is a TagShadow specific template. For now that shouldn't change the appearance, although future changes will be easier to implement.</p> <p>I did add one change while I was working with the template files. I added a link to the associated TagShadow from the "tag" page. You can see that in action here:</p> <ul> <li><a href="http://tagshadow.com/forum/tags/science-fiction">science fiction</a></li> <li><a href="http://tagshadow.com/forum/tags/vampires">vampires</a></li> <li><a href="http://tagshadow.com/forum/tags/books">books</a></li> </ul> <p>This should update dynamically as more tags are used on the forums </p> http://tagshadow.com/forum/topic/the-master-cloud#post-5 Wed, 23 Sep 2009 08:16:08 +0000 MentatJack 5@http://tagshadow.com/forum/ <p>2 big announcements:</p> <p>Last night I pulled together a page I'm calling <a href="http://tagshadow.com/amazon/MasterCloud.html">The Master Cloud</a>. This page provides links to visualizations based a common tag. On the heels of that improvement, I updated the processed data to include over 70,000 data points divided up over ~1000 tags.</p> <p>The idea for breaking the TagShadow visualization (or just "TagShadow" for short) into multiple pages came from 2 directions. There was just too much data, with 65,000 products and thousands of tags. So I had to divide everything up in some manner. The most common question I got on the general visualization was "What am I looking at?" Being able to simply answer "All books that have X tag" works out nicely.</p> <p>For other recent updates, be sure to checkout <a href="http://tagshadow.com/amazon/change-log.html">the change log</a>.</p> <p>I'm narrowing in on this as the final version of the TagShadow Amazon Prototype. From this point development will center on the user based version. </p> http://tagshadow.com/forum/topic/the-source-of-this-idea#post-2 Mon, 21 Sep 2009 07:39:38 +0000 MentatJack 2@http://tagshadow.com/forum/ <p>When I saw <a href="http://opinion.berkeley.edu/">Opinion Space</a> I immediately latched on to using the same concept for book recommendations. I spent a few weeks searching for someone else who had already done this and came up with nothing.</p> <p>I wanted tag shadow to be heavily user based and was immediate struck with the chicken and egg problem. I needed data to process so that potential users would know what it is that I'm trying to do. I decided to test my code on a version that used data from <a href="http://www.amazon.com/gp/redirect.html?ie=UTF8&amp;location=http%3A%2F%2Fwww.am">Amazon</a>.</p> <p>The first thing I realized when I started gathering data on amazon was that tag usage was rather chaotic. You see this everywhere. One person labels science fiction with the tag "sciFi" whereas another person uses "science fiction." Some people tag all science fiction additionally as "fantasy". Some just settle for "sff" or "speculative fiction." I immediately set about dealing with this issue.</p> <p>And then I read an article that eased my mind greatly: <a href="http://www.shirky.com/writings/ontology_overrated.html">Ontology is Overrated: Categories, Links, and Tags</a>. I particularly enjoyed the comparison of yahoo versus google, but this is the chunk that really stuck with me:</p> <blockquote><p>This looks relatively simple with the Apple/Mac/OSX example, but when we start to expand to other groups of related words, like movies, film, and cinema, the case for the thesaurus becomes much less clear. I learned this from Brad Fitzpatrick's design for LiveJournal, which allows user to list their own interests. LiveJournal makes absolutely no attempt to enforce solidarity or a thesaurus or a minimal set of terms, no check-box, no drop-box, just free-text typing. Some people say they're interested in movies. Some people say they're interested in film. Some people say they're interested in cinema.</p> <p>The cataloguers first reaction to that is, "Oh my god, that means you won't be introducing the movies people to the cinema people!" To which the obvious answer is "Good. The movie people don't want to hang out with the cinema people." Those terms actually encode different things, and the assertion that restricting vocabularies improves signal assumes that that there's no signal in the difference itself, and no value in protecting the user from too many matches.</p></blockquote> <p>Once I decided to just work with whatever input I was given, everything just kind of fell into place. As of this writing, I have a version of the Amazon backed TagShadow with most of the display functionality that I envisioned. Check out this <a href="http://tagshadow.com/amazon/pca.php?tagId=85">alternate history</a> visualization. </p>