TagShadow Forum » Recent Topics

TagShadow Forum » Recent Topics http://tagshadow.com/forum/ a quantitative visual SFF book recommendation ... thingy en-US Thu, 18 Mar 2010 12:28:31 +0000 http://bbpress.org/?v=1.0.2 <![CDATA[Search]]> q http://tagshadow.com/forum/search.php MentatJack on "Winter Progress" http://tagshadow.com/forum/topic/winter-progress#post-17 Mon, 07 Dec 2009 00:10:16 +0000 MentatJack 17@http://tagshadow.com/forum/ This weekend I made some pretty massive progress on the user driven iteration of Tag Shadow. This is a checklist and where I stand: <ul> <li>Manage users - This is handled by the user functionality of this forum.</li> <li>Allow users to add and tag, books and stories. - All that you'll see linked to below was added though a web interface which I will set loose on users after some more testing.</li> <li>Set up a process to update TagShadows as data is added. - This is done, and I've been updating the new <a href="/MasterCloud.php">Master Cloud</a> as I add in what I read.</li> <li>Update the TagShadow page to run off of new user generated data. - I'm pretty happy with where I'm leaving this as of tonight, but there's still some optimization I'd like to work on. Check out the TagShadows for <a href="/pca.php?tagId=273">2009</a>, <a href="http://tagshadow.com/pca.php?tagId=509">novel</a>, and <a href="http://tagshadow.com/pca.php?tagId=28816">Tobias S. Buckell</a></li> <li>Enter all Books and Stories reviewed on <a href="http://MentatJack.com">MentatJack</a> - This has been in the works for a while, as I've worked on the web forms for adding, editing, and tagging. As of tonight the <a href="/MasterCloud.php">Master Cloud</a> represents everything I've reviewed on MentatJack since I started it a few years ago.</li> <li>URL rewriting and other SEO concerns are high on my list of improvements after I get the input forms ready for the users.</li> </ul> MentatJack on "The lesser used tenses" http://tagshadow.com/forum/topic/the-lesser-used-tenses#post-3 Tue, 22 Sep 2009 08:56:29 +0000 MentatJack 3@http://tagshadow.com/forum/ I find it most important to record the things that distinguish a small subset of books from the larger collection. A great example is tense. The overwhelming majority of fiction is told in past tense. This makes sense as the reader is inevitably reading a book that was published in the past, written in the past and conceived in the past. The beginning writer that doesn't know this convention can often be identified as such even if their tense usage is consistent and their prose otherwise adequate. However, there are some great examples of the present tense used well, and I make it a habit to tag these books as such whenever the opportunity presents itself. This is my list, and I'd love to find more examples. <ul> <li><a href="http://www.amazon.com/gp/product/0553380958?ie=UTF8&tag=tagshadow-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=0553380958">Snow Crash</a> by Neal Stephenson</li> <li><a href="http://www.amazon.com/gp/product/0553591568?ie=UTF8&tag=tagshadow-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=0553591568">The Mirrored Heavens</a> by David J. Williams</li> <li><a href="http://www.amazon.com/gp/product/0553385429?ie=UTF8&tag=tagshadow-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=0553385429">The Burning Skies</a> by David J. Williams</li> <li><a href="http://www.amazon.com/gp/product/0307454355?ie=UTF8&tag=tagshadow-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=0307454355">Jack Wakes Up</a> by Seth Harwood</li> </ul> As the title implies, I'm sure someone out there has gotten a book published in something other than present or past. I'd suspect that a work translated into English might be the place to look for such a beast. Surprise me! MentatJack on "Guess the work" http://tagshadow.com/forum/topic/guess-the-work#post-16 Wed, 11 Nov 2009 08:16:43 +0000 MentatJack 16@http://tagshadow.com/forum/ Everything I read seems to keep inspiring me to return to this project. Instead of just saying what's inspired me recently I'll Just list the tags I'd use to describe it and see if you can figure it out. In no particular order: <ul> <li>street urchin</li> <li>biological architecture</li> <li>reincarnation</li> <li>augmented vision</li> <li>memory upload</li> </ul> I'll give you the hint that this is a story that I read in a single author collection. MentatJack on "Classification and Spoilers" http://tagshadow.com/forum/topic/classification-and-spoilers#post-14 Fri, 09 Oct 2009 08:42:11 +0000 MentatJack 14@http://tagshadow.com/forum/ Spoilers have always annoyed me. Both when reading and when writing reviews of books, the line between knowing what a book is about and knowing enough to not even need to read the book can be rather thin. This happens quite a bit when you tag things. The main categorization of a book is usually safe. Tagging <a href="http://www.amazon.com/gp/product/076534825X?ie=UTF8&tag=tagshadow-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=076534825X">Spin</a> by Robert Charles Wilson science fiction is in no way a spoiler, but tagging it <a href="http://en.wikipedia.org/wiki/Biological_uplift">uplift</a> probably is. It's a perfect tag, but that aspect of the story appears rather late in the timeline and might thus be considered a spoiler. I can think of other books where tagging them science fiction instead of fantasy could be considered spoilers. The original <a href="http://tagshadow.com/amazon/pca.php?tagId=4326">Pern</a> trilogy comes to mind. There are plenty of examples where the question of science vs magic or alien vs religion is the central mystery. It strikes me that one benefit of the TagShadow method of recommendation (browsing through clusters of similar books) to an extent protects the reader from the potential spoilers contained in a tag cloud about the book. MentatJack on "Favorite TagShadow" http://tagshadow.com/forum/topic/favorite-tagshadow#post-13 Tue, 06 Oct 2009 17:46:07 +0000 MentatJack 13@http://tagshadow.com/forum/ I'm still exploring all the prototype TagShadows linked from the <a href="http://tagshadow.com/amazon/MasterCloud.html">Master Cloud</a>. Occasionally I run across one that particularly jumps out at me. One with interesting structure and a coherent collection of books. One that has neither too many nor too few books. The last one I found that I really like is <a href="http://tagshadow.com/amazon/pca.php?tagId=4049">first contact</a>. It looks a bit sparse till you zoom in on the cluster near the top right. What are your favorites? others: <ul> <li><a href="http://tagshadow.com/amazon/pca.php?tagId=372">apocalypse</a></li> </ul> MentatJack on "Small Improvements" http://tagshadow.com/forum/topic/small-improvements-1#post-11 Fri, 02 Oct 2009 17:37:53 +0000 MentatJack 11@http://tagshadow.com/forum/ I've rearranged the forum files so that there is a TagShadow specific template. For now that shouldn't change the appearance, although future changes will be easier to implement. I did add one change while I was working with the template files. I added a link to the associated TagShadow from the "tag" page. You can see that in action here: <ul> <li><a href="http://tagshadow.com/forum/tags/science-fiction">science fiction</a></li> <li><a href="http://tagshadow.com/forum/tags/vampires">vampires</a></li> <li><a href="http://tagshadow.com/forum/tags/books">books</a></li> </ul> This should update dynamically as more tags are used on the forums MentatJack on "Bookseller Affiliate Programs" http://tagshadow.com/forum/topic/bookseller-affiliate-programs#post-9 Fri, 02 Oct 2009 08:55:05 +0000 MentatJack 9@http://tagshadow.com/forum/ Every dollar spent on books does numerous things. It lets us as a consumers speak to what interests us. It supports the bookseller, publisher, author and everyone who depends on these industries for their livelihood. By placing ourselves into the middle of this transaction, we're able to influence the books, authors and genres we love. We also have a choice to make as to where we point our users. There are MANY factors to consider, but an option many settle on is to link somewhere that provides a commission. I do this and it helps offset the costs associated with the services I'm developing. Amazon is the elephant in the room and they have a great affiliate program, but there are plenty of others out there. This list is by NO means comprehensive. Of the options below I've had the most experience with amazon and mysterious galaxy. <ul> <li><a href="https://affiliate-program.amazon.com/gp/associates/join/landing/main.html">amazon</a></li> <li><a href="http://www.barnesandnoble.com/affiliate/index.asp?">Barnes and Nobel</a></li> <li><a href="http://www.borders.com/online/store/BGIView_affiliateprogram">Borders</a></li> <li><a href="http://mysteriousgalaxy.booksense.com/NASApp/store/IndexJsp;jsessionid=bacrQNlipZwbEeeBC6sqs?s=becomeaffiliate">Mysterious Galaxy</a></li> <li><a href="http://powells.com/partners/partners.html">Powells</a></li> <li><a href="http://www.abebooks.com/books/AffiliateProgram/">Abe Books</a></li> <li><a href="http://www.booksinc.net/affiliate">Books Inc</a></li> <li><a href="http://www.betterworldbooks.com/Custom.aspx?f=affiliate">Better World books</a></li> <li><a href="http://www.alibris.com/affiliates/home">Alibris</a></li> <li><a href="http://www.biblio.com/affiliate_program/">Biblio</a></li> <li><a href="http://ebooks.com/information/affiliates.asp">eBooks.com</a></li> <li><a href="http://www.buzzymultimedia.com/afpr.html">Buzzy Multimedia</a></li> <li><a href="http://www.overstock.com/20789/static.html?TRACK=FOOT_OI_L6">Overstock</a></li> <li><a href="http://affiliates.ebay.com/odcs/custom.htm?template=Payout">eBay</a> - I'd use this to link to things like: <ul> <li><a href="http://shop.ebay.com/subpress1/m.html">Subterranean Press</a></li> </ul> </li> <li><a href="http://www.google.com/ads/affiliatenetwork/">Google</a></li> </ul> I continue to collect these links on <a href="http://delicious.com/MentatJack/affiliate">delicious</a>. If you run across any good suggestions I've missed, be sure to list them here. Also, as I implement various of these for TagShadow, I'll be sure to share my experiences. The sites that don't specialize in books or collect multiple book sellers may actually provide a way for you to support a very specific bookseller that doesn't have an online presence outside of such aggregators. MentatJack on "TagShadow for blogs" http://tagshadow.com/forum/topic/tagshadow-for-blogs#post-8 Wed, 30 Sep 2009 06:27:30 +0000 MentatJack 8@http://tagshadow.com/forum/ One of the new features I have planned is to apply this analysis to user's blogs. It'll be similar to the standard TagShadow, except that instead of books I'll be plotting blog posts. Hopefully this will be a general service with widgets and such in the future, but the initial iteration will be a rather manual process. I'll be testing with <a href="http://MentatJack.com">My Blog</a>, but if you have a SFF related blog ( and you tag your posts ) link to it below. I'll generate plots for the first dozen and see how this works. MentatJack on "TagShadow - A simple explanation" http://tagshadow.com/forum/topic/tagshadow-a-simple-explanation-1#post-7 Thu, 24 Sep 2009 20:11:53 +0000 MentatJack 7@http://tagshadow.com/forum/ Tag shadow is a an intersection of art and geekiness. The goal is to visualize a group of books into beautiful and unusual groupings. The following description is a work in progress and I'd very much like input on how clearly this explains the concepts. Two Dimensions To understand what's going on, think about 2 of your favorite things. I'm going to start with science fiction and vampires. One way to organize all your books about Science Fiction and vampires would be to place the science fiction on one bookshelf and the vampires on another bookshelf. Obviously the problem is that some books are about both science fiction and vampires to one degree or another. Lets put all of those books on one book shelf together. Place the books that are about MOST about vampires on the top shelf and as you move down the shelves add the books that are less and less about vampires. Now look at each of these shelves individually. From left to right, order the books by how much they feel like science fiction. Once you're done you have a rather odd book shelf, but it serves one purpose perfectly. It helps you find exactly the mix of science fiction and vampires that you're in the mood for at a given point. And if a friend comes over, they can browse around until they find a book they've read and liked and it's likely they'll like the books shelved near by. Three Dimensions Now lets assume you're organizing a convention for aficionados of science fiction, monsters, and magic. In the dealer room you decide to arrange the tables such that the science fiction is more concentrated at the front and monsters are more concentrated to the left. So, if you're wanting to find a table with the a lot of science fiction but no monsters, you'd make your way to the front of the room and to the right. At every table we're going to organize stacks of books such that the more magic they have in them the higher they are in a stack. Some vendors might use a shelf for the same purpose, but the idea is the same. What we now have is a way to organize 3 ways of describing books using the 3 dimensions we're used to interacting with. Computer Screens Are Flat We use many different ways to visualize 3 dimensions on a flat display. Think about the previous example, but imaging it in a weightless environment where books could use the entire height of the room to describe how much magic they had in them. One way to understand in 2 dimensions what's happening in 3 dimensions would be to shine a bright light through the room of books and see what the SHADOW looked like on the wall. You'd have to move the light around a while until you found a direction which best captured the 3 dimensional groupings in the shadow. I'm not going to go into details here, but that's a problem that's got a fairly good mathematical solution. That solution isn't limited to 3 dimensions. In fact, we can figure out a 2 dimensional plot derived from hundreds (or even thousands) of descriptive tags. Tags + Shadow I decided to call the groupings of books that I derive from all the tag information a TagShadow. Hopefully you have a general idea what I'm trying to accomplish at this point. Click on tags, hover over points and just explore. MentatJack on "The Master Cloud" http://tagshadow.com/forum/topic/the-master-cloud#post-5 Wed, 23 Sep 2009 08:16:08 +0000 MentatJack 5@http://tagshadow.com/forum/ 2 big announcements: Last night I pulled together a page I'm calling <a href="http://tagshadow.com/amazon/MasterCloud.html">The Master Cloud</a>. This page provides links to visualizations based a common tag. On the heels of that improvement, I updated the processed data to include over 70,000 data points divided up over ~1000 tags. The idea for breaking the TagShadow visualization (or just "TagShadow" for short) into multiple pages came from 2 directions. There was just too much data, with 65,000 products and thousands of tags. So I had to divide everything up in some manner. The most common question I got on the general visualization was "What am I looking at?" Being able to simply answer "All books that have X tag" works out nicely. For other recent updates, be sure to checkout <a href="http://tagshadow.com/amazon/change-log.html">the change log</a>. I'm narrowing in on this as the final version of the TagShadow Amazon Prototype. From this point development will center on the user based version. MentatJack on "The creative team" http://tagshadow.com/forum/topic/the-creative-team#post-4 Tue, 22 Sep 2009 12:37:14 +0000 MentatJack 4@http://tagshadow.com/forum/ When I think about the tags I might use to describe a book, the first that comes to mind is author. There are plenty of books, such as anthologies, that feature many writers and prominently mention the editor. Even with books by a single author there are plenty of other creative people associated with the publication. Here are a few that come immediately to mind. <ul> <li>agent</li> <li>editor</li> <li>cover artist</li> </ul> All of these provide meaningful groupings that are not necessarily just groupings of authors. MentatJack on "The source of this idea" http://tagshadow.com/forum/topic/the-source-of-this-idea#post-2 Mon, 21 Sep 2009 07:39:38 +0000 MentatJack 2@http://tagshadow.com/forum/ When I saw <a href="http://opinion.berkeley.edu/">Opinion Space</a> I immediately latched on to using the same concept for book recommendations. I spent a few weeks searching for someone else who had already done this and came up with nothing. I wanted tag shadow to be heavily user based and was immediate struck with the chicken and egg problem. I needed data to process so that potential users would know what it is that I'm trying to do. I decided to test my code on a version that used data from <a href="http://www.amazon.com/gp/redirect.html?ie=UTF8&location=http%3A%2F%2Fwww.am">Amazon</a>. The first thing I realized when I started gathering data on amazon was that tag usage was rather chaotic. You see this everywhere. One person labels science fiction with the tag "sciFi" whereas another person uses "science fiction." Some people tag all science fiction additionally as "fantasy". Some just settle for "sff" or "speculative fiction." I immediately set about dealing with this issue. And then I read an article that eased my mind greatly: <a href="http://www.shirky.com/writings/ontology_overrated.html">Ontology is Overrated: Categories, Links, and Tags</a>. I particularly enjoyed the comparison of yahoo versus google, but this is the chunk that really stuck with me: <blockquote>This looks relatively simple with the Apple/Mac/OSX example, but when we start to expand to other groups of related words, like movies, film, and cinema, the case for the thesaurus becomes much less clear. I learned this from Brad Fitzpatrick's design for LiveJournal, which allows user to list their own interests. LiveJournal makes absolutely no attempt to enforce solidarity or a thesaurus or a minimal set of terms, no check-box, no drop-box, just free-text typing. Some people say they're interested in movies. Some people say they're interested in film. Some people say they're interested in cinema. The cataloguers first reaction to that is, "Oh my god, that means you won't be introducing the movies people to the cinema people!" To which the obvious answer is "Good. The movie people don't want to hang out with the cinema people." Those terms actually encode different things, and the assertion that restricting vocabularies improves signal assumes that that there's no signal in the difference itself, and no value in protecting the user from too many matches.</blockquote> Once I decided to just work with whatever input I was given, everything just kind of fell into place. As of this writing, I have a version of the Amazon backed TagShadow with most of the display functionality that I envisioned. Check out this <a href="http://tagshadow.com/amazon/pca.php?tagId=85">alternate history</a> visualization.