Duplicate Content

About a year ago, there was a big debate in the forum world about duplicate content. Did your archive count as duplicate content? Was each thread view considered a duplicate? Nobody really knew at the time but we had our suspicions. I posted at TheAdminZone suggesting that forum owners add any duplicate pages to robots.txt. I have had good luck with this technique and Google recently confirmed this suggestion. Google says they will try to index and display the “best” page, but if you want to avoid having the wrong page displayed in the SERPS, just block it in robots.txt. I personally don’t want google displaying weird thread views or my archived pages as they don’t convert nearly as well as the full, original page.

Laura Atler commented on this at ForumTrends and quotes (sort of) Aaron Wall of SEOBook.com saying he also suggests choosing one version and blocking the rest. So go forth and robots.txt those duplicate pages. Keep Google happy and maybe they’ll return the favor.