[Alex Pepple]

There was some recent concern brought to my attention that the Google protection of the workshop threads was no longer working. I was alarmed and looked into it, and after exhaustive tests, I can assure one and all that the protection is working as well as it always did, and nothing has changed.

However, it seems that I need to again remind/clarify everyone about which threads are protected and how.

First, all the workshop forums are automatically protected (Metrical, TDE, Non-Met, Translation, and Fiction). When posting there, there is no need to enter any additional protection code to prevent Google from indexing your poem or fiction thread. However, if you're posting in any of the non-workshop forums that are not protected, there is still a way to protect what you don't want Google to see by using the following protection code:

[noindex][/noindex]

Note that no value is required between the tags. Simply enter it exactly as indicated above as the very first thing in your thread.

Note, however, that tests have shown that this protection code only works on the first thread-page. Thus, if your thread is popular and begins to span multiple pages and you want the extra pages protected as well, send a PM to the poster of the first reply in each of the subsequent pages with a request to edit his or her post and add the protection code as the first thing in that reply. Alternatively, you may also contact the forum moderator with this request. Again, note that you don't need to do anything for workshop forums as listed earlier -- these protection instructions apply only to threads in non-workshop forums that you feel the need to protect from Google indexing.

And now, for some additional instructions -- and this is nothing new since the Sphere has always functioned this way. Yes, the workshop forums are protected, however, Google has access to the main Eratosphere page (i.e. www.ablemuse.com/erato/). As any regular Eratosphere user knows by now, there's a column there entitled 'Last Post'. This column lists the title of the latest post that has bubbled up to the top of each forum, either because it's a new post or because it has just been commented on. In that brief to long timespan when the post's title appears on the main page, it's likely that a Google bot will pass by and capture it, which might result in a search later based on that title and poster name succeeding (but linking to the main page, NOT the actual thread). This might be the case even though that post title is no longer in the Last Post column of the main page, having been superseded by something else that has bubbled to the top in the forum where the post occurred.

I have to state again that all of the above are nothing new, and most Eratosphereans who workshop frequently are already aware of this.

So, what do you do if you can't even bear the thought of Google indexing the title of your poem when it briefly appears on the main page? By faking it! That is, when posting your poem/translation/story, give a faux title that makes no difference to you should it get indexed by Google. Essentially, you'll never submit the poem with that title, so editors searching with the real title will never find the fake one that Google might have indexed. Note though that you can still enter the real title in the body of your post, just before the rest of your poem, because that entire field will never be seen or indexed by Google

I may not be explaining myself well so, I’ll try to illustrate:

When creating a new thread, the fields that the system displays, and what should be entered in them are as follows:

TITLE: (enter a fake title here)

MESSAGE: (start with the real title, followed by the rest of the poem or story)

TAGS: (I can’t confirm it, but this field is most likely visible to the search engines, so refrain from entering any keywords that will cause your poem to be successfully searchable in Google).

That may be quite a lot of tedious explaining, but I hope everything is crystal clear to everyone now, and that there is no more issue as to how or where to protect the post you want to make non-indexable and unavailable to Google bots.

Cheers,
…Alex

[David Rosenthal]

Alex,

I not terribly bothered by it but my entry in the French forms event is utterly googleable even though the "noindex" code is entered in the top of the post. If I type the first line of the poem into google, I am delivered directly to the first page of the thread. As I said, it doesn't bother me because the poem has already been published and is not under consideration anywhere, but if it were...

I wonder if it would be trouble to protect DG the same way you protect the workshop threads since that is a forum in which we often post good work that might be under submission.

David R.

[Alex Pepple]

Hi David,

We're not applying a blanket protection to DG (and only protecting the actual event poems) because that's one of the draws of the Sphere, and blacking out from Google will drastically cut down on the quality and quantity of new membership requests.

However, I've just did a quick search, and no, it doesn't go into your poem thread which was indeed protected by Susan, but rather to the 'Thanks' thread where all the titles were listed without protection. That's why it's showing up. Anyhow, I've just added the protection code there as well so that it eventually rolls off the Google index.

Indeed, the protection for workshop and non-workshop forums works exactly as I've explained at length in the first post of this thread. I'll advise everyone to read it carefully so as not give me too many false positives to spend time testing unnecessarily.


Cheers,
...Alex

[David Rosenthal]

Alex,

I don't know what to say, but I just googled "the integers appear and disappear" and it went right to the poem thread, not the thanks thread. I see the code in Susan's post when I use the quote button, but the search definitely landed in the poem there.

Interestingly, googling the entire first line doesn't work. However, googling the title, "Birthday Villanelle," returns the poem thread as the fourth hit on google's list.

By the way, when I posted my revision on the second page, I included the code, and that page does not come up on the search.

Meanwhile, I get what you're saying about hiding DG, I bet a lot of interesting searches land there. In fact some such search is how I found Eratosphere, so I retract that ill-thought suggestion.

David R.

[Stephen Collington]

David, Alex, if I may . . .

The most likely reason, David, why your poem shows up on a Google search is that the "noindex" tags were not added with the original posting; they were edited in later by Susan, presumably after someone reminded her of the need for them. But in the meantime, some thirty-six hours had passed from posting to editing, and that's more than enough time for Google to come by with its bots. Once something is indexed, it will stay that way for some time; however, when Google returns to recrawl the pages (it does so, apparently, on a periodic basis) it should see the "noindex" tags, and note the page as protected. With that information in hand, Google will then cease updating the page info, and that should eventually lead to the page being eliminated from its cache. How long that will take, I'm not sure--when I investigated the issue a couple of years ago, the ballpark answer was "a couple of months." At any rate, that's how it's all supposed to work; your thread should, eventually, disappear.

If you ever find yourself in a situation where you must make a page disappear in a hurry, there are ways of doing so, apparently; anyone interested can read up on the matter here.

Anyway, I haven't checked all of them, but I tried a few search strings from later entries in the bake-off (examples edited out here) and nothing popped up on Google from the Sphere. (A couple of things appear elsewhere on the web, previously published in webzines, etc.--but that's not our problem.) In other words, once Susan started posting with the "noindex" tags in place from the start, Google stayed out.

The take-home message: "noindex" tags work, but you've got to have them in place from the get-go. Google is always around the corner, and its bots work fast.

.

p.s. I just realized that my own examples of search strings will show up on Google from this unprotected thread . . . which kinda kills the experiment. (Duh!) So I've edited them out. (If Google's managed to pick them up all the same, in the three or so minutes that they were "live" on the page here, then we'll know just how fast it really is.) Anyway, suffice it to say that I really did do some searches from later threads (#7 onwards) and none of them produced hits.

.

[Marcia Karp]

Originally Posted by Alex Pepple

Note, however, that tests have shown that this protection code only works on the first thread-page. Thus, if your thread is popular and begins to span multiple pages and you want the extra pages protected as well, send a PM to the poster of the first reply in each of the subsequent pages with a request to edit his or her post and add the protection code as the first thing in that reply.

Dear Alex,
   This doesn't make sense to me. A page of posts is an indeterminate, fluid measure. Each user can determine how many posts are on a page and can change the number at anytime.

   Is there a measure that is set internally?

Marcia

[W.F. Lantry]

Originally Posted by Marcia Karp

This doesn't make sense to me. A page of posts is an indeterminate, fluid measure.

Marcia,

If you log out, and then come back in, the way an anonymous spider would, you'll see the default is 10 pages. It's just the way the software works, and messing with it is likely not worth the trouble, especially when it comes to upgrades, patches, etc. And one should never underestimate community confusion...

Amusingly, a few months ago, I tried to give a poem a different title on the index page and in the thread, for precisely this reason. Boy, did I catch heck for it in the crits! Sometimes, one can't win for losing!

Thanks,

Bill

[Alex Pepple]

David - thanks for pointing out the exact search string, and I now see what you're referring to. Stephen is right about the point he made of having the noindex code right from the start. As much as possible, don't wait and edit them in later. I now remember that Susan wasn't sure when she started posting the finalist poems how to properly add the protection code, and it was only after I'd PM'd her the how-to that things fell in place.

Stephen - thanks for that clear explanation about the need to add the noindex code right from the beginning. I'll also add that Google bots can be deemed honorary Spherians because they really like Eratosphere, and honor us with several visits daily. So, waiting to add the protection code means that Google will most likely get to it before you get around to edit it in later.

Marcia - You'll notice that on popular posts that span several pages, the forum thread listings will show that thread title as a link, followed right on the same line by all the sub-page, also as links -- i.e. in the format, "Thread Title ( 1 2 3 ... Last Page)", where the thread title and the numbers to the sub-pages are clickable links. For the non-protected, non-workshop pages, the problem for a thread you want protected in its entirety is that each of these links will be captured by the search engine bot which will then iteratively index all the non-protected sub-pages. So, adding the link to just the initial page is not sufficient to protect the additional sub-pages, unless when it gets to that sub-page, there's the 'noindex' protection code in place that tells it not to index. Indeed, each thread, no matter how long, is conceptually one fluid unit as you point out. However, when the software splits them for display purposes into several sub-pages, including links to these sub-pages that are accessible to search engines, then, adding the protection code is necessary (as described in my initial post in this thread) for any sub-page you don't want indexed by Google. I hope that clarifies that segment you quoted.

Cheers,
...Alex

PS: Cross-posted with Bill

[Marcia Karp]

Bill,
I said nothing about messing with any code. Neither you nor Alex seem to understand: when you say page it means nothing except what a given user has determined a page on her screen will be (see "Edit Options"). And what do titles have to do with this?

Are you saying that Google requires the noindex tag on every post ending with a 1? That is clear. Is it so?

Marcia

[W.F. Lantry]

Marcia,

I understand your confusion, I did indeed mistype. What I should have said is "the default is ten posts per page."

You're right that it's fluid in the sense that you can change your individual options. But the default is what the spider sees. Only people with accounts can change their options, thus making the number 'fluid.' And for the most part, spiders don't have accounts.

Thus, it's pretty safe to assume that if you use the tag Alex suggested on post one, eleven, twenty-one, etc., you'll be spider proofing the thread. There are, sadly, never any perfect guarantees in computing, but this particular practice has held up for many years. When we get deep into HTML6, we may need to revisit this one.

Sorry for my inexact wording. I should have been clearer.

Thanks,

Bill