I'm happy to report on some of the changes that we're making at Wikia to improve our ranking on search engines. Some of our wikis show up very well in searches, but many don't, and that's really frustrating. People are working hard on building up amazing, detailed content -- but if it doesn't show up in the first page of search results, then it's hard for readers to know that the content exists!

Now, the basic rule for how to get a good search ranking is just to make a good site -- lots of pages with lots of interesting, unique content, which all link to each other in meaningful ways. So that's always the first step for our wikis -- if you want your search rank to go up, write more content!

MediaWiki issues

There are lots of wikis on Wikia that have thousands of pages of great content, but they're not showing up as high in search engine results as they should. That's because there are some quirks in the MediaWiki software that make it harder for search engines to index our wikis properly. We've been making some improvements to the software that resolve these issues.

(Before I get into specifics, just a quick sidebar: The obvious question is, if there are problems with MediaWiki, then why is Wikipedia the #1 result for practically everything? The answer is: We're not sure. Search engines may treat Wikipedia as an exception and index it differently. It's also possible that Wikipedia has so much search power at this point that it's not affected by the quirks in MediaWiki.)

Redirects

One issue that we're fixing is the way that MediaWiki treats redirects. On Wookieepedia, "Darth Vader" is a redirect to the Anakin Skywalker page. When you click on a link to Darth Vader, you get the Anakin Skywalker page... but you see the URL for the Darth Vader page. As you can see in the images, search engines index both URLs for the same page.

Having the same page show up as multiple URLs means that we're essentially competing against ourselves. One page shows up when you search for "darth vader"; the other when you search for "anakin skywalker". The links are split between the two pages, so it doesn't rank as high as it should. And when you think about the hundreds (or thousands!) of redirect pages on most wikis... that's a huge chunk of "search power" that we're losing.

So we've got a solution, which is a change to the way we handle redirects. We've been testing the new technique on a handful of wikis for six weeks, and it's worked great -- and now we're ready to turn it on site-wide. This change will only be visible for people reading the wiki in Monaco; people who use Monobook won't see any difference.

The change is that now, when you click on a link to a redirect page, you get sent to the actual URL instead of the redirect URL. So clicking on a link to "Darth Vader" will take you to wiki/Anakin_Skywalker. That means that search engine robots sees just one URL for each page, and we stop competing with ourselves. The "Redirected from Darth Vader" message now appears as a Javascript message which is served on the reader's computer -- so you see the link to the redirect page, but the search robots don't.

This change also means that if there's a double redirect, you'll go through to the correct page without getting the broken redirect page. You'll still get the broken link if there's a circular loop of redirects; otherwise, you'll just go to the correct page.

For the user, this is a really minimal change -- you'll see the correct URL instead of the redirect one, and you may notice a slight delay before the Redirect message pops up if things happen to be running slow. Otherwise, it works the same way that it always has. And the good news is, it makes a big difference in the search ranking. We tested the new redirects on Muppet Wiki, and found that all of the redirect URLs dropped out of Google's index within two weeks. Within four weeks, there was a noticeable improvement for every search term we looked at. We're excited to see what the impact will be once we turn this on site-wide.

There was an accidental site-wide release a few weeks ago, which we didn't intend -- but it was actually a great opportunity to spot a few bugs in the code, and fix them. I don't expect to see many problems with this when we turn it on as default -- but if you see any, please report it below and we'll take care of it. If it turns out that a particular community doesn't like the new redirects, it's also possible to turn the feature off by wiki -- but that means that wiki won't get the search benefits. Still, the option's there if anybody wants it.

Behind-the-scenes changes

We're also working on some back-end changes that should also help our rankings. We're adding a "noindex" tag to a lot of pages that we don't need to have indexed in search results -- like special pages, edit pages, templates and history pages. There are literally millions of those pages in Google's index right now, and cutting those out will help to lift all of the actual content and discussion pages up.

This is all behind-the-scenes stuff, so you won't notice any difference on your end... until you notice a whole bunch of new readers and contributors showing up a month from now!

So we're hoping that this will be a big step forward in helping raise the profile of all of the amazing wikis on Wikia. If you've got questions, comments or suggestions, please post them below! -- Danny (talk) 22:01, 17 September 2008 (UTC)

Discussion

Please post your thoughts here!

Javascript

Hello Danny, that very easy to see why some sites are not top.

The layout has to many Javascript. Disable Javascript on your browser and try to walk thought all pages. Additional remember, which pages are disabled by "noindex". So, if you disable Special:SpecialPages, or all the "Special:...", then "Special:AllPages" are also disabled.

But that is not all. Just use a text only browser (w3m, Lynx) and view pages: Very ugly! Many links are not working. I feel the complete sidebar menu is a "Javascript only" box, for example

• Top Content --> links back to "Home_Page"
• Editors pick, Most visited, ... --> links to {{SERVER}}/wiki/

Only sites with full working Categories and with enough links in wikitext can good indexed.

"write more content" is absolutely no guarantee to have the top place. I have a personally home page with 1 static entry page and round about 20 sub pages. I'm not often change any article. More is not better all times. Less is better with an interesting content.

For Mediawiki: They have Monobook as default skin, and they have "Printable version". Both are better for robots. —HenryNe 22:04, 18 September 2008 (UTC)

Please, dont confuse Javascript with Java, they're really not the same ! — TulipVorlax 13:36, 11 March 2009 (UTC)

Hi, Henry! I know that you've had lots of concerns about the amount of Javascript that we're using in the Monaco skin. In this case, I don't think that's affecting our search engine rankings too much -- this is a problem that predates Monaco. Monaco launched in March, and there were still a bunch of large wikis using Monobook as a default up through June. I've been tracking our search engine rankings for a year and a half, and our wikis have always struggled to get the ranking that seems warranted by the quality of the content.

There were some search engine problems early on when we switched to Monaco, but they seem to have been related to the number of links embedded in the skin. Adding "nofollow" to those extra links has helped a lot with that. We're working on tweaking the nofollow piece, and we may also try moving some of the code around to make the text come up higher.

You're absolutely right when you say that sites with working categories and links in wikitext are the ones that will be indexed well. From the wiki creator's point of view, the best thing you can do to get a good search engine ranking is to make a good wiki. That's why I said above that people should write "lots of pages with lots of interesting, unique content, which all link to each other in meaningful ways."

The problem has been that there are a lot of wikis on Wikia that have been doing exactly that, but they're still ranking lower than they should, and they have been for a long time. So we're working on the technical fixes that should help everyone get more attention for the content that they write.

By the way, for the folks who are interested in this and may be checking your rankings to see what happens -- a little note about what we're expecting to see. We're going to be rolling out these changes over the next month or so -- we turned on the redirects as default this week, and we'll be turning on the back-end stuff one at a time. When you make a change that can affect search ranking, it's very common to see a little dip in the ranking at first, as the system notices the change. If the change works, then you'll see it rise back up, and settle higher than it was before. When we tested the new redirects on Muppet Wiki, it took a month before we saw a consistent rise.

So if you notice a temporary drop in your wiki's ranking, don't panic too much! Keep an eye on it for a little while, and (if all goes well), that drop will be followed by an improvement. I'm working on tracking how we're doing -- please report anything interesting that you notice here! -- Danny (talk) 20:26, 20 September 2008 (UTC)

Non contents pages

I'm really glad to read that templates and all will be "removed" from search engines. same for others non content pages. I think it will really help. — TulipVorlax 13:40, 11 March 2009 (UTC)