MSN is working on a Web crawler, known as the MSNBot, that it will likely make a part of its next MSN search engine.

A growing number of Web sites are reporting sightings of the MSNBot as it crawls their Web sites.

"Looks like the competition will shortly be h(ea)ting up in the search engine world," says one poster, Martin Belam. "Microsoft have started sending their own MSNBot out scouring the web to build a new index for MSN search."

Microsoft has created a Web page, "MSN Search Prototype Web Crawler" to a number of questions about its bot.

"MSNBOT is still in the development and research stage," notes Microsoft on its site.

"This crawl is a prototype work. MSNBOT is not currently indexing for the MSN Search Engine, so your site may or may not show up in MSN Search results today," according to the site. "Although we have not set a date, it is our intention to eventually integrate the crawled contents into MSN Search results."

A number of industry watchers have been expecting Microsoft to build its own bot since Yahoo purchased Inktomi, one of Microsoft's OEM search partners, earlier this year.

"With Yahoo grabbing Inktomi from out under MSN, I suspect that they're deciding to build/buy their own Internet crawler," said one search-engine consultant. "And I'm curious as to whether they view paid inclusion as a business model they're interested in."

The current MSN Search engine does not include its own built-in spider, or bot. Instead, MSN has relied on LookSmart and Inktomi for its directory and search results. MSN also works with Overture on paid placements and sponsorships for its site. MSN has denied talk that it plans to dump Overture as its paid-search partner and get into that business itself.

When asked for comment on the timing of MSNBot, an MSN spokeswoman had this to say: "While MSN does not have any specific timelines or plans to discuss at this time, they are committed to delivering the most relevant search results for consumers. Today, MSN Search takes an approach that utilizes both internal technology, as well as the technology of third-party companies, including Looksmart, Inktomi, Overture and Girafa. MSN is strongly committed to continuing to improve their search experience through developing technology internally and continuing to work with partners."

But Microsoft has made no bones about the fact that it wants to "out-google Google" — both with its next-generation MSN Search engine, as well as with new search technology that it is building into Longhorn Windows client and other future Microsoft product releases.

Yusuf Mehdi, Corporate Vice President, MSN Personal Services and Business, recently told attendees of the Goldman Sachs Internet conference that Microsoft is focusing more on the algorithmic part of search than on paid placements.

"What Google has done in terms of doing a great end-user experience — and what's going on with paid click — has led us to basically go back and redouble our efforts.," Mehdi said. "So, one of the things that we are doing now is really looking at this algorithmic search area, and we are investing a lot to go and build what we expect and hope will be the best-in-class search service in the near future."