Not signed in (Sign In)

Vanilla 1.0.1 is a product of Lussumo. More Information: Documentation, Community Support.

    • CommentAuthorespc
    • CommentTimeMay 2nd 2014
     
    As a property portal, we get crawled by various vertical (meta) search engines

    http://www.wiju.co.uk/
    http://home.co.uk/
    http://trovit.co.uk/

    We filter the following strings when counting human traffic:
    WijuBot/1.0+(+http://www.wijutools.com/bot.html)
    trovitBot

    I haven't yet identified the crawler used by home.co.uk

    I wonder if it would it make sense to maintain lists of bots operating in specific markets (property, travel, recruitment etc.)?