Skip to content

This module provides simple robots.txt generation for Silverstripe, with various configuration options available.

License

Notifications You must be signed in to change notification settings

gorriecoe/silverstripe-robots

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

8 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Silverstripe Robots.txt generation

This module provides simple robots.txt generation for Silverstripe, with various configuration options available.

When a site is not in live mode (such as on a testing domain) it will respectively block the entire domain, ensuring that (at least respectful) search engines will refrain from indexing your test site.

Installation

Composer is the recommended way of installing SilverStripe modules.

composer require gorriecoe/silverstripe-robots

Requirements

  • silverstripe/cms ^4.0

Maintainers

Credit

This module is heavily inspired by silverstripe-robots by Damian Mooyman

Configuration

You can add a page or pattern to be blocked by adding it to the disallowedUrls configuration

gorriecoe\Robots\Robots:
  disallowed_urls:
    - 'mysecretpage.html'
    - '_private'
    - 'Documents-and-Settings/Ricky/My-Documents/faxes/sent-faxes'

Also by default, any page with 'ShowInSearch' set to false will also be excluded. This can be useful for hiding auxiliary pages like "thanks for signing up", or error pages.

You can turn this off (if you really absolutely think you need to) using the below.

gorriecoe\Robots\Robots:
  disallow_unsearchable: false

By default the module will check for a sitemap file in /sitemap.xml. You can set a custom file location using the below configuration.

gorriecoe\Robots\Robots:
  sitemap: '/sitemap.xml'

For multiple sitemaps.

gorriecoe\Robots\Robots:
  sitemap:
      - '/sitemap_index.xml'
      - 'http://www.gstatic.com/s2/sitemap.xml'

About

This module provides simple robots.txt generation for Silverstripe, with various configuration options available.

Resources

License

Stars

Watchers

Forks

Packages

No packages published