99ccc4d103
- Updated page titles, descriptions, and keywords across all main pages to enhance local SEO targeting "roofer in Accrington". - Implemented advanced Schema.org structured data for improved search engine visibility, including local business schema and page-specific schemas. - Created a new local business schema file to encapsulate detailed business information and service offerings. - Added a robots.txt file to optimize crawler access and protect sensitive directories. - Updated sitemap with change frequencies and priorities for better crawling efficiency. - Fixed various typos in URLs and standardized variable naming for consistency. - Enhanced technical SEO with improved site architecture and structured data integration.
36 lines
584 B
Plaintext
36 lines
584 B
Plaintext
User-agent: *
|
|
Allow: /
|
|
|
|
# Sitemap location
|
|
Sitemap: https://warmseal-roofing.co.uk/sitemap.xml
|
|
|
|
# Block access to admin and config files
|
|
Disallow: /includes/
|
|
Disallow: /meta/
|
|
Disallow: /scss/
|
|
Disallow: /*.php$
|
|
|
|
# Allow important CSS and JS files for rendering
|
|
Allow: /css/
|
|
Allow: /js/
|
|
Allow: /images/
|
|
Allow: /fonts/
|
|
Allow: /fontawesome/
|
|
|
|
# Crawl delay to be respectful
|
|
Crawl-delay: 1
|
|
|
|
# Specific directives for major search engines
|
|
User-agent: Googlebot
|
|
Allow: /
|
|
|
|
User-agent: Bingbot
|
|
Allow: /
|
|
|
|
# Block unwanted bots
|
|
User-agent: AhrefsBot
|
|
Disallow: /
|
|
|
|
User-agent: MJ12bot
|
|
Disallow: /
|