@Roxzfr wrote:
https://preview.webflow.com/preview/marship?preview=a6e5f4afdb5b10d87fa5efa22f81838b
Hi, I recently submitted the above site (my first in Webflow) to a company called 'TotalSEO' it scored as I expected in most areas but they have raised a couple of issues that concern me. I'll post a topic for each!
They say:-
Robots.txt is missing
It is important to ensure your website has a robots.txt file.
A robots.txt file allows you to restrict the access of search engine robots that crawl the web and it can prevent these robots from accessing specific
directories and pages you may not want them to see. It also specifies where the XML sitemap file is located.
One of the first things we do when we start a new SEO campaign is ensure the robots.txt file is in place and correctly set up.I thought this was built in to Webflows functionality? Could someone please let me know how to rectify this?
Thank you
Posts: 5
Participants: 2