Add robots.txt setting. #5584
Add robots.txt setting. #5584
Conversation
✅ Deploy Preview for plone-components canceled.
|
✅ Deploy Preview for volto ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
Co-authored-by: Steve Piercy <[email protected]>
|
@robgietema could you take a look at the suggestions and move this forward? Thanks! |
* main: (741 commits) Improve the usability of the ObjectBrowser when inputting a manual value, checking it on blur, and adding a local validator (#6576) fixing Markdown heading (#6588) fix site logo issue (#6591) Route registry (#6600) Release @plone/client 1.0.0-alpha.21 Export the getContent bare fetcher (#6594) fix: incorrect copied state useClipboard (#6585) [RR7] Update to latest RR7 and conventions, fix index page (#6589) Release 18.6.0 Release @plone/slate 18.1.0 Revert "added swedish translation" (#6578) Slate Italian translations (#6563) Release 18.5.0 Fix robots.txt in devmode (#6571) added swedish translation (#6557) Depth search issue (#6558) Block examples documentation (#6560) Fixed folder contents issues with persistent selection (#6554) Fix redirects to MDN responsive images (#6552) Bugfix remove query string inclusion in body class generation logic (#6547) ...
|
@robgietema @ericof @davisagli Let's resurect this, I've merged it with main and solved conflicts. |
There was a problem hiding this comment.
@sneridagh @ericof For the most part this is just updating the implementation details, but I see one breaking change that needs to be resolved, and the changelog says it does something it doesn't.
| const url = `${proto}://${host}${port}`; | ||
| text = text.replace(internalUrl, url); | ||
| // Replace the sitemap with the sitemap index. | ||
| text = text.replace('sitemap.xml.gz', 'sitemap-index.xml'); |
There was a problem hiding this comment.
@sneridagh @ericof Removing this replacement is a breaking change. We should keep it here, or handle it instead in plone/plone.volto#183
There was a problem hiding this comment.
Fixed, however I think it has to go to plone.volto too. Although that can be a breaking change, since it can mean an action needed in Google Search Consoles of people.
| @@ -0,0 +1 @@ | |||
| Expose robots.txt setting in Volto control panel, and render robots.txt based on REST API call. @robgietema | |||
There was a problem hiding this comment.
Where is it exposed in the Volto control panel? I don't see it there. I expect it needs to be removed from
There was a problem hiding this comment.
Done, but related to the other comment:
The backend answer with the .gz, which we will override it afterwards in a shady way in code (btw, we are doing it right now)
Fixes #5580