robots.txt
Publish https://yourdomain.com/robots.txt. Allow important marketing and product paths. Disallow only what must stay private.
Include a Sitemap: line pointing at your XML sitemap. Re-check after every WAF or bot-fight change — Attack Mode can block auditors and Googlebot alike.
XML sitemap & canonicals
List canonical HTTPS URLs that return 200. Keep lastmod honest. Submit in Google Search Console and Bing Webmaster Tools.
Every indexable page should declare a preferred absolute HTTPS URL with a canonical link element.