Why can PWA Studio hide Adobe Commerce's configured robots.txt?

Definition: Adobe Commerce Cloud generates robots.txt content on demand, but a PWA Studio storefront can route the request through UPWARD instead of the default Magento theme. The Front Name Allowlist in UPWARD PWA Configuration sends a named route back to Magento, so adding robots.txt there is Adobe's documented fix when the configured file cannot be reached.

Key facts

• Adobe's cloud sitemap and robots documentation says that robots.txt content is generated on demand and stored in the database.
• The same documentation says you can view that content at <domain.your.project>/robots.txt or <domain.your.project>/robots.
• Adobe explicitly says that a PWA Studio deployment unable to access its configured robots.txt should add robots.txt to the Front Name Allowlist at Stores > Configuration > General > Web > UPWARD PWA Configuration.
• The Magento 2 UPWARD connector README describes the allowlist as a line-separated list of routes forwarded to the default Magento theme.

What is being hidden?

The configured file is a Magento response, even when it is requested at the public storefront hostname. Adobe's cloud documentation says that the platform generates robots.txt content on demand and stores it in the database. It also names /robots.txt and /robots as browser URLs for viewing that content.

PWA Studio changes the request path through the storefront layer. The Magento 2 UPWARD connector documentation describes the connector as a module that routes requests to UPWARD-PHP and replaces the default Magento frontend theme with a PWA Studio storefront application. The configured robots response can therefore be present in Adobe Commerce while the public request is handled by the storefront route.

How does the Front Name Allowlist change routing?

The Front Name Allowlist is an admin setting under Stores > Configuration > General > Web > UPWARD PWA Configuration. The connector README defines it as a line-separated list of routes to forward to the default Magento theme. Its example is:

contact
privacy-policy-cookie-restriction-mode

The README says that a visitor requesting either example route lands on a page rendered by Magento instead of the storefront application. The same routing rule explains why the robots.txt entry matters: it identifies the public route that must reach Magento's generated response.

Adobe's cloud page gives the deployment-specific instruction directly: if PWA Studio cannot access the configured robots.txt file, add robots.txt to the Front Name Allowlist. The relevant value is:

robots.txt

This setting concerns request routing. It does not edit the custom instruction stored in Adobe Commerce's Search Engine Robots configuration. Adobe documents that content-editing path as Content > Configuration in the Design section, then Search Engine Robots, followed by the Edit custom instruction of robots.txt field and Save Configuration.

How should you check the result?

Adobe's documentation says to verify the <domain.your.project>/robots.txt file or the <domain.your.project>/robots URL in a browser after saving the robots configuration. For a PWA Studio deployment, check the public robots.txt route after adding the allowlist entry, because the documented fix is tied to access through the storefront routing layer.

The cloud page was last updated on 8 October 2026. It also notes that a 404 error at the public robots.txt URL requires an Adobe Commerce Support ticket to remove the redirect from /robots.txt to /media/robots.txt.

Related terms

• robots.txt
• Does a physical robots.txt file override Adobe Commerce's generated rules?

FAQ

Does the allowlist rewrite the robots.txt contents?

No. It controls which route is forwarded to the default Magento theme. Adobe Commerce continues to generate the content on demand and store it in the database.

Should the allowlist value include a leading slash?

Adobe's instruction names robots.txt as the value to add. The connector documentation describes entries as routes and shows them without a leading slash.

Why can /robots and /robots.txt behave differently?

Adobe documents both URLs as ways to view the generated content, but its PWA Studio troubleshooting instruction specifically names robots.txt for the Front Name Allowlist.

Sources

1. Adobe, "Add site map and search engine robots", 8 October 2026
2. Magento, "Magento 2 UPWARD connector module", undated

---

Check how a page reads to answer engines. Open the GEO score analyzer