Escape characters not allowed by URL specs but not escaped by URL::toString

[JoHaHu]

Describe the bug

Whe using vite in dev mode behind a proxy illegal characters in URL can lead to errors.
As an example Nuxt 3 uses [foo] as pattern for dynamic routes. During development url for modules aren't properly url-encoded.

Reproduction:
open stackblitz example and navigate to /test/id. In the browser network trace there is a request for /test/[id].vue which fails when used behind certain proxies.

nuxt/nuxt#15049

Reproduction

https://stackblitz.com/edit/github-tjxq2a-k3p4ab

System Info

System:
    OS: Linux 5.19 Fedora Linux 36 (Workstation Edition)
    CPU: (12) x64 Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
    Memory: 21.48 GB / 31.04 GB
    Container: Yes
    Shell: 5.8.1 - /bin/zsh
  Binaries:
    Node: 16.16.0 - ~/.nvm/versions/node/v16.16.0/bin/node
    npm: 8.11.0 - ~/.nvm/versions/node/v16.16.0/bin/npm
  Browsers:
    Firefox: 105.0.1

Used Package Manager

npm

Logs

No response

Validations

• Follow our Code of Conduct
• Read the Contributing Guidelines.
• Read the docs.
• Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
• Make sure this is a Vite issue and not a framework-specific issue. For example, if it's a Vue SFC related bug, it should likely be reported to vuejs/core instead.
• Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
• The provided reproduction is a minimal reproducible example of the bug.

[sapphi-red]

nuxt/nuxt#15049

[sapphi-red]

Summary of what was discussed in the above issue

The "[" in the path part is not allowed in RFC 3986 (successor to RFC 2396), but is allowed in the URL Standard.

Both Node.js and browsers parses relative import specifiers in compliance with the URL Standard.

So this is a valid JS in both browser and Node.js.

However, the Java server used rejects this URL because it is compliant with RFC3986, not the URL Standard.

[sapphi-red]

Umm, rereading the URL Standard spec, it seems I'm wrong.
"[" (0x5B) in the path part is allowed in the URL "representation" section, but it's not allowed in the "URL writing" section.

Because new URL('https://example.com/[foo]').href returns https://example.com/[foo], I thought "[" is allowed in the URL Standard spec. But it seems .href may return a invalid URL. I didn't expect that to happen. 🫠

The output of the URL serializer is not always a valid URL string.
https://url.spec.whatwg.org/#urls:~:text=The%20output%20of%20the%20URL%20serializer%20is%20not%20always%20a%20valid%20URL%20string.

related: whatwg/url#379, whatwg/url#753

[blazmrak]

Hey, it seems like I got here by the same path as you (Java proxy in dev mode lol). Any idea what should be changed to fix this? 😄

[blazmrak]

Ok... This is impossible to solve outside of Vite. The only way I see this ever being fixed is by providing a plugin pair in Vite that decodes all URLs and then encodes them at the end of the build. This cannot come from user land, because import analysis runs in post and encoding breaks that plugin.

Something like this:

      const decodingPlugin = {
        name: 'decode-imports',
        transform(code, id, options) {
          // ... run only on csr and javascirpt files

          const staticImportRE = /^\s*import\s+(?:[\w*{}\s,]*\s+from\s+)?['"]([^'"]%[^'"])['"]\s*;?\s*$/

          return {
            code: code.replaceAll(staticImportRE, (match, importUrl) => {
              return match.replace(importUrl, decodeURI(importUrl))
            })
          }
        },
        enforce: 'pre'
      },