HA VPE fails to connect during onboarding or audibly reply if HTTPS internal network URL is used

[the-rexmundi]

Core 2025.1.2
Supervisor 2024.12.3
Operating System 14.1
Frontend 20250109.0
ESPHome Firmware 2024.12.2
Using Let's Encrypt certificate for internal web UI

Steps to reproduce:

1. Follow VPE onboarding procedure using Android Companion app (https://voice-pe.home-assistant.io/getting-started/)
2. After step 6, wizard says that the device cannot connect to HA and sends me to this troubleshooting page (https://voice-pe.home-assistant.io/faq/#i-dont-get-a-voice-response)
3. The trouble shooting guide tells me to check that the internal URL is correct in "Settings > System > Network". The internal URL is correct and is a FQDN that matches the Let's Encrypt certificate
4. Continue on regardless and issue a voice command "turn off laundry lights" which is executed, but there is no verbal response from VPE
5. Checking the VPE logs (attached) shows that it is using the external HA Cloud URL to attempt to retrieve the .flac file containing the verbal response which fails with a TLS error "esp-tls-mbedtls: mbedtls_ssl_handshake returned -0x7280"

Steps to work-around

1. Remove all HTTPS/TLS options from configuration.yaml
2. Change the internal URL to HTTP. Not that nothing else has changed with the URL except removing the S in HTTPS and adding the port, it is still a FQDN with :8123 added to the end
3. Reboot HA
4. Issue voice command to VPE, command executes and verbal response is given

This seems to be a poor outcome in that it forces a security downgrade just to get VPE working. I would not call downgrading from HTTPS to HTTP a solution, just a work around.

FAIL HTTPS logs_home-assistant-voice-0908af_logs.txt
SUCCEED HTTP logs_home-assistant-voice-0908af_logs.txt

[jonofe]

I have mostly the same problem. I'm using a self signed certificate with an own managed CA.
The registration of the VPE device worked, but the TTS voice output only works in ~20% of the cases.
The Assist is configured to use Home Assistant Cloud for STT, Processing, TTS.
If I look at the debug page, the command is understood and the response is correct and also the audio file is generated, but it's not played by the VPE device in ~80% of the cases.
Local URL is configured manually in the way: https://my.domain.home:8123
I'm completely confused, why it works sometimes but in most cases not.
I already nulled the local URL entry in /config/.storage/core.config but without success.

I checked the ESPHOME debug output and found out, that the voice response is called with external nabu.casa URL instead of the configured local URL. Could that be the reason?

Sending VoiceAssistantEventResponse: event_type: VOICE_ASSISTANT_TTS_END
data {
  name: "url"
  value: "https://########################.ui.nabu.casa/api/tts_proxy/9gEnP1vJkF7B2U-nZPgBPA.flac"
}

Currently I'm lost how to fix that problem without deactivating SSL.
Any suggestions?

Logs of a failed voice response:
You can see there are only 300ms between MEDIA_PLAYER_STATE_PLAYING and MEDIA_PLAYER_STATE_IDLE

2025-01-16 21:18:12.526 DEBUG (MainThread) [aioesphomeapi.connection] home-assistant-voice-092666 @ 10.0.20.28: Sending VoiceAssistantEventResponse: event_type: VOICE_ASSISTANT_RUN_END

2025-01-16 21:18:12.526 DEBUG (MainThread) [aioesphomeapi._frame_helper.base] home-assistant-voice-092666 @ 10.0.20.28: Sending frame: [00025c0802]
2025-01-16 21:18:12.598 DEBUG (MainThread) [aioesphomeapi.connection] home-assistant-voice-092666 @ 10.0.20.28: Got message of type MediaPlayerStateResponse: key: 2232357057
state: MEDIA_PLAYER_STATE_PLAYING
volume: 0.7

2025-01-16 21:18:12.880 DEBUG (MainThread) [aioesphomeapi.connection] home-assistant-voice-092666 @ 10.0.20.28: Got message of type MediaPlayerStateResponse: key: 2232357057
state: MEDIA_PLAYER_STATE_IDLE
volume: 0.7

2025-01-16 21:18:12.928 DEBUG (MainThread) [aioesphomeapi.connection] home-assistant-voice-092666 @ 10.0.20.28: Got message of type VoiceAssistantAnnounceFinished: success: true

Logs of a working voice response:

You can see there are now ~7 seconds between MEDIA_PLAYER_STATE_PLAYING and MEDIA_PLAYER_STATE_IDLE. That is the time, where the VPE gives the voice response

2025-01-16 21:28:39.924 DEBUG (MainThread) [aioesphomeapi.connection] home-assistant-voice-092666 @ 10.0.20.28: Sending VoiceAssistantEventResponse: event_type: VOICE_ASSISTANT_RUN_END

2025-01-16 21:28:39.924 DEBUG (MainThread) [aioesphomeapi._frame_helper.base] home-assistant-voice-092666 @ 10.0.20.28: Sending frame: [00025c0802]
2025-01-16 21:28:39.924 DEBUG (MainThread) [homeassistant.components.esphome.assist_satellite] Pipeline finished
2025-01-16 21:28:39.992 DEBUG (MainThread) [aioesphomeapi.connection] home-assistant-voice-092666 @ 10.0.20.28: Got message of type MediaPlayerStateResponse: key: 2232357057
state: MEDIA_PLAYER_STATE_PLAYING
volume: 0.7

2025-01-16 21:28:46.289 DEBUG (MainThread) [aioesphomeapi.connection] home-assistant-voice-092666 @ 10.0.20.28: Got message of type MediaPlayerStateResponse: key: 2232357057
state: MEDIA_PLAYER_STATE_IDLE
volume: 0.7

2025-01-16 21:28:46.336 DEBUG (MainThread) [aioesphomeapi.connection] home-assistant-voice-092666 @ 10.0.20.28: Got message of type VoiceAssistantAnnounceFinished: success: true

[Silvenga]

Same issue, kind of sucks to need to bypass HTTPS.

Any device that is designed to work locally should really support a locally managed CA.

[Silvenga]

@the-rexmundi this suggests that -0x7280 might be related to the server closing the connection from a bad client hello (e.g. unsupported version).

Just double checking you are just using the native HTTP server in your HA instance?

[the-rexmundi]

@Silvenga thanks for the link, for some reason I did not find that thread when I googled the error code. I am a bit lost by the conversation there because "[sysizlayan] [on May 17, 2019]" suggests that the server rejected the connection, but we are not talking about client certificates here. What does a server care if you do not trust it's CA? Is it that the client process of ESP does not trust Let's Encrypt CA?

I followed the instructions here: https://community.home-assistant.io/t/how-to-configure-lets-encrypt-ssl-certificates-for-home-assistant-completely-100-free-updated-for-2022-2023/508329

The certificate PEM is installed in configuration.yaml as per the instructions in the linked article for internal use only, Home Assistant Cloud is used for external connections.

Based on the instructions, I believe that I am using the native HTTPS server and not a proxy solution like NGINX.

[Silvenga]

@the-rexmundi client certificates are sent after the Server Hello. Your logs suggests what appears to be a failure after the client sends it's Client Hello and before the Server sends it's Hello (the MBEDTLS_ERR_SSL_CONN_EOF).

TLS Socket is Open
...
Client Hello -> Server

<failure here somewhere>

Server Hello -> Client
...
Rest of the TLS handshake
e.g. client certificates

It appears the server is rejecting what the client is proposing (in the Client Hello), which is typically things like which cipher suites the client supports (which could mean the ESP is requesting an older/less secure cipher that our servers are rejecting).

But @jonofe's issue might be completely different. I am wondering if it's actually CA related, since normally the client will tell the server that the CA isn't trusted, and I'm not seeing that at all on the server side (I'm not discounting Let's Encrypt as a possibly untrusted CA, they are relatively new still).

So I'm wondering if it's something a bit more lower level (maybe a esp-idf limitation or bug). I was planning on playing with the cipher suites to see if that fixes this.

So @the-rexmundi you are using the native HTTPS server. @jonofe, are you also using the native HTTPS server too?

(I'm personally using Nginx for TLS termination)

[jonofe]

@jonofe, are you also using the native HTTPS server too?

Yes, I'm using the native SSL support of HA.

http:
   ssl_certificate: /ssl/homeassistant.feld.home.crt
   ssl_key: /ssl/homeassistant.feld.home.key

I have my own CA and the SSL certificate is signed by that CA.

On my clients (Win,Android, iOS) I have imported the CA certificate, so my clients trust my CA.
I suppose, the Voice PE simply checks the certificate and does not trust it.
Otherwise it either would have to ignore the certificate verification by default or you would need an option to configure your own CA, means give access to the CA certificate to the ESPHome device, that this can trust the HA SSL certificate.

And I can imagine, that in case of a failed verification of the SSL certificate Voice-PE tries then the public URL via nabucasa (in case you have a subscription). Timing problems could lead to the situation, that this works in few cases, but in most cases not. But that's only my imagination. Hope the developers can clarify that soon.