GET https://owner-api.teslamotors.com/api - error "timeout"

[amit-nz]

Hi All

Since earlier today, my car (Model 3, software 2024.8.9) was showing "unavailable" in teslamate. So, I docker pull'd teslamate (There was an update so figured might as well).

I found that it still showed unavailable The logs showed

[error] GET https://owner-api.teslamotors.com/api/1/vehicles/1492932704197477 -> error: "timeout" (5158.890 ms)

So, I opened the Tesla iOS App. It also showed car offline - last seen ~8 hours ago which is when I last drove it. So I went out to my garage, everything looked OK (said it was connected to my home wi-fi). I rebooted the car anyways (holding both steering buttons). About 15 minutes later, the car now shows up in the iOS App (says online, I can start/stop charging etc just fine) however in Teslamate it is still unavailable. I have attached the log of the container starting up. Any ideas what this could be?

2024-05-28 09:25:37.913 [info] Migrations already up
2024-05-28 09:25:41.279 [info] System Info: Erlang/OTP 26 (jit)
2024-05-28 09:25:41.279 [info] Version: 1.29.1
2024-05-28 09:25:42.088 [info] POST https://auth.tesla.com/oauth2/v3/token -> 200 (773.942 ms)
2024-05-28 09:25:42.088 [info] Refreshed api tokens
2024-05-28 09:25:42.092 [info] Scheduling token refresh in 6 h
2024-05-28 09:25:42.097 [info] Running TeslaMateWeb.Endpoint with cowboy 2.10.0 at :::4000 (http)
2024-05-28 09:25:42.100 [info] Access TeslaMateWeb.Endpoint at http://localhost
2024-05-28 09:25:47.260 [error] GET https://owner-api.teslamotors.com/api/1/products -> error: "timeout" (5159.562 ms)
2024-05-28 09:25:47.261 [warning] TeslaApi.Error / timeout
2024-05-28 09:25:47.261 [warning] Could not get vehicles: :unknown
2024-05-28 09:25:47.269 [warning] Using fallback vehicles:

[
  %TeslaApi.Vehicle{
    id: 1234567890123456,
    vehicle_id: 555555555,
    vin: "ABCD1234567890EF",
    tokens: [],
    state: "unknown",
    option_codes: [],
    in_service: false,
    display_name: "Model 3",
    color: nil,
    calendar_enabled: nil,
    backseat_token: nil,
    backseat_token_updated_at: nil,
    api_version: nil,
    charge_state: nil,
    climate_state: nil,
    drive_state: nil,
    gui_settings: nil,
    vehicle_config: nil,
    vehicle_state: nil
  }
]
2024-05-28 09:25:47.269 [info] Starting logger for 'Model 3'
2024-05-28 09:25:47.459 [info] MQTT connection has been established
2024-05-28 09:25:52.452 [error] GET https://owner-api.teslamotors.com/api/1/vehicles/1234567890123456 -> error: "timeout" (5155.395 ms)
2024-05-28 09:25:52.453 [warning] TeslaApi.Error / timeout
2024-05-28 09:25:52.453 car_id=1 [error] Error / :unknown
2024-05-28 09:26:27.610 [error] GET https://owner-api.teslamotors.com/api/1/vehicles/1234567890123456 -> error: "timeout" (5156.819 ms)
2024-05-28 09:26:27.610 [warning] TeslaApi.Error / timeout
2024-05-28 09:26:27.610 car_id=1 [error] Error / :unknown
2024-05-28 09:27:02.771 [error] GET https://owner-api.teslamotors.com/api/1/vehicles/1234567890123456 -> error: "timeout" (5160.059 ms)
2024-05-28 09:27:02.772 [warning] TeslaApi.Error / timeout
2024-05-28 09:27:02.772 car_id=1 [error] Error / :unknown
2024-05-28 09:27:11.374 [info] GET /
2024-05-28 09:27:11.379 [info] Sent 200 in 4ms
2024-05-28 09:27:13.528 [info] GET /
2024-05-28 09:27:13.533 [info] Sent 200 in 4ms
2024-05-28 09:27:13.679 [info] GET /
2024-05-28 09:27:13.682 [info] Sent 200 in 3ms
2024-05-28 09:27:13.843 [info] GET /
2024-05-28 09:27:13.845 [info] Sent 200 in 2ms
2024-05-28 09:27:37.933 [error] GET https://owner-api.teslamotors.com/api/1/vehicles/1234567890123456 -> error: "timeout" (5160.068 ms)
2024-05-28 09:27:37.933 [warning] TeslaApi.Error / timeout
2024-05-28 09:27:37.933 car_id=1 [error] Error / :unknown
2024-05-28 09:27:41.135 [notice]     :alarm_handler: {:set, {TeslaMate.Vehicles.Vehicle_1_api_error, :fuse_blown}}
2024-05-28 09:28:13.098 [error] GET https://owner-api.teslamotors.com/api/1/vehicles/1234567890123456 -> error: "timeout" (5164.975 ms)
2024-05-28 09:28:13.098 [warning] TeslaApi.Error / timeout
2024-05-28 09:28:13.098 car_id=1 [error] Error / :unknown

[amit-nz]

Alright -- I think this is solved now. I believe the cause of the issue was an incorrect MTU setting on my router's LAN interface. I just switched from PPPoE to IPoE (DHCP) earlier today, and all my interfaces had a 1492-byte MTU hard coded (to allow for the PPPoE overhead) however I have now increased this to 1500, and as soon as I did that everything came back to life.

[amit-nz]

Update/ it’s now nearly 12 hours later and everything is working just as it should. Looks like I had an MTU issue after all.

I don’t quite understand why though. Normally if the MTU is smaller than what we try to send, it’s problems. Never the other way around.

Since the Tesla API auth worked (code 200) Maybe the issue was with the car’s networking stack because it was on my home wifi when all this was happening. It can’t scale MTU itself?

[brianmay]

That does seem strange. Wondering if something else happened at the same time you changed the MTU (e.g. resetting the connection) that caused it to come good.

I currently am on PPPoE, pretty sure I have the reduced MTU (it was a while since I set this up), and I do not have any problems accessing the Tesla API.

[segdy]

I am having exactly the same issue. For me, I have a wireguard tunnel with 1420 MTU in between:

Server with TeslaMate (MTU=1500) ----> Mikrotik Router Ethernet (MTU=1500)
                                                       |
                                                      \ /
                                                       .
                                       Mikrotik Router Wireguard (MTU=1420)
                                                       |
                                                      \ /
                                                       .
                                       Mikrotik Router ISP Ethernet (MTU=1500)   --> Internet

Same as you, it works for me when I change TeslaMates host from 1500 to 1420. But I do not understand why the heck it doesn't work otherwise:

1. Every other connection (incl SSL) to everything else I tried works
2. While there might be performance penalty, IP is supposed to work if there's a low-MTU connection in between somewhere. It will just fragment

In my case, my Mikrotik router should fragment everything to 1420. In the packet sniffer I can confirm that Mikrotik doesn't send any package larger than 1420, no fragmentation is needed.

However, the Tesla server responds with a large packet (2788 bytes, probably server cert) when MTU=1420. With MTU=1500, it's just 664 bytes and from here it hangs.

I do not see anything else. No ICMP messages, nothing. I also temporarily turned off my firewall but issue persists.

Any further idea what the heck is going on here?

[ngardiner]

HTTPS clients/servers set the DF (Do not Fragment) bit in IP communications

[segdy]

That’s right. I did more investigation now and the issue seems to be that then Tesla server just filters/ignores ICMP need fragmentation messages.

The way different MTUs work with TCP is that all connections have DF bit and each side sets MSS based on their local MTU. If there’s a lower MTU in between, that packet will be discarded and the sender will receive the ICMP message. That gives the sender the opportunity to fragment (or decrease MSS, not sure).

if the sender blocks these ICMP messages, as Tesla does, the packages will always exceed the intermediate MTU and be discarded (as they are DF)

[brianmay]

I think MSS clamping is the solution/hack to solve this problem. And it isn't Tesla specific.

Curiously I don't appear to have MSS clamping enabled. But it is possible that my modem has it enabled anyway.

[segdy]

that’s right, I set up MSS clamping on my Mikrotik router which fixed it.

isn’t Tesla specific but it’s specific to anyone who decides to drop/ignore ICMP messages.

Curiously Tesla was the first server in months where it occurred.

Maybe also your internet uplink is actually 1500 MTU as well? Like FTTH? Then it would work as well

[brianmay]

No, only have FTTN. Which is using PPPoE. FTTP supposedly available end of the year. The modem has a setting for MTU which is set to 1492, which I am guessing might silently do MSS clamping also.

As you can guess I don't exactly trust VDSL modems; last I researched this all them seemed terrible or excessively expensive.