Data 1 hour off?

[kikediazfernandez]

I have v1.0.4 with only the historical sensor enabled and I see data is 1 hour off compared with data collected with Shelly EM.
Also, if you look at the history, and compares the graphic with the one that Iberdrola (not I-DE) shows, data from 11:00 to 12:00 for example is shown as the 12:00-13:00 interval in HA
The API seems to return the accumulated consumption in the previous hour. My last (redacted) log for 'successful' update:

2023-05-31 14:41:32.416 DEBUG (MainThread) [custom_components.ideenergy.datacoordinator] Updating data set: <DataSetType.HISTORICAL_CONSUMPTION: 2>
2023-05-31 14:41:32.416 DEBUG (MainThread) [custom_components.ideenergy.datacoordinator] HISTORICAL_CONSUMPTION update allowed
2023-05-31 14:41:33.314 DEBUG (MainThread) [custom_components.ideenergy.datacoordinator] HISTORICAL_CONSUMPTION successfully updated
2023-05-31 14:41:33.314 DEBUG (MainThread) [custom_components.ideenergy.datacoordinator] Finished fetching [email protected]/xxxxxxxx coordinator data in 0.898 seconds (success: True)
2023-05-31 14:41:33.316 DEBUG (MainThread) [custom_components.ideenergy.historical_sensor] sensor.es0021xxxxxxxxxxxxxxsg_historical_consumption: Got 168 historical states from API
2023-05-31 14:41:33.389 DEBUG (DbWorker_0) [custom_components.ideenergy.historical_sensor] Deleted 0 invalid states
2023-05-31 14:41:33.399 DEBUG (DbWorker_0) [custom_components.ideenergy.historical_sensor] sensor.es0021xxxxxxxxxxxxxxsg_historical_consumption: lastest state found: 0.439 @ 2023-05-30 22:00:00+00:00
2023-05-31 14:41:33.399 DEBUG (DbWorker_0) [custom_components.ideenergy.historical_sensor] sensor.es0021xxxxxxxxxxxxxxsg_historical_consumption: no new states from API

The latest state, corresponding to 23:00-24:00, is collected as 00:00 (22:00 UTC)
"Successful" because yesterday's data still does not appear, by the way.

Originally posted by @kikediazfernandez in #34 (comment)

[ldotlopez]

Hi,

"Successful" because yesterday's data still does not appear, by the way.

Yes, there is a bug which makes new data loss, only the first run is imported into statistics. I'm looking at it to figure out what is causing it but it's hard to debug, hopefully I will fix it soon.

About the 1 hour off…

Tracing some random spike from my account:

1. i-de.es web shows 0,783 value at hour 22 (no clues if it's start or end)

2. That value comes from internal API with this JSON:

 {
	"21": {
		"valor": "783.00",
		"tipo": "null",
		"excluido": false
	}
}

Currently I use some API endpoints different from web UI: https://github.com/ldotlopez/ideenergy/blob/main/ideenergy/client.py#L32

3. Using ideenergy command it gets translated as:

{'end': datetime.datetime(2023, 6, 13, 22, 0),
 'start': datetime.datetime(2023, 6, 13, 21, 0),
 'value': 783.0},

You can test it with:
python -m ideenergy --get-historical-consumption -u xxxx -p xxxx

4. That data shows up in my home assistant panel as:
   

What is happening?

I think I'm handling the data correctly, I have no Shelly or other sensors but that data matches with my usage pattern.

May be homeassistant has incorrectly configured country or timezone?

[kikediazfernandez]

My homeassistant is running on docker, and the timezone is correct.
Running the python command shows correct data, but in homeassistant data appears wrong, with points using the end date instead of the start date:

The info un i-DE web. It uses the end hour:

The same info in Iberdrola

[ldotlopez]

Hi @kikediazfernandez

That's correct. Your first graph shows 0,47 at 22:00 point which translates to "at 22:00 I get a 0,47 reading from the hourly consumption sensor. since previous reading was at 21:00 point this means that in the 21:00-22:00 period 0,47 kwh was consumed".

If you have doubts add the historical sensor as energy source at the energy panel, bars will show correct values.

Home Assistant historic graphs can be a little confusing with this type of sensors.

Let me know if it works ok

[kikediazfernandez]

Yes, they are, indeed. Energy panel shows correct values. Now I have to realize how to make a graph that shows the correct matching between i-de and shelly EM. My current one has the 1 hour mismatch.

[kikediazfernandez]

Ok, using statistics I managed to make the graph with apexchart match. Closing the issue.