I received a smartmeter from the local powercompany and recenently they allowed me to enable the (IR) userinterface to read out data.
The problem is that they are not very forthcoming with information regarding this interface so I'm not 100% sure how to implement the readout of the meter.
Here are the facts and what I know so far:
Hardware:
ISKRA (Iskraemeco) AM550
It has an optical interface (IR LEDs)
I have a USB to Serial to IR LED interface Cable
Infos:
This is the only information I received from the powercompany:
– Protocoll: IEC 62056 – 21
– optical interface (IR)
– DLMS IDIS CII
Additional Info I could gather from the webinterface where you can enable the customerinterface:
"Information is sent every second over the IR Interface"
and also a 32 char HEX encyption key
If I use e.g. Hterm with "9600 Data 8 Stop 1 Parity None" I see that every second I receive data over the IR-Serial Adaper. (sending the DLMS handshake does nothing)
I would like to create a python script (with the help of the gurux-dlms library) to gather and decode/decrypt the data to read the meter values.
Could you please give me some directions because it seems this meter does not use the regular way of handshake etc but instead just sends bursts of (encrypted)data.
...still waiting for WienerNetze to answer my request.
Last response was on Wednesday December 23rd: "we have not yet received a [internal] feedback regarding your request. We will actively contact you when there is an answer from our technical department".
In the meantime I checked:
- provided key on web portal is still unchanged
- provided data from smartmeter is still garbage when trying to decrypt with that key.
... I got some responses from Wiener Netze and wrote some additional messages with a new bug oft the meter firmware.
Finally I got the message that failures were found and that the developers are working on it.
Ok everyone can write such a messge :-) Maybe that is true or not.... I don't know but thats the first message where the say that something is not working as expected.
The will provide an update as soon as possible. Normally they need weeks for a response. We will see how much weeks they will need for bugfixing :D
Thanks for the info Stefan. So lets meet up again here in one year and see if they actually did something. (last time it took almost a year for the option to reappear in the webinterface)
I just got an info from a digitalgridsolutions' support representative: He told he opened an "official support ticket at Iskraemeco" and will get back to me as soon as he receives feedback.
That same guy recently asked for a couple of HDLC frames of my meter, as he wanted to test some issue suspect. But I did not get more details on that, anyway I suppose he found/proved something and used his findings within the "official support ticket".
@pocki
till now i reportet three bugs at wiener netze:
1) hdlc frame header is sometimes shorter than expected and wrong
2) length fields are calculated in wrong way
3) Push messages are not decryptable
if you need more infos on that, you can have examples i received from my meter where you can see the bugs.
I just received a statement from WienerNetze, telling they found some issue related to the AES key and admit it to be wrongly shown in the web portal. Their software developers are working on an appropriate fix, which might be rolled out already this week.
That email was polite, apologetic and included an appreciative value honoring my (our) aim to help on this encryption problem :-)
Well, sounds good. So let's stay tuned and check the key after some days. If I understood right, thw web portal will show another AES key after some days, which will - hopefully - work to dechipher the payload in the HDLC frames.
yes indeed, the key has changed and now I can decrypt all the packages (also past ones)
now I just need to make sense of the notification body contents... anybody want to team up?
that first octet-string is the timestamp:
0x07e4 is year 2020
0x0b is month 11
0x16 is day 22
0x07 is weekday (7=sunday)
0x10, 0x05, 0x22 is hh,mi,ss
following 8 values are 4byte counters.
First is the total real energy pulled off the power grid in Wh.
0x00610ED6 is 0x00*256^4 + 0x06*256^3 + 0x10*256^2 + 0x0e*256 + 0xde = 6360790 Wh. Divide by 1000 and that's the value 6360 or 6361 what you may read from the smartmeter's display.
5th value 0x00001A26 ist the current power consumption in W: that is 6694W.
last night I also got the update and now the output of my smart meter looks like yours with the updated key.
That are very good news for me, because now I can start with my Homematic energy sensor integration.
Gentleman it was a pleasure to push wiener netze forwards with you ;-)
Hi - after some playing with de decrypted data of my WienerNetze Iskraemeco AM550 smartmeter I realized something: each data packet (sent each second) includes a timestamp. In the moment of data transmission, that timestamp is 4 or 5 seconds in the past.
That means: at 10:00:00 i receive and decrypted data including the timestamp 09:59:55. Next second (at 10:00:01) the next packet includes timestamp 09:59:56.
Can anybody confirm this?
Does it mean, WienerNetze does not properly sync their smartmeters' system time to a NTP server?
I cannot imagine that these smart meters buffer their packets 5 seconds prior sending to infrared interface.
DLMS meters usually don't use NTP. The time of the meter is usually updated when meter is read by the meter company. There are several reasons for this, but one of them is that there is no active TCP/IP connection and the meter simply can't use NTP.
I believe that this will change and meters will start to use NTP in the future.
I see. Thx for explanation. As WienerNetze polls the data every night, I guess their either do not frequently update the smartmeter's system clock, or their own internal system time is -4 to -5 seconds off sync.
Another question to WienerNetze SmartMeter interface data:
I recieve real power pulled from the grid (+P in W) and pushed into the grid (-P in W) as seperated values.
During the last 3 days I captured for 2 individual moments a value on -P, that I cannot explain for any reason:
Tue 2 Feb 07:47:00 CET 2021: +P: 0W, -P 702W
Wed 3 Feb 10:20:30 CET 2021: +P: 0W, -P 397W
In both cases, the second before and after that "false peak" hat +P=~500W and -P=0W for several minutes. And: -P=0W almost all time, as my two small photovoltaic modules don't ever produce enough power to fullfil the house base consumption (dispite they are not strong enough to produce more than 200W at all during winter times and cloudy, dark sky)
What could cause such a "negative peak" in power measurement?
Is there any device that might de-charge with such a peak (i.e. a fully charged notebook that is plugged in)?
Looks like a measurement error?
It's possible that date-time is not updated on every reading. The clock is usually read and checked what is the deviation. If the deviation is in the given range it's not updated. Max range depends on the country. In some countries, it might be even 5 mins.
Hi. Are you talking about "reactive power" in the meaning of unit "volt-amperes-reactive" and "unused power"? If yes, that is reported as +Q and -Q in two seperate values.
I get -P in Watts (not volt-amperes-reactive) and as a unit of real/active power, that is pushed to the grid.
What other kind of device could cause such a (negative) power peak in an usual one-family-house? There's no big engine, but I have connected: an ups (620VA), a small electrical fan for our bathroom, an induction cooking plate, a daikin air condition unit, a 600Wp PV panel with micro-inverter, and other typical household stuff like an iMac, Notebook, fridge, vakuum cleaner, water-pump for heating,...
Well, an 3-phase-AC power grid is quite a complex thing :-)
I checked the data of these 2 moments once again and realized that in parallel to the negative real-power value of -702W and -397W, the negative reactive power peaked as well at 2256var and 1173var. Values before/after that one-second-peak are at level of -200 to -400var. So -Q peaked by -2000var and -900var in the same moment:
please look at my posting below:
It shows P jumping from +387W to -397W backto +420W, whereas Q does the same from -245var to -1173var back to -261var.
So, for that second in between: P peaks down -800W and Q peaks down -900var.
Hi - I got the Landis & Gyr meter from WienerNetze some month ago and since end of Jannuary I am also able to decrypt data (with the new key ;-)
And yes, I can confirm a time offset of about 4 seconds.
not that far... the thing is that HA integration need to use external libs for all communications with external devices... there is the gurux serial package but there are still some issues with reading the message (see http://gurux.fi/node/17804)
Thanks to the discussion I was looking for, too.
At the moment the web portal of Wiener Netze does not show any key at all. Al lI got is a 500 Internal Server Error-answer for some seconds.
Where did you knock on the door? Wiener Netze do not seem to be prepared for answers regarding this topic. Al the answers regarding the Server Error (which of course belongs to them) were incompetent.
Telephone call didn't help, info@w..., kundendienst@w... did not help. E-control told me to make an official request.
Thanks a lot for the decryption codes you provided. I think I wouldn't have got it.
Dear Pocki,
since I read that you are decrypting with php I tried to read the /dev/ttyUSB0-stream by php - without any success (not for the Wiener Netze Iskra AM 550 - damned lockdown, but) for the L&G E350. With my bash-script I can read, but with php - no chance. I am not able to listen to /dev/ttyUSB0 byte by byte.
Any hint appriciated.
ps: I got the code for Wiener Netze, but I have to wait to come to Vienna again.
I did not try to read the serial data using php. So sorry, I am not able to share my experience on how to read the meter just with php. I only succeeded in decrypting the data with php, that was read before already.
Instead, I read the serial input using python (v3). That works fine for me.
I would like to get in contact with you Stephan. I found a post from you in the ELV forum about an Arduino Sketch encrypting a SmartMeter from Wiener Netze. Unfortunately you didn't reply there. Maybe you can give some sign!
ok, thank you that you've
ok, thank you that you've tried it!
LG Stefan
...still waiting for
...still waiting for WienerNetze to answer my request.
Last response was on Wednesday December 23rd: "we have not yet received a [internal] feedback regarding your request. We will actively contact you when there is an answer from our technical department".
In the meantime I checked:
- provided key on web portal is still unchanged
- provided data from smartmeter is still garbage when trying to decrypt with that key.
... I got some responses from
... I got some responses from Wiener Netze and wrote some additional messages with a new bug oft the meter firmware.
Finally I got the message that failures were found and that the developers are working on it.
Ok everyone can write such a messge :-) Maybe that is true or not.... I don't know but thats the first message where the say that something is not working as expected.
The will provide an update as soon as possible. Normally they need weeks for a response. We will see how much weeks they will need for bugfixing :D
Have a nice day
Stefan
Thanks for the info Stefan.
Thanks for the info Stefan. So lets meet up again here in one year and see if they actually did something. (last time it took almost a year for the option to reappear in the webinterface)
I just got an info from a
I just got an info from a digitalgridsolutions' support representative: He told he opened an "official support ticket at Iskraemeco" and will get back to me as soon as he receives feedback.
That same guy recently asked for a couple of HDLC frames of my meter, as he wanted to test some issue suspect. But I did not get more details on that, anyway I suppose he found/proved something and used his findings within the "official support ticket".
@poki
@pocki
till now i reportet three bugs at wiener netze:
1) hdlc frame header is sometimes shorter than expected and wrong
2) length fields are calculated in wrong way
3) Push messages are not decryptable
if you need more infos on that, you can have examples i received from my meter where you can see the bugs.
I just received a statement
I just received a statement from WienerNetze, telling they found some issue related to the AES key and admit it to be wrongly shown in the web portal. Their software developers are working on an appropriate fix, which might be rolled out already this week.
That email was polite, apologetic and included an appreciative value honoring my (our) aim to help on this encryption problem :-)
Well, sounds good. So let's stay tuned and check the key after some days. If I understood right, thw web portal will show another AES key after some days, which will - hopefully - work to dechipher the payload in the HDLC frames.
Great, I'm getting excited ;)
Great, I'm getting excited ;) then I can finally start working on a home assistant integration.
Hi,
Hi,
Thank you for this information. I am also interested in the outcome. :-)
BR,
Mikko
Now I see a new key in the
Now I see a new key in the web portal. Decryption works with it now.
Hurray :-)
yes indeed, the key has
yes indeed, the key has changed and now I can decrypt all the packages (also past ones)
now I just need to make sense of the notification body contents... anybody want to team up?
Descrambled it already. Mine
Descrambled it already. Mine looks like this:
0f 00 59 7f c4 -> header
0c 07e5011b030d271d00ffc400 ->timestamp
02 09 09 -> inner header
0c 07e5011b030d271d00ffc400 -> timestamp
06 00448222 -> Wh+
06 0000053e -> Wh-
06 0001004b -> varh+
06 001c1f5f -> varh-
06 000001b4 -> W+ (current)
06 00000000 -> W- (current)
06 00000000 ->var+ (current)
06 000000dc ->var- (current
with gurux py decoding gave
with gurux py decoding gave me:
<DataValue>
<Structure Qty="09" >
# 22.11.2020 16:05:34+01:00
<OctetString Value="07E40B160710052200FFC400" />
<UInt32 Value="00610ED6" />
<UInt32 Value="00000001" />
<UInt32 Value="00000ABB" />
<UInt32 Value="0021C1F9" />
<UInt32 Value="00001A26" />
<UInt32 Value="00000000" />
<UInt32 Value="00000000" />
<UInt32 Value="00000096" />
</Structure>
</DataValue>
not really sure what to do with that...
that first octet-string is
that first octet-string is the timestamp:
0x07e4 is year 2020
0x0b is month 11
0x16 is day 22
0x07 is weekday (7=sunday)
0x10, 0x05, 0x22 is hh,mi,ss
following 8 values are 4byte counters.
First is the total real energy pulled off the power grid in Wh.
0x00610ED6 is 0x00*256^4 + 0x06*256^3 + 0x10*256^2 + 0x0e*256 + 0xde = 6360790 Wh. Divide by 1000 and that's the value 6360 or 6361 what you may read from the smartmeter's display.
5th value 0x00001A26 ist the current power consumption in W: that is 6694W.
I tried to put that into a
I tried to put that into a python script (2.7):
See https://gist.github.com/pocki80/941fa090a8d6269a9b3b68c195f8750f
Oder falls es mit Python oder
Oder falls es mit Python oder dessen Crypto-Modul bröselt: hier eine BASH version (ohne CRC check):
https://gist.github.com/pocki80/8d5aed9a5b99e9731130819229e15a37
last night I also got the
last night I also got the update and now the output of my smart meter looks like yours with the updated key.
That are very good news for me, because now I can start with my Homematic energy sensor integration.
Gentleman it was a pleasure to push wiener netze forwards with you ;-)
Thank you very much!
It works in PHP as well:
It works in PHP as well:
https://gist.github.com/pocki80/36144abf1582ec1cb0b5aa30f677494f
Hi - after some playing with
Hi - after some playing with de decrypted data of my WienerNetze Iskraemeco AM550 smartmeter I realized something: each data packet (sent each second) includes a timestamp. In the moment of data transmission, that timestamp is 4 or 5 seconds in the past.
That means: at 10:00:00 i receive and decrypted data including the timestamp 09:59:55. Next second (at 10:00:01) the next packet includes timestamp 09:59:56.
Can anybody confirm this?
Does it mean, WienerNetze does not properly sync their smartmeters' system time to a NTP server?
I cannot imagine that these smart meters buffer their packets 5 seconds prior sending to infrared interface.
can't confirm, my packages
can't confirm, my packages are exactly on time.
HI,
HI,
DLMS meters usually don't use NTP. The time of the meter is usually updated when meter is read by the meter company. There are several reasons for this, but one of them is that there is no active TCP/IP connection and the meter simply can't use NTP.
I believe that this will change and meters will start to use NTP in the future.
BR,
Mikko
I see. Thx for explanation.
I see. Thx for explanation. As WienerNetze polls the data every night, I guess their either do not frequently update the smartmeter's system clock, or their own internal system time is -4 to -5 seconds off sync.
Another question to
Another question to WienerNetze SmartMeter interface data:
I recieve real power pulled from the grid (+P in W) and pushed into the grid (-P in W) as seperated values.
During the last 3 days I captured for 2 individual moments a value on -P, that I cannot explain for any reason:
Tue 2 Feb 07:47:00 CET 2021: +P: 0W, -P 702W
Wed 3 Feb 10:20:30 CET 2021: +P: 0W, -P 397W
powerplus powerminus
1612248420: 0.0000000000e+02 7.0200000000e+02
1612344030: 0.0000000000e+02 3.9700000000e+02
In both cases, the second before and after that "false peak" hat +P=~500W and -P=0W for several minutes. And: -P=0W almost all time, as my two small photovoltaic modules don't ever produce enough power to fullfil the house base consumption (dispite they are not strong enough to produce more than 200W at all during winter times and cloudy, dark sky)
What could cause such a "negative peak" in power measurement?
Is there any device that might de-charge with such a peak (i.e. a fully charged notebook that is plugged in)?
Looks like a measurement error?
Hi,
Hi,
It's possible that date-time is not updated on every reading. The clock is usually read and checked what is the deviation. If the deviation is in the given range it's not updated. Max range depends on the country. In some countries, it might be even 5 mins.
BR,
Mikko
Hi,
Hi,
A big engine might cause reactive power peak value (-Q). It's hard to say but I'm sure that it isn't the notebook.
BR,
Mikko
Hi. Are you talking about
Hi. Are you talking about "reactive power" in the meaning of unit "volt-amperes-reactive" and "unused power"? If yes, that is reported as +Q and -Q in two seperate values.
I get -P in Watts (not volt-amperes-reactive) and as a unit of real/active power, that is pushed to the grid.
What other kind of device could cause such a (negative) power peak in an usual one-family-house? There's no big engine, but I have connected: an ups (620VA), a small electrical fan for our bathroom, an induction cooking plate, a daikin air condition unit, a 600Wp PV panel with micro-inverter, and other typical household stuff like an iMac, Notebook, fridge, vakuum cleaner, water-pump for heating,...
Well, an 3-phase-AC power grid is quite a complex thing :-)
I checked the data of these 2
I checked the data of these 2 moments once again and realized that in parallel to the negative real-power value of -702W and -397W, the negative reactive power peaked as well at 2256var and 1173var. Values before/after that one-second-peak are at level of -200 to -400var. So -Q peaked by -2000var and -900var in the same moment:
timestamp: realpowerplus realpowerminus reactivepowerplus reactivepowerminus
1612344029: 3.8700000000e+02 0.0000000000e+00 0.0000000000e+00 2.4500000000e+02
1612344030: 0.0000000000e+00 3.9700000000e+02 0.0000000000e+00 1.1730000000e+03
1612344031: 4.2000000000e+02 0.0000000000e+00 0.0000000000e+00 2.6100000000e+02
Hi,
Hi,
Yes, What is -Q value? Is it zero?
BR,
Mikko
please look at my posting
please look at my posting below:
It shows P jumping from +387W to -397W backto +420W, whereas Q does the same from -245var to -1173var back to -261var.
So, for that second in between: P peaks down -800W and Q peaks down -900var.
Hi - I got the Landis & Gyr
Hi - I got the Landis & Gyr meter from WienerNetze some month ago and since end of Jannuary I am also able to decrypt data (with the new key ;-)
And yes, I can confirm a time offset of about 4 seconds.
How far are you with your
How far are you with your home assistant integration? I even struggle with the ESP8622 firmware to send MQTT topics.
not that far... the thing is
not that far... the thing is that HA integration need to use external libs for all communications with external devices... there is the gurux serial package but there are still some issues with reading the message (see http://gurux.fi/node/17804)
Thanks to the discussion I
Thanks to the discussion I was looking for, too.
At the moment the web portal of Wiener Netze does not show any key at all. Al lI got is a 500 Internal Server Error-answer for some seconds.
Where did you knock on the door? Wiener Netze do not seem to be prepared for answers regarding this topic. Al the answers regarding the Server Error (which of course belongs to them) were incompetent.
Telephone call didn't help, info@w..., kundendienst@w... did not help. E-control told me to make an official request.
Thanks a lot for the decryption codes you provided. I think I wouldn't have got it.
I wrote to smartmeter@ and
I wrote to smartmeter@ and beschwerden@
BR
Since I read that you are
Dear Pocki,
since I read that you are decrypting with php I tried to read the /dev/ttyUSB0-stream by php - without any success (not for the Wiener Netze Iskra AM 550 - damned lockdown, but) for the L&G E350. With my bash-script I can read, but with php - no chance. I am not able to listen to /dev/ttyUSB0 byte by byte.
Any hint appriciated.
ps: I got the code for Wiener Netze, but I have to wait to come to Vienna again.
I did not try to read the
I did not try to read the serial data using php. So sorry, I am not able to share my experience on how to read the meter just with php. I only succeeded in decrypting the data with php, that was read before already.
Instead, I read the serial input using python (v3). That works fine for me.
Hi!
Hi!
I would like to get in contact with you Stephan. I found a post from you in the ELV forum about an Arduino Sketch encrypting a SmartMeter from Wiener Netze. Unfortunately you didn't reply there. Maybe you can give some sign!
Thanks,
Michael ...