Reading from Serial interface with GuruxDLMS and Python

Forum Rules

Before commenting read Forum rules

Don't comment the topic if you have a new question.

You can create a new topic selecting correct category from Gurux Forum and then create a new topic selecting "New Topic" from the top left.

By mrphoenix , 7 March, 2020
Forums
Gurux.DLMS

Hi there,
This is my first post in this forum, so please be patient in asking dump questions.

I'm trying to connect my Siemens IM350 Smartmeter to my Raspberry (which is running a volkszaehler.org-middleware).

I'm "learning" python since yesterday and you can imagine that I don't have that much clue what I'm actually doing :-)

By now I'm able to get data via a serial interface on my Raspberry by utilizing a simple python-script that I found somewhere:

############
import serial

ser = serial.Serial(
port='/dev/ttyUSB0',
baudrate = 115200,
parity=serial.PARITY_NONE,
stopbits=serial.STOPBITS_ONE,
bytesize=serial.EIGHTBITS,
timeout=1
)

while 1:
x=ser.readline().encode('hex')
print x
############

When I redirect the output of that to a text-file and paste the content of it to the Gurux DLMS Transfer-Tool, I get readable content. Of course the DLMS-Packets (?) are somewhat truncated, but the Transfer-Tool seems to be able to handle that.

When feeding the same text-file to the python-script

############
data = open("fromSerial.txt","r")

# decrypting received data
decrypt = GXDLMSTranslator(TranslatorOutputType.SIMPLE_XML)
decrypt.comments = True
decrypt.blockCipherKey =GXByteBuffer.hexToBytes("my key")

xml = decrypt.messageToXml(GXByteBuffer.hexToBytes(data))

# output
print(xml)
############

I get an "invalid hex-string"-error.

So long story made short:
Can someone please help me in reading data from a serial interface so that the GXDLMSTranslator is able to decrypt?
My Smartmeter doesn't need to be initialized or something like that. it simply pushed (decrypted) data out once per second.

Thank you very much for any help,
Cheers,
Philipp

mrphoenix

6 years 3 months ago

Hi Mikko,
Thx for coming back to me that fast.
I've installed gurux.serial from github and read through your code a few times. Finally I found the place to to set the correct serial-port.
After trying a few things I have to admit that I don't understand how this works. I get 4 lines of response from my server. After that the example-code bails with the exception "Failed to receive reply from the server".
Maybe it's better to wait for next week or find someone who actually understands python.
Cheers,
Philipp

Kurumi

6 years 3 months ago

Hi,

Gurux.DLMS.Push.Listener.Example.python is added to GitHub.
run it and received DLMS data is print to the console.

BR,
Mikko

mrphoenix

6 years 3 months ago

Hi Mikko,
Thx for letting me know about the update!
I've installed it today. The code works, but the output of it looks odd.

Simply be running "main.py -S /dev/ttyUSB0" I get that output:

gurux_dlms version: 1.0.53
gurux_net version: 1.0.12
gurux_serial version: 1.0.13
<gurux_serial.GXSerial.GXSerial object at 0x76175d30>
Press any key to close the application.
Media state changed. MediaState.OPENING
trace:20:21:22 16 53 65 74 74 69 6E 67 73 3A 20 50 6F 72 74 3A 20 2F 64 65 76 2F 74 74 79 55 53 42 30 20 42 61 75 64 20 52 61 74 65 3A 20 42 61 75 64 52 61 74 65 2E 42 41 55 44 5F 52 41 54 45 5F 39 36 30 30 20 44 61 74 61 20 42 69 74 73 3A 20 38 20 50 61 72 69 74 79 3A 20 50 61 72 69 74 79 2E 4E 4F 4E 45 20 53 74 6F 70 20 42 69 74 73 3A 20 53 74 6F 70 42 69 74 73 2E 4F 4E 45 20 45 6F 70 3A 31 32 36
Media state changed. MediaState.OPEN
New data is received. /dev/ttyUSB0:91
New data is received. /dev/ttyUSB0:6C 05 E7 A3 8A FD
New data is received. /dev/ttyUSB0:F5
New data is received. /dev/ttyUSB0:1A B3 4A C1 32 DF

None
Server address:0 Client Address:0
New data is received. /dev/ttyUSB0:D5
New data is received. /dev/ttyUSB0:13 D3 C5 65 2C F8
New data is received. /dev/ttyUSB0:58
New data is received. /dev/ttyUSB0:C5 AF DF 95 F3 91 F8

None
Server address:0 Client Address:0
New data is received. /dev/ttyUSB0:B1
New data is received. /dev/ttyUSB0:89 75 48 D1 B9 A3
New data is received. /dev/ttyUSB0:BD
New data is received. /dev/ttyUSB0:8D D5 EA 51 EC EF

None
Server address:0 Client Address:0
New data is received. /dev/ttyUSB0:91
New data is received. /dev/ttyUSB0:C2 55 E2 A9 2B 1E FF
New data is received. /dev/ttyUSB0:31

By specifying different settings for the Serial-Port (python main.py -S /dev/ttyUSB0:115200:8None1
) the output doesn't look any different to me:

gurux_dlms version: 1.0.53
gurux_net version: 1.0.12
gurux_serial version: 1.0.13
<gurux_serial.GXSerial.GXSerial object at 0x760a4d30>
Press any key to close the application.
Media state changed. MediaState.OPENING
trace:20:34:13 16 53 65 74 74 69 6E 67 73 3A 20 50 6F 72 74 3A 20 2F 64 65 76 2F 74 74 79 55 53 42 30 20 42 61 75 64 20 52 61 74 65 3A 20 31 31 35 32 30 30 20 44 61 74 61 20 42 69 74 73 3A 20 38 20 50 61 72 69 74 79 3A 20 50 61 72 69 74 79 2E 4E 4F 4E 45 20 53 74 6F 70 20 42 69 74 73 3A 20 31 20 45 6F 70 3A 31 32 36
Media state changed. MediaState.OPEN
New data is received. /dev/ttyUSB0:51
New data is received. /dev/ttyUSB0:D9 ED 7A B3 B9 1E
New data is received. /dev/ttyUSB0:7A
New data is received. /dev/ttyUSB0:85 2A 64 EB 22 76

None
Server address:0 Client Address:0
New data is received. /dev/ttyUSB0:78
New data is received. /dev/ttyUSB0:33 5C 2E 45 AF F6
New data is received. /dev/ttyUSB0:BD
New data is received. /dev/ttyUSB0:0D 0F 7D 7B B7 A2

None
Server address:0 Client Address:0
New data is received. /dev/ttyUSB0:B8
New data is received. /dev/ttyUSB0:C5 C3 D5 FE 2E 15
New data is received. /dev/ttyUSB0:31
New data is received. /dev/ttyUSB0:E3 D4 67 47 09 FF

The received data looks a little short and isn't decoded by the DLMS-Decoder provided within Gurux DLMS Director.

Do you have any clue on what's going on here? :-)

Cheers,
Philipp

Kurumi

6 years 3 months ago

Hi,

Update gurux_serial to the latest version (1.0.14).
run pip install -r reguirements.txt

It shows used serial port settings at the command line.
Your baudrate is different. Try with this:

main.py -S /dev/ttyUSB0:115200:8None1

BR,
Mikko

mrphoenix

6 years 3 months ago

Hi Mikko,
Updating didn't fix the issue either. Don't know what's going on here.
When using my own code to read the interface it works pretty well. I just need to figure out how to interpret the correct eop-character.
I will let you know about my findings on the weekend. Going out isn't possible anymore due to things happening right now. So I have enough time to fumble with this little project :-)
Have a great weekend,
Philipp

Kurumi

6 years 3 months ago

Hi Philipp,

Thank you for this info. Can you share what kind of USB connector you are using (manufacturer and model)? We'll order the same. Are you receiving any data from the meter?

BR,
Mikko

mrphoenix

6 years 3 months ago

Good morning Mikko,
I'm using a CP2102-Adapter connected to a Raspberry Pi 3:
https://www.amazon.de/gp/product/B07TFSZ3ZP/ref=ppx_yo_dt_b_asin_title_…

I'm able to receive messages by using the code below:
----------------------------------------------------------------------------------------
#!/usr/bin/env python
import time
import serial

ser = serial.Serial(
port='/dev/ttyUSB0',
baudrate = 115200,
parity=serial.PARITY_NONE,
stopbits=serial.STOPBITS_ONE,
bytesize=serial.EIGHTBITS,
timeout=1
)

message = ""
breaker = 0

while breaker != 1:
# reading serial
readT0=ser.read().encode('hex')
message += readT0

if readT0 == "7e":
readT1 = ser.read().encode('hex')
if readT0 == readT1:
#Breaking the loop
print "-line-"
#breaker = 1
message = "7e" + message
print message
message = ""
----------------------------------------------------------------------------------------
Not every packet works because my end of packet-detection doesn't work as intended.

The Push-Example you provided delivers data like I posted in reply #5 regardless of what gurux_serial version I use. Maybe because the highest baudrate I could find in your code (GXSerial.py) is defined as gurux_common.io.BaudRate.BAUD_RATE_38400?

cheers,
Philipp

Kurumi

6 years 3 months ago

Hi Philipp,

Good news. We have same module so we can test this. I have couple of questions.
1. What PINs you have connected (TX, RX and GRN or also something else)?
2. What python version you are using?

BR,

Mikko

mrphoenix

6 years 3 months ago

Hi!
Only RX is connected since I'm only receiving push-messages from my meter.
Python 2.7 is installed.
Cheers,
Philipp

Kurumi

6 years 3 months ago

Hi Philipp,

Get the latest version (1.0.58). There were a few issues for Raspberry PI Python 2.7 version.

BR,
Mikko

mrphoenix

6 years 2 months ago

Hi Mikko,
I've removed Python 2.7 completely and (re-)installed Python 3.7.3 as well as updated to Version 1.0.58.
The output doesn't look any better though:

gurux_dlms version: 1.0.58
gurux_net version: 1.0.17
gurux_serial version: 1.0.14
/dev/ttyUSB0:115200 8NONE2
Press any key to close the application.
Media state changed. MediaState.OPENING
trace:20:36:33 TraceTypes.INFO Settings: Port: /dev/ttyUSB0 Baud Rate: 115200 Data Bits: 8 Parity: Parity.NONE Stop Bits: 1 Eop:126
Media state changed. MediaState.OPEN
New data is received. /dev/ttyUSB0:B0
New data is received. /dev/ttyUSB0:99 08 EF DA A7 D3
New data is received. /dev/ttyUSB0:B8
New data is received. /dev/ttyUSB0:C9 CD 47 7B 35 A6
New data is received. /dev/ttyUSB0:51
New data is received. /dev/ttyUSB0:58 AE 74 3F 32 F8
New data is received. /dev/ttyUSB0:31
New data is received. /dev/ttyUSB0:BB A3 4E 75 ED 91
New data is received. /dev/ttyUSB0:51
New data is received. /dev/ttyUSB0:8E F4 24 A0 67 FE
New data is received. /dev/ttyUSB0:78
New data is received. /dev/ttyUSB0:33 C0 57 91 B7 DE A6
New data is received. /dev/ttyUSB0:7A
New data is received. /dev/ttyUSB0:B3 BA 74 F4 B3 C5 FC
New data is received. /dev/ttyUSB0:3A
New data is received. /dev/ttyUSB0:25 F9 A9 64 96 9E FF
New data is received. /dev/ttyUSB0:91
New data is received. /dev/ttyUSB0:C6 BA B8 2F 94 FF
New data is received. /dev/ttyUSB0:31
New data is received. /dev/ttyUSB0:49 2E 51 DD 05 68 FF
New data is received. /dev/ttyUSB0:91
New data is received. /dev/ttyUSB0:F3 5E 8E 6A BA F2 FE
^C
Media state changed. MediaState.CLOSING
Media state changed. MediaState.CLOSED

Regardless of that, pressing <CTRL>+<C> does actually stop the code now. It didn't do that before :-)

Maybe a type in the code, maybe something else: I called your code with "python3 main.py -S /dev/ttyUSB0:115200:8None1". The Output states "/dev/ttyUSB0:115200:8NONE2"

Cheers,
Philipp

Kurumi

6 years 2 months ago

Hi Philipp,

Thank you for your observation. There was an issue with the example. Stop bits enum value was wrong.
Get the latest version. I'm not quite sure why our example listener works correctly.

BR,

Mikko

mrphoenix

6 years 2 months ago

Hey,

Updated the code, no change in the behavior though.
Maybe because "Parity is set to "Parity.NONE" instead of just "NONE"?

Cheers,
Philipp

Kurumi

6 years 2 months ago

Hi,

Parity.NONE is defined enum value. That is not the case here.
Are you sure from the baud rate 115200? Usually, it's 9600.

BR,
Mikko

mrphoenix

6 years 2 months ago

Hi!
I'm not entirely certain but as my own code is able to retrieve full packages with 115200 and calling your code with 9600 delivers the same results I think I'm on the right track. Maybe my Siemens IM350 is delivering some oddities?
Cheers,
Philipp

Kurumi

6 years 2 months ago

Hi Philipp,

The first and last byte of the packet should be 0x7e. I believe that one of your serial port settings is not correct and that is causing this issue. I believe that when you start to receive packets that start with 7E this is solved. All our meters are using 9600 so we can't test this.

BR,
Mikko