While executing example java meter server, I noticed that while I have created a single thread to listen on 1 socket but there are other ~24 threads executing.
Please let me know whether these 24 threads are mandatory ? how can i remove them? I need to simulate millions of example java meter server.
The network component for java is not designed to serve millions of clients.
You need to implement a network client that uses IOCP if you want to do it.
Do you think IOCP (Asynchronous I/O) will have better performance than Non-Blocking I/O?
IOCP and Non blocking I/O are definitely better than blocking I/O in terms of system resources.
Hi here, I am same task. Simulate "millions" of DLMS meters virtually. I have started some proof of concept using java nio (I think that it is equivalent to IOCP) and I have no problem when listening n multiple IP addresses and ports (100000 endpoints setup in just a second or so).
Do you have some working? I am in phase of "implement decoding and answering for requests".
I have seen given page. I do not have windows environment to try it easily, but according given page, given simulators purpose is to "record" some real device and than simulate given device in single instance.
I have similar need as in first post. I need to simulate "millions" of virtual devices listening on TCP/IP. So for example listening on ports from 2000 till 62000 on few IP addresses on single machine.
Is your simulator able to do something like that? Can I use your simulator as an library/framework and put my code (which is able to listen on millions of sockets) together to parse/reply on requests?
PS: According my knowledge, using "thread per connection" will result in at most few thousands sockets with poor latency. Using epoll (linux syscall), we can scale much higher with single thread.
If I am not wrong (assumptions based on serial bus example located on github, https://github.com/Gurux/gurux.dlms.java/tree/master/gurux.dlms.server… ), I need to implement my own IGXMedia interface and bind it to my implementation of virtual mettering device (some implementation of GXDLMSSecureServer2. for example GXDLMSBase in case of sample github code).
My question is, which methods are crucial to IGXMedia interface? It seems that there are many (from my task point of view) functionless methods :)
Side question what purpose does IGXMedia2 interface have?
I have implemented "NIO" version of GXNet partially and found out, that there are perhaps more performance issues that I can have in my tiny testing project. I have decided to go "hardcoded" way and I am simulating DLMS device just by having one map of requests and responses. This way I have done some testing and I am now before dilema, how to handle paged profiles. I am able to code some simple state machine (because of "next page" packets, which are same for all pages), but I feel that using GXDLMS would solve many of my issues (like parsing clock filter).
Two things:
* how can I parse DLMS packet (in wrapper frame) using GXDLMS library? (i.e. not using GXDLMSServer2 and implementing something like it by hand)
* are you willing to replace GXByteBuffer by java NIO Buffer class? Do you like this idea? I would consider implementing it and making pull request for this. It seems it would be a bit big change, because of wrappers like GXServerReply ReceiveEventArgs and simmular.
Hardcoding the messages will work when you have a small number of devices and you want to test only communication channels. You will have problems when messages are lost and client will try to re-read data.
Parsing server-side messages is implemented to GXDLMSServer2. You can parse server-side data to XML format using GXDLMSTranslator messageToXml-method, but I'm afraid that it might too slow for this purpose.
I need to think if GXBytebuffer would use NIO buffer internally.
Hi, thanks for info. I see now where is parsing done. I would like to have a small philosophic call with you about code architecture through :)
PS: Now I have task to simulate just single power meter type. I know that hardcoding is not effective way and so I would like to find a way to contribute to gx framework and make my work available for all users...
Hi,
Hi,
There is a thread for each client connection.
The network component for java is not designed to serve millions of clients.
You need to implement a network client that uses IOCP if you want to do it.
BR,
Mikko
BR,
Mikko
Thanks Mikko. Do you have any
Thanks Mikko. Do you have any plans to implement IOCP?
Regards,
Utkarsh
Hi,
Hi,
At the moment no. Our clients are not interested in it.
BR,
Mikko
Hi,
Hi,
Do you think IOCP (Asynchronous I/O) will have better performance than Non-Blocking I/O?
IOCP and Non blocking I/O are definitely better than blocking I/O in terms of system resources.
Regards,
Utkarsh
Hi,
Hi,
IOCP is using a lot of less memory than Non-Blocking I/O and it's much faster.
BR,
Mikko
Hi here, I am same task.
Hi here, I am same task. Simulate "millions" of DLMS meters virtually. I have started some proof of concept using java nio (I think that it is equivalent to IOCP) and I have no problem when listening n multiple IP addresses and ports (100000 endpoints setup in just a second or so).
Do you have some working? I am in phase of "implement decoding and answering for requests".
Hi,
Hi,
Have you checked this?
http://gurux.fi/Gurux.DLMS.Simulator
BR,
Mikko
I have seen given page. I do
I have seen given page. I do not have windows environment to try it easily, but according given page, given simulators purpose is to "record" some real device and than simulate given device in single instance.
I have similar need as in first post. I need to simulate "millions" of virtual devices listening on TCP/IP. So for example listening on ports from 2000 till 62000 on few IP addresses on single machine.
Is your simulator able to do something like that? Can I use your simulator as an library/framework and put my code (which is able to listen on millions of sockets) together to parse/reply on requests?
PS: According my knowledge, using "thread per connection" will result in at most few thousands sockets with poor latency. Using epoll (linux syscall), we can scale much higher with single thread.
If I am not wrong
If I am not wrong (assumptions based on serial bus example located on github, https://github.com/Gurux/gurux.dlms.java/tree/master/gurux.dlms.server… ), I need to implement my own IGXMedia interface and bind it to my implementation of virtual mettering device (some implementation of GXDLMSSecureServer2. for example GXDLMSBase in case of sample github code).
My question is, which methods are crucial to IGXMedia interface? It seems that there are many (from my task point of view) functionless methods :)
Side question what purpose does IGXMedia2 interface have?
Hi,
Hi,
You are right. There are a lot of methods that you don't need. I believe that best way to start is if you take GXNet as a base and modify it.
IGXMedia2 is used with some custom medias to improve asynchronous communication.
You don't need to implement that.
BR,
Mikko
I have implemented "NIO"
I have implemented "NIO" version of GXNet partially and found out, that there are perhaps more performance issues that I can have in my tiny testing project. I have decided to go "hardcoded" way and I am simulating DLMS device just by having one map of requests and responses. This way I have done some testing and I am now before dilema, how to handle paged profiles. I am able to code some simple state machine (because of "next page" packets, which are same for all pages), but I feel that using GXDLMS would solve many of my issues (like parsing clock filter).
Two things:
* how can I parse DLMS packet (in wrapper frame) using GXDLMS library? (i.e. not using GXDLMSServer2 and implementing something like it by hand)
* are you willing to replace GXByteBuffer by java NIO Buffer class? Do you like this idea? I would consider implementing it and making pull request for this. It seems it would be a bit big change, because of wrappers like GXServerReply ReceiveEventArgs and simmular.
Hi,
Hi,
Hardcoding the messages will work when you have a small number of devices and you want to test only communication channels. You will have problems when messages are lost and client will try to re-read data.
Parsing server-side messages is implemented to GXDLMSServer2. You can parse server-side data to XML format using GXDLMSTranslator messageToXml-method, but I'm afraid that it might too slow for this purpose.
I need to think if GXBytebuffer would use NIO buffer internally.
BR,
Mikko
Hi, thanks for info. I see
Hi, thanks for info. I see now where is parsing done. I would like to have a small philosophic call with you about code architecture through :)
PS: Now I have task to simulate just single power meter type. I know that hardcoding is not effective way and so I would like to find a way to contribute to gx framework and make my work available for all users...