Convert/forward P1 port and optical sensor data

Node-red Smartmeter

Node-red flow for converting/forwarding smartmeter P1 port and optical sensor for water meter data.

This flow feeds three applications:

• the Toon thermostat through;
  • the serial out node (which is connected to the Toon Meetmodule) for energy and gas usage.
  • the http in and http response nodes for water usage.
    Toon needs to be rooted and have the toonWater app installed for this to work.
    See also https://domoticaforum.eu/viewtopic.php?f=99&t=13090.
• Domoticz through the tcp out node
  Use the P1 Smart Meter with LAN interface hardware device in Domoticz and set the connection parameters to the IP address of Node-red at port 2000.
  See also https://www.domoticz.com/wiki/Dutch_DSMR_smart_meter_with_P1_port.
• Node-red Dashboard through the mqtt out node
  The telegram is received by Node-red and is translated to display as depicted above.
  Note that the above displayed dashboard is not part of the flow!

P1 converting and forwarding

The flow takes input from a P1 to USB cable (https://www.sossolutions.nl/slimme-meter-kabel) which is forwarded to TCP and MQTT nodes.
It also converts ESMR 5.0 telegrams to DSMR 2.2 spec to feed for example old Toon Meetmodules through a seral out node.
This is useful when the firmware of the module cannot be updated due to hardware limitations and therefor requires old style telegrams.
This flow is running on a PC with real serial ports and a great advantage of the Toon Meetmodule is that it accepts real serial data (RS-232) as input, provided the connection is setup at 9600bps, 7E1.

Using a Toon optical (gas)sensor to read an analog water meter.

The optical sensors that come with a Toon thermostat can be connected directly to a Raspberry Pi and the pulses can be read with Node-red either through the pi gpiod or the rpi-gpio in nodes.

In this flow a pi gpiod node is used and configured like below:

Make sure the resistor is set to Pull-up and given enough Debounce time.
Change the host to whatever IP address your Pi is running at or use the rpi-gpio in node if the pins are local to the Node-red installation.

The node will output a 1 or 0 everytime the metal part on the wheel in the meter passes along the optical sensor.
The sensor will generate a pulse when going from red plastic to metal and vice versa.
This means that 2 pulses equal 1 liter of water, so the number of pulses is divided by 2 to get water consumption in liters.

This is what happens in the flow:

• Node "PIN 7" feeds the "counterGlobal" node with pulses in the form of ones and zeroes and these are added up to the meter total and stored in memory.
• From that total the previous reading from 15 seconds ago is subtracted, giving us the usage in liters.
• This usage is multiplied by 4 to get a flowrate of liters per minute, which is also stored in memory.
• Whenever the http in node receives a request, the stored values are returned as a JSON string: {"waterflow":"0","waterquantity":"438730"}.

Setup the hardware

Place de optical sensor on the water meter:

The meter depicted is a Sensus 620 from Vitens.

The 3.5 mm jack electrical assignment:

A 3.5 mm female jack connector is used onto which some Dupont jumper wires with the colors in the picture were soldered.
Then the original white wire that came with the sensor is plugged into both the sensor itself and the female jack.

Connect the jumper wires to the Pi:

Using Dupont jumper wires might not be the best kind of wires to use, since they are very thin and are susceptible to electrical interference causing false readings.

Copy[{"id":"944dc21e.5aec3","type":"tab","label":"Smart meter I/O","disabled":false,"info":""},{"id":"410389d7.f2bf38","type":"serial out","z":"944dc21e.5aec3","name":"","serial":"e610f32d.1775d","x":700,"y":260,"wires":[]},{"id":"ada9848d.ca2a98","type":"serial in","z":"944dc21e.5aec3","name":"","serial":"47dde4ca.2e265c","x":110,"y":260,"wires":[["4a9aad66.e30864","f7a5282f.6576e8","299105d5.eb36aa","87305046.afcc7","4c831f7e.541e8","999ddc59.cd6d8"]]},{"id":"18cb131b.78a4cd","type":"delay","z":"944dc21e.5aec3","name":"","pauseType":"rate","timeout":"5","timeoutUnits":"seconds","rate":"1","nbRateUnits":"10","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":true,"x":520,"y":260,"wires":[["410389d7.f2bf38","d2cd5f7b.8afe6"]]},{"id":"4a9aad66.e30864","type":"function","z":"944dc21e.5aec3","name":"DSMR 5 => 2.2","func":"node.status({fill:\"red\",shape:\"dot\",text:'Off'})\nvar active = global.get('ttyS0');\nif (active) node.status({fill:\"green\",shape:\"dot\",text:'On'});\nelse return;\n\nvar data = \"\";\nvar telegram=msg.payload;\nvar arr=telegram.split('\\r\\n');\n\nfor (i=0;i<arr.length;i++){\n    if(i===0) {\n        data=arr[i] + '\\r\\n\\r\\n'\n    }\n    if(arr[i].startsWith(\"0-0:96.1.1\")) data += arr[i] + '\\r\\n';\n    if(arr[i].startsWith(\"1-0:1.8.1\")) data += arr[i] + '\\r\\n';\n    if(arr[i].startsWith(\"1-0:1.8.2\")) data += arr[i] + '\\r\\n';\n    if(arr[i].startsWith(\"1-0:2.8.1\")) data += arr[i] + '\\r\\n';\n    if(arr[i].startsWith(\"1-0:2.8.2\")) data += arr[i] + '\\r\\n';\n    if(arr[i].startsWith(\"0-0:96.14.0\")) data += arr[i] + '\\r\\n';\n    if(arr[i].startsWith(\"1-0:1.7.0\")) data += arr[i] + '\\r\\n';\n    if(arr[i].startsWith(\"1-0:2.7.0\")) data += arr[i] + '\\r\\n';\n    //if(arr[i].startsWith(\"0-0:96.13.1\")) data += arr[i] + '\\r\\n';\n    if(arr[i].startsWith(\"0-0:96.13.0\")) data += \"0-0:96.13.1()\\r\\n\" + arr[i] + '\\r\\n';\n    if(arr[i].startsWith(\"0-1:24.1.0\")) data += \"0-1:24.1.0(3)\" + '\\r\\n';\n    if(arr[i].startsWith(\"0-1:96.1.0\")) data += arr[i] + '\\r\\n';\n    if(arr[i].startsWith(\"0-1:24.3.0\")) data += arr[i] + '\\r\\n';\n    if(arr[i].startsWith(\"0-1:24.2.1\")) {\n        gasData = arr[i].split('(');\n        gasData[1] = gasData[1].substring(0, gasData[1].length - 2);\n        gasData[2] = gasData[2].substring(0, gasData[2].length - 4);\n        data += '0-1:24.3.0(' + gasData[1] + ')(08)(60)(1)(0-1:24.2.1)(m3)' + '\\r\\n';\n        data += '(' + gasData[2] + ')' + '\\r\\n';\n        data += '0-1:24.4.0(1)' + '\\r\\n';\n    }\n    if(i===arr.length-1) data += \"!\\r\\n\"\n}\n\nreturn [{payload:data}];","outputs":1,"noerr":0,"initialize":"","finalize":"","x":320,"y":260,"wires":[["18cb131b.78a4cd"]],"inputLabels":["P1 telegram"],"outputLabels":["P1 telegram"],"icon":"font-awesome/fa-plug"},{"id":"d2cd5f7b.8afe6","type":"debug","z":"944dc21e.5aec3","name":"","active":false,"tosidebar":true,"console":false,"tostatus":true,"complete":"payload","targetType":"msg","statusVal":"payload","statusType":"auto","x":710,"y":320,"wires":[]},{"id":"f7a5282f.6576e8","type":"tcp out","z":"944dc21e.5aec3","host":"","port":"2000","beserver":"server","base64":false,"end":false,"name":"","x":300,"y":140,"wires":[]},{"id":"299105d5.eb36aa","type":"mqtt out","z":"944dc21e.5aec3","name":"","topic":"p1/esmr50","qos":"2","retain":"false","broker":"ea38fbc.8a35708","x":310,"y":200,"wires":[]},{"id":"87305046.afcc7","type":"debug","z":"944dc21e.5aec3","name":"","active":false,"tosidebar":true,"console":false,"tostatus":true,"complete":"payload","targetType":"msg","x":310,"y":320,"wires":[]},{"id":"7854c379.d0020c","type":"comment","z":"944dc21e.5aec3","name":"Smart meter In","info":"","x":100,"y":220,"wires":[]},{"id":"102f57ae.ba7648","type":"comment","z":"944dc21e.5aec3","name":"Smart meter Out","info":"","x":680,"y":220,"wires":[]},{"id":"ab420a78.527778","type":"inject","z":"944dc21e.5aec3","name":"","repeat":"","crontab":"","once":true,"onceDelay":0.1,"topic":"ttyS0","payload":"true","payloadType":"bool","x":700,"y":140,"wires":[["f258941d.5617e8"]]},{"id":"7b3ac861.785298","type":"inject","z":"944dc21e.5aec3","name":"","repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"ttyS0","payload":"false","payloadType":"bool","x":700,"y":180,"wires":[["f258941d.5617e8"]]},{"id":"f258941d.5617e8","type":"change","z":"944dc21e.5aec3","name":"","rules":[{"t":"set","p":"ttyS0","pt":"global","to":"payload","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":900,"y":160,"wires":[[]]},{"id":"b177f722.d63718","type":"catch","z":"944dc21e.5aec3","name":"","scope":null,"uncaught":false,"x":100,"y":40,"wires":[["428671e.9bb119"]]},{"id":"428671e.9bb119","type":"debug","z":"944dc21e.5aec3","name":"Error - Smart meter I/O","active":true,"tosidebar":true,"console":true,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":310,"y":40,"wires":[]},{"id":"4c831f7e.541e8","type":"trigger","z":"944dc21e.5aec3","name":"","op1":"","op2":"{\"error\":true}","op1type":"nul","op2type":"json","duration":"2","extend":true,"overrideDelay":false,"units":"s","reset":"","bytopic":"all","topic":"topic","outputs":1,"x":300,"y":380,"wires":[[]]},{"id":"999ddc59.cd6d8","type":"function","z":"944dc21e.5aec3","name":"countersGlobal","func":"var ts = new Date().getTime();\nvar energy = {ts:ts, usage:0, return:0};\nvar gas = {ts:ts, value:0};\nvar arr=msg.payload.split('\\r\\n');\n\nfor (i=0;i<arr.length;i++){\n    if(arr[i].startsWith(\"1-0:1.8.1\")) energy.usage += parseFloat(arr[i].substring(10,20))*1000;\n    if(arr[i].startsWith(\"1-0:1.8.2\")) energy.usage += parseFloat(arr[i].substring(10,20))*1000;\n    if(arr[i].startsWith(\"1-0:2.8.1\")) energy.return += parseFloat(arr[i].substring(10,20))*1000;\n    if(arr[i].startsWith(\"1-0:2.8.2\")) energy.return += parseFloat(arr[i].substring(10,20))*1000;\n\n    if(arr[i].startsWith(\"0-1:24.2.1\")) {\n        gasData = arr[i].split('(');\n        gas.value = parseFloat(gasData[2].substring(0, gasData[2].length - 4))*1000;\n        dateString = gasData[1].substring(0, gasData[1].length - 2);\n        d = new Date();\n        d.setFullYear(parseInt(dateString.substring(0,2))+2000);\n        d.setMonth(parseInt(dateString.substring(2,4))-1);\n        d.setDate(parseInt(dateString.substring(4,6)));\n        d.setHours(parseInt(dateString.substring(6,8)));\n        d.setMinutes(parseInt(dateString.substring(8,10)));\n        d.setSeconds(parseInt(dateString.substring(10,12)));\n        d.setMilliseconds(0);\n        gas.ts = d.valueOf();\n    }\n}\n\nglobal.set(\"energy\", energy);\nglobal.set(\"gas\", gas);\n\nreturn [{payload:{energy, gas}}];","outputs":1,"noerr":0,"initialize":"","finalize":"","x":320,"y":100,"wires":[[]]},{"id":"84716a7d.f9e408","type":"pi-gpiod in","z":"944dc21e.5aec3","name":"","host":"192.168.1.202","port":8888,"pin":"4","intype":"PUD_UP","debounce":"750","read":true,"x":130,"y":540,"wires":[["43f6d31c.c814fc","5412107b.a83fe"]]},{"id":"7c01b293.877f4c","type":"inject","z":"944dc21e.5aec3","name":"setCounter","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"water/setcounter","payload":"436.615","payloadType":"num","x":120,"y":500,"wires":[["43f6d31c.c814fc"]]},{"id":"43f6d31c.c814fc","type":"function","z":"944dc21e.5aec3","name":"counterGlobal","func":"ts = new Date().getTime();\nvalue = parseFloat(msg.payload);\n\npulses = context.get(\"pulses\");\nif (typeof pulses === 'undefined' || pulses === null || pulses === \"\" || msg.topic === \"water/resetpulses\") {\n    pulses = 0;\n    context.set(\"pulses\", pulses);\n} else {\n    if (msg.topic !== \"water/setcounter\") {\n        pulses += value;\n        context.set(\"pulses\", pulses);\n    }\n}\n\nif (value === 0) return;\nif (value === 1) value = 0.5;\n\ncounter = global.get(\"water\");\nif (typeof counter === 'undefined' | counter === 0 | counter === null | counter === \"\") {\n    global.set(\"water\", {\"ts\": ts, \"value\": value});\n    return;\n} else {\n    if (msg.topic == \"water/setcounter\") {\n        counter.value = 0;\n        value = value * 1000;\n    }\n    global.set(\"water\", {\"ts\": ts, \"value\": (counter.value + value)});\n}\n\nnode.status({fill:\"gray\",shape:\"dot\",text:\"pulses: \" + pulses + \" counter: \" + ((counter.value + value)/1000).toFixed(5)})\nreturn [{payload: value}];\n","outputs":1,"noerr":0,"initialize":"","finalize":"","x":320,"y":540,"wires":[[]]},{"id":"9eb87f.d2ed778","type":"mqtt in","z":"944dc21e.5aec3","name":"","topic":"water/setcounter","qos":"2","datatype":"auto","broker":"ea38fbc.8a35708","x":100,"y":600,"wires":[["43f6d31c.c814fc"]]},{"id":"2de606ed.c471ba","type":"comment","z":"944dc21e.5aec3","name":"Water meter In","info":"","x":100,"y":420,"wires":[]},{"id":"36524bd7.ae96e4","type":"inject","z":"944dc21e.5aec3","name":"15 sec.","props":[{"p":"payload"}],"repeat":"15","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":700,"y":480,"wires":[["fa5602cb.01b8a","a97a01f6.aaae9"]]},{"id":"a97a01f6.aaae9","type":"function","z":"944dc21e.5aec3","name":"domoticz","func":"/*\n/json.htm?type=command&param=udevice&idx=IDX&svalue=COUNTER\nIDX = id of your device (this number can be found in the devices tab in the column \"IDX\")\nCOUNTER = Integer of the overall total volume.\n\nWhen there is a counter created, there is a possibility to change the units by clicking on \"Change\" at the utility tab.\n\nEnergy (kWh)\nGas (m3)\nWater (m3)\nCounter (no unit)\nThe counter will be treated with the divider which is defined in the parameters in the application settings. For example if the counter is set to \"Water\" and the value is passed as liters, the divider must set to 1000 (as the unit is m3). The device displays 2 values:\nThe status is the overall total volume (or counter).\nThe volume (or counter) of the day (in the top right corner).\nThe today's volume (or counter) is calculated from the total volume (or counter).\n*/\n\nvar currCntr = global.get('water');\n\nvar params = {};\nparams.idx = 58;\n//params.nvalue = 0;\nparams.svalue = currCntr.value.toFixed(0);\n\nreturn [{payload: params, topic: 'domoticz/in'}];\n","outputs":1,"noerr":0,"initialize":"","finalize":"","x":880,"y":480,"wires":[["1bd18d6b.41ddb3"]]},{"id":"fa5602cb.01b8a","type":"function","z":"944dc21e.5aec3","name":"flowRate","func":"/*\n Output every 10 seconds;\n - the current counter value\n - the current flow rate in liters per minute\n*/\n\nvar currCntr = global.get('water');\nvar prevCntr = context.get('prevCntr');\nif (typeof prevCntr === 'undefined') prevCntr = currCntr;\nflowRate = (currCntr.value - prevCntr) * 4;\ncontext.set('prevCntr', currCntr.value);\nflow.set('flowRate', flowRate.toFixed(0));\n\nnode.status({fill:'gray',shape:'dot',text:flowRate.toFixed(1)});\nreturn [{payload: {counter: currCntr, flowRate: flowRate}, topic: 'water/usage'}];\n","outputs":1,"noerr":0,"initialize":"","finalize":"","x":880,"y":520,"wires":[["1bd18d6b.41ddb3"]]},{"id":"1bd18d6b.41ddb3","type":"mqtt out","z":"944dc21e.5aec3","name":"PWM-MQT1","topic":"","qos":"0","retain":"false","broker":"ea38fbc.8a35708","x":1070,"y":520,"wires":[]},{"id":"8145bf0a.0a5cb","type":"comment","z":"944dc21e.5aec3","name":"Water meter Out","info":"","x":680,"y":440,"wires":[]},{"id":"5412107b.a83fe","type":"debug","z":"944dc21e.5aec3","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":290,"y":600,"wires":[]},{"id":"603aef80.cfd2b","type":"inject","z":"944dc21e.5aec3","name":"resetPulses","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"water/resetpulses","payload":"0","payloadType":"num","x":110,"y":460,"wires":[["43f6d31c.c814fc"]]},{"id":"f3e391a8.e8149","type":"http in","z":"944dc21e.5aec3","name":"","url":"/water.html","method":"get","upload":false,"swaggerDoc":"","x":680,"y":580,"wires":[["4a0fc01a.4438b"]]},{"id":"c16bda7a.571208","type":"http response","z":"944dc21e.5aec3","name":"","statusCode":"200","headers":{},"x":1060,"y":580,"wires":[]},{"id":"4a0fc01a.4438b","type":"change","z":"944dc21e.5aec3","name":"","rules":[{"t":"set","p":"payload.waterflow","pt":"msg","to":"flowRate","tot":"flow"},{"t":"set","p":"payload.waterquantity","pt":"msg","to":"$string($floor($globalContext(\"water.value\")),false)","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":880,"y":580,"wires":[["c16bda7a.571208","d807203.20680e"]]},{"id":"d807203.20680e","type":"debug","z":"944dc21e.5aec3","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":1070,"y":620,"wires":[]},{"id":"d5bc5837.34a108","type":"http in","z":"944dc21e.5aec3","name":"","url":"/setnew","method":"get","upload":false,"swaggerDoc":"","x":110,"y":660,"wires":[["113cfdc7.b3f302"]]},{"id":"113cfdc7.b3f302","type":"debug","z":"944dc21e.5aec3","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":290,"y":660,"wires":[]},{"id":"e610f32d.1775d","type":"serial-port","serialport":"/dev/ttyS0","serialbaud":"9600","databits":"7","parity":"even","stopbits":"1","waitfor":"","dtr":"none","rts":"none","cts":"none","dsr":"none","newline":"500","bin":"false","out":"interbyte","addchar":"","responsetimeout":"10000"},{"id":"47dde4ca.2e265c","type":"serial-port","serialport":"/dev/ttyUSB0","serialbaud":"115200","databits":"8","parity":"none","stopbits":"1","waitfor":"","dtr":"none","rts":"none","cts":"none","dsr":"none","newline":"500","bin":"false","out":"time","addchar":"","responsetimeout":"10000"},{"id":"ea38fbc.8a35708","type":"mqtt-broker","name":"PWM-MQT1","broker":"PWM-MQT1","port":"1883","clientid":"","usetls":false,"compatmode":false,"keepalive":"60","cleansession":true,"birthTopic":"","birthQos":"0","birthPayload":"","closeTopic":"","closeQos":"0","closePayload":"","willTopic":"","willQos":"0","willPayload":""}]