Node Project Flow Splitter - Export flows to separate YAML files

About

This flow is intended to be used in conjunction with the projects feature and git version control. The flow will be triggered on any change to the flows.json file and will create a subdirectory with separate exports of each flow/tab contained in the current Node-RED project.

These files can then be be managed in git independently of each other (commit, revert, etc.)

Flows are stored in YAML format for improved readability when managing. Function nodes are stored multiline so they can be accurately compared line by line in diffs.

The flow uses node-red-contrib-fs-ops for file system operations.

Configuration

For each feature the nodes in the green "Configuration" group need to be adjusted according to your environment with the flows.json path and the hostname/port of your Node-RED instance.

Once the flow has been configured and deployed, the .yml files are kept up to date fully automatically every time any change is deployed.

.yml files will show up in the project history of your Node-RED project and can be includes in commits like any other files.

Restore/Rollback

As Node-RED itself does not interact with the .yml files in any way, rolling back flows to an earlier state is a two step process:

Revert the changes in question within your git repository and pull the branch using the project history tab in Node-RED.
Use the restore or rollback actions of the splitter flow to import the .yml file of the flow you want to revert. During the import the flow will be converted from YAML back to a JavaScript object and integrated into your flows.json file.

Note: Do not deploy changes to any other flow between these two steps as this would cause the .yml file to be overwritten with the currently deployed state of the flow again.

Note: some third-party nodes may appear with blank styling, and not as they appear in the Node-RED Editor.

[{"id":"2d17751b2e961b97","type":"tab","label":"Flow splitter","disabled":false,"info":"","env":[]},{"id":"547e4f042c7ab792","type":"group","z":"2d17751b2e961b97","name":"","style":{"label":true,"fill":"#000000"},"nodes":["12c03f9dac277564","5944d955dee206fd","c25eddc94757ac68","c802ed30dcee9c3f","8d95b243082b170a","aa2815e4bede1cce"],"x":28,"y":19,"w":884,"h":708},{"id":"6cd281117eb396be","type":"group","z":"2d17751b2e961b97","name":"","style":{"label":true,"fill":"#000000"},"nodes":["b3a51a090c2a72bc","c858e555a2917f4c","78b22525afa66c7d","b6a0d4628676bb41","d2c00cf78ca93bfe","c2cc2e6456adef31","16b9ca225a6c544f"],"x":28,"y":759,"w":1278,"h":188},{"id":"f275b7976d06bdf5","type":"group","z":"2d17751b2e961b97","name":"","style":{"label":true,"fill":"#000000"},"nodes":["2141231fd280640a","5df8de587faba460","1fc03371f827c6f0","308a390849388d87","c91ada61c56cfb99","137a4acc3af2ea3b","e0168c10a646f735"],"x":28,"y":979,"w":1278,"h":188},{"id":"12c03f9dac277564","type":"group","z":"2d17751b2e961b97","g":"547e4f042c7ab792","name":"Configuration","style":{"label":true,"fill":"#92d04f","stroke":"#92d04f","stroke-opacity":"0.5"},"nodes":["31136e327d6c7c09","0c82a98f7baa9d23","965906b0cfe6a4b9","1fcee61a1f6135a2"],"x":54,"y":99,"w":592,"h":122},{"id":"5944d955dee206fd","type":"group","z":"2d17751b2e961b97","g":"547e4f042c7ab792","name":"Delete old files","style":{"label":true},"nodes":["40902e8e73bc3833","8fd61f2f229d433a","fde4ee3e7096a8af","5bf2b9b153d7907f"],"x":54,"y":259,"w":492,"h":82},{"id":"c25eddc94757ac68","type":"group","z":"2d17751b2e961b97","g":"547e4f042c7ab792","name":"Get flow IDs and names","style":{"label":true},"nodes":["27a07350381294ae","b9b7542eeb194fdd","8fabab91600d37e0","a8935cd030059d23","85e874ab171c3fab","825d5bda3805e622"],"x":54,"y":379,"w":832,"h":82},{"id":"c802ed30dcee9c3f","type":"group","z":"2d17751b2e961b97","g":"547e4f042c7ab792","name":"Get flow details","style":{"label":true},"nodes":["0af0a5a888825843","e8d38f4ed8a7fcb5","b08e5395b2dbce75","4669cafc26060cdc"],"x":54,"y":499,"w":522,"h":82},{"id":"8d95b243082b170a","type":"group","z":"2d17751b2e961b97","g":"547e4f042c7ab792","name":"Write {flow}.yml","style":{"label":true},"nodes":["59201f932b963a6f","3fc7f5bbe469e981","a5bd61412368ab45","eaf9d6ac08367cdc"],"x":54,"y":619,"w":652,"h":82},{"id":"c858e555a2917f4c","type":"group","z":"2d17751b2e961b97","g":"6cd281117eb396be","name":"Configuration","style":{"label":true,"fill":"#92d04f","stroke":"#92d04f","stroke-opacity":"0.5"},"nodes":["5c8af20914dad296"],"x":54,"y":839,"w":252,"h":82},{"id":"5df8de587faba460","type":"group","z":"2d17751b2e961b97","g":"f275b7976d06bdf5","name":"Configuration","style":{"label":true,"fill":"#92d04f","stroke":"#92d04f","stroke-opacity":"0.5"},"nodes":["dc3c90fb7aead744"],"x":54,"y":1059,"w":252,"h":82},{"id":"965906b0cfe6a4b9","type":"inject","z":"2d17751b2e961b97","g":"12c03f9dac277564","name":"Run manually","props":[],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","x":290,"y":180,"wires":[["0c82a98f7baa9d23"]]},{"id":"8fabab91600d37e0","type":"split","z":"2d17751b2e961b97","g":"c25eddc94757ac68","name":"","splt":"\\n","spltType":"str","arraySplt":1,"arraySpltType":"len","stream":false,"addname":"","x":570,"y":420,"wires":[["a8935cd030059d23"]]},{"id":"a8935cd030059d23","type":"switch","z":"2d17751b2e961b97","g":"c25eddc94757ac68","name":"Select type","property":"payload.type","propertyType":"msg","rules":[{"t":"eq","v":"tab","vt":"str"}],"checkall":"false","repair":false,"outputs":1,"x":730,"y":420,"wires":[["85e874ab171c3fab"]]},{"id":"31136e327d6c7c09","type":"watch","z":"2d17751b2e961b97","g":"12c03f9dac277564","name":"","files":"/data/projects/Node-RED/flows.json","recursive":"","x":220,"y":140,"wires":[["0c82a98f7baa9d23"]]},{"id":"e8d38f4ed8a7fcb5","type":"change","z":"2d17751b2e961b97","g":"c802ed30dcee9c3f","name":"Build API request","rules":[{"t":"set","p":"url","pt":"msg","to":"http://localhost:1880/flow/{flowId}","tot":"str"},{"t":"change","p":"url","pt":"msg","from":"{flowId}","fromt":"str","to":"payload.id","tot":"msg"},{"t":"set","p":"flowName","pt":"msg","to":"payload.label","tot":"msg"},{"t":"delete","p":"payload","pt":"msg"},{"t":"delete","p":"parts","pt":"msg"},{"t":"delete","p":"headers","pt":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":230,"y":540,"wires":[["0af0a5a888825843"]]},{"id":"b9b7542eeb194fdd","type":"http request","z":"2d17751b2e961b97","g":"c25eddc94757ac68","name":"Get flows","method":"GET","ret":"obj","paytoqs":"ignore","url":"","tls":"","persist":false,"proxy":"","insecureHTTPParser":false,"authType":"","senderr":false,"headers":[],"x":420,"y":420,"wires":[["8fabab91600d37e0"]]},{"id":"0af0a5a888825843","type":"http request","z":"2d17751b2e961b97","g":"c802ed30dcee9c3f","name":"Get flow","method":"GET","ret":"obj","paytoqs":"ignore","url":"","tls":"","persist":false,"proxy":"","insecureHTTPParser":false,"authType":"","senderr":false,"headers":[],"x":420,"y":540,"wires":[["b08e5395b2dbce75"]]},{"id":"59201f932b963a6f","type":"yaml","z":"2d17751b2e961b97","g":"8d95b243082b170a","property":"payload","name":"Convert to YAML","x":430,"y":660,"wires":[["a5bd61412368ab45"]]},{"id":"3fc7f5bbe469e981","type":"function","z":"2d17751b2e961b97","g":"8d95b243082b170a","name":"Build filename","func":"let newmsg = {};\nconst regex = /[\\W]/g;\nnewmsg.fileName = msg.exportPath + msg.flowName.replace(regex, '_').toLowerCase() + \".yml\";\nnewmsg.payload = msg.payload;\n\nreturn newmsg;","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":220,"y":660,"wires":[["59201f932b963a6f"]]},{"id":"a5bd61412368ab45","type":"file","z":"2d17751b2e961b97","g":"8d95b243082b170a","name":"Write files","filename":"fileName","filenameType":"msg","appendNewline":true,"createDir":true,"overwriteFile":"true","encoding":"utf8","x":620,"y":660,"wires":[[]]},{"id":"40902e8e73bc3833","type":"fs-ops-delete","z":"2d17751b2e961b97","g":"5944d955dee206fd","name":"Delete files","path":"exportPath","pathType":"msg","filename":"files","filenameType":"msg","x":390,"y":300,"wires":[["5bf2b9b153d7907f"]]},{"id":"8fd61f2f229d433a","type":"fs-ops-dir","z":"2d17751b2e961b97","g":"5944d955dee206fd","name":"List contents","path":"exportPath","pathType":"msg","filter":"*","filterType":"str","dir":"files","dirType":"msg","x":210,"y":300,"wires":[["40902e8e73bc3833"]]},{"id":"0c82a98f7baa9d23","type":"change","z":"2d17751b2e961b97","g":"12c03f9dac277564","name":"","rules":[{"t":"set","p":"exportPath","pt":"msg","to":"/data/projects/Node-RED/flows/","tot":"str"},{"t":"set","p":"apiHost","pt":"msg","to":"http://localhost:1880","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":480,"y":140,"wires":[["1fcee61a1f6135a2"]]},{"id":"b3a51a090c2a72bc","type":"http request","z":"2d17751b2e961b97","g":"6cd281117eb396be","name":"Create flow","method":"POST","ret":"obj","paytoqs":"ignore","url":"","tls":"","persist":false,"proxy":"","insecureHTTPParser":false,"authType":"","senderr":false,"headers":[],"x":1050,"y":880,"wires":[["16b9ca225a6c544f"]]},{"id":"27a07350381294ae","type":"change","z":"2d17751b2e961b97","g":"c25eddc94757ac68","name":"Build API request","rules":[{"t":"set","p":"url","pt":"msg","to":"{apiHost}/flows","tot":"str"},{"t":"change","p":"url","pt":"msg","from":"{apiHost}","fromt":"str","to":"apiHost","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":230,"y":420,"wires":[["b9b7542eeb194fdd"]]},{"id":"1fcee61a1f6135a2","type":"link out","z":"2d17751b2e961b97","g":"12c03f9dac277564","name":"link out 4","mode":"link","links":["fde4ee3e7096a8af"],"x":605,"y":140,"wires":[]},{"id":"fde4ee3e7096a8af","type":"link in","z":"2d17751b2e961b97","g":"5944d955dee206fd","name":"link in 4","links":["1fcee61a1f6135a2","cd02867626a32bb6"],"x":95,"y":300,"wires":[["8fd61f2f229d433a"]]},{"id":"5bf2b9b153d7907f","type":"link out","z":"2d17751b2e961b97","g":"5944d955dee206fd","name":"link out 5","mode":"link","links":["825d5bda3805e622"],"x":505,"y":300,"wires":[]},{"id":"825d5bda3805e622","type":"link in","z":"2d17751b2e961b97","g":"c25eddc94757ac68","name":"link in 5","links":["5bf2b9b153d7907f"],"x":95,"y":420,"wires":[["27a07350381294ae"]]},{"id":"85e874ab171c3fab","type":"link out","z":"2d17751b2e961b97","g":"c25eddc94757ac68","name":"link out 6","mode":"link","links":["4669cafc26060cdc"],"x":845,"y":420,"wires":[]},{"id":"4669cafc26060cdc","type":"link in","z":"2d17751b2e961b97","g":"c802ed30dcee9c3f","name":"link in 6","links":["85e874ab171c3fab"],"x":95,"y":540,"wires":[["e8d38f4ed8a7fcb5"]]},{"id":"b08e5395b2dbce75","type":"link out","z":"2d17751b2e961b97","g":"c802ed30dcee9c3f","name":"link out 7","mode":"link","links":["eaf9d6ac08367cdc"],"x":535,"y":540,"wires":[]},{"id":"eaf9d6ac08367cdc","type":"link in","z":"2d17751b2e961b97","g":"8d95b243082b170a","name":"link in 7","links":["b08e5395b2dbce75"],"x":95,"y":660,"wires":[["3fc7f5bbe469e981"]]},{"id":"5c8af20914dad296","type":"inject","z":"2d17751b2e961b97","g":"c858e555a2917f4c","name":"Insert YAML file path","props":[{"p":"path","v":"/data/projects/Node-RED/flows/example.yml","vt":"str"},{"p":"apiHost","v":"http://localhost:1880","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","x":190,"y":880,"wires":[["b6a0d4628676bb41"]]},{"id":"78b22525afa66c7d","type":"yaml","z":"2d17751b2e961b97","g":"6cd281117eb396be","property":"payload","name":"Convert YAML to object","x":610,"y":880,"wires":[["d2c00cf78ca93bfe"]]},{"id":"b6a0d4628676bb41","type":"file in","z":"2d17751b2e961b97","g":"6cd281117eb396be","name":"Read file","filename":"path","filenameType":"msg","format":"utf8","chunk":false,"sendError":false,"encoding":"utf8","allProps":false,"x":400,"y":880,"wires":[["78b22525afa66c7d"]]},{"id":"d2c00cf78ca93bfe","type":"change","z":"2d17751b2e961b97","g":"6cd281117eb396be","name":"Build API request","rules":[{"t":"set","p":"url","pt":"msg","to":"{apiHost}/flow","tot":"str"},{"t":"change","p":"url","pt":"msg","from":"{apiHost}","fromt":"str","to":"apiHost","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":850,"y":880,"wires":[["b3a51a090c2a72bc"]]},{"id":"2141231fd280640a","type":"http request","z":"2d17751b2e961b97","g":"f275b7976d06bdf5","name":"Update flow","method":"PUT","ret":"obj","paytoqs":"ignore","url":"","tls":"","persist":false,"proxy":"","insecureHTTPParser":false,"authType":"","senderr":false,"headers":[],"x":1050,"y":1100,"wires":[["e0168c10a646f735"]]},{"id":"dc3c90fb7aead744","type":"inject","z":"2d17751b2e961b97","g":"5df8de587faba460","name":"Insert YAML file path","props":[{"p":"path","v":"/data/projects/Node-RED/flows/example.yml","vt":"str"},{"p":"apiHost","v":"http://localhost:1880","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","x":190,"y":1100,"wires":[["308a390849388d87"]]},{"id":"1fc03371f827c6f0","type":"yaml","z":"2d17751b2e961b97","g":"f275b7976d06bdf5","property":"payload","name":"Convert YAML to object","x":610,"y":1100,"wires":[["c91ada61c56cfb99"]]},{"id":"308a390849388d87","type":"file in","z":"2d17751b2e961b97","g":"f275b7976d06bdf5","name":"Read file","filename":"path","filenameType":"msg","format":"utf8","chunk":false,"sendError":false,"encoding":"utf8","allProps":false,"x":400,"y":1100,"wires":[["1fc03371f827c6f0"]]},{"id":"c91ada61c56cfb99","type":"change","z":"2d17751b2e961b97","g":"f275b7976d06bdf5","name":"Build API request","rules":[{"t":"set","p":"url","pt":"msg","to":"{apiHost}/flow/{flowId}","tot":"str"},{"t":"change","p":"url","pt":"msg","from":"{apiHost}","fromt":"str","to":"apiHost","tot":"msg"},{"t":"change","p":"url","pt":"msg","from":"{flowId}","fromt":"str","to":"payload.id","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":850,"y":1100,"wires":[["2141231fd280640a"]]},{"id":"137a4acc3af2ea3b","type":"comment","z":"2d17751b2e961b97","g":"f275b7976d06bdf5","name":"Roll back flow","info":"Run this to roll back an existing flow to the state described in the YAML file.","x":130,"y":1020,"wires":[]},{"id":"c2cc2e6456adef31","type":"comment","z":"2d17751b2e961b97","g":"6cd281117eb396be","name":"Restore flow","info":"Run this to import the flow described in the YAML file if it doesn't exist in Node-RED anymore. This will fail if a flow with the same ID still exists in the system.","x":130,"y":800,"wires":[]},{"id":"16b9ca225a6c544f","type":"debug","z":"2d17751b2e961b97","g":"6cd281117eb396be","name":"Result","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":1210,"y":880,"wires":[]},{"id":"e0168c10a646f735","type":"debug","z":"2d17751b2e961b97","g":"f275b7976d06bdf5","name":"Result","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":1210,"y":1100,"wires":[]},{"id":"aa2815e4bede1cce","type":"comment","z":"2d17751b2e961b97","g":"547e4f042c7ab792","name":"Split flows.json","info":"On update of flows.json, all flows will be exported into separate YAML files.","x":140,"y":60,"wires":[]}]

Flow Info

Created 1 month, 3 weeks ago

Updated 1 month, 2 weeks ago

Rating: not yet rated

view on github

Owner

04nd01

Actions

share flow

Node Types

Core

change (x5)
comment (x3)
debug (x2)
file (x1)
file in (x2)
function (x1)
http request (x4)
inject (x3)
link in (x4)
link out (x4)
split (x1)
switch (x1)
watch (x1)
yaml (x3)

Other

fs-ops-delete (x1)
fs-ops-dir (x1)
group (x10)
tab (x1)