Openwhisk Sliding Window Action Monitor

This is a subflow node in order to monitor the latest performance (sliding window) of Openwhisk actions. The flow pings periodically the target Openwhisk installation in order to retrieve the last executed actions and extract statistics from their execution. The flow has been extended in order to create a third output which includes the detailed logs of failed actions during the observed window.

Configuration includes:

• the polling time of the monitor (default 30 seconds, msg.pollingPeriod)
• the target Openwhisk endpoint (msg.targetEndpoint)
• credentials for that endpoint (msg.creds) in the form of user:key
• the window of time (in minutes) in the past for which you want to retrieve results (msg.window)
• an optional action name (msg.action), if one wants to filter specific action activations

The configuration can be set either via the flow UI or the incoming message. The incoming message prevails over the UI-set value.

The monitor can be stopped via a msg.stop=true message. The subflow uses a flow variable in order to control starting and stopping, however subflow context seems not to be shared in the flow context. Hence multiple OW monitors can be used in each flow.

The outputs are 4:

• output 1-MONITOR is the normal performance report
• output 2-STOP is the notification of the STOP operation
• output 3-ERRORS is used for filtering ERRORS and according logs in the executed functions
• output 4-UNREACHABLE is used when the Openwhisk endpoint is unreachable, either due to msg.statusCode>200 value in the Openwhisk API call for getting the activation or othe error cases such as ECONNREFUSED or ENOTFOUND

The results in Output 1 include the raw data acquired (msg.payload.rawData) in the last window of time, as well as the moving averages of duration (msg.payload.results.movingAverageDuration), init time (msg.payload.results.movingAverageInitTime) and wait time (msg.payload.results.movingAverageWaitTime) of the window. The number of cold starts are also included (msg.payload.results.coldStarts) as well as the percentage with relation to the total calls (msg.payload.results.coldStartPercentage). If no activations are found, then the averages return NaN values.

An example of the outputs appears below.

Copy[{"id":"8262c8e75cfbabdd","type":"subflow","name":"OW monitor","info":"\n\nThis is a subflow node in order to monitor the latest performance (sliding window) of Openwhisk actions. The flow pings periodically the target Openwhisk installation in order to retrieve the last executed actions and extract statistics from their execution.\n\nConfiguration includes:\n - the polling time of the monitor (default 30 seconds, `msg.pollingPeriod`)\n - the target Openwhisk endpoint (`msg.targetEndpoint`)\n - credentials for that endpoint (`msg.creds`) in the form of user:key\n - the window of time (in minutes) in the past for which you want to retrieve results (`msg.window`)\n - an optional action name (`msg.action`), if one wants to filter specific action activations\n\nThe configuration can be set either via the flow UI or the incoming message. The incoming message prevails over the UI-set value.\n\nThe monitor can be stopped via a `msg.stop=true` message. The subflow uses a flow variable in order to control starting and stopping, hence only one OW monitor should be used in each flow.\n\nThe results include the raw data acquired (`msg.payload.rawData`) in the last window of time, as well as the moving averages of duration (`msg.payload.results.movingAverageDuration`), init time (`msg.payload.results.movingAverageInitTime`) and wait time (`msg.payload.results.movingAverageWaitTime`) of the window. The number of cold starts are also included (`msg.payload.results.coldStarts`) as well as the percentage with relation to the total calls (`msg.payload.results.coldStartPercentage`).\n\n\n","category":"PHYSICS PEF","in":[{"x":60,"y":80,"wires":[{"id":"408b32b6eb687a08"}]}],"out":[{"x":820,"y":180,"wires":[{"id":"fdcd7c09df04efbb","port":0}]},{"x":560,"y":80,"wires":[{"id":"e3862e86d11b08d5","port":0}]},{"x":1080,"y":320,"wires":[{"id":"14511977ba16aed2","port":0}]}],"env":[{"name":"targetEndpoint","type":"str","value":"http://10.100.59.182:3233/api/v1","ui":{"label":{"en-US":"Target Endpoint"}}},{"name":"pollingPeriod","type":"num","value":"30","ui":{"label":{"en-US":"Polling Period (seconds)"}}},{"name":"window","type":"num","value":"10","ui":{"label":{"en-US":"Window of Time (minutes)"}}},{"name":"namespace","type":"str","value":"guest","ui":{"label":{"en-US":"Namespace"}}},{"name":"action","type":"str","value":"","ui":{"label":{"en-US":"Optional Action Name"}}},{"name":"creds","type":"cred","ui":{"label":{"en-US":"Credentials (user:key)"}}}],"meta":{"module":"node-red-contrib-owmonitor","version":"0.0.1","author":"George Kousiours <[email protected]>","desc":"This is a subflow node in order to monitor the latest performance (sliding window) of Openwhisk actions. The flow pings periodically the target Openwhisk installation in order to retrieve the last executed actions and extract statistics from their execution.","keywords":"openwhisk, performance, monitor, sliding window","license":"Apache-2.0"},"color":"#E9967A","icon":"node-red/status.svg"},{"id":"f81580b2bf9ad891","type":"http request","z":"8262c8e75cfbabdd","name":"","method":"use","ret":"obj","paytoqs":"ignore","url":"","tls":"df3a306242e21be5","persist":false,"proxy":"","authType":"","x":290,"y":180,"wires":[["b61d171594e2cdf3"]]},{"id":"408b32b6eb687a08","type":"function","z":"8262c8e75cfbabdd","name":"defaults","func":"if (msg.hasOwnProperty('window')){\n    \n} else {\n    msg.window=env.get('window');\n}\n\nif (msg.hasOwnProperty('targetEndpoint')){\n    \n} else {\n    msg.targetEndpoint=env.get('targetEndpoint');\n}\n\nif (msg.hasOwnProperty('pollingPeriod')){\n    \n} else {\n    msg.pollingPeriod=env.get('pollingPeriod');\n}\n\nif (msg.hasOwnProperty('creds')){\n    \n} else {\n    msg.creds=env.get('creds');\n}\n\nif (msg.hasOwnProperty('namespace')){\n    \n} else {\n    msg.namespace=env.get('namespace');\n}\n\nif (msg.hasOwnProperty('action')){\n    \n} else {\n    msg.action=env.get('action');\n}\n\nif (msg.hasOwnProperty('stop')){\n    flow.set('stop',msg.stop);\n} else {\n    flow.set('stop',false);\n}\n\nmsg.headers={};\nvar auth = 'Basic ' + new Buffer(msg.creds).toString('base64');\nmsg.headers = {\n    \"Authorization\": auth\n}\n\n\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":180,"y":80,"wires":[["e3862e86d11b08d5"]]},{"id":"e3862e86d11b08d5","type":"function","z":"8262c8e75cfbabdd","name":"produce rate","func":"\nfunction sayHi(input) {\n  \n  if (flow.get('stop')) {\n    msg.payload=\"STOPPED\";\n    node.send([msg,null]);\n    clearTimeout(timerId);\n    \n  } else {\n      msg.since=Date.now()-msg.window*60*1000; //remove window of time from timestamp\n      //add since filter\n      msg.url=msg.baseurl+'?since='+msg.since;\n      msg.url=msg.url+'&limit=0';//by default, OW returns 30 results. We need this to return without limit\n      //add optional action name filter. This needs to be second since it may not always exist\n      if (msg.action!==\"\"){\n          msg.url=msg.url+'&name='+msg.action;\n      }\n\n      node.send([null,msg])\n      \n  }\n}\n\nmsg.baseurl=msg.targetEndpoint+'/namespaces/'+msg.namespace+'/activations';\n\nmsg.method='GET';\nvar timerId=setInterval(sayHi, msg.pollingPeriod*1000,msg);\n\n\n","outputs":"2","noerr":0,"initialize":"","finalize":"","libs":[],"x":370,"y":80,"wires":[[],["f81580b2bf9ad891"]]},{"id":"b61d171594e2cdf3","type":"function","z":"8262c8e75cfbabdd","name":"preprocess results","func":"\n\nvar resultsArray=[];\n\nvar results={};\n\nvar errorsArray=[];\n\nfor (k=0;k<msg.payload.length;k++){\n    //from OW\n    results.activationId=msg.payload[k].activationId;\n    results.duration=msg.payload[k].duration;\n    results.start=msg.payload[k].start;\n    results.end=msg.payload[k].end;\n    results.action=msg.payload[k].name;\n    results.namespace=msg.payload[k].namespace;\n    results.statusCode=msg.payload[k].statusCode;\n    results.version=msg.payload[k].version;\n    \n    if (msg.payload[k].statusCode!=0){\n        errorsArray.push(msg.payload[k].activationId);\n    }\n    \n    //results.success=msg.payload[k].response.success; This is not included in the summary results\n    for (i=0;i<msg.payload[k].annotations.length;i++){\n        if (msg.payload[k].annotations[i].key==='waitTime'){\n            results.waitTime=msg.payload[k].annotations[i].value;\n        }\n        if (msg.payload[k].annotations[i].key==='initTime'){\n            results.initTime=msg.payload[k].annotations[i].value;\n        }\n        \n    }\n    \n    if (!(results.hasOwnProperty('initTime'))){\n        results.initTime=0;\n    }\n    if (msg.action===\"\"){\n        resultsArray.push(results);    \n    } else {\n        if (results.action===msg.action){\n            resultsArray.push(results);\n        }\n    }\n    \n    results={};\n    \n}    \nmsg.payload={};\nmsg.payload=resultsArray;\nmsg.errors=errorsArray;\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":490,"y":180,"wires":[["fdcd7c09df04efbb","9a0a8ecbe2031562"]]},{"id":"fdcd7c09df04efbb","type":"function","z":"8262c8e75cfbabdd","name":"averages","func":"var rawData=msg.payload;\nmsg.payload={};\nmsg.payload.rawData=rawData;\nvar samples=msg.payload.rawData.length;\n\nmsg.results={};\nmsg.results.averageDuration=0;\nmsg.results.averageWaitTime=0;\nmsg.results.averageInitTime=0;\nmsg.results.coldStarts=0;\nmsg.results.successPercentage=0;\n\nvar duration= [];//=new Array();\nvar waitTime=[];\nvar initTime=[];\n\n\nfor (i=0;i<msg.payload.rawData.length;i++){\n    msg.results.averageDuration=msg.results.averageDuration+msg.payload.rawData[i].duration;\n    duration.push(msg.payload.rawData[i].duration);\n    \n    msg.results.averageWaitTime=msg.results.averageWaitTime+msg.payload.rawData[i].waitTime;\n    waitTime.push(msg.payload.rawData[i].waitTime);\n    \n    msg.results.averageInitTime=msg.results.averageInitTime+msg.payload.rawData[i].initTime;\n    initTime.push(msg.payload.rawData[i].initTime);\n    if (msg.payload.rawData[i].initTime!==0){\n        msg.results.coldStarts=msg.results.coldStarts+1;\n    }\n    if (msg.payload.rawData[i].statusCode===0){\n        msg.results.successPercentage=msg.results.successPercentage+1;\n    }\n    \n\n}\nmsg.results.successPercentage=(msg.results.successPercentage/msg.payload.rawData.length)*100;\nmsg.results.averageDuration=toFixedNumber(msg.results.averageDuration/samples,2);\nmsg.results.averageWaitTime=toFixedNumber(msg.results.averageWaitTime/samples,2);\nmsg.results.averageInitTime=toFixedNumber(msg.results.averageInitTime/samples,2);\nmsg.results.coldStartPercentage=toFixedNumber((msg.results.coldStarts/samples)*100,2);\n\nmsg.results.movingAverageDuration=msg.results.averageDuration;\ndelete msg.results.averageDuration;\n\nmsg.results.movingAverageWaitTime=msg.results.averageWaitTime;\ndelete msg.results.averageWaitTime;\n\nmsg.results.movingAverageInitTime=msg.results.averageInitTime;\ndelete msg.results.averageInitTime;\n\n\nmsg.payload.results=msg.results;\nreturn msg;\n\n\nfunction toFixedNumber(num, digits, base){\n  var pow = Math.pow(base||10, digits);\n  return Math.round(num*pow) / pow;\n}\n","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":680,"y":180,"wires":[[]]},{"id":"b3d54c50ea53f87c","type":"split","z":"8262c8e75cfbabdd","name":"","splt":"\\n","spltType":"str","arraySplt":1,"arraySpltType":"len","stream":false,"addname":"","x":370,"y":320,"wires":[["6be2109aa289fde8"]]},{"id":"9a0a8ecbe2031562","type":"change","z":"8262c8e75cfbabdd","name":"","rules":[{"t":"set","p":"payload","pt":"msg","to":"errors","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":220,"y":320,"wires":[["b3d54c50ea53f87c"]]},{"id":"6be2109aa289fde8","type":"function","z":"8262c8e75cfbabdd","name":"prepare call","func":"msg.url=msg.targetEndpoint+'/namespaces/'+msg.namespace+'/activations/'+msg.payload;\nmsg.method='GET';\nmsg.headers={};\nvar auth = 'Basic ' + new Buffer(msg.creds).toString('base64');\nmsg.headers = {\n    \"Authorization\": auth\n}\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":510,"y":320,"wires":[["29a1f27bad73001c"]]},{"id":"29a1f27bad73001c","type":"http request","z":"8262c8e75cfbabdd","name":"","method":"use","ret":"obj","paytoqs":"ignore","url":"","tls":"df3a306242e21be5","persist":false,"proxy":"","authType":"","credentials":{},"x":670,"y":320,"wires":[["75fded98834b7338"]]},{"id":"14511977ba16aed2","type":"join","z":"8262c8e75cfbabdd","name":"","mode":"auto","build":"object","property":"payload","propertyType":"msg","key":"topic","joiner":"\\n","joinerType":"str","accumulate":"false","timeout":"","count":"","reduceRight":false,"x":970,"y":320,"wires":[[]]},{"id":"75fded98834b7338","type":"function","z":"8262c8e75cfbabdd","name":"logs string","func":"msg.payload.logs=msg.payload.logs.join('\\r\\n');\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":830,"y":320,"wires":[["14511977ba16aed2"]]},{"id":"d26d863e7351f69d","type":"comment","z":"8262c8e75cfbabdd","name":"STOP","info":"","x":650,"y":80,"wires":[]},{"id":"5dd2549588fa70e5","type":"comment","z":"8262c8e75cfbabdd","name":"PERFORMANCE STATISTICS","info":"","x":990,"y":180,"wires":[]},{"id":"fd5f7e1e8c09478f","type":"comment","z":"8262c8e75cfbabdd","name":"ERROR LOGS","info":"","x":1000,"y":360,"wires":[]},{"id":"df3a306242e21be5","type":"tls-config","name":"","cert":"","key":"","ca":"","certname":"","keyname":"","caname":"","servername":"","verifyservercert":false,"alpnprotocol":""},{"id":"a9c56d857051cf6c","type":"subflow:8262c8e75cfbabdd","z":"67840c79d73a38a4","name":"Openwhisk MONITOR AND LOGGER","env":[{"name":"window","value":"10000","type":"num"},{"name":"creds","type":"cred"}],"credentials":{"creds":"__PWRD__"},"x":510,"y":460,"wires":[["1c16e9187544e5d2"],["693844663215b5b6"],["0914815e5be0a7e6"]]},{"id":"355abdad73b4cc62","type":"inject","z":"67840c79d73a38a4","name":"start","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payloadType":"date","x":250,"y":420,"wires":[["a9c56d857051cf6c"]]},{"id":"56d3e80171b8995b","type":"inject","z":"67840c79d73a38a4","name":"stop","props":[{"p":"stop","v":"true","vt":"bool"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payloadType":"str","x":250,"y":500,"wires":[["a9c56d857051cf6c"]]},{"id":"1c16e9187544e5d2","type":"debug","z":"67840c79d73a38a4","name":"MONITOR","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":750,"y":420,"wires":[]},{"id":"693844663215b5b6","type":"debug","z":"67840c79d73a38a4","name":"STOP","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":730,"y":460,"wires":[]},{"id":"0914815e5be0a7e6","type":"debug","z":"67840c79d73a38a4","name":"ERRORS","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":740,"y":500,"wires":[]}]