node-red-contrib-facial-recognition 0.30.105
Provides a node-red node for Facial Detection & Facial Recognition.
node-red-contrib-facial-recognition
Node-RED - Provides a node-red node for Facial Detection & Facial Recognition.
Table of Contents
- Requirements
- Install
- About
- Usage
- Node_Properties
- Example_Flows
- Simple_Queue_Method
- Heavy_image_processing_or_mjpeg_video_stream
- FaceRecognition_Persistant_labeledFaceDescriptors
- Bugs_Feature_request
- License
- Work
- Contributor
Requirements
Node version 12.x or higher. Use node -v in terminal to find out
Node-red version 1.x or higher.
Install
Install with node-red Palette Manager or,
Run the following command in your Node-RED user directory - typically ~/.node-red:
npm install node-red-contrib-facial-recognition
Windows users: If your having issues and use the trouble shooting guide. Run the commands in the Windows troubleshooting guide from within your Node-RED user directory - typically ~/.node-red
Docker users: Node-red's official docker will not work. It is based on alpine and is not libc6 compatible. Please see this thread for how to install a docker version of node-red that works. https://discourse.nodered.org/t/announce-node-red-contrib-facial-recognition/37384/53
About
At its core it uses @vladmandic/face-api and @tensorflow/tfjs-node ver.2.7.x and it can use @tensorflow/tfjs-node-gpu for the CUDA crazy amongst you.
vladmandic was a big help for us nodejs guys. After finding a bug and fielding questions, he took the time to make a nodejs build specific to tfjs-node. If you like this node-red-contrib-facial-recognition, I highly recommend you take the time to goto vladmandic's github page https://github.com/vladmandic/face-api and throw his repo a star.
Usage
Takes a buffered image and runs TensorFlow Facial Detection and/or Facial Recognition to Detect:
- Faces in an image
- Facial landmarks
- Compute Face Descriptors
- Face Expressions
- Age & Gender Recognition
- Face Recognition by Matching Descriptors
Node_Properties
Name
Define the msg name if you wish to change the name displayed on the node.
Image
You can change the msg property value that you send a buffered image of your choice to.
Example: msg.NameOfYourChoice
Image size and the avalible memory on the device your using. Sending a 4mb image on a raspberry pi will crash the node. If your running other stuff this size will be lower. The tensorfolow core only sends out a Standard Error when you push the limits of your devices memory. If your unsure look at your node-red console log to see if you unsure if the image size is too big.
Settings
This is optional, you do not have to send it anything. Used to override settings in the nodes config Properties menu.
You can change the msg property value that you send an object of your choice to.
Example: msg.NameOfYourChoice
Sending a object to this msg property value will override any settings in the nodes config Properties menu. Great for using input messages to change settings on the fly.
Please see/inport the example Flow section of this documentation for better understanding.
Example:
msg.settings = {
FaceDetector :
{
SsdMobilenetv1 :
{
maxResults : 4,
minConfidence : 0.6
}
},
Tasks :
{
detectAllFaces :
{
withFaceLandmarks : true,
withFaceExpressions : true,
withAgeAndGender : true,
withFaceDescriptors : true
}
},
FaceRecognition :
{
enabled :
{
KnownFacesPath : "/example/known_face",
distanceThreshold : 0.6,
ReInitializeFaceMatcher : false
}
}
};
return msg;
You do not have to fill out every option. You can omit any object key and its value. This node will then use the settings found in the nodes config Properties menu for that omitted object key and its value.
Note: ReInitializeFaceMatcher
Set this value to true if you have changed/edited/added images or image folders to your KnownFacesPath to ReInitialize the FaceMatcher. Used to process all the images into Labeled Face Descriptors for each dir name and individual descriptions for images. Do not leave set to true! it takes significant time to process. Once its ran after you have made changes to images or image folder it is saved to context and used for Facial Recognition.
Else you can just re-deploy node-red and ReInitializeFaceMatcher will run one time only on the first image you send.
Bindings
By default it is set to CPU - @tensorflow/tfjs-node, this will use your CPU to process images. However you may choose to install @tensorflow/tfjs-node-gpu to utilize your video card to process images. This is not an easy process to get CUDA working. However if you go down this rabbit hole the benefits in time to process images are significant.
Good luck.
FaceDetector
SsdMobilenetv1 - A Single Shot Multibox Detector; based on MobileNetV1.
Computes the locations of each face in an image and returns the bounding boxes with it's probability for each face. High accuracy in detecting face bounding boxes at the cost of time to compute.
- maxResults - The max number of faces to return
- minConfidence - returns results for face(s) in a image above Confidence threshold
tinyFaceDetector - a fast realtime face detector, and less resource consuming compared to the SSD Mobilenet V1 face detector. It is poor at detecting small faces. Best face detector on resource limited devices.
- inputSize - size at which image is processed, the smaller the faster, number must be divisible by 32. Common sizes are 128, 160, 224, 320, 416, 512, 608
- scoreThreshold - returns results for face(s) in a image above Confidence threshold
Tasks
detectAllFaces - Returns an array of objects. Utilize the selected FaceDetector to detect multiple faces in a buffered image sent in message by user
detectSingleFace - Returns an object. Utilize the selected FaceDetector to detect a single face in a buffered image sent in message by user. If image contains multiple faces it will only detect one of them, hopefully the highest probability one.
- withFaceLandmarks - computes landmarks for each detected face(s)
- withFaceExpressions - recognize face expressions of each face(s)
- withAgeAndGender - estimate age and recognize gender of each face(s)
- withFaceDescriptor - computes the face descriptors for each face(s)
FaceRecognition
disabled - Don't use any facial recognition
enabled - Performs face recognition, by comparing reference face descriptor(s) to determine the similarity to query face descriptor(s).
Note: FaceDetector minConfidence Properties affect the labeledFaceDescriptors. If you have a minConfidence of .9 you may miss a bunch of faces when building your labeledFaceDescriptors. after you run the node once or supply it a labeledFaceDescriptors from a persistant storage you can set the value to any level you wish for screening the input image you send
- KnownFacesPath - The location of the main folder that contains all subfolders, labeled with persons name. If you change the value of this path you mush use a full path. Example: /home/myuser/name/TheFolderIPutAllTheLabledFaceFolderIn.
Subfolders should contain close-up face images of the individual. The actual name of the file in this folder does not matter. Please look at the structure of the example folder for more understanding. The name of the subfolder is what is used to label the faces for facial recognition.Note: If you have changed/edited/added images or image folders to your KnownFacesPath you must redeploy the node to ReInitialize the FaceMatcher. On the first image you send, FaceMatcher is ran to process all the images into Labeled Face Descriptors for each dir name and individual descriptions for images. Then it is saved to context and used for Facial Recognition for all future images you send in a message.
- distanceThreshold - returns results based on measures of how far away, Euclidean distance of face descriptor, the user submitted image is compared to how far away, Euclidean distance of face descriptors, it is to all the faces found in the labeled subfolder that are above the distanceThreshold. Simply put: return names for person if it is this distanceThreshold of confidence. The lower the distanceThreshold is the more likely you are to get a incorrect match. The higher the distanceThreshold is the more likely it is that a person will not be recognized.
Example_Flows
BASIC:
Advanced:
NOTE: other node-red nodes required
Simple_Queue_Method
If your having issues with overloading the node with too many images that tensorflow can't handle you can always add a Queue
This will take up ram as the queue grows and tensorflow is ram hungry. Only use this method if the queue will decrease in time. Don't let it grow out of control.
Example Flow:
NOTE: other node-red nodes required
node-red-contrib-simple-message-queue
Click Me For Simple_Queue_Method example
Heavy_image_processing_or_mjpeg_video_stream
Every output object message from this node has sec_to_complete with the amount of time it took to process the image.
Based on your device/processing speed/CPU or CUDA will vary the amount of time it takes to process the image.
So if it takes 0.623 seconds to do a facialrecognition and your sending it 15 frames a second you will create a backlog of work and overflow the node.
The best thing to do is use multiple facial-recognition nodes to process the images as individual workers.
then check to see if it is keeping up.
Example Flow:
NOTE: other node-red nodes required
Click Me For Heavy_image_processing_or_mjpeg_video_stream example
FaceRecognition_Persistant_labeledFaceDescriptors
Note: FaceDetector minConfidence Properties affect the labeledFaceDescriptors. If you have a minConfidence of .9 you may miss a bunch of faces when building your labeledFaceDescriptors. after you run the node once or supply it a labeledFaceDescriptors from a persistant storage you can set the value to any level you wish for screening the input image you send.
You can save the FaceRecognition labeledFaceDescriptors to persistant storage solution so you don't have to load all the images in your KnownFacesPath folder every time you deploy node red or if node-red/your device restarts.
This could be a HUGE time savings if you have thousands of people in your KnownFacesPath folder.
Also this shows how the msg.settings is used. Just un-comment anything you wish to override in the Properties menu of the node.
Example Flow:
Click Me For FaceRecognition_Persistant_labeledFaceDescriptors example
Bugs_Feature_request
Please report bugs and feel free to ask for new features directly on GitHub.
License
This project is licensed under Apache 2.0 license.
All photos are free to use and provided by: unsplash.
Work
_Need a node? _Need automation work? _Need computers to flip switches?
Contact me at [email protected]
Contributor
Thanks to:
The @tensorflow/tfjs-node team for supporting and maintaining a repo that allows us JS guys to create cool stuff.
vladmandic and his @vladmandic/face-api for help and support in creating a nodeJS specific build for the face-api.
protocolus for his work on finding images for the user example.
Joshua Rondeau for free use of his photos.
Brandon Atchison for free use of his photos.
zenofmud for bug fixes and new feature.
M0ebiu5 for helping change the logic of KnownFacesPath
Heavy_image_processing_or_mjpeg_video_stream
release notes
0.0.0 = (majorchange) . (new_feature) . (bugfix-simple_mod)
version 0.30.105 - new_feature - Updated to use tfjs-node 3.7.0 or higher, face-api 1.3.1 or higher, fixes the agent requierment of 6.x or higher
version 0.29.105 - documentation - Added queue documentation, added documentaion for array / object
version 0.29.104 - new_feature - Checks for node version greather than 12, added checks for @vladmandic/face-api
version 0.28.104 - documentation - Added Docker use info and Requirements section
version 0.28.103 - documentation - Switched to using msg-router for Heavy_image_processing_or_mjpeg_video_stream
version 0.28.102 - bug - fix how README.md is displayed on web pages
version 0.28.101 - documentation - Update FaceRecognition_Persistant_labeledFaceDescriptors to version based on zenofmud
version 0.28.100 - bug - M0ebiu5 change the logic of KnownFacesPath
version 0.28.99 - bug - fix faceapi = require for gpu, so the gpu option will work and up date documentation for path
version 0.28.98 - bug - docker post install script
version 0.28.97 - documentation - fix missing'<' for details
version 0.28.96 - documentation - add expand for examples
version 0.28.95 - documentation - correction
version 0.28.94 - documentation - Windows troubleshooting guide
version 0.28.93 - bug - fix disc in JSON file
version 0.28.92 - new_feature - persistent storage of labeledFaceDescriptors
version 0.27.92 - bug - remove github code mistake
version 0.27.91 - new_feature - zenofmud submitted a fix for MacOS .DS_Store file removal from files
version 0.26.91 - bug/new_feature - zenofmud submitted fix for MacOS .DS_Store file removal from folders, and a spelling error
version 0.26.90 - bug - file location fix
version 0.26.89 - new_feature - Removed eval and other optimizations
version 0.25.89 - bug - UnhandledPromiseRejection - added catch to handle these
version 0.25.88 - new_feature -added error checks for user submiting bad/unkown images to the folder or as msg.input
version 0.24.88 - bug -typo fix in documentation
version 0.24.87 - documentation - added Heavy_image_processing_or_mjpeg_video_stream section and example for it
version 0.24.86 - documentation - Fix documentation, added examples and intro image to top of page.
version 0.24.85 - documentation - Fix documentation, updating examples!
version 0.24.84 - First Public release
