ComfyUI Standard Nodes + α: BiRefNet, Depth-Anything-V3, SAM 3.1 & Lama Remover Guide
- BiRefNet separates the background
- Depth-Anything-V3 estimates depth
- Combine SAM 3.1 and LaMa to remove unwanted objects
Introduction
Hello, this is Easygoing.
In this post, I’ll introduce image editing using ComfyUI’s standard nodes with just a few custom nodes added.
SAM 3.1 Is a General-Purpose Model
In the previous article, I introduced background removal using the SAM 3.1 model.
How to Use the SAM 3.1 Model
The SAM 3.1 model is a general-purpose model that can detect almost anything from text input.
While general-purpose models are very convenient because they can recognize a wide variety of subjects, when the task is limited to simply separating foreground and background, specialized models tend to deliver higher accuracy.
So first, let’s look at how to use the BiRefNet model, which is available through ComfyUI’s standard nodes.
BiRefNet Is a Specialized Model for Background Removal!
BiRefNet is a specialized model for binary image segmentation (separating foreground from background).
There are several variants in the BiRefNet series, but the standard BiRefNet model is officially distributed by Comfy Org, so we’ll use that one this time.
Workflow
Model
- background_removal: birefnet.safetensors
Placement of birefnet.safetensors
- ComfyUI\models\background_removal (create this folder if it doesn’t exist)
Custom Node
Background Removal with BiRefNet
Let’s compare background removal results between SAM 3.1 and BiRefNet.
Input Image
Background Removal with SAM 3.1
Background Removal with BiRefNet
Compared to SAM 3.1, BiRefNet detects fine details such as hair much more accurately.
BiRefNet Boundaries Are Jagged
Now let’s zoom in on the boundary areas produced by BiRefNet.
Enlarged View of BiRefNet
When you enlarge the BiRefNet detection, the fine hair strands appear jagged and somewhat broken.
In ComfyUI’s implementation, the BiRefNet model processes images at a fixed resolution of 1024 × 1024 . For high-resolution illustrations, this resolution is insufficient, which leads to this kind of result.
Including Semi-Transparent Areas in the Mask
To ensure that hair is fully included in the foreground without gaps, let’s treat all the semi-transparent regions that BiRefNet judged as being between foreground and background as foreground.
Connect the mask created by BiRefNet to a Threshold Mask node and set the Threshold to the minimum value of 0.01.
Threshold Mask = 0.01
Compared to the previous result, the hair is now solidly included in the foreground. However, thin strips of background have now crept into the boundary areas.
Adjusting the Mask Region with the Mask Refine Node!
Next, we’ll use a custom node to further refine the mask region.
The Mask Refine node in ComfyUI-AutoMask uses a method called pymatting to compare the original illustration with the mask and adjust the mask region accordingly.
Adjusting Boundaries with the Mask Refine Node
Using the Mask Refine node allows us to clean up the mask boundaries considerably.
On the other hand, because Mask Refine applies blur before adjusting the boundaries, fine hair strands become thinner, and sharp corners enclosed by the mask tend to get rounded off. These are its drawbacks.
Automatic mask adjustment nodes each have their strengths and weaknesses, so it’s best to choose the right one depending on the use case.
BiRefNet Also Has Subjects It Struggles With
BiRefNet primarily separates foreground and background based on color and texture. When the boundary areas have similar structures, its separation accuracy drops.
Let’s try the same workflow on a different illustration.
Input Image 2
BiRefNet → Mask Threshold → Mask Refine
In this illustration, parts of the fireworks that have similar color and brightness to the hair were incorrectly classified as foreground.
In addition, the fine sea area enclosed by the hair in the lower right was not properly recognized as background and ended up being included in the foreground.
So let’s explore a way to improve separation accuracy by using a different model.
Depth-Anything-V3 Estimates Depth
The Depth-Anything-V3 model is a monocular depth estimation model (estimating depth from a single illustration) released by China’s ByteDance .
Accurate depth estimation normally requires at least two viewpoints, but by training on a large number of images, the Depth-Anything-V3 model can perform depth estimation with quite high accuracy even from a single image.
Like BiRefNet, Depth-Anything-V3 can also be used with ComfyUI’s standard nodes.
Trying Out the Depth-Anything-V3 Model!
Let’s actually perform depth estimation using the Depth-Anything-V3 model.
Workflow
Model
- geometry_estimation: depth_anything_3_mono_large.safetensors
Placement Folder
- ComfyUI\models\geometry_estimation (create this folder if it doesn’t exist)
Monocular Depth Estimation with DepthAnythingV3
When we create a depth map with the Depth-Anything-V3 model, we can see that the person is correctly placed in the foreground, while the fireworks and the sea area enclosed by the hair are correctly recognized as distant background.
Determining the Foreground from the Depth Map
Now let’s actually determine the foreground using the depth map.
Workflow
Models
-
background_removal: birefnet.safetensors
-
geometry_estimation: depth_anything_3_mono_large.safetensors
Custom Node
First, we use the region detected by BiRefNet to black out unnecessary parts of the depth map created by Depth-Anything-V3.
Next, since the depth map exists as a gradient from black to white, we look for a suitable threshold to separate foreground from background.
After converting the depth map to a mask with the Convert Image to Mask node, we connect it to the Auto Mask Threshold node.
The Auto Mask Threshold node analyzes the histogram (frequency distribution) of the mask and proposes eight candidate thresholds.
In this case, the middle (5th) output looked good, so we set threshold to 5 and run it.
Finally, we pass the result through Mask Refine to adjust the boundaries.
By combining the outputs of BiRefNet and Depth-Anything-V3 in this way, we were able to correctly separate the areas enclosed by the fireworks and hair as background.
Removing Unwanted Objects with Lama Remover!
Finally, let’s look at how to use Lama Remover.
Lama Remover is a custom node that brings the image inpainting model LaMa (which I introduced previously) into ComfyUI.
About Image Inpainting with the LaMa Model
Workflow
Models
-
checkpoints: sam3.1_multiplex_fp16.safetensors
-
lama: big-lama.pt
Placement Folder for sam3.1_multiplex
- ComfyUI\models\checkpoints
Placement Folder for big-lama.pt
- ComfyUI\models\lama (create this folder if it doesn’t exist)
Custom Nodes
-
comfyui-lama-remover (revived)
Detecting the Objects to Remove with SAM 3.1
First, we detect the objects we want to remove using the SAM 3.1 model.
This time, we’ll try removing the fireworks and bokeh lights from the following illustration.
Input Image 3
Prompt
fireworks:10, bokeh lights:30When you want SAM 3.1 to detect multiple subjects at the same time, writing a number after a colon tells it to detect up to that many instances of the object in one go.
Removing Unwanted Objects with Lama Remover!
Next, we connect the original illustration and the mask region to the Big Lama Remover node and actually remove the objects.
By combining the SAM 3.1 and LaMa models, we were able to remove the unwanted objects with quite high accuracy.
Both SAM 3.1 and LaMa are lightweight and fast models, so they can be very useful as preprocessing steps in illustration editing workflows.
Summary: Try Out Various AI Models!
- BiRefNet separates the background
- Depth-Anything-V3 estimates depth
- Combine SAM 3.1 and LaMa to remove unwanted objects
In this article, I introduced how to use the SAM 3.1, BiRefNet, Depth-Anything-V3, and LaMa models in ComfyUI.
Nowadays, a large number of AI models are being released with open weights. Even when limited to segmentation techniques for detecting regions in images, it is impossible to simply rank the many publicly available models as better or worse.
Bracer Jack’s Segmentation Model Comparison Video
AI models each have their own personalities. By skillfully combining multiple models and incorporating them into a workflow, they can cover one another’s shortcomings.
Recently, official support in ComfyUI has expanded, making it easier to try new models. By combining just a few custom nodes, infinite possibilities open up in ComfyUI.
Why not take this opportunity to challenge yourself with some new models?
Thank you for reading all the way to the end!