Image Editing with ComfyUI Standard Nodes! How to Use SAM 3.1 and Qwen3-VL

Introduction

Hello, this is Easygoing. In this article, I’ll introduce image editing that can be done using standard nodes in ComfyUI.

ComfyUI Has High Extensibility!

ComfyUI allows anyone to freely create custom nodes, which has given rise to its unparalleled extensibility.

About the Strength of ComfyUI

Among ComfyUI’s custom nodes, those that are especially useful and frequently used are gradually being incorporated into ComfyUI’s standard nodes. Once they become official implementations, both stability and performance improve.

In this article, I’ll show you how to use SAM 3.1 and Qwen3-VL with these now-standard nodes.

SAM 3.1 Is a Region Detection Model

The SAM 3.1 model is a region detection model released by Meta.

Meta’s SAM 3.1 Model

The SAM 3.1 model is mainly an improved version of the previously introduced SAM 3, with better performance especially for video use.

About the SAM 3 Model

When downloading Meta’s SAM 3 model from its Hugging Face page, contact information was required. However, the sam3.1_multiplex model published by Comfy Org no longer requires this, making it freely downloadable by anyone.

SAM 3.1 Model Published by Comfy-Org

This time, I’ll introduce how to use the SAM 3.1 model with ComfyUI’s standard nodes.

Detecting a Person by Instruction

Let’s actually use the SAM 3.1 model to remove the background from the following illustration.

Workflow

• sam3_1_remove_background_20260731.json

Model

• sam3.1_multiplex_fp16.safetensors

First, the sam3.1_multiplex_fp16.safetensors model published by Comfy Org is distributed in Checkpoint format, so place the downloaded model in the same location as regular Checkpoints.

For ComfyUI

• comfyui\models\checkpoints

For Stability Matrix

• Models\StableDiffusion

Then load the SAM 3.1 model with the Load Checkpoint node.

Next, connect a CLIP Text Encode node just like a normal image generation prompt and write the target you want to detect.

Then connect it to the model and conditioning inputs of the SAM3 Detect node. Note that the SAM 3.1 model does not include a VAE, so there is no need to connect a vae.

With this, we were able to detect the person using the SAM 3.1 model.

Removing the Background Other Than the Person!

Currently, the person detected by SAM 3.1 is selected. However, what we actually want to remove is the opposite side—the background—so we invert the region with the Invert Mask node.

Then, by inputting the original image and the background mask region into the Join Image with Alpha node, the background becomes transparent.

Here, “Alpha” in the node name refers to the Alpha Channel (transparency channel).

This way, we were able to cut out the illustration with a transparent background using the SAM 3.1 model.

Qwen3-VL Understands Images Too!

In the second half, I’ll introduce how to use the Qwen3-VL model.

The Qwen series is a large language model (chat AI) released by China’s Alibaba.

• Large Language Model (LLM): understands text
• Vision Language Model (VLM): understands both images and text

Qwen3-VL is a model that adds image recognition capability to the large language model Qwen3, allowing it to understand images in addition to text.

Due to its high performance and open license, the Qwen series is increasingly being adopted as a text encoder (the AI that analyzes prompts) in image generation AI. The Qwen3-VL model is used as the text encoder in the previously introduced Krea 2 model.

About the Krea 2 Model

Describing Images with Qwen3-VL!

Let’s use the Qwen3-VL model to add a description (caption) to an image.

We’ll try it with the anime illustration whose background we just removed.

Workflow

Models

• qwen3vl_4b_bf16.safetensors
• qwen3vl_4b_INT8_ConvRot_HQ.safetensors (lightweight version created by the author)

First, when loading the Qwen3-VL model, use the Load CLIP node just like a regular text encoder.

Then input the clip and the image into the Generate Text node, and write the instruction in the Prompt field.

Please describe the following image in English using about 50 words.

Anime Illustration of a Woman in Purple Clothing

A vibrant anime girl with brown hair streaked in neon pink and purple, blue eyes, and a gentle smile. She wears a dark top with a beaded necklace. The background features glowing bokeh lights, creating a dreamy, energetic atmosphere. Her expression is warm and inviting.

Using the Qwen3-VL model, we were able to create a caption that also describes the colors and atmosphere.

Creating Prompts with Qwen3-VL

Next, let’s use the Qwen3-VL model to create an image generation prompt from two images.

First, separately from the previous illustration, we’ll create a caption for the following sunflower field photograph.

Photograph of a Sunflower Field

A vast field of sunflowers stretches under a soft blue sky, their vibrant yellow petals and dark centers creating a radiant sea of gold. Green leaves peek through, adding contrast. The scene exudes warmth and natural beauty, capturing the essence of summer's bounty in a serene, picturesque landscape.

Next, we combine the caption created from the first image, add the quality prompt that the author usually uses, and generate a prompt with Qwen3-VL.

The instruction given to Qwen3-VL here is taken directly from the prompt enhancer used in the official Krea-2 workflow.

• qwen3vl_4b_prompt_combine_20260731.json

A vibrant anime girl with long, flowing brown hair streaked in purple and pink, striking blue eyes, and a gentle little smile, captured in a close-up upper body shot from a Dutch angle, wearing a dark blue top adorned with a beaded necklace, her colorful aura radiating energy with dynamic glowing highlights around her hair, as she turns around gracefully against a simple background with soft bokeh blur and transparent elegance, while behind her, a vast field of sunflowers stretches under a soft blue sky, their vibrant yellow petals and dark centers forming a radiant sea of gold, green leaves peeking through for contrast, all bathed in warm, natural summer light to evoke serene beauty and lively dreaminess.

Generating Images with the Completed Prompt

Now let’s generate an illustration with the completed prompt using the Krea-2-Turbo_clear model.

(The workflow is provided at the end of the article.)

Input Images

Output Image

This way, we were able to generate a completely new illustration that inherits elements from the two input images.

No Need for Ollama or LM Studio!

Previously, when using large language models such as Qwen in ComfyUI, it was necessary to simultaneously launch and synchronize external tools like Ollama or LM Studio.

Naturally, these tools compete with ComfyUI for VRAM, making them slow, and memory shortages frequently caused crashes, making stable operation difficult.

With ComfyUI officially supporting models such as SAM 3.1 and Qwen, model movement according to ComfyUI’s algorithms has become possible. This has greatly improved performance and stability, enhancing user convenience.

ComfyUI is evolving into the strongest generative AI tool that can handle almost anything on its own.

Will ComfyUI Swallow the Custom Node Community?

At the same time, however, the author holds another perspective.

Many of the excellent features that currently support ComfyUI—such as SAM 3, Qwen, and INT8_ConvRot—were originally implemented by the custom node community.

The creator of the ComfyUI-INT8-Fast custom node, BobJohnson24, commented at the beginning of the repository as follows:

🎉 INT8 is now officially supported in ComfyUI 🎉

(omitted)

No, I had nothing to do with this and was not involved at all. My existing quants are likely not to work due to quant name mismatches. On the other hand, silveroxides’ ones are more likely to work because he was quite deeply involved in making this happen.

Existing INT8 fast quants can be converted to the official native format using the following script

(omitted)

I’m happy to be able to retire to the beach with a piña colada in hand. In the future, I may slim this node down to a dedicated node for pre-LoRA application (if it doesn’t become a default Comfy feature).

From Comfy Org’s stance of extracting excellent ideas from the custom node community, excluding compatibility with tools other than ComfyUI for the sake of “performance optimization,” and aiming for hegemony in generative AI tools, I feel that the collaboration with the community that the company once advocated is gradually fading.

Comfy Org Blog Post dated 2025.9.5

(preceding text)

since so much of what makes ComfyUI great comes from the community, we are exploring different ways to do revenue share with custom node developers from Comfy Cloud. It’s important to us that everyone benefits economically. We are still working through the details of how this would work.

(following text)

Of course, most of ComfyUI’s custom nodes are published under open licenses such as MIT or Apache-2.0, so there is no rights issue with ComfyUI itself using them, and it may even be an honor for the custom node creators.

As a witness to 2026, I would like to watch ComfyUI transform from a community tool into the strongest commercial tool.

Workflows

Finally, I’ll introduce two of the author’s everyday workflows created with SAM 3.1 and Qwen3-VL.

SDXL -> Upscale -> Detailer

• SDXL_Upscale_Detailer_20260801.json

Models

• checkpoint: mellow_pencil-XL_clear_v2.safetensors
• upscaler: 4x_IllustrationJaNai_V1_ESRGAN_135k.pth (non-commercial model)

Generating One Illustration from Two Images with Krea-2-Turbo_clear (Workflow from This Article)

• Krea-2-Turbo_clear_image_combine_20260801.json

Models

• text_encoder: qwen3vl_4b_bf16.safetensors
• diffusion_model: Krea-2-Turbo_clear.safetensors
• vae: qwen_image_vae.safetensors
• upscaler: 4x_IllustrationJaNai_V1_ESRGAN_135k.pth (non-commercial model)

Summary: Let’s Try SAM 3.1 and Qwen3-VL

• SAM 3.1 and LLMs have been integrated into ComfyUI itself
• Performance and stability have improved
• Evolving into the strongest generative AI tool

In this article, I introduced how to use the SAM 3.1 and Qwen3-VL models with ComfyUI’s standard nodes.

ComfyUI accurately senses users’ needs, expands its basic functions, and has made great progress as a professional-oriented tool.

ComfyUI will continue to develop as an indispensable tool for the author and many other creators.

Thank you for reading to the end!