← Back
Hugging Face
Hugging Face launches Modular Diffusers, offering composable building blocks for custom image generation pipelines
· featureapisdkopen-source · huggingface.co ↗

Overview

Hugging Face has released Modular Diffusers, a new approach to constructing diffusion pipelines that prioritizes flexibility and reusability. Rather than building entire pipelines from scratch or modifying monolithic DiffusionPipeline classes, developers can now compose workflows from self-contained blocks — each with defined inputs, outputs, and computation logic.

Key Features

Composable Architecture: Pipelines are constructed from discrete, pluggable blocks such as text encoders, VAE encoders, denoisers, and decoders. Blocks can be:

  • Run independently as standalone pipelines
  • Added, removed, or swapped without rewriting pipeline logic
  • Dynamically recomposed to adapt to remaining components

Familiar API: The public API remains consistent with standard DiffusionPipeline usage, lowering the learning curve. Load and run pipelines with minimal code while accessing granular control under the hood.

Custom Block Development: Developers can create custom blocks by defining:

  • expected_components: Model dependencies with default Hub repositories
  • inputs and intermediate_outputs: Data contracts for block communication
  • __call__: Computation logic executed during pipeline execution

The post includes an example of a DepthProcessorBlock that extracts depth maps using Depth Anything V2, demonstrating how custom blocks integrate into existing workflows like ControlNet pipelines.

Integration & Lazy Loading

Modular Diffusers includes:

  • ComponentsManager for memory-efficient lazy loading and component reuse across blocks
  • Modular Repositories: Pre-configured block collections for popular models (e.g., FLUX.2 Klein 4B)
  • Community Pipelines: Shared custom workflows from the community
  • Mellon Integration: Visual node-based interface for wiring blocks together without code

Developer Action Items

Developers building custom diffusion workflows should:

  1. Review the Modular Diffusers documentation for detailed block type and composition patterns
  2. Consider migrating complex pipeline customizations to modular blocks for improved maintainability
  3. Share custom blocks with the community via Modular Repositories