je-suis-tm/icespice_lora_flux_nf4

IceSpice Lora Flux NF4

Prompt

Training With QLoRA: IceSpice with vibrant, short, curly red hair confidently poses for a photograph. Centrally framed, she wears a black blazer over a `Miu Miu` crop top and a striking silver eye pendant necklace. Behind her, a blue wall is playfully decorated with green leaves. A partially visible banner in the background in bold yellow lettering against a blue backdrop, adding a unique, urban feel to the scene.

Prompt

Training Without QLoRA: IceSpice with vibrant, short, curly red hair confidently poses for a photograph. Centrally framed, she wears a black blazer over a `Miu Miu` crop top and a striking silver eye pendant necklace. Behind her, a blue wall is playfully decorated with green leaves. A partially visible banner in the background in bold yellow lettering against a blue backdrop, adding a unique, urban feel to the scene.

Prompt

Testing With QLoRA: Surreal 4k, a beautiful elf princess called IceSpice with expressive black eyes and cosmic features. Her skin appears to be composed of intertwined bioluminescent particles, surrounded by neon lights and floating, colorful orbs in a fantastical forest environment, with exotic animals creating a mesmerizing and otherworldly atmosphere, cinematic composition. full body portrait

Prompt

Testing Without QLoRA: Surreal 4k, a beautiful elf princess called IceSpice with expressive black eyes and cosmic features. Her skin appears to be composed of intertwined bioluminescent particles, surrounded by neon lights and floating, colorful orbs in a fantastical forest environment, with exotic animals creating a mesmerizing and otherworldly atmosphere, cinematic composition. full body portrait

The QLoRA fine-tuning process of icespice_lora_flux_nf4 takes inspiration from this post (https://huggingface.co/blog/diffusers-quantization). The training was executed on a local computer with 1000 timesteps and the same parameters as the link mentioned above, which took around 6 hours on 8GB VRAM 4060. The peak VRAM usage was around 7.7GB. To avoid running low on VRAM, both transformers and text_encoder were quantized. All the images generated here are using the below parameters

• Height: 512
• Width: 512
• Guidance scale: 5
• Num inference steps: 20
• Max sequence length: 512
• Seed: 0

Usage

import torch
from diffusers import FluxPipeline, FluxTransformer2DModel
from transformers import T5EncoderModel

text_encoder_4bit = T5EncoderModel.from_pretrained(
    "hf-internal-testing/flux.1-dev-nf4-pkg", subfolder="text_encoder_2",torch_dtype=torch.float16,)

transformer_4bit = FluxTransformer2DModel.from_pretrained(
        "hf-internal-testing/flux.1-dev-nf4-pkg", subfolder="transformer",torch_dtype=torch.float16,)

pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-dev", torch_dtype=torch.float16,
                                    transformer=transformer_4bit,text_encoder_2=text_encoder_4bit)

pipe.load_lora_weights("je-suis-tm/icespice_lora_flux_nf4",
                       weight_name='pytorch_lora_weights.safetensors')

prompt="Surreal 4k, a beautiful elf princess called IceSpice with expressive black eyes and cosmic features. Her skin appears to be composed of intertwined bioluminescent particles, surrounded by neon lights and floating, colorful orbs in a fantastical forest environment, with exotic animals creating a mesmerizing and otherworldly atmosphere, cinematic composition. full body portrait"

image = pipe(
            prompt,
            height=512,
            width=512,
            guidance_scale=5,
            num_inference_steps=20,
            max_sequence_length=512,
            generator=torch.Generator("cpu").manual_seed(0),            
        ).images[0]

image.save("icespice_lora_flux_nf4.png")

Trigger words

You should use IceSpice to trigger the image generation.

Download model

Download them in the Files & versions tab.