fofr commited on
Commit
2b62f2e
1 Parent(s): 5cb768c

Upload folder using huggingface_hub

Browse files
Files changed (4) hide show
  1. README.md +77 -0
  2. embeddings.pti +0 -0
  3. lora.safetensors +3 -0
  4. special_params.json +1 -0
README.md ADDED
@@ -0,0 +1,77 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: creativeml-openrail-m
3
+ tags:
4
+ - text-to-image
5
+ - stable-diffusion
6
+ - lora
7
+ - diffusers
8
+ base_model: stabilityai/stable-diffusion-xl-base-1.0
9
+ pivotal_tuning: true
10
+ textual_embeddings: embeddings.pti
11
+ instance_prompt: <s0><s1>
12
+ inference: false
13
+ ---
14
+ # sdxl-emoji LoRA by [fofr](https://replicate.com/fofr)
15
+ ### An SDXL fine-tune based on Apple Emojis
16
+
17
+ ![lora_image](https://replicate.delivery/pbxt/a3z81v5vwlKfLq1H5uBqpVmkHalOVup0jSLma9E2UaF3tawIA/out-0.png)
18
+ >
19
+
20
+ ## Inference with Replicate API
21
+ Grab your replicate token [here](https://replicate.com/account)
22
+ ```bash
23
+ pip install replicate
24
+ export REPLICATE_API_TOKEN=r8_*************************************
25
+ ```
26
+
27
+ ```py
28
+ import replicate
29
+
30
+ output = replicate.run(
31
+ "sdxl-emoji@sha256:dee76b5afde21b0f01ed7925f0665b7e879c50ee718c5f78a9d38e04d523cc5e",
32
+ input={"prompt": "A TOK emoji of a man"}
33
+ )
34
+ print(output)
35
+ ```
36
+ You may also do inference via the API with Node.js or curl, and locally with COG and Docker, [check out the Replicate API page for this model](https://replicate.com/fofr/sdxl-emoji/api)
37
+
38
+ ## Inference with 🧨 diffusers
39
+ Replicate SDXL LoRAs are trained with Pivotal Tuning, which combines training a concept via Dreambooth LoRA with training a new token with Textual Inversion.
40
+ As `diffusers` doesn't yet support textual inversion for SDXL, we will use cog-sdxl `TokenEmbeddingsHandler` class.
41
+
42
+ The trigger tokens for your prompt will be `<s0><s1>`
43
+
44
+ ```shell
45
+ pip install diffusers transformers accelerate safetensors huggingface_hub
46
+ git clone https://github.com/replicate/cog-sdxl cog_sdxl
47
+ ```
48
+
49
+ ```py
50
+ import torch
51
+ from huggingface_hub import hf_hub_download
52
+ from diffusers import DiffusionPipeline
53
+ from cog_sdxl.dataset_and_utils import TokenEmbeddingsHandler
54
+ from diffusers.models import AutoencoderKL
55
+
56
+ pipe = DiffusionPipeline.from_pretrained(
57
+ "stabilityai/stable-diffusion-xl-base-1.0",
58
+ torch_dtype=torch.float16,
59
+ variant="fp16",
60
+ ).to("cuda")
61
+
62
+ pipe.load_lora_weights("fofr/sdxl-emoji", weight_name="lora.safetensors")
63
+
64
+ text_encoders = [pipe.text_encoder, pipe.text_encoder_2]
65
+ tokenizers = [pipe.tokenizer, pipe.tokenizer_2]
66
+
67
+ embedding_path = hf_hub_download(repo_id="fofr/sdxl-emoji", filename="embeddings.pti", repo_type="model")
68
+ embhandler = TokenEmbeddingsHandler(text_encoders, tokenizers)
69
+ embhandler.load_embeddings(embedding_path)
70
+ prompt="A <s0><s1> emoji of a man"
71
+ images = pipe(
72
+ prompt,
73
+ cross_attention_kwargs={"scale": 0.8},
74
+ ).images
75
+ #your output image
76
+ images[0]
77
+ ```
embeddings.pti ADDED
lora.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5934fe70ec5f06be145670315a4cd20334abb51ace38bee4f9265fc82d035f54
3
+ size 185968776
special_params.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"TOK": "<s0><s1>"}