使用Stable Diffusion和Pokedex的描述生成神奇宝贝图片

还记得我们以前使用GAN、Clip、DALL-E生成神奇宝贝的文章吗，现在是时候使用Stable Diffusion了

在本文中，我将展示如何从神奇宝贝系列不同游戏中的Pokedex条目中获取神奇宝贝描述，并使用Stable Diffusion根据这些藐视生成图片，这样可以看看AI如何解释这些描述的。这篇文章中，我只生成了最初的150个神奇宝贝，如果需要其他的可以自行尝试。

第一件事是获得Pokedex的描述。这些Pokedex的描述将作为生成图片的文本提示。我最初的想法是为pokemon.com编写一个webscraper，Pokedex Number (NPN)执行搜索查询。这并不是很难做到，但是有一个叫做PyPokedex的小Python库非常的好用，如果你对编程和神奇宝贝感兴趣，我建议你去看看!

PyPokedex使用PokeAPI来获取神奇宝贝数据，所以可以使用PyPokedex获得各种信息，如NPN，名称，身高，体重，类型，基础统计等。除此之外PyPokedex还有一个叫做“get_description”的方法，它以字典的形式返回每个不同游戏的Pokedex描述。

一个基本的PyPokedex查询如下所示:

pokemon = pypokedex.get(dex=poke_id)

这个查询会返回一个实例化的对象，该对象将包含基于NPN值的给定神奇宝贝的所有信息。

poke_name = pokemon.name

下一步就是用get_descriptions方法获取Pokedex描述。这里我们使用《Pokemon Yellow》的Pokedex中的描述，因为这代大家都应该知道

yellow_description = pokemon.get_descriptions()[ver]

然后将这两个结果结合起来，这样给AI的提示就好了:

prompt = poke_name + " " + yellow_description

这里我们使用神奇宝贝的名字作为提示的一部分，因为有一些Pokedex条目有点模糊，并不总是产生好结果。所以神奇宝贝的名字，能够创造出更像是原版的“真正变体”。如果你不喜欢也可以去掉，但我对这样的结果很满意。

我们已经有文本了，下面开始准备Stable Diffusion模型。用下面这行代码安装必要的包:

%pip install — quiet — upgrade diffusers transformers accelerate mediapy scipy ftfy spacy

还需要包含xformers包来帮助我们创建图像。这个需要根据你的python版本训责相应的wheels包，一定不要错了

github_url = "https://github.com/brian6091/xformers-wheels"xformer_id = "0.0.15.dev0+4c06c79"xformers_wheels = f"xformers-{xformer_id}.d20221205-cp38-cp38-linux_x86_64.whl%pip install -q {github_url}/releases/download/{xformer_id}/{xformers_wheels}

接下来，指定想要使用的模型。这里选择dreamlike-photoreal-2.0。

model_id = “dreamlike-art/dreamlike-photoreal-2.0”

我们这里使用简单，快速的方法：直接使用StableDiffusionPipeline生成图片

import mediapy as media
import torch
from diffusers import StableDiffusionPipeline
device = "cuda"
if model_id.startswith("stabilityai/"):
model_revision = "fp16"
else:
model_revision = None
if scheduler is None:
pipe = StableDiffusionPipeline.from_pretrained(
model_id,
torch_dtype=torch.float16,
revision=model_revision,
)
else:
pipe = StableDiffusionPipeline.from_pretrained(
model_id,
scheduler=scheduler,
torch_dtype=torch.float16,
revision=model_revision,
)
pipe = pipe.to(device)
pipe.enable_xformers_memory_efficient_attention()
if model_id.endswith('-base'):
image_length = 512
else:
image_length = 768

现在已经准备好所有步骤了，下面开始正式生成了：

remove_safety = False
num_images = 4
if remove_safety:
negative_prompt = None
else:
negative_prompt = "nude, naked"
images = pipe(
prompt,
height = image_length,
width = image_length,
num_inference_steps = 25,
guidance_scale = 9,
num_images_per_prompt = num_images,
negative_prompt = negative_prompt,
).images
media.show_images(images)

上面的代码已经可以根据单个查询生成单个图片。但是我们需要处理150个。所以我将他们整合成一个函数，循环调用。

def makePokemonFromPokedex(ver,nPokemon):
#Loop over nPokemons to get the descritptions and generate images for each
#poke_id = 1
for poke_id in range(1, nPokemon+1, 1):
#print(poke_id)
#Specify which Pokemon we want to query using its ID number
pokemon = pypokedex.get(dex=poke_id)
#print(pokemon)
#This is the name of the Pokemon we are querying
poke_name = pokemon.name
#This is the PokeDex desciption for the current Pokemon
yellow_description = pokemon.get_descriptions()[ver]
#This is the prompt I'll feed to the AI
prompt = poke_name + " " + yellow_description
#print(prompt)
remove_safety = False
num_images = 4
if remove_safety:
negative_prompt = None
else:
negative_prompt = "nude, naked"
images = pipe(
prompt,
height = image_length,
width = image_length,
num_inference_steps = 25,
guidance_scale = 9,
num_images_per_prompt = num_images,
negative_prompt = negative_prompt,
).images
fname = 'poke_' + str(poke_id)
get_concat_h_multi_blank(images).save(fname + '.jpg')

下面看看结果

铁甲蛹

Metapod — The prompt for this is “It is waiting for the moment to evolve. At this stage, it can only harden, so it remains motionless to avoid attack”.

独角虫

Weedle —T he prompt for this is “Beware of the sharp stinger on its head. It hides in grass and bushes where it eats leaves”.

大针蜂

Beedrill— The prompt for this is “It has three poisonous stingers on its forelegs and its tail. They are used to jab its enemy repeatedly”.

阿柏蛇

Ekans— The prompt for this is “The older it gets, the longer it grows. At night, it wraps its long body around tree branches to rest.”.

穿山鼠

Sandshrew— The prompt for this is “It loves to bathe in the grit of dry, sandy areas. By sand bathing, the Pokémon rids itself of dirt and moisture clinging to its body.”. Image by author.

穿山王

Sandslash— The prompt for this is “The drier the area Sandslash lives in, the harder and smoother the Pokémon’s spikes will feel when touched.”. Image by author.

尼多力诺

Nidorino— The prompt for this is “With a horn that’s harder than diamond, this Pokémon goes around shattering boulders as it searches for a moon stone.”. Image by author.

九尾

Ninetales— The prompt for this is “It is said to live 1,000 years, and each of its tails is loaded with supernatural powers”. Image by author.

走路草

Oddish— The prompt for this is “If exposed to moonlight, it starts to move. It roams far and wide at night to scatter its seeds.”. Image by author.

地鼠

Diglett— The prompt for this is “If a Diglett digs through a field, it leaves the soil perfectly tilled and ideal for planting crops.”. Image by author.

蚊香蝌蚪

Poliwag— The prompt for this is “For Poliwag, swimming is easier than walking. The swirl pattern on its belly is actually part of the Pokémon’s innards showing through the skin.”. Image by author.

毒刺水母

Tentacruel— The prompt for this is “When the red orbs on Tentacruel’s head glow brightly, watch out. The Pokémon is about to fire off a burst of ultrasonic waves.”. Image by author.

这个要比我们以前生成的图像好太多了，对吧

https://avoid.overfit.cn/post/7f072f5e107145d0b6502a956d4e3ede

作者：Victor Murcia