Prompting as Practice

Generative filament bloom

vibecon 2026 · workshop

prompt engineering for image & video generation
Derrick Schultz · Canyon NYC · June 17–18 2026

Derrick Schultz

  • artist & filmmaker
  • build custom models on titles.xyz
  • teach generative AI at NYU & SVA
  • lead creative tech with artists

structuralist filmmaking by structuring prompts

err me z · 2026

Agentic filmmaking system generates sequences and prompt modifications for stop-motion imagery and video motion.

Phantoms of Endless Day

2025 · LUMA Arles

Ho Tzu Nyen’s four-channel installation at LUMA Arles — built from an algorithmic editing system and 10+ hours of generative AI footage.

training models @ titles

Selfie Song generated stillSelfie Song generated stillSelfie Song generated still

titles.xyz

Trained artist models so anyone can prompt with their work.

I fucking hate prompt engineering

Ultra-realistic vertical photo (9:16).a stylish young man with a lean,toned physique (around 57 kg,5’6 tall) — his beard remains unchanged,and his head is clean-shaven,giving him a modern,confident bald look.He wears one small,silver hoop earring in each ear,adding a subtle yet refined touch of individuality and style.sits on indoor stairs beside a matte concrete wall.A rectangular beam of golden sunlight from a window hits the wall,creating a crisp shadow silhouette inside the bright frame.He wears a black ribbed knit sweater,tapered grey chinos,and chunky white sneakers.Pose: seated,elbows on thighs,hands loosely clasped,chin slightly lifted,eyes looking toward the light,calm and confident expression.Lighting: hard warm sunlight from camera-right as key,soft ambient bounce fill,high contrast with long shadows,cinematic golden-hour mood.Camera & look: low-mid angle from a few steps below,50–85mm f/2.2 lens,shallow depth of field,clean optics,realistic skin texture,fine film grain,subtle vignette.Style: minimalist background,no clutter,fashion editorial realism.Exclude: cartoon,CGI,AI-artifacts,over-smoothing,plastic skin,excessive sharpening,motion blur,warped anatomy,extra fingers,disfigured hands,double shadow,blown highlights,banding,watermark,logo,text,bad perspective,dirty wall,clutter.

I fucking hate can stand prompt engineering exploring

schedule

01  how prompting works
02  explore
03  expand
04  more to explore

01

how prompting works

Branching dendritic structure

diffusion

Diffusion denoising sequence — pure noise refining step by step into a clear image

how prompting works

The model starts from random noise and refines it, step by step, into an image.

conditioning

Diffusion denoising sequence — pure noise refining step by step into a clear image

how prompting works

Your prompt conditions each step — it biases what the noise resolves into. You don’t draw the image; you steer the denoising.

02

explore

Microscopic bloom of glowing orange and blue filaments

use any site that offers you more than one model

Multi-model generation canvas with model picker and image gallery

where to explore

Titles, Krea, Fuser, Flora, Runway, etc. (Get $5 free when you use titles.xyz 😁)

before we jump to the 200-word prompt, let’s use this time to explore the breadth of these models

no prompt

Output of an empty promptSDXL
Output of an empty promptNano Banana Pro
Output of an empty promptSeedream 5.0 Lite

explore

Type nothing — on titles, a single comma. See what the model makes from nothing: its raw default.

Note: Nano Banana requires at least 3 characters.

always generate multiple images

Output from the same model and promptOutput from the same model and promptOutput from the same model and prompt

explore

Image generation models are non-deterministic. They generate new images every time they are run. Don’t rely on a single output to define the entire model.

Flux Klein 9B Base

a short prompt

Output of a short promptSDXL
Output of a short promptFlux.1 Dev
Output of a short promptNano Banana Pro

explore

A subject with a style — two or three words is enough to point it somewhere.

03

expand

Kirlian-style electric glow around a leaf

the basics

  • Order matters
  • Some (not all) models support negative prompts — what you don’t want to see

subject: what’s in frame
style: medium & aesthetic
environment: where it sits
lighting: how it’s lit
composition: framing & camera
color: the palette
mood: the overall feeling

Every model prefers of prompts in a specific format.
The right prompt (and the right format) steers the model toward the output you want.

red fox, misty pine forest, dawn, backlit, soft fog, 35mm, shallow depth of field
A lone red fox stands in a misty pine forest at dawn, backlit by soft light through the fog. Shot on 35mm, shallow depth of field.
{
  "subject": "red fox",
  "scene": "misty pine forest",
  "time": "dawn",
  "light": "soft backlight through fog",
  "camera": "35mm, shallow depth of field",
  "mood": "quiet, cinematic",
  "palette": "amber, sage, cool grey"
}

how prompting works

tags — keywords · SD1.5, SDXL
natural language — sentences · Flux, Z-Image, Midjourney
structured data — JSON · Nano Banana, Gemini

structured data

  • Skip it for quick exploration — it’s verbose and slows you down
  • Only some models truly parse it (Nano Banana, Gemini); most diffusion models (SDXL, Flux) just read it as plain text

Reach for JSON when you want precise, repeatable control — one field per attribute.
Change one field, hold the rest constant.

expand to JSON

Paper sculpture from a JSON promptPaper sculpture from a JSON promptPaper sculpture from a JSON prompt

expand

Structured, repeatable prompts — change one field, hold the rest constant.

most prompting has turned to AI to solve prompt expansion

[ images coming ]

expand

This is a lot of information to hold at one time.

one click, many directions

Screenshot — the expand tool’s grid of style/direction icons

expand

Pick a direction instead of writing it all out by hand.

live

the expand tool

prompt-expander-dvsmethid.replit.app

rough idea in · expanded prompt out

Prompt Workspace — turn a simple idea or image into a detailed generation prompt

add styles

before — plain short prompt
after — expanded with styles

expand

Lighting, lens, mood, composition — added for you, without writing it all out.

04

more to explore

Soft coral-like fractal bloom

image → prompt

Image-to-prompt tool — upload an image, get an extracted prompt, regenerate

bonus

Upload an image; a VLM extracts a base prompt you can edit, then regenerate.

Goldfish in a glass aquarium

inversion

the prompt a VLM extracted from this image

Hyperrealistic close-up of a vibrant orange goldfish and a smaller companion in a square glass aquarium, lush green seaweed and mossy gravel around a single smooth stone, tiny rising bubbles and subtle glass reflections against a deep, nearly black backdrop. Centered tight composition with high-value contrast and soft cinematic rim lighting, crisp fine-detail linework and airbrushed shading for velvety scales and glass highlights, saturated warm oranges against cool deep greens in a National Geographic-style macro photography aesthetic.

pipeline

image → VLM (“describe as a prompt for this model”) → base model → generate → compare → revise the query.

failure mode

VLMs default to content — “a woman in a red coat.”
You often want style — “grainy 35mm, blown highlights, teal shadows.”
Specify which to extract.

constrain the VLM

“describe only lighting and color.”
“ignore the subject; capture rendering style.”
“output as sdxl tags.”

The VLM’s output is controllable — constrain it.

live demo

image → prompt → image

prompt-expander-dvsmethid.replit.app

image in · formatted prompt out · regenerate

applications

• reproduce a target look
• maintain one style across many images
• convert a film still into a generatable prompt

video prompts

more to explore

Now you direct motion and camera — not just the frame, but what happens over time.

text → video

popular video models

Runway · Kling · Veo 3 · Seedance · Wan

action: what happens, not just what’s there
camera: dolly, pan, tracking, or locked-off
one beat: a single clear action per shot
pacing: how it moves through time
style: cinematic, film stock, mood

image → video

motion, not the scene — the frame already set the look
name what moves — camera, subject, or both
keep it plausible — motion the composition allows

train a model with Titles

Custom model output — dog riding a dolphinCustom model output — dog riding a dolphinCustom model output — dog riding a dolphin

more to explore

Bake a style or subject into the model so you barely have to describe it.

train a model with Titles

Custom model output — dog riding a dolphinCustom model output — dog riding a dolphinCustom model output — dog riding a dolphin

more to explore

One model, one subject — endlessly re-promptable.

thanks

find me