Ideagram Studio

How it works

SigLIP2 (Sigmoid Loss for Language-Image Pre-Training 2) from Google was trained on huge batches of image-caption pairs scraped from the internet.

When you upload an image, the SigLIP2 embedding algorithm’s image encoder splits the image up into a 14x14 grid of tiles, each of which gets turned into a 768-dimensional vector which is a rich numerical fingerprint capturing color, texture, edges and more. These vectors are initially independent (meaning, each one has no knowledge of the others) but are then fed into 12 transformer layers in which each vector gets iteratively updated based on context from the others.

After this process, the 196 tiles are fed into a single pooling layer which asks each tile “what do you know about this image?” and composes a single 768-dimensional vector summarizing the responses.

Simultaneously, that same embedding process is happening on a curated set of 261 descriptive vocabulary tags covering 8 distinct regions of “feel”, including color, light, mood, valence, arousal, texture, energy and style. Because SigLIP2 was trained on captions instead of individual words, each tag gets wrapped into 4 short template sentences:

• “a photo that feels {tag}”
• “a {tag} image”
• “{tag} atmosphere”
• “a picture with a {tag} quality”

Each set of these template phrases gets batched and fed into the text encoder, after which they are stacked and averaged resulting in a single 768-dimensional embedding vector per tag. After encoding, the cosine similarity between the image vector and each tag vector is calculated and fed into a sigmoid (hence the name) with a scale k and bias b learned directly from SigLIP2’s pre-training, to get a final score between 0 and 1:

Here’s how to understand this: if you showed the image to the system and asked “does the phrase ‘a photo that feels {tag}’ genuinely describe this image, yes or no?”, then the final score would be how confident the system is that the answer is yes.

Final scores for tags are ranked and categorized by their distinct region of “feel” (referred to in the code as “facet”). The top few tags per facet (specifically: 2 for “color”, 1 for “light”, 3 for “mood”, 1 for “valence”, 1 for “arousal”, 2 for “texture”, 1 for “energy”, and 1 for “style”) are used to compose a prompt for the Suno v5.5 model, which is simply the list of tags in order from highest to lowest score. That prompt is then fed to Suno v5.5 to produce two highly mood-accurate songs for that image.

Through testing across a diverse image corpus, I have found this system capable of accurately capturing a wide range of unique moods and aesthetics from pleasant to whimsical to punk. It is really fun to play with - please try your own images and see what comes up!

Optional LLM-Evaluation

By default, language models aren’t being used anywhere in the tag extraction process. If you’d like, you can turn on LLM-judges for a run for slightly more refined tag lists.

LLM-judge #1 has Claude Sonnet 5 look at the image, then each tag in the list, and decide “whether it captures the mood or feeling the image evokes - not whether it’s a literal, physically accurate description” (taken directly from the system prompt). If not, the tag gets flagged and then dropped from the prompt.

LLM-judge #2 has Claude Sonnet 5 scan the tag list for any seeming contradictions between tags across different facets (ex: “joyful” from valence and “haunting” from mood). If it identifies a contradictory pair, it then checks whether each tag’s "feeling" is already covered by some other tag elsewhere in the list. Whichever of the two tags has coverage gets dropped, since two somewhat similar tags are each relatively less descriptive in the final Suno v5.5 prompt. While this is somewhat uncommon, there were instances in my testing of the system that justified the creation of this judge.

These LLM-judges sort of “hone-in” the sometimes unpredictability of the tags from the embedding model and thus the unpredictability of the generated tracks. But the reason I set these to off by default is because it sometimes feels like there is some magic contained within these otherwise seemingly unfitting tags (which shine through in the audio) - if you play around with this enough, you’ll see exactly what I mean!

Other embedding algorithms

While this project was optimized for and built around the SigLIP2 embedding algorithm, I also wanted to explore other embedding algorithms to see how they compared. Specifically, CLIP (SigLIP’s predecessor and the OG joint embedding algorithm from OpenAI), ALIGN from Google (adaptation of CLIP that uses EfficientNet for the image encoder and BERT for the text encoder) and MetaCLIP from Meta (adaptation of CLIP that favors more rare, specific concepts due to how it was trained).

From my testing, CLIP and MetaCLIP felt like less refined versions of SigLIP, producing tracks that sort of had the right idea but ultimately tended to sound somewhat generic. ALIGN would occasionally knock it out of the park but I found it to be unreliable. I should stress again that this project was built around SigLIP2, so it’s very likely that a system like this could work well for any of these algorithms if optimized properly.

Specific checkpoints (all loaded through HuggingFace transformers.AutoModel):

Genre Knob

You may have noticed the “Genre Knob”. This is a fun addition I came up with that adds a genre tag to the start of a tag list to essentially force the generated song to conform to a particular genre. You can select from one of 17 genres: rock, psychedelic rock, prog rock, math rock, metal, heavy metal, thrash metal, dream pop, synth-pop, idm, house, techno, trance, jazz, jazz fusion, orchestral, and reggae (this selection of genres is not comprehensive by any means, but from my testing these were the genres that Suno v5.5 was most consistently responsive to).

The genre tag tends to overpower the nuances that the embedding algorithm captures, which is why this is positioned as an extra feature. It is really fun to mess around with though, and the quality of the tracks produced is a testament to how good the Suno model is.

Vocals

All prompts sent to Suno have “Instrumental=On”, but the app will occasionally produce songs with vocals anyway. Since Suno v5.5 is the same model for generating songs both with and without vocals, the instrumental option can’t ever be a hard toggle, only a strong suggestion.

Suno v5.5

This app uses the Suno v5.5 model for audio generation. All rights to the underlying model belong to Suno; no ownership is claimed by this app.