At a glance
A new system called PrompTHis, co-authored by University of Nottingham associate professor Kai Xu, turns an artist's prompt history into a navigable graph showing which edits moved an AI-generated image. The study reveals both how much people can read from that map and where they still struggle.
Eleven people sat down with an AI image generator and were then asked to explain what they had just done. They could name the distinct visual themes they had explored 82.21% of the time, and they could describe how one cluster of generated images differed from another 96.92% of the time. When the question narrowed to which particular word in a prompt drove a particular change, accuracy fell to 78.12%.
Those figures come from the evaluation of a system called PrompTHis, built by researchers at Peking University and the University of Nottingham. The paper, by Yuhan Guo, Hanning Shao, Can Liu, Kai Xu and Xiaoru Yuan, appears in IEEE Transactions on Visualization and Computer Graphics and was presented at IEEE VIS 2024. Its premise is that the most useful artifact in generative art isn't the final image. It's the record of what the artist tried, changed and threw away.
That record usually evaporates. Most text-to-image tools keep the last prompt that worked and little else, so the deleted words, reweighted phrases, abandoned branches and half-finished side experiments behind a finished picture are gone. The artist is left guessing at their own process, and often re-running an experiment they already ran two hours earlier.
The trail of edits that nobody keeps
Kai Xu is an associate professor in the School of Computer Science at the University of Nottingham and co-director of the school's visualization research group. He took his undergraduate degree in computer engineering at Shanghai Jiaotong University and a PhD in computer science at the University of Queensland, and much of his career has gone into data-driven computer graphics and 3D vision. PrompTHis is a turn toward the human side of the generative boom. "Following this idea, we created the PrompTHis to help artists better understand the capability of generative AI models and how to control them," he wrote on his research blog when the work was published.
The design started with two working artists, not with a dataset. One built images through Disco Diffusion in a Google Colab notebook; the other used Stable Diffusion and Midjourney. The researchers watched them work and interviewed them, then coded the transcripts thematically. What emerged was a workflow with no clean ending: artists often begin without a fixed idea and adjust as images arrive, and the loop stops when they're satisfied, when they no longer know how to improve the prompt, or when time runs out. The authors note that the last two happen far more often than the first.
The misalignment between what an artist means and what the model renders is where the anecdotes get specific. One artist's prompt contained the words "head" and "shoulder" and the model produced a bottle of shampoo. Another ended up with robotic animals nobody asked for, liked them, and folded the term into the next prompt. Both directions matter: the failures are as informative as the lucky accidents, and neither survives in a tool that only stores the final text.
A third finding pushed the project's scope. Artists wanted the model to make suggestions tuned to their own taste, covering everything from automatic parameter exploration to judging an output against their preferences. The authors decided that sort of personalization requires knowing how a person actually explores, so they started with the exploration record first and left recommendations for later.
A family tree where every branch carries its own edit
The system's centrepiece is the Image Variant Graph. Each generated image becomes a node. Each line between two nodes represents a difference between their prompts, so the graph is built from edits rather than from images alone. Edits are colour-coded: blue for an added word or an increased weight, red for a removal or a reduced weight, green for a reordering. Every line carries a short annotation naming the change, something like removing "1boy" and adding "1girl". Distance on the canvas reflects how far apart the images sit, and clusters of similar outputs collect into bubbles.
The easiest way to picture it is a family tree for images in which every line is labelled with the exact edit that produced the child, and the length of the line tells you how far the child moved from its parent. Some edits barely nudge the picture; others send it somewhere else entirely, and the graph makes that difference visible at a glance.
Getting there takes two passes. On the text side, the system computes Jaccard similarity between every pair of prompts to build a distance matrix, and only compares pairs that are close enough, with a default threshold of 0.6 that the user can loosen or tighten. Word alignment uses the Myers algorithm, which identifies insertions and deletions, treats a deletion followed by an insertion as a reorder, and also catches the numeric weights that diffusion models let users attach to individual terms.
On the image side, each picture is passed through both the text and image encoders of CLIP, the contrastive model that maps text and images into a shared space, producing 512-dimensional vectors. Those vectors are squashed to two dimensions with t-SNE using cosine distance, which yields two separate embeddings, one shaped by text and one by pixels. Procrustes analysis aligns the two spaces, and by default they're averaged, with a slider letting users shift the balance. Images are then grouped by hierarchical agglomerative clustering, and the connecting lines are bundled by cluster before their weights are redistributed to reflect how much each edit actually moved the output.
Around the graph sit three supporting panels: a creation panel for entering prompts and opening sessions, a history box that shows each prompt with its changes highlighted against the previous attempt, and a navigation mini-map that summarises the whole session. A full description of the method and the study is available in the preprint on arXiv.
Photo by Egor Komarov on Unsplash
What six interviewees did with the graph
The qualitative work involved six people. Four were university professors who teach and practise visual art, each with more than twenty years of professional experience, and two of them had taken part in the original design interviews. The fifth was a designer with seven years in interaction design. The sixth was a computer science postgraduate with no art training. Each session began with fifteen minutes of training, included a short creative run on a stripped-back version of the tool containing only the history box and mini-map, then roughly twenty minutes of iterative image making, ten minutes of recounting the session, and a semi-structured interview.
Reviewing came first. Everyone leaned on the history box to find earlier images worth refining. The graph entered later, as an overview. "The graph enables me to ignore the prompts and look at the images on their own merits," one participant said. Another pointed at the bubbles that gather images made from an identical prompt: "The images generated by one prompt can be quite different. I believe the visualization helps to identify more stable prompts."
The most concrete result involved a phrase collision. One participant had spent a baseline session mixing stylistic and descriptive terms and could not work out any rule behind the outputs. Reconstructing the session on the graph, he saw that "light red" was pulling against "Chinese painting", introducing modern elements into pictures meant to stay traditional. In a later session he got stuck, traced the run of disappointing images back to a single insertion, the phrase "album cover", and removed it. The outputs improved.
Memory turned out to be part of the payload. Another participant had once tried an apple in a painter's style and disliked the result. Reviewing a different session, a cartoon pig where less familiar style references had been swapped for "Monet", he recalled that the model handled Monet well and applied the lesson to the apple. A third participant described how the graph changed the shape of exploration: "When creating with the baseline system, I often focused on the most recent steps and was unwilling to branch out." With the graph in view, he said, "I could imagine where the desired results are in the embedding space and fine-tune prompts accordingly."
The separate quantitative study with eleven amateur participants tested whether people could read the visualisation correctly rather than whether they liked it. Identifying the themes of a session and the differences between clusters proved straightforward. Attributing influence to individual words did not, and the authors explain why: participants tended to fix on the most obvious variation in an image and miss the same word's effect in a different context.
The honest limits
That 78.12% is the number to hold onto. A word in a prompt does not act in isolation, and a graph that summarises an edit with one label can only ever tell part of the story. The authors also acknowledge that the system was not designed for live use: the current version mainly supports reviewing and recounting a session after the fact, not steering one in real time.
Sample size is the other constraint. Eleven people in the quantitative study and six in the interviews is normal for visualization research, where deep observation of a few expert users carries more weight than a broad survey. It still means the findings describe what a small group did in a controlled, remote setting, not what a studio of forty artists would do over a month. Two of the six interviewees had helped shape the design, which makes their enthusiasm worth reading with some care.
The authors describe PrompTHis as "an initial step towards understanding and utilizing individual creation history". Their own list of what comes next is unglamorous and specific: tighten the graph's layout and encoding so it can support people mid-session rather than afterwards; capture provenance completely, including the context that prompts and images alone leave out; and extend the system beyond the professional artists it was built for. The artists, for their part, asked for features that don't exist yet. They wanted to rate and pin images, annotate attempts, arrange a session along a story line when the goal is a film, and use their own sketches as seeds for generation.
Why researchers outside visualization are paying attention
The paper's framing reaches past art. The authors treat a prompt history as a case of analytic provenance, a term from visual analytics for the full trail of an analysis: the inputs, the interactions, the operations performed, the intermediate and final results, and the reasoning behind each decision. Provenance is what makes an analysis reproducible and auditable, and generative models have made it scarce. A published image rarely arrives with the sequence of edits that produced it.
That gap has consequences beyond studios. Anyone teaching with these tools has to reconstruct process from memory. Journals and galleries fielding AI-assisted work have few ways to verify how a picture was made. And the record of an artist's exploration is exactly the kind of data that could train better assistance, which is where the authors' longer ambition sits. They argue the system can also serve researchers who want to understand how artists actually interact with these models, rather than how vendors say they do.
The published version of the work is available through the paper's DOI at IEEE. The research was supported by the National Natural Science Foundation of China under grant 62272012, with additional support acknowledged from the Wuhan East Lake High-Tech Development Zone national experimental base for governance of intelligent society. Some of the prompts and images used in the study were supplied by Richard Ramchurn, and the authors thank Rebecca Ruige Xu and the other interview participants.
Photo by Adelade Mbuyazi on Unsplash
What happens next is a harder problem than plotting history. The authors' long-term goal is to infer creative intention itself, which can only be guessed at from the prompts and images a person leaves behind. If that inference works, even narrowly in text-to-image generation, it would unlock recommendation and adaptive behaviour that respond to a specific artist's taste instead of a generic average. Suggestion engines that currently ask what you want would instead read what you have been circling for the past hour.
The practical version is already visible. A tool that shows an artist which word broke a promising image is useful on a Tuesday afternoon. A decade of such records, kept across projects and studios, is something else: a searchable account of how a visual idea was formed, by whom, and with what model at what version. That account doesn't exist yet. PrompTHis is a first attempt at keeping it.
- PrompTHis
- Kai Xu
- text-to-image generation
- information visualization
- prompt engineering

