What is the value of human work in a world of generative AI? In her paper, Learning to Prompt: Human Adaptation in Production with Generative AI, Economics PhD candidate Sijie Lin investigated the role human creativity might play to answer that question. Lin investigated how people use Midjourney, an AI image generator, the users of Discord can access to create images based on text prompts. To Lin, the tool provided an interesting setting to conduct research.
“Midjourney allowed me to observe the creative process,” Lin said. “In many previous settings we only observed inputs and outputs, but in this setting, I can see exactly how people interact with AI, going back and forth to come up with the image they want.”
What Lin discovered is that using AI tools to produce quality work that meets specifications requires iteration and human effort.
“It’s not trivial to get what you want from AI,” she explained. “You need to write the right prompts, so the output goes in the direction you want. That judgment is very important and it’s mentally demanding. You can’t just press a button and magically get whatever you want.”
During the period when Lin observed how Midjourney users interacted with AI, several different versions were released. In economic terms those improvements to the tools represented an exogenous shock that enabled her to see how user behaviour changed in response to the improvements to AI.
“When the AI tools became better, people started to write different words in their prompts, which changes the final output, the images,” Lin explained.
In the first part of the paper, Lin studies human adaptation across different AI versions. As each new version of the tool was released, Lin could collect the prompts that users submitted to the old AI version and resubmit them to the new AI, and vice versa, to see how the image output changed across versions. This exercise shows that when the AI improves, most output changes occur because people use different words, while a smaller fraction is due to changes in the AI itself.
In the second part of the paper, Lin studies the creative process of an artwork. “Without human adaptation, users would need three times more prompts to achieve the same results,” she said. “Human judgment and adaptation are quantitatively important in the creative process.”
Her research has led Lin to express her ideas about the very nature of art.
“Even if AI can create images that instantly fill an entire room, you still want a human to walk around and pick out the ones that matter,” she said. “I would say that judgment is the soul of art.”
As AI’s emergence continues, Lin’s research is remarkable for its discoveries about the key role humans play in securing valuable output with its tools.
“Using a creative approach to modelling the creative process, and cutting-edge techniques for evaluating images and analyzing natural language, Sijie evaluates the impact of human adaptability both to new versions of AI and within the process of creating an artwork,” said Professor Heski Bar-Isaac, Lin’s Supervisor. “Her results imply that human adaptation is quantitatively important to the production process even as generative AI improves.”
Lin believes that, as she moves forward with this work, her interdisciplinary approach will take on added dimensions.
“I reached out to some artists online who use AI to create art and also know how to draw without AI,” she remembered. “From talking to them, I learned so much about the industry. I think these interactions with people in other fields are extremely helpful for research because we want research to be directly related to real life.”
She has also found her students to be a source of inspiration. Through her role as a course instructor of Applied Game Theory, Lin came to appreciate undergraduate learners for their creativity.
“Students have so many new ideas when I talk about well-established concepts in game theory,” she said. “I learn so much from them.”
Return to the Department of Economics website.
Scroll more news.