Take Your AI Know-How to the Next Level: Here's What You Need to Know About Prompt Engineering

An Artist in Residence on A.I.’s Territory

At a reception for OpenAI’s first developer conference in San Francisco last month, a crowd mingled, wine in hand, as withering criticism of art created with artificial intelligence flashed on a blue wall at the front of the room. “I’ve seen more engaging art from a malfunctioning printer,” one critic jabbed. “The fine-art equivalent of elevator music,” huffed another. “Inoffensive, unmemorable and terminally dull.”

It might seem an odd strategy for OpenAI, the company behind widely used generative A.I. tools like ChatGPT and DALL-E, to promote scorn of A.I. art, until you catch the twist: A.I. itself wrote the criticism. Alexander Reben, the M.I.T.-educated artist behind the presentation, combined his own custom code with GPT-4, a version of the large language model that powers the ChatGPT online chatbot.

Next month, Mr. Reben, 38, will become OpenAI’s first artist in residence. He steps in as generative A.I. advances at a head-spinning rate, with artists and writers trying to make sense of the possibilities and shifting implications. Some regard artificial intelligence as a powerful and innovative tool that can steer them in weird and wonderful directions. Others express outrage that A.I. is scraping their work from the internet to train systems without permission, compensation or credit.

In late November, a group of visual artists filed an amended copyright lawsuit against Stability AI, Midjourney and other makers of A.I. tools after a federal judge dismissed parts of the original complaint, which accused the companies of misusing the artists’ creations to train generative A.I systems. Mr. Reben said he couldn’t speak to the specifics of A.I. and the law, “but like with any new creative technology, the law needs to catch up to the unpredictable future.”

(The New York Times sued OpenAI and Microsoft for copyright infringement on Wednesday.)

Tech companies including Google, Autodesk and Microsoft have welcomed artists in residence. And for the last several years, artists have tested products like GPT and the DALL-E image generator, offering insight into the tools’ creative potential before their public release. But the OpenAI residency, which is giving Mr. Reben a front-row view of the company’s work, is a first for the start-up that is at the center of the debate over art and A.I.

“Alex is one of the first people we share our new models with,” said Natalie Summers, a spokeswoman for OpenAI.

Sam Altman, OpenAI’s chief executive, has long acknowledged that the technologies created by his company will change the nature of art. But he insists that no matter how good the technology gets, artists — human artists — will always matter.

“There was a real moment of fear where people asked, ‘Is this a tool we have built or a creature we have built?’” he said last month during an appearance in front of more than 300 artists and art lovers packed into an abandoned warehouse in downtown Oakland, Calif. “People now view these things as a new set of tools.”

After the digital artist Android Jones said at the event that many artists were still very angry over the rise of A.I. image generators and the way it reduced the value of their own art, Mr. Altman said people would always seek art created by other people.

“There is clearly going to be more competition,” he said. “But, awash in a sea of A.I.-generated art, that desire for human connection will go up, not down.”

Ge Wang, an associate director of Stanford’s Institute for Human-Centered Artificial Intelligence and an associate professor of music and computer science at the school’s Center for Computer Research in Music and Acoustics, wonders how receptive OpenAI will be to considering the tough questions about A.I.’s impact on art. What’s the right balance between machine output and human curation? Will the instantaneous results produced by the likes of DALL-E discourage people from developing the kinds of skills that require study and time?

“Asking these questions is kind of bad for business, and OpenAI is a business,” Dr. Wang said. “You might have a wonderful artist there in residence asking questions. Are you willing to receive them?”

Nonetheless, Dr. Wang — who is also a musician and designed two music-making apps, Ocarina and Magic Piano, for Apple’s iPhone — said he was heartened that Mr. Reben was open to engaging with the questions about A.I.’s impact on the art community.

Mr. Reben said that as a technologist who had studied the impact of innovations like photography and recorded music on creativity, “I usually stay on the cautiously optimistic side.”

“But like any other technology of the past, there are both sides to the coin,” he added.

The New York native moved to Berkeley, Calif., a decade ago to become director of technology and research at Stochastic Labs, an incubator for creative scientists and engineers that is housed in a three-story 19th-century Victorian. Mr. Reben’s highly conceptual art lines the walls of the main hallway and fills work spaces packed with printers, headphones, cables, capacitors, soldering supplies, and other bits and bobs.

On a rainy Thursday, Mr. Reben relaxed on a couch at Stochastic after a meeting at OpenAI to continue working out details of what he’ll do during the residency, which will last three months.

“If I come out of it and make my art better, or even come up with some new questions or new directions to present to the world, that would be very valuable,” said Mr. Reben, who researched human-machine symbiosis as a graduate student at the M.I.T. Media Lab, an interdisciplinary research center.

The residency overlaps with Mr. Reben’s first major retrospective, titled “AI Am I?” and on display through April at Sacramento’s Crocker Art Museum. DALL-E and other image generators like Midjourney and Stability AI’s Stable Diffusion have captivated the internet by allowing anyone to instantly retrieve custom visual imagery simply by typing a few words into a box. But while much A.I.-generated art exists as pixels, Mr. Reben often manifests physical structures from ideas he hones with the help of artificial intelligence.

“I like a lot of absurdity and humor in my work, even if the underpinning question is serious,” Mr. Reben said.

One sculpture at the exhibit presents six toilet plungers queued up like a bizarre police lineup. A.I.-generated text on the wall placard explains that the work represents all that remains of the Plungers, an apocryphal ’70s art collective. Its fake artists adhered to “plungism,” a fictional philosophy “wherein the mind of an artist is in a state of flux and able to be influenced by all things, even plungers.”

Plungism arose from Mr. Reben’s extensive back and forth with GPT-3: He’d enter a prompt (an input aimed at producing a desired response), and then tinker with his favorite responses, sometimes feeding the edited language back to the A.I. until he landed on just the right wording.

Then there’s “Dreams of the Cheese-Faced Gentleman,” which depicts a man whose face could be mistaken for a wheel of Swiss cheese. Mr. Reben worked with GPT-4 to find the right prompts to craft a compelling description of a painting, then fed the curated text into an image generator. He’s not a painter himself, so he commissioned one to make the artwork.

A large language model capable of ingesting both images and text then studied the painting and described it in language that would fit in at any museum. “The combination of psychedelic surrealism and whimsicality lends the painting an air of playfulness, challenging the viewer to engage with the work’s complex layers of meaning,” the wall label reads.

Janisy Lagrue, the A.I.-imagined name for the real-life painter who produced the oil on canvas, explained: “I use cheese because it is so perfect a symbol of the American dream. Cheese is a commodity, not a food. It is totally artificial, and it is delicious.”

The exhibit provokes more questions than answers, a reflection of Mr. Reben’s belief that as machines produce better outputs, humans need to ask better questions — about bias and ownership, among other things.

“Given how young this creative tool is, much still needs to be solved, and confronting these problems falls on the shoulders of everyone involved, from its developers to its users,” Mr. Reben said. “The more people thinking about these questions the better.”

Mr. Reben doesn’t profess to speak for all artists as OpenAI’s first artist in residence. But he does understand their concerns. Artists and writers worry that A.I. could steal their jobs, but Dr. Wang of Stanford said the nervousness extended beyond the possibility of lost livelihood.

The fear is “not only are we going to be replaced as artists, it’s that we’ll be replaced by something far more generic, far less interesting,” he said. “Maybe generic is enough to make a ton of money.”

Cade Metz contributed reporting.

Perplexity AI valued at $520 million in funding from Bezos, Nvidia