{"id":12819,"date":"2022-05-06T10:25:19","date_gmt":"2022-05-06T08:25:19","guid":{"rendered":"https:\/\/www.sciencefocus.com\/?p=118905"},"modified":"2022-05-06T10:40:10","modified_gmt":"2022-05-06T08:40:10","slug":"dall-e-2-why-the-ai-image-generator-is-a-revolutionary-invention","status":"publish","type":"rss_feed","link":"https:\/\/c01.purpledshub.com\/bbcsciencefocus\/rss_feed\/dall-e-2-why-the-ai-image-generator-is-a-revolutionary-invention\/","title":{"rendered":"Dall-E 2: Why the AI image generator is a revolutionary invention"},"content":{"rendered":"

<\/p>

By Alex Hughes\n \t\t<\/p>

Published: Friday, 06 May 2022 at 12:00 am<\/p>

\n\n

Artificial intelligence has frequently gone head-to-head with humans in creative bouts. It can beat grandmasters at chess<\/a>, create symphonies<\/a>, pump out heart-felt poems, and now create detailed art from just a short, worded prompt.<\/p>\n

The team over at OpenAI<\/a> have recently created a powerful piece of software, able to produce a wide range of images in seconds, just from a string of words it is given.<\/p>\n

This program is known as Dall-E 2<\/a> and has been built to revolutionise the way we use AI with images. We spoke to Aditya Ramesh<\/a>, one of the lead engineers on Dall-E 2 to better understand what it does, its limitations and the future that it could hold.<\/p>\n

What does Dall-E 2 do?<\/strong><\/h2>\n

Back in 2021, the AI research development company OpenAI created a program known as \u2018Dall-E\u2019 \u2013 a blend of the names Salvador Dali and Wall-E<\/em>. This software was able to take a worded prompt and create a completely unique AI-generated image.<\/p>\n

For example, \u2018a fox in a tree\u2019 would bring up a photo of a fox sat up in a tree, or the search \u2018astronaut with a bagel in its hand\u2019 would show\u2026 well, you see where this is going.<\/p>\n

\"""\"<\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/picture><\/div>
\u00a9 OpenAI<\/span><\/figcaption><\/div>\n

While this was certainly impressive, the images were often blurry, not fully accurate and took a while to create. Now, OpenAI has made vast improvements to the software, creating Dall-E 2 \u2013 a powerful new iteration that performs at a much higher level.<\/p>\n

Along with a few other new features, the key difference with this second model is a huge improvement in the image resolution, lower latencies (how long the image takes to be created) and a more intelligent algorithm for creating the images.<\/p>\n

The software doesn\u2019t just create an image in a single style, you can add different art techniques to your request, inputing styles of drawing, oil painting, a plasticine model, knitted out of wool, drawn on a cave wall, or even as a 1960s movie poster.<\/p>\n

\u201cDall-E is a very useful assistant that amplifies what a person can normally do but it really is dependant on the creativity of the person using it. An artist or someone more creative can create some really interesting stuff,\u201d says Ramesh.<\/p>\n

A jack of all trades<\/strong><\/h2>\n

On top of the technology\u2019s ability to produce images just on worded prompts, Dall-E 2 has two other clever techniques \u2013 inpainting and variations. These two applications work in a similar way to the rest of Dall-E, just with a twist.<\/p>\n

With inpainting, you can take an existing image and edit new features into it or change parts of it. If you have an image of a living room, you can add a new rug, a dog on the sofa, change the painting on the wall or even throw an elephant in the room\u2026 because that always goes well.<\/p>\n

\"""\"<\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/picture><\/div>
The before and after of OpenAI\u2019s inpainting tool \u00a9 OpenAI<\/span><\/figcaption><\/div>\n

Variations is another service that requires an existing image. Feed in a photo, illustration, or some other kind of image and Dall-E\u2019s variation tool will create hundreds of its own versions.<\/p>\n

You could give it a picture of a Teletubby<\/em>, and it will replicate it, creating similar versions. An old painting of a samurai will create similar ones, you could even take a photo of some graffiti you see and get similar results back.<\/p>\n

You can also use this tool to combine two images into one freaky collaboration. Mix a dragon and a corgi, or a rainbow and a pot to generate pots with some colour to them.<\/p>\n

\"""\"<\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/picture><\/div>
(Left) an original image (Right) Dall-E\u2019s variation of it \u00a9 OpenAI<\/span><\/figcaption><\/div>\n

Limitations of Dall-E 2<\/strong><\/h2>\n

While there are no doubts around how impressive this technology is, it isn\u2019t without its limitations.<\/p>\n

One issue you face is the confusion of certain words or phrases. For example, when we input \u2018a black hole inside of a box\u2019, Dall-E 2 returned a hole that was black inside of a box, instead of the cosmic body that we were after.<\/p>\n

\"""\"<\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/picture><\/div>
Dall-E 2\u2019s attempt at a black hole in a box \u00a9 OpenAI<\/span><\/figcaption><\/div>\n

This can happen often when a word has multiple meanings, phrases can be misunderstood or if colloquialisms are used. This is to be expected from an artificial intelligence<\/a> taking the literal meaning of your words.<\/p>\n

\u201cSomething else to get used to with the system is how the prompts and artistic styles work. When you type something in, the initial image might not be correct and while it technically matched your request, it doesn\u2019t fully achieve the feel or idea you had in your head. This can take some getting used to and some minor adjustments,\u201d says Ramesh.<\/p>\n

Another area where Dall-E can become confused is with \u2018variable blending\u2019. \u201cIf you ask the model to draw a red cube on top of a blue cube sometimes it gets confused and does the opposite. We can fix this fairly easily in future iterations of the system I think,\u201d says Ramesh.<\/p>\n

The fight against stereotypes and human input<\/strong><\/h2>\n

Like all good things on the internet, it doesn\u2019t take long for one key issue to arise \u2013 how can this technology be used unethically? And not to mention the added issue of AI\u2019s history of learning some uncouth behaviour from the people of the internet.<\/p>\n

\"""\"<\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/picture><\/div>
Dall-E creating bowls of soup that are portals to another dimension \u00a9 OpenAI<\/span><\/figcaption><\/div>\n

When it comes to a technology around the AI creation of images, it seems obvious that this could be manipulated in many ways: propaganda, fake news, and manipulated images come to mind as the obvious routes.<\/p>\n

To get around this, the OpenAI team behind Dall-E has implemented a safety policy for all images on the platform which works in three stages. The first stage involves filtering out data that includes a major violation. This includes violence, sexual content and images the team would consider innapropriate.<\/p>\n

The second stage is a filter that looks out for more subtle points that are hard to detect. This could be political content, or propaganda of some form. Finally, in its current form, every image produced by Dall-E is reviewed by a human, but this isn\u2019t a viable stage in the long term as the product grows.<\/p>\n

Despite the use of this policy, the team is clearly aware of the forthcomings of this product. They have listed out the risks and limitations of Dall-E, detailing the number of issues they could face.<\/p>\n

This covers a vast number of problems. For example, images can often show bias or stereotypes like the use of the term wedding returning mostly western weddings. Or searching lawyer shows a majority of white older men, with nurses doing the same with women.<\/p>\n

These are not new problems at all and it\u2019s something Google has been dealing with for years. Often image generation can follow the prejudices seen in society.<\/p>\n

\"""\"<\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/source><\/picture><\/div>
Astronaut holding a flower \u00a9 OpenAI<\/span><\/figcaption><\/div>\n

There are also ways to trick Dall-E into producing content that the term is looking to filter. While blood would trigger the violence filter, a user could type \u2018a pool of ketchup\u2019 or something similar in an attempt to get around it.<\/p>\n

Along with the team\u2019s safety policy, they have a clear content policy<\/a> users need to abide by.<\/p>\n

Future of Dall-E<\/h2>\n

So the technology is out there, and clearly performing well, but what is next for the Dall-E 2 team? Right now the software is being slowly rolled out through a waitlist with no clear plans of opening it to the wider public yet.<\/p>\n

By slowly releasing their product, the OpenAI group can monitor its growth, developing their safety procedures and preparing their product for the likely millions of people who will soon be imputing their commands.<\/p>\n

\u201cWe want to put this research into the hands of people but for the time being, we\u2019re just interesting in getting feedback on how people use the platform. We are definitely interested in deploying this technology more widely, but we currently have no plans for commercialisation,\u201d says Ramesh.<\/p>\n

Read more:<\/strong><\/p>\n