Satellite-Like Landscape Visuals: Art and Machine Learning
September 20, 2024
How can we harness ML to generate terrains that are not only functional but also artistically compelling? Moreover, what role do engineers play in bridging the gap between complex algorithms and artistic expression? As machine learning continues to blur the lines between technology and creativity, these questions become increasingly pertinent. Certain algorithmic creations captivate us as art, while others leave us indifferent.
As part of our work laying the foundations for digital worlds, PLAYERUNKNOWN Productions is developing an ML model that creates images resembling realistic satellite views of landscapes. Interestingly, while still in development, this model has produced images our staff have interpreted as distinctly artistic. This blog offers some thoughts on this phenomenon through an overview of what you're seeing when observing these images. It's partly technical but also somewhat speculative.
We are also launching a daily feed of this ML-generated art on our Discord, and Instagram. Enjoy!
Author: Patrick McCaffrey, Senior Machine Learning Engineer
Art created using a machine learning tool
How does an AI learn to paint?
At a basic level, our machine learning model learns by seeing a large dataset of landscape images—mountains, coastlines, deserts, and various other natural environments. We use a training process called "Diffusion” to teach it to generate new unique images. This is an iterative process in which noise (think TV static) is added to an image, and the model learns by reducing it step by step, and in doing so refines the details of the image with each iteration.
Diffusion models have proven to be state of the art when it comes to image generation. Most leading edge image generation models like Stable diffusion and DALL-E are based upon this process. Below is an example of this process, with the most “noisy” images on the far left, gradually transforming into more plausible satellite views on the right.
Diffusion image generation in progress
Once our model has learned about millions of images it begins to understand how to produce its own images that look realistic. It looks promising, but theres a catch: every image it creates is random. How can we get a reasonable snapshot of what the model can do?
This is one of the challenges of working with machine learning. Assessing the performance of a model is a process of discovery. Lets start by generating thousands random images.
Thousands of unsorted images generated by the tool
Exploring the data
Rather than look through thousands of images manually, we can query through them with words. We use an open-source model called CLIP to transform an image into a list of numbers that captures its essential features, like shapes, colors, or objects within it. Crucially, CLIP also allows us to transform text into the same representation.
We can then sort the images by search terms like “desert” or “island,” returning results based on how aesthetically similar the generated images are to those concepts.
Search terms being entered into the tool
Interestingly, the tool also responds to fictional and fanciful terms. For example, searching for “Arrakis” might produce alien-like desert scenes, while “galaxy” could yield images that blur the line between landscapes and outer space. But, the model never saw a picture of a galaxy. Rather the noisy and nebulous shapes are the model breaking down in beautiful unintended ways. From a research perspective, these abstract terms can be of help in identifying when the model is generating images that diverge from our goal of creating realistic satellite views.
Search prompt: “Mordor”
Is there art in the machine?
Is the model solving the problem in a creative way? Perhaps the galaxy glitches are just colouful deserts. If a human created these when tasked to paint an alien landscape, would we consider it a success? Art is in the eye of the beholder after all. This subjectivity allows us to find beauty and meaning in unexpected places—even in the glitches produced by machine learning algorithms
This perhaps helps us see the connection of ML to the artistic process. Before we explored them, our random images were just that, random. It was only when a human sees the glitch that we can assign it value as art. Art is created in the symbiosis of the artist and AI. The model we trained was just a ball of potential creativity before we explored its ability.
Search prompt: “Coastal”
Schrödinger's Art
Maybe we can draw an analogy between Schrödinger's cat and the latent artistic potential of an AI model. In Schrödinger's thought experiment, a cat inside a sealed box is simultaneously alive and dead until an observer opens the box and observes its state. The act of observation collapses the cat's quantum superposition into a single reality.
Similarly, an AI model's outputs—especially unexpected results or glitches—exist in a state of latent potential. These outputs are neither art nor non-art until a human interacts with them. It's the observer's perception and interpretation that "collapses" this potential, transforming the output into a recognized piece of art. Just as the cat's fate is sealed upon observation, the AI-generated content becomes art when someone views it and labels it as such.
Search prompt: "Nice smooth moon desert"
Anyone can cook... but only the fearless can be great.
Following this train of thought, it appears increasingly crucial to give everyone the ability to explore this model. Partly because access to the means of creativity ought not to be a privilege, but also because the development of this tool can only stand to benefit from more minds and more creativity. This isn’t to say that the role of engineers, artists, or even humans is diminished. This is not a passive mode of exploration. Rather, it is an active, playful valorization of human creativity - all the more reason to have as many people as possible involved!
This training process is only the first step in our journey. We are continually researching ways to leverage machine learning to empower everyone to shape and create our worlds.
We invite you to join the conversation and share your thoughts on our Discord, where we'll be sharing a daily feed of these images further and discussing their development.
