Something different happens when art and music merge

This is a long read. But it describes a project that has taken me two years.

To begin at the beginning: visual art is one thing. It uses specific senses and parts of the brain. Music or auditory art is another thing. It uses other senses and parts of the brain.

When you paint, you use your sight and your touch.
You think in terms of composition, shapes, colours, texture, light, lines.
When you compose music, you use your hearing, your sight, and your sense of motion.
You think in keys, notes, pitch, rhythms, waveforms, time.
When you do both, you make things that…look right, and rhyme.

Binary or on the spectrum?

I work at two forms of art; music and painting. And I see images of music, and vice versa. As a result, I have written songs about paintings, have made paintings that depict songs, poems to describe paintings, and songs from poems, about paintings. You get my meaning? The common element in all these art forms, is narrative.

As it has always been, every song, every painting, every poem, and every book tells a story and has a message. Every story has a creator who sends it to someone in some form, who receives the message and discovers the story. The language and medium differ, but the intention is the same – to communicate. It’s literally a case of art speaking to you.

Without narrative, meaning, and sharing, I’m not sure I’d call something art, or even real. If no-one hears your song, reads your book, or sees your painting, does it exist, does it function? Is it worth the time, and does it matter?

And is this a binary situation, where something either is or isn’t, or is it on a spectrum of significance and existence? Well, these are the conundrums that keep me awake at night.

A song about painting, and paintings about a song

While producing our song, Perfect You, in 2024. I started painting a portrait of my songwriting partner, the man who inspired the song. It wasn’t successful, but I kept trying. A  year and many flops later, I had painted more than forty portraits of him, mostly in acrylics. By now, there are over a hundred. He has done me a great honour by allowing me to paint him.

This journey culminated at the end of 2025, when I painted a new version of that first portrait, for our song, The Portrait. Painting is what I try to do every day now, as a form of meditation. I switch off my ears and switch on my eyes. This one, called #101 Classic Redone, was a high point.

The paintings happened because I hear as well as see the music that we are working on. Since the paintings are of a real person, telling the narrative of a real person, I wanted the portraits to be realistic and to absolutely look like him, regardless of the technique I used. And, I wanted to animate the photos of the paintings to show the portraits coming to life, like the lyrics of the song describe.

Eyes that follow you

The sensation of a painting coming to life is basically the result of perspective in portrait painting, and our brains interpreting the 2D artwork as 3D. Before artists started using oil paints that allowed for blending and shaping of surfaces, the eyes of paintings were so flat that these effects didn’t happen – think Medieval church art versus Renaissance portraits. As opposed to using pigments without oil, painted on absorbent surfaces like tempera, oil paintings took longer to dry and artists could use modelling, shading, and layering to create depth, rather than being limited to flat shapes and surfaces.

Using oil paint became prevalent in the 13^th century in Europe, and the change was driven by demands for greater naturalism and detail in paintings. It was no longer only the clergy and royalty who commissioned paintings. Ordinary people wanted their portraits painted, and they wanted them to look real and be filled with the details of their lives.

The Dutch-American artist Willem de Kooning is known for saying “Flesh is the reason oil paint was invented”.

If you think that using technology to achieve realism is a modern thing, remember that Leonardo da Vinci, as far back as 1502, described using a camera obscura device to trace a projected image, thereby producing a highly accurate representation and proper graphical perspective.

Criteria for portrait paintings

Leonardo da Vinci emphasized the importance of accurate representation and perspective in visual art, especially figure studies and portraits. That is one of top three criteria for realistic portraiture: 1) close or recognizable resemblance to the subject, 2) accurate technique, including perspective, and 3) the narrative or message – giving the painting meaning through what you are saying about the subject.

These criteria apply regardless of the style or technique used by the artist. Meeting all three in one work is very difficult. Also, a portrait painting is completely personal. Its meaning and value come from the relationship between the subject and the artist, which represents another criterion: the subject has to find the portrait of them acceptable. So, in the portraits in the gallery, above, consider what the artists were saying about their subjects.

The eyes are important

Using these new materials and tools, it became possible to create the illusion of life, in 2D. In portraiture, if a painting is sufficiently realistic, and the person is portrayed as looking forward at the viewer, it may seem as if the eyes of the sitter are looking back at you and even following you around the room as you go by.

You are fooled by the light in the iris of the painted eye – the chiaroscuro that popularized portraiture centuries ago. For this reason, it is far easier to paint a three-quarter profile, or a side view, than a straight-on view of a face.

The portrait, above, that I chose to be the most-featured image in the my music video of The Portrait, tells you something about the subject (the third criterion for a portrait) by the fact that he is not looking at the viewer. He is avoiding meeting their gaze.

The story of a portrait coming to life, and the subject looking at the viewer, is what we tell in the lyrics of The Portrait:

---

Paintings that come to life

For the music video of The Portrait, I wanted to my paintings to come to life. To do this, I imported photos of six of my completed paintings into the Grok program to generate animated versions as six-second video clips. Grok is a generative artificial intelligence (generative A.I.) chatbot developed by xAI. It was launched in November 2023 by Elon Musk as an initiative based on the large language model (LLM) of the same name. Grok is named after the verb grok, coined by American author Robert A. Heinlein to describe a form of understanding.

There’s something about Grok

Grok delivers images and video that, as far as I can see, have none of the problems of other A.I. programs, such as six-fingered hands, double-iris eyes, or limbs spouting from odd places. I subscribed as a superuser, allowing me to make more animations in higher resolution, with more complex prompts.

---

Grok animated my photos precisely according to my prompts, which were to not change, delete, or add anything. This was difficult, since it is contrary to Grok’s functionality. It meant that the program recognized the images, even with my visible paint strokes, as being of humans, but could not add human details to the image from its database or my prior inputs or prompts. It could not learn, nor expand.

It could add light, change the position or pose, (un)sharpen parts of the image, or smooth the texture of the skin, removing paint strokes. It could not demonstrate, as its name implies, understanding. So most of the time my prompts started with “Do not…”

What you see in the animations used in the video of the song, is what there is.  What is in the animations, is in the paintings. 

The movements that Grok imposed on the images – basically moving the pixels around – are movements and gestures that are either already suggested by the pose of the person in the image, or almost identical to his gestures in real life.  Does he smoke? Yes. Does he lift up a rose and sniff it? Yes. Does he rest his face on his hands? Yes. Did he sing those two phrases in the song? Yes, he did. Is he any more or less handsome than in real life? No. He is what he is.

---

Checks for realism and fidelity

I wanted my portraits to meet the first criterion: realistic representation. So, I also used Grok to check whether I had made mistakes in the realism and fidelity of my portrayal of the sitter, and what exactly I had done wrong. Since I instructed Grok to not change or add anything, or improve any facial feature, any mistake I had made in the painting – a line, a shadow, a shape, proportion – would remain and become obvious during the animated movement.

When I saw such a mistake – and there were many! – I went back and corrected the painting until it was both as true to life as I could get it, and as close a resemblance to my sitter as I could get it. To date, I have animated most of the 100+ paintings.

The subtleties matter

Interestingly, while Grok’s output is extremely close to the original image,  something does change when it’s animated. Perhaps an expression or an emotion, or a balance or proportion, caused by the change in light. Human faces are complicated and even the smallest detail can change the meaning. It is very easy for a painting of a person to fall into the Uncanny Valley. In some of the animations, he looks 100% like himself. In others, it is almost as though he is expressing something that he rarely feels.  But only he and I are able to pick up on that.

But I left the results like that to remind myself (and others) that these are not characters invented by Grok, built from composite images of other faces. They are not artificial. They are real, and I made them, from a very real person. Like a child, each one was born unique. And, one by one, they document his life and his art.

Real art comes from the heart

What I hope to show with this video is that A.I.-generated images and music cannot replace, or be as good as, real art, because real art comes from inside someone, from their unique thoughts and feelings, and their personal relationships. This is my art, painted by me, with music written by me, and this is how I see my subject.

---

---