From AI That Reads Scores to AI That Reads Performance

Recently, the relationship between music and AI has been attracting attention in many different ways.

Aside from composition, one trend in digital music production can be seen in systems such as NotePerformer, which attempt to read scores and reconstruct them into human-like performances.

Their directions differ, but they share one common goal: to treat music not as a mere collection of sounds, but as something with structure and meaning.

So the next question naturally becomes:

Can AI evaluate a classical music performance ?

This time, I decided to test that idea using ChatGPT.

An Experiment with Beethoven

I sent ChatGPT an audio rendering of my MIDI performance of the opening of Beethoven’s Piano Sonata No.29 in B-flat major, Op.106, the “Hammerklavier.”

Its response surprised me more than I expected.
Some of its comments were strikingly close to points that my own teacher has often made—though expressed differently.

For example:

• the commanding presence of the bass line
• the direction of musical motion
• the declarative force of the opening
• the center of gravity of the phrase
• the overall forward drive of the music

That was a genuinely interesting experience.

How Does AI Judge Music?

So I asked a further question:
“How do you produce that kind of evaluation?”

In summary, the answer was that it seems to understand music on three levels.

1. Physical Layer

First, it looks at the sound itself.

• Changes in dynamics
• timing shifts
• attack characteristics
• register balance
• chord density

These are the surface features of a performance.

2. Structural Layer

Next, it compares these elements with musical structure.

• harmonic progression
• phrase direction
• role of each voice
• contrapuntal weight
• melody vs inner voices

For example, if the bass is weak, the harmonic foundation also sounds weak.

3. Meaning Layer

Beyond that, it begins to recognize musical qualities such as:

• tension
• forward drive
• declarative force
• calmness
• a sense of resolution

This is an area quite close to human musical interpretation.

Why Beethoven May Suit This Kind of Analysis

In Beethoven’s works, especially the later ones, the following become highly important:

• structural strength
• motivic development
• harmonic gravity
• forward-moving energy

Therefore, analysis that captures where the weight lies and where the music is heading, rather than merely the beauty of tone, may be surprisingly well suited to AI.

Something That Made Sense to Me

Having worked with MIDI performances of classical music for many years, I feel that performance ultimately consists of repeated decisions:

• what to bring forward
• what to support
• where to drive ahead
• where to release tension

Whether for piano, strings or orchestra, the essence does not really change.

Seen that way, it may not be so strange that some of AI’s observations overlap with the viewpoints of my teacher.

But AI Does Not "Feel" Music

This, I think, is an important point.

AI does not listen to music with emotion or bodily sensation in the way humans do.


When people listen to music, they may experience:

• being emotionally moved
• feeling tension and release
• nostalgia or excitement
• even changes in breathing or the sense of time

AI has no inner experience in that sense.

Rather, it is more accurate to say that AI returns coherent analysis or advice based on:

• features of the performance
• musical structure
• comparison with vast amounts of prior knowledge
• language-based interpretive models

Even so, the fact that it can point out structures or balances that humans may overlook makes it a very interesting presence.

As a Partner for Self-study

For someone like me, who has not had full formal musical training, it can be a valuable partner in pointing out:

• weaknesses I had not noticed
• imbalances in texture
• lack of musical direction
• unclear voice handling

It cannot replace my teacher.

But during production or practice, I feel it already has real value as a mirror for self-examination.

Final Thoughts

If NotePerformer is an AI that reads scores, then ChatGPT may represent the first steps toward an AI that reads performance

Does AI truly understand classical music?

There may be no simple answer.

After all, even when humans say they understand music, that understanding is not one single thing.

In other words, it can take many different forms.

- grasping musical theory
- understanding performance
- or simply being deeply moved

These may all be different forms of understanding.

Yet at the very least, AI seems to be becoming a partner that can read musical structure and think with us about performance.

That was the impression this experience left me with.