Things AI still does poorly — like teaching me piano

The last post, on Lin Xi, Wyman Wong and AI, actually left a question not fully answered.

AI keeps getting better at writing. It can understand heartbreak, analyze emotions, and quickly generate a beautiful line. So I’m reluctant to explain why some expression still falls well short with “because AI has never really loved.” That answer is too easy, and doesn’t really explain anything.

Following that doubt, I later wanted to do something in the opposite direction.

For the past few years we’ve kept watching what else AI can do; I wanted to take a serious look at:

what does it still do poorly right now? And why?

Not compiling a list of “AI’s 20 shortcomings,” nor assuming up front that some things can only ever be done by people. I’d rather start from the friction I actually run into in real use, then see whether research can explain these experiences, or instead show that my original understanding wasn’t accurate.

In the end I put together about 20 sets of core evidence, covering music perception, randomized education experiments, human–AI collaboration, medical decision-making and AI reliability. I mainly looked at original studies, randomized experiments and meta-analyses; some very recent 2026 studies are still conference papers or preprints, so I treat them only as early evidence and don’t extrapolate too far.

There’s one result of this round of research I quite like: it didn’t simply prove “my feeling was right.”

Some things I thought were AI’s inherent limits might, once broken down, just be that it didn’t get the information it needed; some really are about current model capabilities; and for some, the problem isn’t even AI itself, but how we use it and how we judge the results.

I plan to write it in three posts.

The first starts with an example I feel very personally: practicing piano.

The second looks at whether AI helping us get something done is the same as actually learning it ourselves.

The last comes back to the now very common Human in the Loop: as long as someone reviews at the end, is the system really reliable?

My understanding at first was simple: practicing piano can’t be explained in words alone

I’ve talked with AI about piano quite a lot.

If I ask about a piece’s background, form, harmony, or how a passage is usually practiced, it often explains very well.

That’s not surprising. Music holds a great deal of knowledge already expressed in language and symbols: scores, harmony theory, music analysis, textbooks, practice methods teachers have summed up. Large language models naturally have an advantage with this content.

But once you actually sit down at the piano, things change.

Sometimes I feel a passage is “not quite natural,” my hands a bit tight, or every note is clearly right yet the sound just isn’t nice.

AI can still give suggestions, of course: slow down, relax your wrist, practice hands separately, mind the voicing, listen to the melodic line.

You usually can’t call these suggestions wrong.

But they often sound more like telling me:

“In this situation, a person can usually try these methods.”

Rather than a teacher who actually listened to me play it twice and said:

“The problem here is actually this.”

So my original explanation was very intuitive.

Too much of the important information in piano practice isn’t words.

How much force goes into the keys, how the fingers and wrist work together, that bit of timing between two notes, where exactly the sound turns hard, when the body starts to tense — often even I find these hard to describe accurately.

This often involves tacit knowledge and embodied knowledge: some skills live in the body, in sensation and in long experience. Being able to do something doesn’t mean being able to articulate every judgment you make while doing it.

And ChatGPT, Claude, Gemini — the AI we use so heavily now — really did grow out of LLMs.

Of course, the concepts need stating a bit more precisely here.

AI is a very broad field, and LLMs are just one technology within it.

And today’s general-purpose AI has long since stopped being pure “text in, text out” LLMs. They can handle multimodal input like images and sound, and connect to all kinds of tools and specialized systems.

So after the research, I felt my original statement —

much of piano can’t be described in language, so LLM-based AI can’t teach piano —

was still too crude.

The more accurate question should be:

does AI actually get, and correctly understand, the important information that isn’t in language?

From here, I found that “teaching someone piano” can actually be broken down quite clearly into four layers:

Observe → Diagnose → Intervene → Verify

First observe what happened, then judge why, choose how to intervene, and finally verify whether it really got better.

A problem at any layer can produce a very familiar result:

what AI says is quite right, but it didn’t really help.

1. Observe: first it has to actually hear how I played

A 2025 study specifically tested whether today’s multimodal large models can really “understand” some basic musical information.

The researchers had models like Gemini 2.5 Pro, Gemini 2.5 Flash and Qwen2.5-Omni do tasks such as rhythm, transposition and chord recognition, then compared two kinds of input.

• One had the model listen to the audio directly.
• The other first converted the music to MIDI, explicitly representing structured information like pitch and timing, before handing it to the model.

The results showed a very clear difference.

For example, in one chord-category recognition task, Gemini 2.5 Pro’s zero-shot accuracy was about 47.7% working directly from audio; with the same information as MIDI, it was close to 97.7%.

The gap varies across tasks, so you can’t take one number and conclude “AI can’t understand music.”

But this experiment at least supports a very important distinction:

a model knowing a lot of music doesn’t mean it can just as reliably extract, from real sound, the signals that knowledge needs.

That hit part of my original intuition exactly.

Suppose I finish playing a passage and ask:

why does this part sound wrong?

Before discussing musicality, practice methods or even the wrist, you should really first confirm:

did it actually hear accurately how I just played?

If I only say in words “this part is a bit stiff,” a great deal of the information that really affects the judgment never enters the system at all.

Even given a recording, you can’t assume a multimodal model has heard every important detail in it the way a piano teacher would.

A teacher in the room gets much more information. They don’t only listen; they’re watching my hands and body; where I just hesitated, they may see directly; if unsure, they can have me play it again right away.

So I now think a more accurate statement than “music can’t be expressed in language” is:

for a bodily skill like performing, language is a lossy information interface.

And even if you swap language for audio or video, the first hurdle is the same:

has the system actually observed the reality that really matters?

2. Diagnose: hearing what’s wrong, it still has to know why

Take one more step forward.

AI has already heard accurately:

the rhythm in this bar is unsteady.

It still isn’t over.

Because “unsteady” is just a symptom.

The causes behind it can be completely different.

Maybe I haven’t learned the score well yet, so I hesitate here; maybe it’s a control problem in a few fingers; maybe the tempo is already beyond my current ability; or maybe I was intentionally doing rubato and just didn’t control it well.

In the end, all of them might sound like:

this part isn’t quite steady.

But the way to practice won’t be the same.

So what the second layer really needs is:

going from symptom to cause.

That’s also why I now think it’s best not to lump together “AI can identify errors” and “AI can diagnose technical problems.”

A system can mark:

this is a little behind the standard beat.

That’s a very concrete, fairly easy-to-measure ability.

But going on to judge:

why does this learner slow down here?

has already entered another layer.

When teachers make judgments like this, they often aren’t just applying one explicit rule, but using experience built from seeing many students, hearing a great deal of playing, and practicing for years themselves.

That doesn’t mean AI can never do it.

Just that:

finding a problem and knowing why it happens were always two different things.

3. Intervene: knowing the cause, it still has to know how to teach this person now

Suppose the first two steps were done right too.

AI not only heard the unsteadiness, but judged it’s mainly a control problem in a few right-hand fingers.

Can it teach from here?

Still not necessarily.

Because teaching isn’t outputting:

“Ten Solutions for Unsteady Finger Control.”

For the same problem, different learners may need different methods.

• Some should slow down first.
• Some need to change the motion.
• Some do well with rhythmic variations.
• Or someone may already have done too much mechanical repetition, and practicing twenty more times would just drill the tension in along with it.

So what’s really hard about the third layer isn’t:

how many practice methods AI knows.

But:

whether it can judge which one thing the person in front of it should do first right now.

This is also a big difference between generic advice and teaching.

AI easily gives five suggestions that are “all right.”

Teachers often do something else:

forget the rest for now.

Just change this.

Once this has really changed, then move on.

4. Verify: after teaching once, it still has to know whether the method worked

There’s one more layer at the end.

The teacher has me try a different method and play it again.

• If it got better, they need to judge whether I just happened to play it well once, or it’s starting to stabilize.
• If nothing changed, did I not understand? Is the method unsuitable? Or was the original diagnosis itself wrong?

So real teaching was never a single answer.

It’s a feedback loop:

Observe → Diagnose → Intervene → Verify

Observe, diagnose, intervene, then go back to reality to check the result.

A 2026 piano study called Profy happened to make this boundary especially clear to me.

The researchers collected more than a thousand valid performances from 73 players, recording key motion and audio at the same time, then used a model to find the spots in a performance most worth a learner’s relistening and checking.

The results showed this kind of system can already fairly effectively help people locate performance segments worth attention.

That’s very practical progress.

But the paper itself is very cautious: what it validated is how well these hints line up with expert judgment, and whether this kind of feedback is usable; it didn’t show that students learn better long term, that the skills transfer, or that this kind of system can already replace a teacher.

I really like this distinction.

Between “pointing out more accurately where to look” and “actually teaching someone,” there’s still a long chain.

So why can’t AI teach people piano very well yet?

After the research, my answer now is no longer:

because AI has no body.

Nor:

because music can’t be expressed in language.

It’s closer to these four layers:

1. Observe

First it has to really hear and see how you’re playing now.

2. Diagnose

Having found a problem, it still has to know why it happens.

3. Intervene

Knowing the cause, it has to choose a practice method that suits this specific learner at this stage.

4. Verify

Finally it has to observe again, confirm the intervention really had an effect, then decide the next step.

If any link is weak, you end up with a very similar experience: it knows a lot, and says a lot of right things, but doesn’t really teach you.

These four layers are of course not a master framework for “all of AI’s problems.”

They fit better with this kind of task: one that requires observing the real world, judging causes, intervening individually, and continually adjusting based on feedback.

Piano is just one very concrete example.

But having broken it down clearly, I found it quite helpful for other uses of AI too.

When AI doesn’t help, instead of immediately asking:

is the model just not strong enough?

I now first look at:

• did it get the information the task really needs?
• is it only seeing symptoms, or does it have enough basis to judge the cause?
• is its advice generally correct, or suited to the specific situation at hand?
• in the end, is there new feedback from reality proving the advice actually worked?

Different answers mean completely different fixes.

Sometimes you really should switch to a stronger model.

Sometimes what’s missing is sound, video, sensors or more complete data.

Sometimes you need a system designed specifically for this kind of task, rather than continuing to chat with a general-purpose chatbot.

And sometimes, what the next step really needs isn’t generating another answer, but going back to reality to try once, then bringing the new result back.

That may be the first useful takeaway this round of research gave me.

When AI clearly knows a lot but hasn’t really helped, don’t rush to label it “can’t do it.” First look at where exactly the chain, from observing reality to verifying results, breaks.

The next post will look at another problem I think is more hidden.

Sometimes AI hasn’t obviously done anything wrong at all.

The answer it gives is good, and we finish the task smoothly.

But if the point of the task was to learn:

is AI getting it done for me the same as me really learning it?

About this round of research

This is the first post in the research series “What AI still does poorly.”

The whole round starts from real friction I’ve hit in piano practice, learning, writing, relationships and AI products, then looks for research across fields to see whether the explanations behind these experiences have evidence, and where they need correcting.

The core evidence gathered so far is about 20 sets, mainly from music perception and bodily skill, randomized education experiments, human–AI collaboration meta-analyses, medical decision-making, and AI / agent reliability.

It isn’t a systematic review in the strict sense, and it won’t use one model’s performance today to define AI’s permanent capability limits. Personal experience raises the questions; research tests the explanations; where the evidence can’t answer for now, it stays an open question.

This post mainly drew on two kinds of research.

One tests how multimodal models differ when handling real audio versus structured musical information, helping answer “did the model actually hear it.”

The other studies what dedicated piano feedback systems can already do, and what they haven’t yet shown, helping distinguish “analyzing a performance” from “actually producing a teaching effect.”