The problem with the experiment is that there exists a set of instructions for which the ability to complete them necessitates understanding due to conditional dependence on the state in each iteration.

In which case, only agents that can actually understand the state in the Chinese would be able to successfully continue.

So it's a great experiment for the solipsism of understanding as it relates to following pure functional operations, but not functions that have state changing side effects where future results depend on understanding the current state.

There's a pretty significant body of evidence by now that transformers can in fact 'understand' in this sense, from interpretability research around neural network features in SAE work, linear representations of world models starting with the Othello-GPT work, and the Skill-Mix work where GPT-4 and later models are beyond reasonable statistical chance at the level of complexity for being able to combine different skills without understanding them.

If the models were just Markov chains (where prior state doesn't impact current operation), the Chinese room is very applicable. But pretty much by definition transformer self-attention violates the Markov property.

TL;DR: It's a very obsolete thought experiment whose continued misapplication flies in the face of empirical evidence at least since around early 2023.