They have several opportunities to see the Fremen riding the worms already in the book. As you said Leto discovered they were much more numerous than it was widely believed. And that is with just a short occupation before everything happens.

Harkonnem certainly had plenty of opportunities to discover he was underestimating them.