I dont know much about OpenWRT so dont pay to much attention to me.
I doubt 20ms of extra latency would make much of a noticeable difference. Though its odd its higher now. Does it change, get higher and lower? Are you cable links actually connected at 1000Mbps? Or are some connected at only 100? Can you verify the connection speed?
I have seen IP conflicts with do some weird things depending on hardware. Have a good look to make sure something didn't get missed with static IPs And Your DHCP pool.
How well are those glass panes sealed to the wood? Could silicone the panes in. Doubt it would help you go lower, but I'm curious how much the dehumidifier runs after it gets to 35% to keep it there? I didnt realize they even made dehumidifiers that small, might have to look at them. 🙂