For some strange reason pinging the cable provider's default gateway, instead of a host on the Internet, fixed the issue with ping detection for WAN2. Now it works as it is supposed to. It seems that the default gateway IP doesn't change even if my IP address changes. Problem solved, thanks!

To Draytek: there should really be a hint in the UI that the user should try putting the IP of default gateway for WAN2 for ping detection. It would have saved me (and probably others) many hours of experimentation with the feature.