In a nutshell, the upper river is frozen in many locations like any freestone while the lower river is mostly open like any limestoner. It is ALWAYS worth fishing and not just the Musky! That said, days like today were probably fairly good but as soon as it warms later this week the snow will melt, water will rise and turn sharply colder and that can turn fish off until the snow is gone.
Winter is cruel because once it gets warmer like later this week every angler gets the itch to fish while the snow melt actually makes fishing lousy for the most part. But the fish are there and they'll be there once the water warms. Try chimara caddis, scuds, PTs, etc. and you should do fine once the water temps moderate.