I drive the same way and a vibration like that would be very noticeable and annoying.The shudder goes away. It always appears at about 40 - 45 mph, when I lift the gas pedal and gently apply the brakes (such as when on a county / state highway and the traffic signal turns red much further ahead).
I have replaced brakes and rotors and drums for decades on my other vehicles - but now face physical limitation and can't do brake work.
A loose caliper bolt makes sense.
I'm kind of at the point in life too where my knees can't take the bending to do what I need to do to replace brakes, so I can relate to that.