Yeah I hear you there. The problem I have too is that the rocker panels were pretty rusted out before I took off my boards, and now they are even worse since the boards aren't there to take some of the beating. It also became apparent as I took one side off, that they were not going to go back on even if I wanted them to haha.
That's why I'm kinda 50/50 on if I should take them off. I always tire shine them and they have blue LEDs for the courtesy lights so I think they look nice being on there. But at the same time it would be a clean look without them