Depends. If you're lugging the rifle around in the field, I'd definitly consider it. Better safe than sorry, and if your rifle has facilities to do it (and bands are available) it makes sense to add one. Beware though, it's possible the accuracy will be influenced, and the POI definitly will, so at least rezero your scope.
In a target shooting scenario where all the lugging is done between e.g. gun case and benchrest, I'd not bother. Just make sure the screws which hold the barrel in de breech are secured tightly and there's no movement of the barrel in the breech to ruin the score!