I think that if you have a GW, with a transom live well, you get used to removing it for routine chores, live well pump, bilge pump, through hull maintenance, etc..
If your insulation is brittle, or coming off, you might try spraying with something like Flex Seal.