It depends on how you use it. We use our fridge all the time (in our Sea Ray) as we stay out for long weekends (no genny so it's running on 12V when away from shore power). Heck, we even use it if we're just out for the day. But I wouldn't turn it on and then off and then on again. If you turn it on, leave it on until you're completely finished with it and won't need it for at least another day or two. On the other hand, if you're in a slip (you didn't mention slip/trailer or boat) and like to leave food in the fridge during the week, then of course leave it running on shore power. Many, many people with cruisers leave the fridges on all season - just like you do at home.
The fridge in my Sea Ray really doesn't pull much juice - especially if you stock it up good and limit the times that you open the door. I once went 3 days and nights with just one, group 27 DC battery - engine only ran for a total of about an hour that whole weekend. We used cabin/cockpit lights at night, Vaccuflush toilet, some stereo and of course the anchor light all night long... along with the fridge the whole time. Still had plenty of juice to start the engine for the trip home on the last day.