My recommendation is to go to various marinas and dealers, chat with people, listen to local advice and see if you can take a look at some local Gradys in various models. What style of boats are most common where you are?
Basically they have the bow riders, center consoles, and walk around styles, each with their own merits. WAs have the most storage but if yours will be in the back yard on a lift, that may not be necessary for your needs. WAs also lose out on deck space but provide good weather protection. Bow riders are nice cruisers for the family as are CCs, but CCs are a bit more susceptible to the elements with the design - any of these will comfortably fit 5 adults or less at 20’+. So you need to evaluate the primary uses to fit the needs, including what you expect your season to be (ie June to Sept, or maybe May to Oct). You have a lot of options being in Mid Atlantic and surely you can find what you are looking for within a few hours, but you could also look up to the northeast (generally lower hours on engines due to shorter season) or NC close to where Grady’s are manufactured.
Make sure to evaluate your vehicle towing capacity- I assume you did that already. I’d suggest you and your dad and whoever else would be at the helm to get a boater safety course completed, even if you have some experience.
in terms of negotiation- a surveyor can provide you with leverage if they identify any issues, but know that you likely need a marine mechanic to look at an engine separately from a boat surveyor. If it’s from a reputable dealer, I would say that alleviates some of the concern (but they are selling still, a surveyor hired by you is your advocate). I’d say any used boat is subject to price negotiation anyways - people don’t list prices low, they expect some lower offers off list.