Most likely it is the switch or the pump. Regardless, you'll likely need access. You may be able to remove the entire access port to gain a bit more hole diameter, but from the sound of it, it likely won't do the job. If you do, you can get a replacement O-Ring and Deck plate assembly from Beckson Marine. You just seal in place and get a pop or screw in version.
If you need to take the panel out, you can simply cut the sealant (GE Almond at HD to re-seal later), remove the screws and pull it out. You trim all the edges on the deck and floor piece and can do a good inspection and cleaning of the underside and bilge area. I'd replace the switch and pump as the hassle is getting the access versus the cost in my book. Make sure they work via the float switch and helm switch, make sure all tubes that allow water to travel are cleaned out. Then screw back down and seal up. Use painters tape for a nice clean line on the floor side and deck piece. Use a plastic spoon (wet it) to make a nice smooth edge on the sealant. It really isn't that hard if you take your time.