F18amec,
Here is my take on this topic having gone through a similar analysis a few years back. I bought the Sportsman 180 and have no regrets. It is all about your usage and needs.
There is no doubt the 209 is a bigger boat than the 180. But, just as the 209 is big for a 20' CC, the 180 is big for an 18'. I'm not talking about dimensions, but about ride, features, feel, appearance. I have numerous 20' owners that think my 180 is bigger than theirs, just as some will think the 209 is a 23'.
I comfortably fish two adults in the 180, and fairly comfortably 3. Four is a bit much but doable. My wife and 3 kids are comfortably accomodated.
I don't know your waters, but I boat on the South Shore of Boston and Cape Cod. Wind blown waves, not much of a swell. Sea conditions here are somewhat binary being either 3'+ or 1' and under. Because of my conditions, I don't think the 209 buys me much more capability than the 180. Both are great in calm waters, both can handle the 3'+ I might encounter when the afternoon SW picks up and I'm headed home from PTown. Obviously the 209 will handle it better. It is manageable but not a great deal of fun in the 180.
I doubt that I would leave the dock to venture offshore in the 209 on a day I would not in the 180. Its not that big of a difference. Where the difference comes in is coming home when it kicks up.
So in each case I have to pick my days carefully and keep a keen eye on the weather. If it kicks up, I head in.
For me to really increase the # of days running 10-20 miles out, I think I really would need to go up to 25' or more.
A couple of other things. The 209 is much heavier and will require a 200HP or more engine. I see many with 225HP. My 180 runs great with 130HP and a buddy has one with the 150HP 4S that turns it into a rocket. I tow very easily behind my 97 4Runner V6 with a 5000lb tow rating, I would be a lot closer to that rating with the 209.
The 180 is tight on storage (although it is good for an 18'), no room for a porta potti (not sure on the 209). THe 209 has better livewell, rod and fishbox storage. I also think the 180 is a bit small for a T Top where the 209 may be fine. My 180 fits in my garage, the 209 will not....
They are both superb boats that will seem large compared to other brands of the same size. If you are regularly in 2-3' ft seas, the 209 will likely make a big difference. If you're like me, it may not.
Enjoy the decision!