So I did a ton of research on this, and the issue is twofold: (1) some (but not all alternators) can burn out when supplying too much current. The issue is that they were designed to charge a lead-acid battery which will not accept the high level of current an lithium battery will. More sophisticated alternators don't have this problem however. (2) Lithium batteries typically come with a built-in battery management system which can instantly disconnect the battery in certain error conditions, such as overheating. Apparently the sudden disconnect of the load can also damage alternators.
For these reasons, both the battery manufacturers and inverter makers strongly recommend using a DC-DC charger in between an alternator and a lithium battery. I spoke directly to tech-support at Victron and they confirmed this recommendation.
But just to complicate matters further, some outboards, in particular Yamaha, do not use automotive-style alternators at all. On my motors (4.2L F250's), DC current is created by fixed magnets on the fly-wheel rotating past "lighting" coils. My understanding is that the Yamaha systems generating a constant amount of current depending on the RPM of the engine and they simply sink any unused current to ground through a water cooled shunt. So I don't believe there's anything that can be burned out. As for what can happen if the battery is suddenly disconnected, I'm a lot less certain. I'm definitely unwilling to risk damage to the engine, though, since I do some pretty far offshore trips and engine reliability is the most important thing to me.