How rough is rough? you can certainly tell that the HPDIs are on 4 in idle, once kicked into gear it should sound different and smooth out. Run it next to an ox66 and determine if you still think it is rough
. If you are topping out at 5100, you are likely running on all 6. If you have not changed your VST in a while, that may be preventing full top end.
That being said, here is my write-up of issues and solutions with my HPDIs not making max rpm. you have an idle "issue" so it may not be the same but it is worth knowing how the HPDIs work. There are lots of things that could contribute to that. Great motors that love good fuel and good voltage. Mine fouled a lot of plugs with Yamaha Lube.
there is a neutral switch in the binnacle which prevents the motor from starting in gear and a shift position switch in the motor. There is a slide on a track that closes that switch when the motor is in neutral.
It's part #47 on the diagram. I will be on the starboard side of the motor. Check the continuity of the switch. Manually operate it ON while you have the meter on it. Put it in neutral, check the continuity of the switch. Shift to forward, check the continuity again. It should be closed (ON) when the boat is in neutral, open (off) when shifted into gear.
The shift position switch needs to be ON when the boat is started. if it is in the OFF (open) position when the motor is started, it will only ever run on 4 cylinders (2 and 3 are off) until it is restarted with the switch in the correct position. You should confirm if it is running on 4 or 6. put it in gear (it runs on 4 in neutral) and pull the plug cables one by one (start on #2 and #3 as those are the ones that would be off) if they drop you are running on 6. If no change you are on 4. if it is running on 6 you have ruled out that switch. If you confirm you are on running on 4, hold down the switch and have someone start the boat. Let go of the switch, if it kicks to 6, your switch is not correctly engaged when in neutral. You may be able to adjust the slide or switch to correct the engagement. If you are still on 4 it could be a bad switch (stick in the closed position). Jump the connection, start the boat, pull the jumper, you should hear it kick into 6 cylinders. if it does replace the switch (topping at 2500 happened to me when the motor was running on 4)
if you are running on 6, I would check fuel delivery. Your mid pump on the vst should always read 50 psi. You need to watch it as you get up to your top RPM. If it drops below 50psi, your hose, filter, VST Filter (this get clogged quickly with ethonal fuel), LP fuel pump or mid fuel pump are the likely issue. If it holds, the next check is your inline medium pressure filter then the HP fuel pump. The VDS can log this for you. It should read somewhere around 700psi. if it falls off at your top RPM, that is your likely issue. I had both of mine HP pumps rebuilt by injectorRX, solved my pressure issue.
I still had an issue topping out at 4400 RPM, turns out I had a stuck open injector. Replaced that and I was back in business.
One thing to note is HPDI like good power. If your battery is bad or the stator is not producing enough juice, it can screw everything up.
Hope this helps