I've just upgraded the firmware on my router V2820, which was on 3.3.5_282201 and I was having VPN issues.

I selected the latest firmware (3.3.7.3), double checked it was the correct V2820n device, and upgraded it.

Now, reading the release notes, I see that I should not have done that:
"IMPORTANT NOTE: Do not upgrade directly from 3.3.5 (and earlier) to 3.3.7.3.
Due to differences in the Web UI and functionality the router MUST first be
upgraded to 3.3.5.2 prior to upgrading to 3.3.7.3."

Ok, so I should have read them first, but it's a bit late now, I've done what it says I shouldn't do.

What might happen? What action should I take now (should I downgrade to 3.3.5.2 and then upgrade again)?

As far as I can see my router is still working.

if you read the release notes you would of known about this

We don't know what they changed that prompted that warning notice, but it looks like you got away with it so far so you probably didn't use anything in the areas that were changed. However I'd be concerned about saving unwanted content in any saved configuration files that you make from now onwards using the current configuration. You don't know what might be corrupted, because you don't know what changed beneath you.

If I were in your position I would carefully document all the settings on the router, absolutely all of them, then I'd check them again.
I'd make an emergency-use-only software config backup in case I miss-typed something.
Then I'd restore it to factory settings and enter all those written details again by hand. Avoiding the backup to avoid corruption.
If it doesn't work first time you can restore the config that you saved and start again looking carefuly for whatever spelling mistake caused it to fail.
Finally I'd save a copy of the config using the GUI option to do this and delete all your older copies of the config backups.

At this point your entire router will be working fine, and you'll have a newly created backup of the new config that can be used going forwards. It'd probably take about 1/2hr to do it all though.