I updated to the new 3.6.4 firmware over the weekend & after initially being happy with it I'm noticing that speed wise it's not performing as it should.

If I plug my PC directly into the BT modem I get a solid 73/17 connection
Using the 2850 with the BT modem I get 55/16
Ditching the BT modem & using the 2850 only (as I want to) I only get about 55/13 but this fluctuates wildly

The 3.6.3 firmware had a whole load of bugs in it for me & I was previously using a different release as suggested by Draytek support but I'm seriously considering going back to that now. Anyone else having similar issues with the new firmware?

is that over wireless or wired?

It's good that you've eliminated the VDSL modem (BT or Draytek) as an issue.

Maybe try turning the firewall off temporarily to see what difference it makes?

Use wired connection at wan 1
WAN >> General Setup>>wan 1, enable yes,always ON (DISABLE at this stage WAN 2 & 3) it may stolen power from CPU if are on
You can also try from auto navigation to half & full duplex if is letting you to change this feature
WAN >> Internet Access>> MTU set to 1492(max) and try protocol both PPPoA & PPPoE to see if makes difference
Firewall >> DoS defense Setup disable (but is off by default)
Bandwidth Management >> Quality of Service Untick use priority for VOIP and make sure all are inactive
Let us know if is better to try some more tweaks (if i can call it)