That's no problem you might say, ah but you'd be wrong for if you change the time and wait a few seconds then the time will reset itself. Once again I hear you say "no problem, disable 'synchronisation with the Internet time server" and once again you'd be wrong and the reason for this is that VirtualBox syncs the time between your Host and Guest machines.
To prevent this from happening you need to stop your VM and run the following command:
vboxmanage setextradata "VMs Name" "VBoxInternal/Devices/VMMDev/0/Config/GetHostTimeDisabled" "1"
Just replace VMs Name with the registered name (how it appears in VirtualBox) of your VM and voilà problem solved.
Note: If that doesn't work you can also modify the vbox file of your VM manually. Open the file with your favourite text editor and add the following line
<ExtraDataItem name="VBoxInternal/Devices/VMMDev/0/Config/GetHostTimeDisabled" value="1"/>
under the <ExtraData> element.