Hi Marius,
first of all, virtual machine from ESXi 3.5 (virtual hardware 4) are supported in vSphere 5.1, so there is no needed conversion.
About the move, as long as you move all the files contained in the datastore you should be fine, the important point is to not break the snapshot chain described in the pointer files, so you should do the copy directly from the command line (ssh for example) so you can see all the files, the datastore browser hides the pointer/flat files couple and shows you only one.
For your safety, copy files and remove the old one only when you are sure everything is working in the new ESXi.
Luca.