According to http://docs.nvidia.com/cuda/cuda-installation-guide-mac-os-x/index.html#uninstall
should be enough to uninstall any version of NVidia’s CUDA driver, but … it fails (there’s no such file at all)!
So, I went through the following steps semi-manually to remove it successfully:
sudo rm -rf /System/Library/Extensions/CUDA.kext
sudo rm -rf /Library/Frameworks/CUDA.framework
sudo rm -rf /Library/LaunchAgents/com.nvidia.CUDASoftwareUpdate.plist
sudo rm -rf /Library/PreferencePanes/CUDA/Preferences.prefPane
sudo rm -rf /System/Library/StartupItems/CUDA
sudo rm -rf /Developer/NVIDIA/CUDA*
sudo rm -rf /usr/local/cuda
Restart your system and “right-click” and remove the CUDA preference icon in the system control app.