The graphics card on my old PowerMac G5 gave up, so my Mac is now headless. That is, it runs fine, but it can't display anything on the screen. It's still possible to log on to it through SSH, but I needed to use GUI applications. Getting a replacement graphics card proved to be difficult, so I needed to find another way.
Macs support remote desktop logins through Apple Remote Desktop, which is just a different name for VNC. The only problem was that I had not enabled desktop sharing before the graphics card broke down, and all the instructions I could find were about how to enable it in the Mac's System Preferences. And I can't access the System Preferences since it's a GUI application!
With the help of this Apple Support article, I could enable the VNC server. The VNC clients I tried reported errors like "No matching security types" and "Server did not offer supported security type".
Finally I got it to work. The trick is to enable "legacy mode" and set a password. This is the command that did the trick:
sudo /System/Library/CoreServices/RemoteManagement/ARDAgent.app/Contents/Resources/kickstart -activate -configure -access -on -restart -agent -privs -all -clientopts -setvnclegacy -vnclegacy -yes -setvncpw -vncpw somesecretpassword
I guess there are security implications with this, but as long as I don't expose the VNC server to untrusted computers, I guess I'm fine. Both Vinagre (the VNC client that comes with Ubuntu) and xtightvncviewer now work. The vncviewer command still doesn't connect properly for what seems to be some other reason.