Trying to install Ubuntu to the third partition.

Do you have two or three partitions?

Simple start Ubuntu LiveCD go to System → Administration → Partition Editor (or GParted) and reduce one of Windows partion and on the free space make two new partitons (one for linux-swap and one for system) (if you don't have third partition) or (if you have third partition/free space on disk) make same two partions. Then start installation and select manually edit the partition table (step five i mean) and windows partitions let be and swap select mount point "swap" and for the system select mount point "/".

I have three partitions created. The 3rd has been formatted, NTFS.

And have you there data on the third partition or is it free?

And do you have any data on the third partition or is it free?

Nope it is data-less.

So that is easy - delete (in LiveCD using GParted) it and make two new partitions.

Didn't work. I tried, but it says there is no root file system determined.

Your disk should be partitioned so and with those mount points (more or less):
ntfs /media/something
ntfs /media/something
ext3 /
swap swap

and then what one do I install to, or does it just know?

Does it just automatically know what partition to install it to?

You need to ultimately have 4 partitions minimum. The first two are your Windows Vista and XP partitions. The third will be a partition that should be changed from ntfs to ext3 as the file system, and / as the mount point, which means that this drive is the root file system for linux. The fourth partition will be set up as a swap file system and swap as the mount point.

Check out this webpage, it can walk you through a dual boot windows/linux setup: http://www.psychocats.net/ubuntu/installing#partitioning.