It's a long story as to why, but I needed to use the £ symbol in my web.config
On my local windows 7 IIS setup, it was happy to just use £. On our windows 2003 staging server, the same was true. But under Windows 2008, IIS threw it's teddy out of the pram and just errored saying invalid xml file.
A few places suggested using & pound; instead, but this did not work.
What did work though was using the unicode value of & # 163;