Sometimes you might try to rename a folder in TFS Team Explorer and it tells you that you can’t rename the folder b/c an existing item exists (but it doesn’t).

The reason is that if there is a delete folder with the same name, it won’t allow you to name a current folder with the same name.

To see your deleted folders (so you can undelete, rename, and delete again) you can use the following option under Tools->Options.