What do I do when I turn on my iPod, and I see a folder icon with a exclamation mark?

The cause of this could be several things.

For an explanation of each, and recommended courses of action, read AppleCare Knowledge Base document #61003.