From Aatif's piece (which is pretty definitive, with only a couple of minor facts - hyperfocal is NOT the same as EDoF! - I'd quibble about):
Notice how much grainier the image at ISO 1600 is? The higher the ISO, the more the likelihood of grain in your shot.
Then why ever raise the ISO, you may ask. If you click through each of the images and look at the EXIF data, you will notice that in the image at ISO 1600, the camera shutter was open for 1/55th of a second. That’s a shutter speed of 0.018 seconds. However, on the image. However, the image at ISO 50 had the shutter open for 1/2 a second, or a shutter speed of 0.5 seconds.
That means it took 27 times longer to take the image at low ISO. This is where higher ISO images help. When you are in a place where the lighting conditions aren’t great, the camera shutter automatically has to stay open longer to capture everything. If the shutter is open for longer, there is likelihood of your hand shaking, and blur being induced in the image.
When Shutter Speed is as low as half a second, even a minuscule shake of your hand causes blur. It means you need to use a tripod for good shots.
So summing it up, keep ISO as low as possible, but if you are getting blurry shots, progressively up the ISO one level at a time.
Nicely done. Regarding the quote above, I've had success at gigs trying to take stage shots - crank up the ISO and snap away and auto-magically there are far better 'frozen' shots under stage lighting and, hey, even the extra noise is all part of the stage ambience...
(high ISO, low light, fast moving lights and subjects, handheld - not bad results?)