The risk of the battery overheating appears to be fairly small, Nokia claims 100 minor incidents worldwide out of the 46 million batteries being recalled. Also, the odds are that your battery isn't affected because the recall only applies to 46 million out of 300 million BL-5C produced.
Only Matsushita-made Nokia BL-5C batteries made between December 2005 and November 2006 are affected. Other batteries aren't affected.
As stated above, if you are unsure, take your battery out of your phone and visit the Nokia recall website. Instructions for an automated checker can be found at the bottom of the page, and the checker will tell you whether your battery is part of the batch being recalled.