You can download the new version by opening the application and choosing Help -> Check for updates now, or by going to the Nokia Music website.
Nokia Ovi Player is essentially a re-brand of Nokia Music (PC Client), although there have been a few UI enhancements.
Nokia Music Store (Nokia Ovi Music Store soon?) is available as a tab in Nokia Ovi Player
There is now a post on the Ovi Blog about the name change.
"Ovi Player contains some new views to help you navigate and enjoy your music collection. The ‘recently added’ view is used by many people as a way of keeping track of their new music. This enhanced view shows your recently added music grouped by albums together with cover art. In addition, the introduction of a ‘filter column’ on the detailed view will also help you browse your music collection more easily."