If you ever find yourself wondering why your ScrollViewer does not kick in (as I was wondering!); you have most likely forgotten what I did.
You need to set the height and width of the scrollviewer explicitly to make it work.
<ScrollViewer Width="460" Height="700">
The size of this control (ScrollViewer) is simply a “window” to your data that allows you to scroll the content of this window. Due to this reason – you need to tell WP7 that it should only display a fixed window of data to the User.