I am so happy that I found your post! I have been struggling with this issue for quite some time now, and who would have thought that all I had to do was just sub-class the UIImageView and the UIScrollView. Woohoo!
It made me decide to become a subscriber to iPhone Dev SDK.