only driven m3s, not owned one
but i did have two m coupes. all of my other cars are much smaller (elise and scion xb) but the only time the m5 feels big is when i have to park it or in heavy traffic. on a twisty road it 'shrinks' and is a very willing dance partner, if you know how to lead :)
overall i'm very impressed with mine. it's not perfect by any stretch and ultimately doesn't make sense for me but i'm loving every minute of the experience while it lasts.