Not a huge difference in spec if it's pre-facelift (older than 2000). Elite has leather, cruise control, rear sunblind, auto-dimming mirror, plus a few other bits and pieces. MV6 has better front seats, 17" wheels (compared to 16" on Elite), sports suspension, and is much more likely to be manual than the Elite due to it being the sporty rather than luxury option.
I bought the MV6 primarily because I wanted a manual, but also because I'd rather have sports seats, sports suspension and 17" wheels compared to the other stuff the Elite gets. Only things I really care about from the Elite -- cruise control and auto-dimming mirror -- are easy to fit to the MV6, both of which i've done. The rear sunblind is also an easy retrofit which I have done, although to be honest it's not much benefit anyway.
Bottom line, there isn't a huge difference between them. Most of the Elite bits can be added to the MV6, and the sporty bits from the MV6 can be added to the Elite. Most important is getting one in good condition with the manual gearbox -- it is possible to swap a manual in to replace an automatic (a few members have done it) but it's a pretty big job!
Oh and don't confuse ex-police "MV6" models (most of which will be white), these are actually very basic spec cars with a 3.0 engine and uprated suspension. Very different to the retail MV6 which is much closer to Elite spec. If it doesn't have climate control, a sunroof, sports seats, etc... it's a police spec MV6.