I think the ranking of the stories should be based on the likely score rather than merely the average.
If a story has nine votes of 5 and a 1 it will have a lower average than one that has six 5's but the chances are that it is a better story.
I would also like to see the voting predict the type of story I am likely to like rather than ones that are merely popular among the people who like to read them.