Buying Books at the Supermarket, Good or Bad?

Phil and I have been having a conversation in the comments section of a post below and I thought it would be good to bring that into a separate post. I had made a tongue-in-cheek comment about how buying Harry Potter at the grocery store would be a sign of the end of literature to some people. Phil in turn wondered why this might be. I tried to lay out some of the economic arguments in my comments but I am not sure I am the best person to make the case.

So let me throw that out to the peanut gallery. Would it be bad for literary/serious fiction if supermarkets and stores like Wal-Mart, Mejier, etc. develop a much bigger market share in the book business? If so, how and why?