As a bookseller, I had this conversation a half-dozen times, and now you can see it play out on the unofficial Barnes & Noble bookseller's forum:

We have a customer whose special orders have been brought to my attention by other booksellers. He has purchased books of photography by Sally Mann and Jock Sturges and other titles with the same theme, i.e., naked children (these had to be special ordered b/c we don't keep them in the store). We have also rung him up for unusual magazine purchases, such as Playboy, Maxim...and American Girl Magazine. Together. While this is technically all circumstantial, it really does seem like he's a pedophile or working up to it. So the question is, should we ban him from our store? Or should we just try to keep him out of Kids? Or should we allow him, as a PAYING customer, to do whatever he wants, which is how we usually operate? Granted most of them just want to us to hold their books indefinitely or construct a conference table in the cafe...Anyway, I personally think we should throw him out.

The discussion in the comments pretty much covers all the bases of bookseller conversation about this sort of thing. Do you really want to be the thought police? Where do you draw the line? On the other hand, do you want to be seen as a friendly location for pedophiles? You can refuse business to anyone, but should you?

The resolutions we reached in my bookstores varied on a case-by-case basis depending on the creep factor; I do remember that the guy who was sitting on the floor and reading a Jock Sturges book in the middle of the kids' activity area was immediately booted forever from the bookstore.

This comment, on the other hand, seems a bit extreme:

Men without children are not allowed loiter in the Kids department. Ever. Men who are alone in the Kids department are always approached immediately and offered assistance. If they refuse they are asked what they're shopping for. If they say anything along the lines of, "Just looking," then they are asked to leave the department. If a man who's by himself is sitting in the Kids department, staring at the children, skulking around, or really doing anything that seems mildly weird, he's asked to leave the department. Yes, it's only men (and teenagers, but for completely different reasons). Yes, it could be considered discriminatory, but we'd rather make some man who had no ill intentions wonder why he was told, "Only parents with kids are allowed to hang out in here," than some creepy guy make all the moms uncomfortable.

It's an interesting discussion that doesn't have a clear answer, and it's a debate that will always rage on for as long as there are bookstores.