Depending on the believer’s particular philosophical bent, and upon the journey of the unbeliever to her philosophy, it may be that there is truly not much difference between the believer and the infidel. It is possible, but not likely, that the unbeliever’s world-view is very nearly the same as that of any given believer, except that she believes in one fewer god than does a true believer, to paraphrase the words of Stephen Roberts.
The unbeliever may be more independent than is the believer, a natural result of having researched, pondered, and trod the path less followed. She is certainly more skeptical, less likely to suspend disbelief for anything less than a logical argument, which can make it difficult for her to take miracles, magic, and obvious hokum on faith.
Clearly, then, there are often fundamental (pun not intended) differences in the mindsets of believers and infidels. Those difference do not necessarily fall at the exact center of the lives of either, unless one or both parties are in the lunatic fringe of either of those beliefs, so that they inform every decision made on any subject. Those philosophical outliers, on both sides of the Bell curve, are not informed by either logic or critical thinking. Rather, regardless of which side of the god-hood fence they inhabit, they are zealots who are more likely to throw the first stone than not.
Other than that, if you are a believer, the unbeliever is as likely as the fellow next to you in the line at the DMV to either believe or disbelieve any of the other things that you hold dear. That is to say, the believer and the unbeliever probably have as much in common (save the belief in god) as does the believer and some other random person in their neighborhood.
Further, it may not be just any god that the unbeliever refuses to believe in. It may be your god in particular, which make it personal. But the unbeliever may be unable to acknowledge your particular god simply because he has carefully considered the issue (probably for years) and is of the opinion that your particular god was invented by human beings centuries ago to make them feel better about dying, though she may admit (as do I) that there may or may not be some higher power that she had no tools to recognize, even if they were in the same compact car together.
Once that difference is disposed of, any given unbeliever is just as likely as any believer to watch the same films, to like the same brand of mustard, to wear the same style of underwear, to like the same foods, to read the same authors, to like the same sports, to have the same friends, to go the same places on vacation, to become irritated about the same traits in others and, well, be the same in every other way. Or not. An unbeliever only needs to be different from a believer in the view of one specific god, and may otherwise believe and like the same things as the believer does.
If centrist persons of both (anti-) religious persuasions can digest that single fact, it will be possible to agree more than we disagree, and to therefore leave both groups able to enter into meaningful discourse that is neither sharp nor shrill. We are not likely to change each others minds. Don’t expect it. Do expect, however, that both groups should be able leave behind views that demonize one another, in favor of views that more carefully consider what we have in common.