I've found it to be somewhat effective to point out that dead people don't give testimonials (for things like CAM treatments for cancer or other deadly diseases). Or you can point out that failures in general don't tend to give testimonials. It's often helpful to make this point about something not related to medical treatment, because making people defensive often makes them stop listening. Point out that you wouldn't take testimonials for a weight-loss product at face value, because if there were 1000 people that the diet didn't work for for every one successful testimonial, how could you ever find out about that?

It can also be effective, although less logically robust, to use your own counter-anecdote. You could try the product yourself and then say that it didn't work for you. It doesn't solve the problem of that person relying on faulty "evidence", but it can at least make them aware that testimonials can go either way. Maybe the next time they hear one, they'll consider the possibility that someone else had a different result from that product. Years ago, I used to go to a chiropractor for chronic back pain. I stopped going during college because it just wasn't convenient. Surprisingly, I have back pain less often now than I did when I saw a chiropractor weekly. That testimonial has been more powerful than all the statistics and studies you can come up with. I still debate with myself whether it's intellectually dishonest to use it.

The best answer to personal testimony would be another personal testimony that goes the other way. In this case, you could go back to the story of the little girl that died panifully of eczema because her homeopath dad refused to use real medicine. You can find some details here. http://tinyurl.com/ybyl9xv

I will admitt that it is hard to come up with counteranectodes under the gun though, but generally it's a loosing battle. A personal aenctode appeals to our emotions, while logic purposefully steers clear of our emotions. At the end, while logic is in general better than emotions at finding the truth, emotional pleas have more appeal. That's just the hand we're dealt and we gotta roll with it.

An excellent question. It depends on the tone and how any individual acts with another. For example, I'd be pretty blunt with my sister-in-law, "Science doesn't work this way. Clinical trials control for the mistakes that can be made by just using personal stories". But, it's more difficult with people you aren't as open with. I have said before to people, "I must go by what the data says to account for all the possibilities. The most reliable knowledge we have comes from a huge collection of examples, not just one. Science takes into account the range of people's experiences."

Anecdotes give us a place to start looking for answers; the answers don't end there.

Pithy reply:

Data is not the plural of anecdote.

Response to the inevitable "what do you mean?":

Anecdotes and personal feelings are not reliable indicators of a therapy's effectiveness. Our personal perceptions are swayed and distorted by everything from the mistaken attribution of normal healing processes to the placebo effect. This is why scientists, doctors, medical companies, the FDA or NHS, and so on conduct clinical trials on large scales. If it was as simple as asking a couple of people what their experience was to get a reliable result then no one would waste millions of dollars/pounds on these expensive, time-consuming studies. But precisely because personal stories of medical effectiveness are so unreliable even under the best of circumstances, these trials are essential.

On a slight tangent to the question, here's a brilliant article addressing the issue of why and how people are fooled into thinking that alternative treatments work for them.

• Source: Skeptical Enquirer Sep/Oct 1997
• Article title: Why Bogus Therapies Seem to Work
• Author: Barry L. Beyerstein
• URL : http://www.csicop.org/SI/show/why_bogus_therapies_seem_to_work/

Extracts:

"At least ten kinds of errors and biases can convince intelligent, honest people that cures have been achieved when they have not."

"To people who are unwell, any promise of a cure is especially beguiling. As a result, false hope easily supplants common sense. In this vulnerable state, the need for hard-nosed appraisal is all the more necessary, but so often we see instead an eagerness to abandon any remaining vestiges of skepticism. Erstwhile savvy consumers, felled by disease, often insist upon less evidence to support the claims of alternative healers than they would previously have demanded from someone hawking a used car. Caveat emptor!"

Agree with the previous comments about general methods for countering personal anecdotes. I've noticed something about the specific anecdotal claim in the original example that can be countered...

...one of the people present gave an anecdote about how their son suffered from some kind of skin complaint and that no treatment helped until they visited a homeopath. The treatment cleared it up only for it to return after he stopped taking it (and go away again when he resumed taking it)

... now, they don't say what "medicine" or treatment the homeopath actually gave the son. A lot of homeopaths mix in other kind of alternative medicines and treatments, such as herbal medicine, along with their magic sugar pills. For a skin complaint, it is possible that the homeopath 'prescribed' something like witch hazel, which is actually very soothing for skin problems. {see http://en.wikipedia.org/wiki/Witch_hazel_(astringent) }

Something very much like this happened to me. When I was on holiday in Torquay (Devon, UK), aged about 11, I really overdid it in the sun on the beach and by 6PM, back at the holiday flat we were staying in, I was really burning up and my skin was really itchy. My parents asked the landlord & his wife, a friendly hippyish couple, for help. They mentioned they were into homeopathy and they gave my parents some witch hazel (not diluted billions of times) to put on my sunburnt skin, and it really soothed the burning. Since then, my Dad has been convinced that homeopathy works, but I keep having to remind him the treatment they gave me was not homeopathic.

I can't remember the name of the logical fallacy (I'll come back and edit it in if I do), but it is specific to claims for the efficacy of medical treatments based on anecdotal testimony.

Left untreated many medical problems/symptoms appear and gradually worsen to a point, then start to fade as the body corrects the situation on its own.

What reinforces some forms of quackery is that people typically wait to seek treatment until the symptoms are most unbearable, which often coincides closely with the peak of the condition. More precisely, they get treatment at exactly the time the condition would have started to fade anyway. This reinforces the idea that the "cure" worked. Whereas if they had started the treatment at the onset of the problem they would have seen it continue to worsen then decline and undermine the perceived effectiveness of the cure.

Consider that the people who tell these stories often used the alternative method as a last resort, pushing the treatment even closer to that tipping point and further reinforcing the triumph of alternative medicine over traditional medicine which has already been tried.

For conditions with cyclical episodes as the body fights back then relapses, the effectiveness of the alternative treatment is reinforced with each episode, but the fact that the problem comes back each time is ignored.

A better logical argument, but often a weaker one when trying to argue with a true believer is that anecdotal evidence is not very sound because it is subject to the "post hoc fallacy." That is: Just because B happens after A doesn't necessarily imply A caused B.