Discussion of Migraine Paper

weakness:

Although not necessarily a weakness per se, it seems strange to formally announce to patients that there would be both real and false treatments. While anyone participating in a study is likely aware that researchers will be studying differences through control groups, a formal announcement and anticipation of a questionnaire on patient's randomization arm seems to mitigate the placebo. For example, if patients knew that they were required to fill in a survey on their randomization arm impression they could easily research the differences in both sham and real treatments.

strength:

The researchers presented findings against the efficacy of semi-standardized methods instead of trying to make the data tell another story promoting these treatments. They also included and (recorded) detailed patient flows through the experiment (though not much commentary on whether these dropouts were systematic or random).

Weakness: use of diaries as a form of data collection could result in subjectivity and bias.

Strength: good use of randomization and double blinding to reduce certain bias such as by the evaluator.

strength: the procedure of this trial is very clear by showing the flow graph of participants through each stage of the trial. weakness: in the results of associated symptoms and the adverse effects, only p-values are show, no confidence interval.

Strength: Design for control group and treatment group is rigorous. To insure the accuracy of tests, the study puts lots of efforts on choosing randomization sample set. Weakness: The choice of treatment is related to Chinese medicine, which put limitation of sample selection. So it is possible that sample is not representative enough.

Weakness: From my point of view, the statement "There was only one statistically significant difference between groups, the age mean. The patients in the sham acupuncture group were older than in the real acupuncture group (P = 0.024)." is a very strange statement. Why would we have uncertainty (a p-value) in the difference in means of what are essentially completely observed populations? (the populations being the groups we assigned)? Could you please discuss the motivation behind this?

Strength: It appears that they put thought into their sample size calculations. The use of the results of the unpublished paper indicates they probably had decent estimates. As was mentioned above, the power seems small, however.