Jan 24: Assignment 2 Questions

I took the liberty of starting a discussion. Post your questions here!

The trial used “semi-standardized” acupuncture. It seems like the success of acupuncture in the treatment group would depend on initial diagnostics – since this determined the course of treatment. This might lead to more variable results.

How comparable was the pilot trial used for sample size calculations? It seems like the pilot used TCM, which the paper suggests is more effective than semi-standardized acupuncture.

Is it really true that age has does not impact migraine symptoms and severity? I wonder if the difference in age between the treatment and control groups may have introduced bias into the study.

Due to the strict inclusion and exclusion regulations, the number of subject (sample size) is so small. In this case, does it make sense to compute p values in order to do hypothesis testing?

In both received real acupuncture group and received sham acupuncture group, there are quite a few participants quiting from the study in the treatment process. The author just got rid of the missing data or dropouts. Does this simple approach bring about bias in estimation?

Just to clarify, was the sample size determined with a targeted significance level 0.05 and power 0.80 using the asymptotic result that a difference in proportions is normally distributed when n is large?

If yes, was the sample size, n, deemed large enough?

The authors were not exactly clear on how they did their sample size calculations. Did they calculate a confidence interval (what they call an "equivalence range" (?) ) for the estimate in the pilot study, and then use the end points of that estimate in their sample size calculation?

1. On page 524, "Statistical analysis" section: What do the authors mean by "The equivalence range was 11.8-57.9%"?

2. On page 524, "Patients" section, last paragraph: Why do the authors test for mean age difference between groups? Are there even "true means" here to compare?

3. On page 524, "Efficacy and long-term follow-up" section: The authors found that "pain" had significantly reduced within each group after treatment. Is this due to regression towards the mean? How can we tell?

1. In page 526 (efficacy), the real and the sham acupuncture groups are statistically insignificant for a number of pain parameters just by seeing the graphs (fig 2, 3 and 4) without by using any statistical inference ? It is not clear for me and how is it possible ?

2. In page 524, the author used an unpublished pilot trial to determine the sample size for his study and he don't specify how long ago the pilot study has done. Is the pilot trial valid  ?

1. In "Statistical Analysis" section, p524, what is "equivalence range"?

2. In "Statistical Analysis" section, p524, how did they get beta=0.2?

3. In question 4 of assignment 2, how can we do ANCOVA based on the paper which does not give us the raw data?

My question relates more to the assignment than to the technical details: What is the level of statistical knowledge we should expect of the clients? This may have impact on how we explain things in the report.

1. ANCOVA. In order to suggest a suitable sample size for ANCOVA of the change in headache frequency we need to estimate the Variance of the error term, which can be estimated from the Sum of Squares Error (SSE) after the regression fitting. The paper does not provide paired data on each patient: How do we give a reasonable estimate?

2. ANCOVA. RE:Treatment effect used in power calc. One way to interpret the effect is the difference in migraine frequency between trt/ctrl groups Given the same baseline value: what is a reasonable size of scientific interest? If there is no way of getting this question answered by the time a consulting report is due, is it good enough to present sample sizes calculated under several scenarios? (perhaps show result graphically)

3. Change Scores Analysis. What is an effect size (difference in mean change) of scientific interest?

1. On page 524, in "statistical analysis" section, is the 15% drop-out rate a generally assumed value in such type of trials, or is it based on result from previous acupuncture trials?

2. On page 524, in "associated symptoms" section, how is the Tukey test conducted?

1. The researchers did chi-square and Fisher's exact test for comparisons, and I wonder whether they can fit some models for this study.

2. In page 522, Real acupuncture treatment, it points out that some possible feelings was explained to the patients in the real treatment group. However, not in the sham group. Suppose patients ever talked with each other, would they find whether they received the real or sham treatment? Would it have influence on the outcome?