14th March 2011
Session 2.4
Design Fault Spot
Clinical Trial Design Fault Spot
Case Study: Clinical Trial Design Fault Spot
We have sketched complete designs. We will now critique partial
designs.
In this case study, each part describes a clinical trial design
set-up. Indicate the problem(s) with the approach(es) used in each part.
A large scale AIDS clinical trial
is conducted in a Third World nation, in which the effects of a cheap, low-dose
regimen of AZT(Zidovudine) in pregnant women is compared to the effects of a placebo
in pregnant women. Randomization and Double Blinding is employed. The intended
effect to be evaluated is the prevention of HIV infection in the child carried
by the HIV infected mother.
CDC/WHO actually conducted a trial of this type - the primary
objection was the use of a placebo in subjects with AIDS. The defense provided by the principles in this study were:
Conventional AIDS therapies are simply not available to AIDS patients
in the 3rd world countries involved.
The actual subjects in the study were the developing children -
they might be at risk at higher doses of AZT and the intended purpose of the
AZT is the prevention of HIV transmission to these developing children.
In a comparative clinical trial,
a new surgical method is compared to a standard surgical method. Study
physicians classify subjects by the severity of their disease, and assign only
the "mild" or "moderate" subjects to the new surgical
method. Only the "severe" subjects are assigned to the standard
surgical method.
The subjects should be randomly assigned to treatment groups. Under
this study, subjects in each treatment group differ by treatment type and by
severity. So we wouldn't know whether to attribute difference in outcome to
treatment type, severity, or a combination of both treatment and severity.
Suppose a clinical trial is used
to evaluate the safety of Drug X. The trial uses adult volunteers. The
researchers claim that this trial is sufficient to ensure the safety of Drug X
for pregnant women and children.
Data from men cannot automatically be applied to women (pregnant or
otherwise) and children.
DES(Diethylstibestol) is an artificial hormone, whose intended effect is the prevention of Spontaneous
Abortion in pregnant women. Spontaneous Abortion is a special type of
miscarriage, not due to external factors such as injury. A trial is conducted
in which two volunteer groups are recruited, one set of volunteers is recruited
to try DES, the other group is recruited solely for observation(no
treatment). The DES group knows it is getting DES, and the observation group
receives no treatment. The physicians and nurse know which women are getting
DES, and which are receiving no treatment.
Subjects should be assigned randomly to either DES or Placebo.
Double blinding should be employed. Otherwise, differences in outcome might not
be due to DES.
The groups that are recruited specifically for each treatment might
well differ in important ways.
Disease X Therapeutic Trial
Disease
X is a disease which is caused by an infection. It usually takes five (5) years
for disease X to present symptoms. Left untreated, disease X
produces severe and occasionally fatal symptoms and complications.
Suppose that an effective, standard treatment, oldtreatX, is available. Suppose further that a new treatment, ihopeitworksX is available for evaluation. A basic clinical trial is
proposed.
Use of placebo here is inappropriate.
Disease Y Preventive Trial
Consider
disease Y, which is caused by a bacterial infection, primarily affects children
and which produces severe and occasionally fatal complications. Suppose that a
candidate vaccine, newvaxY,
is available for evaluation. A randomized, double-blinded basic clinical trial
for newvaxY is proposed. This trial will use adult subjects only.
A pediatric (child-focused) trial is required here to establish the
safety and effectiveness of the new vaccine.
The basic trial is unethical, since an effective vaccine is
available, and the consequences of disease Y are potentially nasty.
Cancer Z Prevention Trial
Suppose
cancer Z typically strikes adults who are aged 40-65 years, and suppose further
that no established preventive treatment is available. Suppose that a new
treatment, preventZ, which is intended to help prevent cancer Z is available for evaluation. A basic clinical trial is
proposed. The trial will focus on adult subjects aged 25-30 years, who have no
prior history of cancer Z. Study subjects will be followed for five (5) years
after study entry. The trial will be a double-blinded, randomized basic
clinical trial.
The follow-up time (5 years) is inadequate.
Sample Survey Design
Hypermedia Resources
http://www.pbs.org/fmc/segments/progseg7.htm
http://www.public.iastate.edu/~jhutter/406/lectureoutline.doc
http://www.csudh.edu/dearhabermas/sampling01.htm
http://www.bartleby.com/65/po/poll.html
http://www.stat.ucla.edu/~rgould/m12s01/survey.pdf
http://fly.hiwaay.net/~jmcmulle/450polls.htm
Pew Center Links
http://people-press.org/reports/
http://people-press.org/reports/display.php3?ReportID=12
http://people-press.org/reports/display.php3?ReportID=89
About Sample Surveys
Our brief overview of sample survey
methodology focuses on what is sometimes called “scientific polling.” This type
of survey design relies on a few well-defined features: use of a random
sampling scheme, use of a carefully-designed survey instrument and on
carefully-planned interviewing techniques.
There are a few key concepts in a
well-designed sample survey, each keyed to simple question or questions.
Why? – Why are you conducting
the survey? Is this a brief poll? Is this a detailed survey of a population?
Are you predicting, describing, or both?
What? –What do you want to learn?
Who? – Consider the population of
interest. Who exactly will you sample?
How? – How do you obtain your random
sample of respondent candidates? How do you design your survey instrument? How
do you deliver the instrument/conduct the interview?
When? – What is the time-frame for
your survey? Is this a short-term project, or an ongoing activity? What is your
time-line from beginning to publication?
Sample Survey Process
Define Population(Pop)
Acquire Sampling Frame within Population(Frame Pop)
Design Random Sampling Process(RSP)
Select Random Sample of Candidate
Respondents Using RSP from Frame Pop
Design Survey Interview Instrument
and Interview Protocol (IIP)
Interview Consenting Respondents
using IIP
Compile, Analyze and Report Findings
The 1936 and 1948 Presidential Polls
A number of important survey
failures stressed the need for Random Sampling.The Literary Digest US Presidential Poll of 1936 used a non-random
sample targeting telephone directories, automobile registries and the LD
subscriber list. Even though 2.4 million respondents were acquired for the
poll, the poll failed miserably. Moreover, the bulk of US voters in the 1936
Presidential Election were neither automobile owners nor telephone owners
nor LD subscribers.
1936: Alf Landon (Incumbent Governor
Kansas (R), father of Nancy Kassebaum
) versus Franklin Delano Roosevelt (Incumbent
President (D), previously Governor New York (D) )
Links for the 1936 LD Poll: 1, 2, 3
Three major polling organizations - Crossley, Roper and Gallup, suffered
poll failures for the 1948 US Presidential Election. All three organizations
drew non-random samples which nicely resembled the population of US Voters, and
all three polls still failed, though by a small margin. The problem was with
the use of quota sampling.In quota sampling, poll workers are free to use judgment in
selecting respondents, so long as the selected respondents meet quota requirements.
Gallup did two random-sample based
polls for the 1936 US Presidential Election - both samples were much smaller
than the 2.4 million LD Poll Sample. Gallup’s 1936 polls correctly
predicted both the results of the Literary Digest Poll and the Actual
1936 US Presidential Election Results. The bizarre thing is that the Gallup
people did not use Random Sampling in the 1948
US Presidential Election Poll.
1948: Thomas Dewey(Incumbent Governor New York)
versus Truman (Incumbent President (D)
Links for the 1948 Presidential
Polls: 1, 2, 3
After the 1948 Presidential Poll
failures, the importance of random sampling became clear.
Supplemental Sample Survey Notes
glossary gives a glossary
of terms used in Sample Survey Designs.
Sample Survey Design Faults
Sample Survey Design Fault
Spot
Case Study: Clinical Trial Design Fault Spot
We have sketched complete designs. We will now critique partial
designs.
In this case study, each part describes a clinical trial design
set-up. Indicate the problem(s) with the approach(es) used in each part.
A large scale AIDS clinical trial
is conducted in a Third World nation, in which the effects of a cheap, low-dose
regimen of AZT(Zidovudine) in pregnant women is compared to the effects of a placebo
in pregnant women. Randomization and Double Blinding is employed. The intended
effect to be evaluated is the prevention of HIV infection in the child carried
by the HIV infected mother.
CDC/WHO actually conducted a trial of this type - the primary
objection was the use of a placebo in subjects with AIDS. The defense provided by the principles in this study were:
Conventional AIDS therapies are simply not available to AIDS
patients in the 3rd world countries involved.
The actual subjects in the study were the developing children -
they might be at risk at higher doses of AZT and the intended purpose of the
AZT is the prevention of HIV transmission to these developing children.
In a comparative clinical trial,
a new surgical method is compared to a standard surgical method. Study
physicians classify subjects by the severity of their disease, and assign only
the "mild" or "moderate" subjects to the new surgical
method. Only the "severe" subjects are assigned to the standard
surgical method.
The subjects should be randomly assigned to treatment groups. Under
this study, subjects in each treatment group differ by treatment type and by
severity. So we wouldn't know whether to attribute difference in outcome to
treatment type, severity, or a combination of both treatment and severity.
Suppose a clinical trial is used
to evaluate the safety of Drug X. The trial uses adult volunteers. The
researchers claim that this trial is sufficient to ensure the safety of Drug X
for pregnant women and children.
Data from men cannot automatically be applied to women (pregnant or
otherwise) and children.
DES(Diethylstibestol) is an artificial hormone, whose intended effect is the prevention of Spontaneous
Abortion in pregnant women. Spontaneous Abortion is a special type of
miscarriage, not due to external factors such as injury. A trial is conducted
in which two volunteer groups are recruited, one set of volunteers is recruited
to try DES, the other group is recruited solely for observation(no
treatment). The DES group knows it is getting DES, and the observation group receives
no treatment. The physicians and nurse know which women are getting DES, and
which are receiving no treatment.
Subjects should be assigned randomly to either DES or Placebo.
Double blinding should be employed. Otherwise, differences in outcome might not
be due to DES.
The groups that are recruited specifically for each treatment might
well differ in important ways.
Disease X Therapeutic Trial
Disease
X is a disease which is caused by an infection. It usually takes five (5) years
for disease X to present symptoms. Left untreated, disease X
produces severe and occasionally fatal symptoms and complications.
Suppose that an effective, standard treatment, oldtreatX, is available. Suppose further that a new treatment, ihopeitworksX is available for evaluation. A basic clinical trial is
proposed.
Use of placebo here is inappropriate.
Disease Y Preventive Trial
Consider
disease Y, which is caused by a bacterial infection, primarily affects children
and which produces severe and occasionally fatal complications. Suppose that a
candidate vaccine, newvaxY,
is available for evaluation. A randomized, double-blinded basic clinical trial
for newvaxY is proposed. This trial will use adult subjects only.
A pediatric (child-focused) trial is required here to establish the
safety and effectiveness of the new vaccine.
The basic trial is unethical, since an effective vaccine is
available, and the consequences of disease Y are potentially nasty.
Cancer Z Prevention Trial
Suppose
cancer Z typically strikes adults who are aged 40-65 years, and suppose further
that no established preventive treatment is available. Suppose that a new
treatment, preventZ, which is intended to help prevent cancer Z is available for evaluation. A basic clinical trial is
proposed. The trial will focus on adult subjects aged 25-30 years, who have no
prior history of cancer Z. Study subjects will be followed for five (5) years
after study entry. The trial will be a double-blinded, randomized basic
clinical trial.
The follow-up time (5 years) is inadequate.
Sample Survey Design
Hypermedia Resources
http://www.pbs.org/fmc/segments/progseg7.htm
http://www.public.iastate.edu/~jhutter/406/lectureoutline.doc
http://www.csudh.edu/dearhabermas/sampling01.htm
http://www.bartleby.com/65/po/poll.html
http://www.stat.ucla.edu/~rgould/m12s01/survey.pdf
http://fly.hiwaay.net/~jmcmulle/450polls.htm
Pew Center Links
http://people-press.org/reports/
http://people-press.org/reports/display.php3?ReportID=12
http://people-press.org/reports/display.php3?ReportID=89
About Sample Surveys
Our brief overview of sample survey
methodology focuses on what is sometimes called “scientific polling.” This type
of survey design relies on a few well-defined features: use of a random
sampling scheme, use of a carefully-designed survey instrument and on
carefully-planned interviewing techniques.
There are a few key concepts in a
well-designed sample survey, each keyed to simple question or questions.
Why? – Why are you conducting
the survey? Is this a brief poll? Is this a detailed survey of a population?
Are you predicting, describing, or both?
What? –What do you want to learn?
Who? – Consider the population of
interest. Who exactly will you sample?
How? – How do you obtain your random
sample of respondent candidates? How do you design your survey instrument? How
do you deliver the instrument/conduct the interview?
When? – What is the time-frame for
your survey? Is this a short-term project, or an ongoing activity? What is your
time-line from beginning to publication?
Sample Survey Process
Define Population(Pop)
Acquire Sampling Frame within Population(Frame Pop)
Design Random Sampling Process(RSP)
Select Random Sample of Candidate
Respondents Using RSP from Frame Pop
Design Survey Interview Instrument
and Interview Protocol (IIP)
Interview Consenting Respondents
using IIP
Compile, Analyze and Report Findings
The 1936 and 1948 Presidential Polls
A number of important survey
failures stressed the need for Random Sampling.The Literary Digest US Presidential Poll of 1936 used a non-random
sample targeting telephone directories, automobile registries and the LD
subscriber list. Even though 2.4 million respondents were acquired for the
poll, the poll failed miserably. Moreover, the bulk of US voters in the 1936
Presidential Election were neither automobile owners nor telephone owners
nor LD subscribers.
1936: Alf Landon (Incumbent Governor
Kansas (R), father of Nancy Kassebaum
) versus Franklin Delano Roosevelt
(Incumbent President (D), previously Governor New York (D) )
Links for the 1936 LD Poll: 1, 2, 3
Three major polling organizations - Crossley, Roper and Gallup, suffered
poll failures for the 1948 US Presidential Election. All three organizations
drew non-random samples which nicely resembled the population of US Voters, and
all three polls still failed, though by a small margin. The problem was with
the use of quota sampling.In quota sampling, poll workers are free to use judgment in
selecting respondents, so long as the selected respondents meet quota
requirements.
Gallup did two random-sample based
polls for the 1936 US Presidential Election - both samples were much smaller
than the 2.4 million LD Poll Sample. Gallup’s 1936 polls correctly
predicted both the results of the Literary Digest Poll and the Actual
1936 US Presidential Election Results. The bizarre thing is that the Gallup
people did not use Random Sampling in the 1948
US Presidential Election Poll.
1948: Thomas Dewey(Incumbent Governor New York)
versus Truman (Incumbent President (D)
Links for the 1948 Presidential
Polls: 1, 2, 3
After the 1948 Presidential Poll
failures, the importance of random sampling became clear.
Supplemental Sample Survey Notes
glossary gives a glossary of
terms used in Sample Survey Designs.
Sample Survey Design Faults
Sample Survey Design
Fault Spot
We have sketched complete
designs. We will now critique partial designs.
In this case study, each
part describes a sample survey design set-up. Indicate the problem(s) with the approach(es) used in each part.
The Communications
Department of a University conducts a survey
of its major students(prior to graduation) regarding
their satisfaction levels with their degree program. Random sampling is
employed, and the survey instrument is unbiased and properly written. The
respondents are interviewed face-to-face by department faculty members.
Problems Include:
The face-to-face thing
needs to go - you'll get hopelessly biased responses based on the perceived
expectations of the faculty {modified Hawthorne Effect}.
A sample survey design
targets membership lists of US churches,
and surveys a random sample of members regarding attitudes toward god and
religion. The survey people then claim that their results apply to the general
US population.
Problems Include:
The US genpop cannot be captured by
the specified list of church rolls - not even if this list magically included
every single body of followers of every organised religion in the US.
In 1987, Shere Hite published Women and Love. The author distributed 100,000
questionnaires through various women's groups , asking
questions about love, sex and relations between women and men. Of all the questionnaires
distributed, 4.5% were returned. Hite based her findings based on these
returned surveys. Her gist was that her results represented a general female
population.
Problems Include:
The femgenpop
cannot be captured by women's group rosters - even every single women's group is included.
The response rate (4.5%)
is simply too inadequate - there may be a difference in opinions between those
who responded and those who did not.
A survey regarding the
treatment of interned Japanese-Americans is
conducted sometime during 1941-1945. The surveyed population is the population
of interned Japanese-Americans, and is conducted by uniformed US Army
personnel. A random sample is selected for the survey. Assume that the sample
survey instrument is worded properly.
Problems Include:
This one is hopelessly
hopeless.
The use of US military
personnel is likely to ellicit biased responses from the internees.
The basic status of the
respondents (prisoners) may prevent meaningful responses from emerging.
Euthanasia / Assisted
Suicide Survey
The term euthanasia loosely refers
to a variety of procedures in which a patient's life is actively or passively
ended, or in which the patient is assisted in dying. Suppose a sample survey
design attempts to determine US attitudes towards euthanasia. This design will
use a national (US) random sample of adults who have had a relative die of a
terminal illness, or who have a relative dying of a terminal illness. The
respondents will be interviewed by a specially trained health professional. The
designers hope that the results will reflect general US attitudes regarding
euthanasia.
Problems Include:
Euthanasia is a sensitive
topic, and the topic itself may cause bias in the responses.
The general opinion may
differ systematically from the opinions held by people close to the topic -
people close to the topic may have more extreme opinions (pro and con) than the
genpop.
The sampling cannot
represent the genpop.
Political Candidate
Preference / Voting Survey
Suppose that a sample survey design
is desired which will predict the outcome of a particular election. An election
usually involves the selection of a candidate (or candidates) for a particular
office (or offices). An election might also involve referenda, in which voters
decide a particular question. The design uses a random sample of likely voters
(adults who are registered to vote, or intend to register to vote, and intend
to vote), and is conducted 3 weeks prior to the election.
Problems Include:
3 weeks is potentially a looooooong
time in political science.
We might want to
"tighten up" the definition of "likely voter" to include
only registered voters who report themselves likely to vote.