4th October 2010

Session 2.4

Design Fault Spot

Clinical Trial Design Fault Spot

Case Study: Clinical Trial Design Fault Spot

We have sketched complete designs. We will now critique partial designs.

In this case study, each part describes a clinical trial design set-up. Indicate the problem(s) with the approach(es) used in each part.

A large scale AIDS clinical trial is conducted in a Third World nation, in which the effects of a cheap, low-dose regimen of AZT(Zidovudine) in pregnant women is compared to the effects of a placebo in pregnant women. Randomization and Double Blinding is employed. The intended effect to be evaluated is the prevention of HIV infection in the child carried by the HIV infected mother.

CDC/WHO actually conducted a trial of this type - the primary objection was the use of a placebo in subjects with AIDS. The defense provided by the principles in this study were:

Conventional AIDS therapies are simply not available to AIDS patients in the 3rd world countries involved.

The actual subjects in the study were the developing children - they might be at risk at higher doses of AZT and the intended purpose of the AZT is the prevention of HIV transmission to these developing children.

In a comparative clinical trial, a new surgical method is compared to a standard surgical method. Study physicians classify subjects by the severity of their disease, and assign only the "mild" or "moderate" subjects to the new surgical method. Only the "severe" subjects are assigned to the standard surgical method.

The subjects should be randomly assigned to treatment groups. Under this study, subjects in each treatment group differ by treatment type and by severity. So we wouldn't know whether to attribute difference in outcome to treatment type, severity, or a combination of both treatment and severity.

Suppose a clinical trial is used to evaluate the safety of Drug X. The trial uses adult volunteers. The researchers claim that this trial is sufficient to ensure the safety of Drug X for pregnant women and children.

Data from men cannot automatically be applied to women (pregnant or otherwise) and children.

DES(Diethylstibestol) is an artificial hormone, whose intended effect is the prevention of Spontaneous Abortion in pregnant women. Spontaneous Abortion is a special type of miscarriage, not due to external factors such as injury. A trial is conducted in which two volunteer groups are recruited, one set of volunteers is recruited to try DES, the other group is recruited solely for observation(no treatment). The DES group knows it is getting DES, and the observation group receives no treatment. The physicians and nurse know which women are getting DES, and which are receiving no treatment.

Subjects should be assigned randomly to either DES or Placebo. Double blinding should be employed. Otherwise, differences in outcome might not be due to DES.

The groups that are recruited specifically for each treatment might well differ in important ways.

Disease X Therapeutic Trial

Disease X is a disease which is caused by an infection. It usually takes five (5) years for disease X to present symptoms. Left untreated, disease X produces severe and occasionally fatal symptoms and complications. Suppose that an effective, standard treatment, oldtreatX, is available. Suppose further that a new treatment, ihopeitworksX is available for evaluation. A basic clinical trial is proposed.           

Use of placebo here is inappropriate.

Disease Y Preventive Trial

Consider disease Y, which is caused by a bacterial infection, primarily affects children and which produces severe and occasionally fatal complications. Suppose that a candidate vaccine, newvaxY, is available for evaluation. A randomized, double-blinded basic clinical trial for newvaxY is proposed. This trial will use adult subjects only.

A pediatric (child-focused) trial is required here to establish the safety and effectiveness of the new vaccine.

The basic trial is unethical, since an effective vaccine is available, and the consequences of disease Y are potentially nasty.

Cancer Z Prevention Trial

Suppose cancer Z typically strikes adults who are aged 40-65 years, and suppose further that no established preventive treatment is available. Suppose that a new treatment, preventZ, which is intended to help prevent cancer Z is available for evaluation. A basic clinical trial is proposed. The trial will focus on adult subjects aged 25-30 years, who have no prior history of cancer Z. Study subjects will be followed for five (5) years after study entry. The trial will be a double-blinded, randomized basic clinical trial.

The follow-up time (5 years) is inadequate.

 

 

Sample Survey Design Hypermedia Resources

http://www.pbs.org/fmc/segments/progseg7.htm

http://www.public.iastate.edu/~jhutter/406/lectureoutline.doc

http://www.csudh.edu/dearhabermas/sampling01.htm

http://www.bartleby.com/65/po/poll.html

http://www.stat.ucla.edu/~rgould/m12s01/survey.pdf

http://fly.hiwaay.net/~jmcmulle/450polls.htm

Pew Center Links

http://people-press.org/

http://people-press.org/reports/

http://people-press.org/reports/display.php3?ReportID=12

http://people-press.org/reports/display.php3?ReportID=89

About Sample Surveys

Our brief overview of sample survey methodology focuses on what is sometimes called “scientific polling.” This type of survey design relies on a few well-defined features: use of a random sampling scheme, use of a carefully-designed survey instrument and on carefully-planned interviewing techniques.

There are a few key concepts in a well-designed sample survey, each keyed to simple question or questions.

Why?  – Why are you conducting the survey? Is this a brief poll? Is this a detailed survey of a population? Are you predicting, describing, or both?

What? –What do you want to learn?

Who? – Consider the population of interest. Who exactly will you sample?

How? – How do you obtain your random sample of respondent candidates? How do you design your survey instrument? How do you deliver the instrument/conduct the interview?

When? – What is the time-frame for your survey? Is this a short-term project, or an ongoing activity? What is your time-line from beginning to publication?

 

 

 

Sample Survey Process

Define Population(Pop)

Acquire Sampling Frame within Population(Frame Pop)

Design Random Sampling Process(RSP)

Select Random Sample of Candidate Respondents Using RSP from Frame Pop

Design Survey Interview Instrument and Interview Protocol (IIP)

Interview Consenting Respondents using IIP

Compile, Analyze and Report Findings

 

 

        

 

The 1936 and 1948 Presidential Polls

A number of important survey failures stressed the need for Random Sampling.The Literary Digest US Presidential Poll of 1936 used a non-random sample targeting telephone directories, automobile registries and the LD subscriber list. Even though 2.4 million respondents were acquired for the poll, the poll failed miserably. Moreover, the bulk of US voters in the 1936 Presidential Election were neither automobile owners nor telephone owners nor LD subscribers.

1936: Alf Landon (Incumbent Governor Kansas (R), father of Nancy Kassebaum ) versus Franklin Delano Roosevelt (Incumbent President (D), previously Governor  New York (D) )

Links for the 1936 LD Poll: 1, 2, 3 

Three major polling organizations - Crossley, Roper and Gallup, suffered poll failures for the 1948 US Presidential Election. All three organizations drew non-random samples which nicely resembled the population of US Voters, and all three polls still failed, though by a small margin. The problem was with the use of quota sampling.In quota sampling, poll workers are free to use judgment in selecting respondents, so long as the selected respondents meet quota requirements.

Gallup did two random-sample based polls for the 1936 US Presidential Election - both samples were much smaller than the 2.4 million LD Poll Sample. Gallup’s 1936 polls correctly predicted both the results of the Literary Digest Poll and the Actual 1936 US Presidential Election Results. The bizarre thing is that the Gallup people did not use Random Sampling in the 1948 US Presidential Election Poll.

1948: Thomas Dewey(Incumbent Governor New York) versus Truman (Incumbent President (D)

Links for the 1948 Presidential Polls: 1, 2, 3

After the 1948 Presidential Poll failures, the importance of random sampling became clear.

 

Sample Survey Design Outline

Survey Sampling

Respondent Rights

Supplemental Sample Survey Notes

Sampling Designs

glossary gives a glossary of terms used in Sample Survey Designs.

Sample Survey Design Faults

Sample Survey Design Fault Spot

 

Case Study: Clinical Trial Design Fault Spot

We have sketched complete designs. We will now critique partial designs.

In this case study, each part describes a clinical trial design set-up. Indicate the problem(s) with the approach(es) used in each part.

A large scale AIDS clinical trial is conducted in a Third World nation, in which the effects of a cheap, low-dose regimen of AZT(Zidovudine) in pregnant women is compared to the effects of a placebo in pregnant women. Randomization and Double Blinding is employed. The intended effect to be evaluated is the prevention of HIV infection in the child carried by the HIV infected mother.

CDC/WHO actually conducted a trial of this type - the primary objection was the use of a placebo in subjects with AIDS. The defense provided by the principles in this study were:

Conventional AIDS therapies are simply not available to AIDS patients in the 3rd world countries involved.

The actual subjects in the study were the developing children - they might be at risk at higher doses of AZT and the intended purpose of the AZT is the prevention of HIV transmission to these developing children.

In a comparative clinical trial, a new surgical method is compared to a standard surgical method. Study physicians classify subjects by the severity of their disease, and assign only the "mild" or "moderate" subjects to the new surgical method. Only the "severe" subjects are assigned to the standard surgical method.

The subjects should be randomly assigned to treatment groups. Under this study, subjects in each treatment group differ by treatment type and by severity. So we wouldn't know whether to attribute difference in outcome to treatment type, severity, or a combination of both treatment and severity.

Suppose a clinical trial is used to evaluate the safety of Drug X. The trial uses adult volunteers. The researchers claim that this trial is sufficient to ensure the safety of Drug X for pregnant women and children.

Data from men cannot automatically be applied to women (pregnant or otherwise) and children.

DES(Diethylstibestol) is an artificial hormone, whose intended effect is the prevention of Spontaneous Abortion in pregnant women. Spontaneous Abortion is a special type of miscarriage, not due to external factors such as injury. A trial is conducted in which two volunteer groups are recruited, one set of volunteers is recruited to try DES, the other group is recruited solely for observation(no treatment). The DES group knows it is getting DES, and the observation group receives no treatment. The physicians and nurse know which women are getting DES, and which are receiving no treatment.

Subjects should be assigned randomly to either DES or Placebo. Double blinding should be employed. Otherwise, differences in outcome might not be due to DES.

The groups that are recruited specifically for each treatment might well differ in important ways.

Disease X Therapeutic Trial

Disease X is a disease which is caused by an infection. It usually takes five (5) years for disease X to present symptoms. Left untreated, disease X produces severe and occasionally fatal symptoms and complications. Suppose that an effective, standard treatment, oldtreatX, is available. Suppose further that a new treatment, ihopeitworksX is available for evaluation. A basic clinical trial is proposed.           

Use of placebo here is inappropriate.

Disease Y Preventive Trial

Consider disease Y, which is caused by a bacterial infection, primarily affects children and which produces severe and occasionally fatal complications. Suppose that a candidate vaccine, newvaxY, is available for evaluation. A randomized, double-blinded basic clinical trial for newvaxY is proposed. This trial will use adult subjects only.

A pediatric (child-focused) trial is required here to establish the safety and effectiveness of the new vaccine.

The basic trial is unethical, since an effective vaccine is available, and the consequences of disease Y are potentially nasty.

Cancer Z Prevention Trial

Suppose cancer Z typically strikes adults who are aged 40-65 years, and suppose further that no established preventive treatment is available. Suppose that a new treatment, preventZ, which is intended to help prevent cancer Z is available for evaluation. A basic clinical trial is proposed. The trial will focus on adult subjects aged 25-30 years, who have no prior history of cancer Z. Study subjects will be followed for five (5) years after study entry. The trial will be a double-blinded, randomized basic clinical trial.

The follow-up time (5 years) is inadequate.

 

 

Sample Survey Design Hypermedia Resources

http://www.pbs.org/fmc/segments/progseg7.htm

http://www.public.iastate.edu/~jhutter/406/lectureoutline.doc

http://www.csudh.edu/dearhabermas/sampling01.htm

http://www.bartleby.com/65/po/poll.html

http://www.stat.ucla.edu/~rgould/m12s01/survey.pdf

http://fly.hiwaay.net/~jmcmulle/450polls.htm

Pew Center Links

http://people-press.org/

http://people-press.org/reports/

http://people-press.org/reports/display.php3?ReportID=12

http://people-press.org/reports/display.php3?ReportID=89

About Sample Surveys

Our brief overview of sample survey methodology focuses on what is sometimes called “scientific polling.” This type of survey design relies on a few well-defined features: use of a random sampling scheme, use of a carefully-designed survey instrument and on carefully-planned interviewing techniques.

There are a few key concepts in a well-designed sample survey, each keyed to simple question or questions.

Why?  – Why are you conducting the survey? Is this a brief poll? Is this a detailed survey of a population? Are you predicting, describing, or both?

What? –What do you want to learn?

Who? – Consider the population of interest. Who exactly will you sample?

How? – How do you obtain your random sample of respondent candidates? How do you design your survey instrument? How do you deliver the instrument/conduct the interview?

When? – What is the time-frame for your survey? Is this a short-term project, or an ongoing activity? What is your time-line from beginning to publication?

 

 

 

Sample Survey Process

Define Population(Pop)

Acquire Sampling Frame within Population(Frame Pop)

Design Random Sampling Process(RSP)

Select Random Sample of Candidate Respondents Using RSP from Frame Pop

Design Survey Interview Instrument and Interview Protocol (IIP)

Interview Consenting Respondents using IIP

Compile, Analyze and Report Findings

 

 

        

 

The 1936 and 1948 Presidential Polls

A number of important survey failures stressed the need for Random Sampling.The Literary Digest US Presidential Poll of 1936 used a non-random sample targeting telephone directories, automobile registries and the LD subscriber list. Even though 2.4 million respondents were acquired for the poll, the poll failed miserably. Moreover, the bulk of US voters in the 1936 Presidential Election were neither automobile owners nor telephone owners nor LD subscribers.

1936: Alf Landon (Incumbent Governor Kansas (R), father of Nancy Kassebaum ) versus Franklin Delano Roosevelt (Incumbent President (D), previously Governor  New York (D) )

Links for the 1936 LD Poll: 1, 2, 3 

Three major polling organizations - Crossley, Roper and Gallup, suffered poll failures for the 1948 US Presidential Election. All three organizations drew non-random samples which nicely resembled the population of US Voters, and all three polls still failed, though by a small margin. The problem was with the use of quota sampling.In quota sampling, poll workers are free to use judgment in selecting respondents, so long as the selected respondents meet quota requirements.

Gallup did two random-sample based polls for the 1936 US Presidential Election - both samples were much smaller than the 2.4 million LD Poll Sample. Gallup’s 1936 polls correctly predicted both the results of the Literary Digest Poll and the Actual 1936 US Presidential Election Results. The bizarre thing is that the Gallup people did not use Random Sampling in the 1948 US Presidential Election Poll.

1948: Thomas Dewey(Incumbent Governor New York) versus Truman (Incumbent President (D)

Links for the 1948 Presidential Polls: 1, 2, 3

After the 1948 Presidential Poll failures, the importance of random sampling became clear.

 

Sample Survey Design Outline

Survey Sampling

Respondent Rights

Supplemental Sample Survey Notes

Sampling Designs

glossary gives a glossary of terms used in Sample Survey Designs.

Sample Survey Design Faults

Sample Survey Design Fault Spot

We have sketched complete designs. We will now critique partial designs.

In this case study, each part describes a sample survey design set-up. Indicate the problem(s) with the approach(es) used in each part.

The Communications Department of a University conducts a survey of its major students(prior to graduation) regarding their satisfaction levels with their degree program. Random sampling is employed, and the survey instrument is unbiased and properly written. The respondents are interviewed face-to-face by department faculty members.

Problems Include:

The face-to-face thing needs to go - you'll get hopelessly biased responses based on the perceived expectations of the faculty {modified Hawthorne Effect}.

A sample survey design targets membership lists of US churches, and surveys a random sample of members regarding attitudes toward god and religion. The survey people then claim that their results apply to the general US population.

Problems Include:

The US genpop cannot be captured by the specified list of church rolls - not even if this list magically included every single body of followers of every organised religion in the US.

In 1987, Shere Hite published Women and Love. The author distributed 100,000 questionnaires through various women's groups , asking questions about love, sex and relations between women and men. Of all the questionnaires distributed, 4.5% were returned. Hite based her findings based on these returned surveys. Her gist was that her results represented a general female population.

Problems Include:

The femgenpop cannot be captured by women's group rosters - even every single women's group is included.

The response rate (4.5%) is simply too inadequate - there may be a difference in opinions between those who responded and those who did not.

A survey regarding the treatment of interned Japanese-Americans is conducted sometime during 1941-1945. The surveyed population is the population of interned Japanese-Americans, and is conducted by uniformed US Army personnel. A random sample is selected for the survey. Assume that the sample survey instrument is worded properly.

Problems Include:

This one is hopelessly hopeless.

The use of US military personnel is likely to ellicit biased responses from the internees.

The basic status of the respondents (prisoners) may prevent meaningful responses from emerging.

Euthanasia / Assisted Suicide Survey

The term euthanasia loosely refers to a variety of procedures in which a patient's life is actively or passively ended, or in which the patient is assisted in dying. Suppose a sample survey design attempts to determine US attitudes towards euthanasia. This design will use a national (US) random sample of adults who have had a relative die of a terminal illness, or who have a relative dying of a terminal illness. The respondents will be interviewed by a specially trained health professional. The designers hope that the results will reflect general US attitudes regarding euthanasia.      

Problems Include:

Euthanasia is a sensitive topic, and the topic itself may cause bias in the responses.

The general opinion may differ systematically from the opinions held by people close to the topic - people close to the topic may have more extreme opinions (pro and con) than the genpop.

The sampling cannot represent the genpop.

Political Candidate Preference / Voting Survey

Suppose that a sample survey design is desired which will predict the outcome of a particular election. An election usually involves the selection of a candidate (or candidates) for a particular office (or offices). An election might also involve referenda, in which voters decide a particular question. The design uses a random sample of likely voters (adults who are registered to vote, or intend to register to vote, and intend to vote), and is conducted 3 weeks prior to the election.

Problems Include:

3 weeks is potentially a looooooong time in political science.

We might want to "tighten up" the definition of "likely voter" to include only registered voters who report themselves likely to vote.