Tools
Dear Science,
In the recent HIV-vaccine trial, the reports said the study showed that about 30 percent fewer people who got the vaccine got HIV, and that this difference was statistically significant. What does it mean for something to be "statistically significant"? Does that make it true?
Stranger Personals
Math Monster
Let's think about that HIV-vaccine study. We wanted to know if a combination of vaccines could prevent HIV infection. To answer the question, 16,000 people were recruited from the Thai population and divided into two groups; one half received the vaccination series, the other half received a placebo. Both groups were then followed for a period of five years, with new HIV infections recorded. Seventy-four of the placebo recipients were newly HIV positive versus 51 of the vaccine recipients, or 23 fewer people. With this raw information, we want to know if the vaccine reduced the risk of HIV infection.
The question boils down to this: Are the HIV infection rates between these two groups of 8,000 people really different? (Most medical research often boils down to asking if two groups are really different.) On the surface this seems simple. Twenty-three fewer people got HIV after being vaccinated! Surely the vaccine protected those 23 people. Of course it works.
Not so fast. Every measurement is subject to noise and error. If we repeated this study again, the numbers of newly infected with HIV in both groups would almost certainly be different. This noise can make the two groups look different even if they were the same. Before embarking on a study, a scientist is expected to pick some risk they're willing to accept of finding a false difference between the group; for almost all medical studies, that threshold is a 1 in 20 chance, or 5 percent of getting it wrong. Math—glorious, lovely, and wondrous mathematics—allows us to estimate this noisiness. Using these mathematical estimates of the range of the true counts, we can figure out if the difference between the groups we observed has less than a 5 percent chance of being due to noise alone. For this study, the risk that the difference between the groups was due to noise was just under five percent. Therefore, the study is statistically significant.
That's not to say it's true. We still have a 1 in 20 chance of getting it (honestly) wrong, detecting a difference that isn't actually there. Further, statistical significance does not consider the quality of the study—whether the groups initially picked really represent the general population, if the randomization was fairly done, and so on. Statistical significance is the starting place of a valid study. Calling something true takes a bit more thought.
Samplingly Yours,
Science
Send your science questions to dearscience@thestranger.com.
Share via
1
This is a good article, Science, but I think it misses a key
point, a point which many professional scientists get wrong, too.
Try this thought experiment on for size: You get a bag of 1000
pennies. 999 are honest pennies, one is crooked -- one has two
heads. Now shake up the bag, reach in, and pull out a penny at
random. Don't look at it! Instead, flip it in the air, catch it,
and note if it is heads or tails. Do this five times.
OK, now say that each of the five times you flipped the coin, you
get heads. Now ask yourself: Is it more likely the five heads in a
row is a statistical fluke you got from an honest coin, or is it
more likely that you've picked the one two-headed coin out of the
bag? By your argument above, you would just use math, marvelous
math, to calculate that the odds of flipping five straight heads
are only about 3%, and thus you would say, "yep, only a 3% chance
this is a statistical fluke, and therefore there is a 97% chance
this penny is a two-headed penny."
But you would be wrong. Yes, if you pull out an honest coin,
there is only a one in 32 chance you'll then flip five straight
heads. But, when you first pulled out the coin, you had only a
one in 1000 chance of pulling out a crooked coin. The math gets
a little harder, but it turns out that if you do the experiment
we're talking about here, and get five straight heads, it is
almost certainly the case that you have an honest coin (which is
the analog here of a null result -- an ineffective vaccine). Don't
take my word for it (although you should). Go ask someone who
really knows stats. Not your research advisor, probably, but
someone in the UW stats department maybe. Then write a follow-up
column! And then readers of the Stranger will be better informed
about statistics than many working scientists!
In interpreting the results of an experiment, you have to ask not
just is it "1 in 3 statistics", "1 in 20 statistics" or "1 in 200
statistics". You also have to ask, "does my hypothesis pass the
giggle test?" So for instance, if you take two vaccines which
each been very very thoroughly shown to be ineffective when used
individually, and then you do a study where you give people a
combo shot, you have to ask yourself "when has a combo shot
_ever_ worked, in the history of immunology, in those case when
each individual one doesn't work at all? Or am I instead looking
for the equivalent of an exotic two-headed coin?" If the latter,
then designing an experiment to yield only two-sigma statistics
is a huge waste of money.
More...
point, a point which many professional scientists get wrong, too.
Try this thought experiment on for size: You get a bag of 1000
pennies. 999 are honest pennies, one is crooked -- one has two
heads. Now shake up the bag, reach in, and pull out a penny at
random. Don't look at it! Instead, flip it in the air, catch it,
and note if it is heads or tails. Do this five times.
OK, now say that each of the five times you flipped the coin, you
get heads. Now ask yourself: Is it more likely the five heads in a
row is a statistical fluke you got from an honest coin, or is it
more likely that you've picked the one two-headed coin out of the
bag? By your argument above, you would just use math, marvelous
math, to calculate that the odds of flipping five straight heads
are only about 3%, and thus you would say, "yep, only a 3% chance
this is a statistical fluke, and therefore there is a 97% chance
this penny is a two-headed penny."
But you would be wrong. Yes, if you pull out an honest coin,
there is only a one in 32 chance you'll then flip five straight
heads. But, when you first pulled out the coin, you had only a
one in 1000 chance of pulling out a crooked coin. The math gets
a little harder, but it turns out that if you do the experiment
we're talking about here, and get five straight heads, it is
almost certainly the case that you have an honest coin (which is
the analog here of a null result -- an ineffective vaccine). Don't
take my word for it (although you should). Go ask someone who
really knows stats. Not your research advisor, probably, but
someone in the UW stats department maybe. Then write a follow-up
column! And then readers of the Stranger will be better informed
about statistics than many working scientists!
In interpreting the results of an experiment, you have to ask not
just is it "1 in 3 statistics", "1 in 20 statistics" or "1 in 200
statistics". You also have to ask, "does my hypothesis pass the
giggle test?" So for instance, if you take two vaccines which
each been very very thoroughly shown to be ineffective when used
individually, and then you do a study where you give people a
combo shot, you have to ask yourself "when has a combo shot
_ever_ worked, in the history of immunology, in those case when
each individual one doesn't work at all? Or am I instead looking
for the equivalent of an exotic two-headed coin?" If the latter,
then designing an experiment to yield only two-sigma statistics
is a huge waste of money.
2
What I(@1) left unclear in the second paragraph is that you _know_ ahead of the time that the bag of 1000 pennies has been "salted" with one and only one crooked penny.
3
@1 If they can't answer your question correctly I would revoke their scientist cred.
What Science forgot to tell the public is that there are three types of lies: lies, damned lies and statistics.
What Science forgot to tell the public is that there are three types of lies: lies, damned lies and statistics.
4
@3: Ain't nothing wrong with statistics. I love statistics. They just need to be interpreted with thought and care, is all. Science's column on estimating noise above is an excellent, simplified treatment of a complicated subject, and I came down on it way too hard. I wouldn't have been so grumpy had he not used as his point of departure an HIV the reporting of which he kind of messed up a few weeks ago. It was a rare Golub blunder. He is actually one of my favorite science blogger/journalists.
6
Oh boy. I won't hold it against you, Mr. Golob, but it sure stings to see yet another mis-interpretation of significance testing.
I would go on a tirade a la post #1 about what significance testing actually gets you in these types of studies (not much), but it's much easier to just say that this interpretation is inaccurate and unhelpful and then link to an articulate, fun to read, published version of the tirade I would have made:
http://library.mpib-berlin.mpg.de/ft/gg/…
I would go on a tirade a la post #1 about what significance testing actually gets you in these types of studies (not much), but it's much easier to just say that this interpretation is inaccurate and unhelpful and then link to an articulate, fun to read, published version of the tirade I would have made:
http://library.mpib-berlin.mpg.de/ft/gg/…
7
Oh boy. I won't hold it against you, Mr. Golob, but it sure stings to see yet another mis-interpretation of significance testing.
I would go on a tirade a la post #1 about what significance testing actually gets you in these types of studies (not much), but it's much easier to just say that this interpretation is inaccurate and unhelpful and then link to an articulate, fun to read, published version of the tirade I would have made:
http://library.mpib-berlin.mpg.de/ft/gg/…
I would go on a tirade a la post #1 about what significance testing actually gets you in these types of studies (not much), but it's much easier to just say that this interpretation is inaccurate and unhelpful and then link to an articulate, fun to read, published version of the tirade I would have made:
http://library.mpib-berlin.mpg.de/ft/gg/…
8
Eric, I'm delighted that you affirm my (granted, biased - I work in JILA) stereotype of Boulder as a bastion of scientific literacy. I'll make the semi-informed prediction that there's about a 40% chance you fall into the category of Boulderites that do research at NIST, NOAA, or CU (though only a 0.0044% chance you're Eric Cornell).
I'll send my Mathematica notebook if you need to check the work.
I'll send my Mathematica notebook if you need to check the work.
9
Eric, I'm delighted that you affirm my (granted, biased - I work in JILA) perception that Boulder is a bastion of scientific literacy. I'll make the semi-informed prediction that there's about a 51% chance you belong to the population of Boulderites that does/did research at NIST, NOAA, or CU (though only a 0.0057% chance you're Eric Cornell).
I'll send my .nb if you need to check the numbers.
I'll send my .nb if you need to check the numbers.
10
but I still can't use the internet. well, the second one uses better population numbers, anyways.
11
cripes, forget it. I have no idea which of those comments actually got posted, as either my connection or thestranger.com is going fucking nutter.
12
When doing this kind of science, it comes down to genetic homogeniety as well. Some people are just genetically immune to things, even HIV.
13
I attended a lecture about how the point where scientists decide that something is "statistically significant" is arbitrary, and doesn't necessarily mirror reality. The problems the guy brought up were similar to those that #1 brought up, but he went further to suggest that even some studies that turn out "statistically insignificant" could still be more accurate than studies whose numbers are considered significant.
14
THE CURE for HIV/AIDS.......AMBUSH
THE IDEA that AMBUSH cures AIDS
is being proven by the more than 400 individuals who have taken a dose of 60 ml three times daily for 21 days. The result is that AMBUSH 'KILLS' the virus by causing the protein envelope to rupture and the viral particles are discarded by the white blood cells. AMBUSH is able to 'KILL' the virus that are 'hiding' in the lymph system by its 'natural radioactive' properties. This process allows the body to 'return to normal health' with a corresponding immunity to that or those strains of the virus.
What is AMBUSH ?
AMBUSH is a radioactive isotope of uranium that is found in the 'palm' plant of which there are more than 3000 species. When ingested, AMBUSH causes the body temperature in the trunk area to rise to about 102 degrees when the individual is sleeping. The preparation takes four hours per batch, which is then given to the individuals for consumption 60 ml three times daily for 21 days. AMBUSH is a herbal preparation in this form but it contains an active ingredient which is a 'NEW' crystalline substance, a drug from the 'palm plant' similarly to ASPIRIN originating from the willow tree bark
RESULTS:
After 21 days on AMBUSH, ALL the individuals experienced a decrease in viral load to undetectable, an increase in cd4, increase in RBC, an improvement in general health such as more color to the face, decrease in Buffalo hump, an increase in gluteal muscles, a decrease to having no joint pains whereby individuals can bend to touch their toes, and walk up steps are but a few examples. There is also a dramatic increase in their sexual appetite beginning after the first week of therapy
DISCUSSION:
In any plant concoction such as percolated 'tea', there are 30-40,000 compounds, whi ch would take the scientific community twenty years to isolate one particular ingredient if they knew what they were looking for. The LORD GOD has given me seven steps to isolate the active ingredient, which is soft and metallic in nature and has a carbon- uranium-sulfur-(classified)-phentolamine configuration or structure. This is similar to Federick Kekule and the discovery of the benzene ring where he dreamt the structure.
As an antiviral and 'natural radioactivity' producing agent, AMBUSH is also effective against leukemia, lupus and HPV. Here I am saying that I have 'GIVEN' AMBUSH in the same 'strength' and dosage to patients with leukemia, lupus and HPV. A 35 year old male with HIV found it difficult to impossible to urinate was put on 'green tea' and water while the doctors contemplated prostrate surgery. One of the doctors gave him my number , I sent him a supply of AMBUSH an d he has not been given any more ARV's, since taking AMBUSH 18 months ago, is in 'good' health and has expressed a willingness to be examined by HIV investigators like many others who have taken AMBUSH.
I have sent this 'IDEA' to most HIV research agencies, scientist of the field, universities, hospitals, clinics, politicians and news agencies to which it is REJECTED because the name of THE LORD GOD is mentioned. He has steered me scientifically through the processes such as which plant and how to produce the active ingredient. What are the odds of a Florida Pharmacist picking a plant would contain the CURE for HIV/AIDS ?
I have never charged any of the people for their supply of AMBUSH but a life saving has been spent on the project with NO renumeration from any sources because AMBUSH falls outside the walls of modern medicine and research.
PROPOSAL:
My proposal is that I PROVE that AMBUSH CURES HIV/AIDS by giving it to a number of END-STAGE or DRUG-RESISTANT people and the scientific community watches their recovery. This proposal addresses the problem in that I have already outlaid the results to be obtained.
This IDEA is unconventional in that the scientific community has rejected AMBUSH because I say it is GOD given. Secondly if I wrote it according to certain standards, then it might be peer reviewed. However, THE LORD GOD has also shown me that there are five enzyme systems associated with the virus, reverse transcriptase, protease, fusion and two more of which causes the virus to be AIRBOURNE. This means that without DIVINE intervention mankind and ALL warm- blooded mammals will be extinct in a number of years.
The PROOF of what I am saying is found in scientific papers wherein it is found that when the protease cuts the viral strands, it cuts it at DIFFERENT lengths EVERY time, to which it should always be a valine at the end but is a different amino acid every time. This is why it is IMPOSSIBLE to produce a VACCINE.
Since this is NOT a hypothesis but there are about 400 individuals who have taken AMBUSH, here lies a vast area in which to check, recheck and confirm that AMBUSH CURES AIDS. Let it be mentioned that during the HIV reproductive cycle, reverse transcriptase converts viral RNA into DNA compatible to human genetic materials. Thus the human DNA has been 'hijacked' and since each person has a DIFFERENT DNA, then the new viral copy is unique to that person which shows that each individual has a DIFFERENT STRAIN of the virus. Consider two HIV positive people swapping viral strains and increasing its complexity with multiple partners.
It can also be proposed that they be revisited as proof that the strain or strains that they had were 'killed' at the time of taking AMBUSH considering that a person can catch as many different strains as there are people who are infected by HIV.
I am also willing to work with the scientific community in identifying those individuals who took AMBUSH and wish to be identified with this process notwithstanding that some are stigmatized while others are jubilant,
Once AMBUSH is verified as being able to accomplish that which is aforementioned then the next stage might be the natural and artificial synthesis of the substance.
Finally, if this is accepted or not, believed or not, THE LORD GOD always wins and this is the heavenly truth to which AMBUSH was divinely given to mankind for the CURE of HIV/AIDS and it will be here forever. Apostle Shada Mishe.
apostleshadamishe@gmail.com
Here is a video taped presentation that I gave at t he Martin Luther King library in Washington
http://www.youtube.com/watch?v=8V53D1w__…
http://www.youtube.com/watch?v=vPwuwlVBO…
http://www.youtube.com/watch?v=ZejptOwMT…
http://www.youtube.com/watch?v=CqcTgIAhr…
http://www.youtube.com/watch?v=f7HPKcT_i…
http://www.youtube.com/watch?v=W9iQfgiYA…
http://www.youtube.com/watch?v=i3RzRS6tJ…
More...
THE CURE for HIV/AIDS.......AMBUSH
THE IDEA that AMBUSH cures AIDS
is being proven by the more than 400 individuals who have taken a dose of 60 ml three times daily for 21 days. The result is that AMBUSH 'KILLS' the virus by causing the protein envelope to rupture and the viral particles are discarded by the white blood cells. AMBUSH is able to 'KILL' the virus that are 'hiding' in the lymph system by its 'natural radioactive' properties. This process allows the body to 'return to normal health' with a corresponding immunity to that or those strains of the virus.
What is AMBUSH ?
AMBUSH is a radioactive isotope of uranium that is found in the 'palm' plant of which there are more than 3000 species. When ingested, AMBUSH causes the body temperature in the trunk area to rise to about 102 degrees when the individual is sleeping. The preparation takes four hours per batch, which is then given to the individuals for consumption 60 ml three times daily for 21 days. AMBUSH is a herbal preparation in this form but it contains an active ingredient which is a 'NEW' crystalline substance, a drug from the 'palm plant' similarly to ASPIRIN originating from the willow tree bark
RESULTS:
After 21 days on AMBUSH, ALL the individuals experienced a decrease in viral load to undetectable, an increase in cd4, increase in RBC, an improvement in general health such as more color to the face, decrease in Buffalo hump, an increase in gluteal muscles, a decrease to having no joint pains whereby individuals can bend to touch their toes, and walk up steps are but a few examples. There is also a dramatic increase in their sexual appetite beginning after the first week of therapy
DISCUSSION:
In any plant concoction such as percolated 'tea', there are 30-40,000 compounds, whi ch would take the scientific community twenty years to isolate one particular ingredient if they knew what they were looking for. The LORD GOD has given me seven steps to isolate the active ingredient, which is soft and metallic in nature and has a carbon- uranium-sulfur-(classified)-phentolamine configuration or structure. This is similar to Federick Kekule and the discovery of the benzene ring where he dreamt the structure.
As an antiviral and 'natural radioactivity' producing agent, AMBUSH is also effective against leukemia, lupus and HPV. Here I am saying that I have 'GIVEN' AMBUSH in the same 'strength' and dosage to patients with leukemia, lupus and HPV. A 35 year old male with HIV found it difficult to impossible to urinate was put on 'green tea' and water while the doctors contemplated prostrate surgery. One of the doctors gave him my number , I sent him a supply of AMBUSH an d he has not been given any more ARV's, since taking AMBUSH 18 months ago, is in 'good' health and has expressed a willingness to be examined by HIV investigators like many others who have taken AMBUSH.
I have sent this 'IDEA' to most HIV research agencies, scientist of the field, universities, hospitals, clinics, politicians and news agencies to which it is REJECTED because the name of THE LORD GOD is mentioned. He has steered me scientifically through the processes such as which plant and how to produce the active ingredient. What are the odds of a Florida Pharmacist picking a plant would contain the CURE for HIV/AIDS ?
I have never charged any of the people for their supply of AMBUSH but a life saving has been spent on the project with NO renumeration from any sources because AMBUSH falls outside the walls of modern medicine and research.
PROPOSAL:
My proposal is that I PROVE that AMBUSH CURES HIV/AIDS by giving it to a number of END-STAGE or DRUG-RESISTANT people and the scientific community watches their recovery. This proposal addresses the problem in that I have already outlaid the results to be obtained.
This IDEA is unconventional in that the scientific community has rejected AMBUSH because I say it is GOD given. Secondly if I wrote it according to certain standards, then it might be peer reviewed. However, THE LORD GOD has also shown me that there are five enzyme systems associated with the virus, reverse transcriptase, protease, fusion and two more of which causes the virus to be AIRBOURNE. This means that without DIVINE intervention mankind and ALL warm- blooded mammals will be extinct in a number of years.
The PROOF of what I am saying is found in scientific papers wherein it is found that when the protease cuts the viral strands, it cuts it at DIFFERENT lengths EVERY time, to which it should always be a valine at the end but is a different amino acid every time. This is why it is IMPOSSIBLE to produce a VACCINE.
Since this is NOT a hypothesis but there are about 400 individuals who have taken AMBUSH, here lies a vast area in which to check, recheck and confirm that AMBUSH CURES AIDS. Let it be mentioned that during the HIV reproductive cycle, reverse transcriptase converts viral RNA into DNA compatible to human genetic materials. Thus the human DNA has been 'hijacked' and since each person has a DIFFERENT DNA, then the new viral copy is unique to that person which shows that each individual has a DIFFERENT STRAIN of the virus. Consider two HIV positive people swapping viral strains and increasing its complexity with multiple partners.
It can also be proposed that they be revisited as proof that the strain or strains that they had were 'killed' at the time of taking AMBUSH considering that a person can catch as many different strains as there are people who are infected by HIV.
I am also willing to work with the scientific community in identifying those individuals who took AMBUSH and wish to be identified with this process notwithstanding that some are stigmatized while others are jubilant,
Once AMBUSH is verified as being able to accomplish that which is aforementioned then the next stage might be the natural and artificial synthesis of the substance.
Finally, if this is accepted or not, believed or not, THE LORD GOD always wins and this is the heavenly truth to which AMBUSH was divinely given to mankind for the CURE of HIV/AIDS and it will be here forever. Apostle Shada Mishe.
apostleshadamishe@gmail.com
Here is a video taped presentation that I gave at t he Martin Luther King library in Washington
http://www.youtube.com/watch?v=8V53D1w__…
http://www.youtube.com/watch?v=vPwuwlVBO…
http://www.youtube.com/watch?v=ZejptOwMT…
http://www.youtube.com/watch?v=CqcTgIAhr…
http://www.youtube.com/watch?v=f7HPKcT_i…
http://www.youtube.com/watch?v=W9iQfgiYA…
http://www.youtube.com/watch?v=i3RzRS6tJ…
More Dear Science
RSS
Facebook
Twitter
Comments (14) RSS