Stories tagged scientific method


I'm both a linguist by training and a puzzle enthusiast by inclination, so this story in the New York Times caught my eye. Um, so to speak. The Copiale cipher is an 18th century docment found in Germany, written with a set of symbols that include Roman characters (the ordinary letters that are used to write many European languages, including English), and other symbols that included Greek letters and abstract designs. The text has now been shown to describe an initiation ceremony for a secret society centered around eyes. Eye: The Copiale cipher talks about eyes, but the eye also symbolizes observation and discovery.  See what I did there?
Eye: The Copiale cipher talks about eyes, but the eye also symbolizes observation and discovery. See what I did there?Courtesy Thomas Tolkien

The story about the codebreaking itself is fascinating and worth a read. The cipher was cracked by a team of linguists and computer scientists with the help of software that's been developed for machine translation, which is the automatic translation of a text from one language to another, for example Russian to Japanese. Machine translation, and the problems that it presents, is fascinating in itself. But what really got me thinking as I read the researchers' paper
is the parallels between the process of deciphering this unknown document and the procedure that scientists everywhere go through in making discoveries about the natural world.

The scientific method is a series of steps intended to lead to explanations of previously unknown phenomena. Generally, the method consists of four parts:

1. Noticing something;
2. Making a hypothesis, or educated guess, that explains what you notice;
3. Predicting what will happen if you test your educated guess; and
4. Testing the guess to see if you're right.

If your guess was right, you can make more predictions and do more experiments. If you're wrong, you can learn from that to make a new guess, and then continue with the process. Sometimes being wrong is the most valuable part of the process, because knowing what *isn't* happening can help you discover what *is*.

The researchers working on the Copiale cipher made a lot of wrong guesses at first. For example, they predicted that the Roman letters held the information and the others were filler, when the opposite was true. Still, they might never have found meaning in the other symbols if they hadn't started with a reasonable, though ultimately wrong, hypothesis.

In a way, every scientist is a codebreaker. Information about the natural world, about biology, astronomy, sociology, or any other field of science is about finding the information hidden deep within the phenomena that we observe as we look at the things around us. To decipher it, all we need is the desire to know what's behind the code, and the patience to keep trying until we figure it out.

The Boston Globe reports on a group of political scientists and more classical scientists alike, who suggest that we should submit potential legal changes up for scientific randomized trails, like how the FDA tests new drugs. Would you submit to experimentation in the law lab?


Missing Link - Not

Darwinius masillae
Darwinius masillaeCourtesy University of Oslo
Perhaps taking advantage of the Darwin publicity last year (200th birthday), a scientific paper was published revealing Ida, a 47 million year old fossil classified Darwinius masillae.
The study's lead author, Jørn Hurum of the University of Oslo, variously called the fossil the holy grail of paleontology and the lost ark of archeology. A two-hour documentary called "The Link" was on the History Channel and a book with the same title hit bookstores.

One million dollars

How big money became mixed with science is described in the Guardian post titled Deal in Hamburg bar led scientist to Ida fossil, the 'eighth wonder of the world'.
Now that money has been made, it is time for the scientific process (peer review).

John Fleagle, a professor at Stony Brook University, in New York state, who reviewed the paper for the journal, agrees that the fossil is not a lemur. But Ida's full significance would not be known until other scientists had seen the paper. "That will be sorted out, or at least debated extensively, in the coming years."

Confirmed: Fossil Ida is not a human ancestor

In a paper in the Journal of Human Evolution, Chris Kirk strongly argue(d) that Darwinius is not one of our ancestors. Science blogger, Brian Switek, also explains why ... That "Ida" is Not Our Great-Great-Great-Great-Etc. Grandmother. Dissenting scientists are awaiting a response from Jørn Hurum.

How science should be done

I am reminded of another case where the media was used to hype a story before it was properly reviewed by others. I wrote about it here: Jesus and family found in tomb? What moral is to be learned here?

Don't announce discoveries through the media, but through the tried and tested peer-review process.


Ibex: Capra ibex
Ibex: Capra ibexCourtesy Nino Barbieri
A recent article in the Journal of Archaeological Science reminded me of the importance of the Scientific Method Often we hear new and exciting scientific theories that seem plausible, especially if these ideas are presented in prestigious journals. However, the beauty of the Scientific Method is its verifiability, whether or not the data can be recreated through repetitive testing (If we truly believed everything the first time, our budding young scientists would have nothing to do!)

Michael Campana from the University of Cambridge and colleagues from across the UK and Ireland recently ran a sequence of DNA tests on 18th and 19th century parchments made from animal skins in order to reveal the complexities of ancient parchment analysis. Parchment is one of the most valuable archaeological and historical artifacts that can be used to understand not only language and history, but DNA testing on it can reveal clues to animal population studies, animal husbandry, different historical animal breeds, and provenance (where the animal or skins originated from). In the case of the Dead Sea Scrolls, DNA testing on the parchment could reveal what type of animal was used and possibly where it came from, providing additional data for questions regarding who wrote the scrolls.

Campana and colleagues analyzed both mitochondrial and autosomal genetic data using stable isotope, genetic, phylogenetic and ion beam analysis. All samples were considered to be well preserved and ideal samples for accurate testing. All but one parchment produced multiple DNA sequences that matched several different species including cow, goat, sheep, and even human. In other words, a parchment assumed to be made from one individual of one species, gave conflicting results as more than one species or more than one individual. Of course it can be assumed the parchment was not made of human skin and therefore human genetic data must have came from handling and processing of the parchment, but parchments can also be contaminated in long-term storage or contact with each other. Testing results can also be skewed by glues and inks or other preparatory treatments used to improve the surface. All of these factors need to be considered when testing truly ancient parchment like the Dead Sea Scrolls.

Previous DNA test results from 2001 and 1996 on the Dead Sea Scrolls produced results pointing to a single species, either ibex (Capra ibex) or domestic goat. While these results may indeed be correct, the likelihood that the results were so exact, when testing such as Campana's and colleagues on better preserved and more recent parchment were so complex, questions the accuracy of the earlier DNA testing. Of course we must not forget, precious artifacts like the Dead Sea Scrolls can not be needlessly dissected to offer unlimited samples for DNA testing labs. But as, Campana states, “Improving our understanding of parchment's DNA content would allow us to develop a predictive model for sampling of historic manuscripts.”

So the messages for today, bravo for the Scientific Method and go see the Dead Sea Scrolls at the Science Museum! Learn the science, archaeology, history and more that surround these amazing artifacts. Ask questions like: did the scroll writers choose ibex for some scrolls over goat because they thought these documents were so special or was ibex as readily available as any other animal species? Did the handling of the scrolls by shepherds who supposedly found them contaminate the actual scroll DNA with sheep, human or goat DNA? What can DNA testing tell us about other ancient artifacts? As long as there are unanswered questions, no matter how small, there will be a need for scientific investigation; which is good news for our future scientists!


There is this chart that we found, and it's quite interesting. It breaks down the process of science in general. This chart also help you understand how the world connects, once you put it in a certain order by the chart. It goes from a broad idea and once you hover over it it goes into detail. For example a teenager had to create a lab and not a clue what is expected of it, so then she can locate the chart and see testing ideas. which then lead to the other processes and provide her with her needs for the lab... . Below is the website and check it out!


A display on adult stem cells, here at the SMM: In fact, this exhibit features Catherine Verfaillie herself.    (Good looking out, BK)
A display on adult stem cells, here at the SMM: In fact, this exhibit features Catherine Verfaillie herself. (Good looking out, BK)Courtesy bryankennedy
Following the results of an evaluation by a panel of experts at the University of Minnesota, the magazine New Scientist published an article last week announcing that some of the data used in a groundbreaking study on adult stem cells had been falsified.

The study, performed at the University of Minnesota under the supervision of Catherine Verfaillie, is part of a line of research that seemed to indicate that adult stem cells, taken from bone marrow, are pluripotent—that is that they have the potential to develop into any type of cell. Previously, only embryonic stem cells were thought to be pluripotent, and Verfaillie’s research looked like it could eventually offer an alternative to the ethically complicated use of embryonic cells for research (which requires the destruction of an embryo).

Unfortunately, other scientists had trouble replicating Verfaillie’s results, which were published in the journal Nature. New Scientist began examining the research done by Verfaillie and her team, and found that key images in the research appeared several times in papers for different experiments, and, in the case of a related study in the publication Blood, were used twice in the same paper, but had been visually altered slightly, and flipped 180 degrees. New Scientist reported their findings to the University, which began a formal investigation of the matter.

The University just recently completed the investigation, and found that data in the blood article had indeed been falsified (the images in particular), by a former PhD student of Verfaillies’, Morayma Reyes. The University and Catherine Verfaillie have asked Blood to redact the study.

Verfaillie has stated that she was unaware of the problems with the published study, and while she didn’t believe that the data was deliberately falsified, she takes ultimate responsibility for the errors.

Reyes, who now works as an assistant professor at the University of Washington, denies that the images represent deliberately altered data, and blames the errors on inadequate supervision and training. She claims that she had neither the equipment (photo editing software) nor knowledge required to alter the images. The differences in the reoccurring images were likely the result of the inadvertent use of the image adjusting tools built into lab equipment, she says, and the duplication of a figure within the Blood paper was accidental. Reyes also feels that she has been treated unfairly by the University, and that the expert panel in the investigation demonstrated a clear “lack of expertise” in the field of stem cell biology.

Reyes’ full position can be read here. The University’s response can be found here.

The altered images, Reyes asserts, shouldn’t change the results of the paper, but the whole incident brings up some interesting issues on the process of vetting science. While the errors in the paper never should have made it past Verfaillie and the rest of her team, the process of peer review should have caught them anyway. Generally, before research is published in a scientific journal, the editors select several scientists in the particular field of the paper to evaluate and comment (often anonymously) on the paper. The review panel is meant to confirm that the methodology of the experiments and the interpretation of the results are sound. Research can then be recommended (or not) for publication.

Publishing research essentially formally submits it to the scientific community, and it’s common for other scientists to attempt to replicate experiments, especially if a study makes particularly striking claims (like adult stem cells being pluripotent). The work of other scientists in replicating results is, obviously, essential to the scientific method—in this case is was what finally drew attention to some of the irregularities in Verfaillie’s team’s work.

Reproducibility can be a tricky thing, though—difficulty in repeating results doesn’t necessarily mean that they aren’t reproducible. (Here’s a good article on repeating and reproducing results.) But the problems in reproducing these results drew attention to the questionable data, which brought up another aspect of scientific vetting: the University’s investigation into academic misconduct. If the problems with reproducibility seem to come from data being changed, added, or omitted to strengthen a conclusion, then there could be a serious problem. This sort of misconduct undermines scientific progress, and can call into question the reputation of the institution it came out of and the validity of other research performed there. And if Morayma Reyes seems a little extra defensive in her letter, it’s understandable, because being accused of academic misconduct is a big deal, and no good for your career and future work.

The subject of the research further complicates the situation—this isn’t the first time issues of academic dishonesty have come up with regards to stem cell research. In 2006, a Korean scientist’s claims that he had cloned human embryos (thereby eliminating the need to destroy new embryos for stem cells) turned out to be based on lies. There’s a fear that the potentially huge medical payoff of stem cell research, as well as the ethical debate surrounding the use of human embryonic stem cells, could lead to science that is less than completely thorough, or even situations like the Korean controversy. And that’s bad for science in general. There’s also the thought that errors that are unintentional (as may be the case with Reyes’ images) could be the result of “pathological science,” where results are steered in a particular direction by scientists because of “subjective effects, wishful thinking, or threshold interactions.” It doesn’t have the same ethical problems, but pathological results aren’t a whole lot better for science than straight-out misconduct, and it’s a serious potential pitfall with the benefits of stem cell research waiting out there as temptations.

So there you go. It looks like things are, for the most part, being handled appropriately in this situation, but it’s an interesting window into scientific process.

Any thoughts? Does it seem like the vetting process of science is lacking in some way? Or is it maybe too thorough? Professor Reyes, I imagine, would argue that too much has been made of this situation, and there are many who argue that the process of peer review limits the communication and dissemination of scientific ideas.

Or, even better, does it seem like I got something wrong here?

Let’s have it, Buzzketeers.

The process of science is often portrayed as deliberate and successful. However, it's good to remember that discovering things about our natural world can often be trying and full of failure. In an effort to understand how walruses near Greenland migrate, eight of the animals were satellite tagged. Alas, all but one tag has now stopped working or has fallen off its walrus. Track the one remaining walrus.


Last fall I attended a talk by one of the other students at my university (Harvard). He was discussing recent results from a perception experiment he had posted online. He said he had over a thousand subjects. "How long have you had this experiment online," I asked him. "Just over a week," he responded.

"Holy crap!" I thought. There are many experiments I would love to do except they require hundreds or thousands of subjects -- something that just isn't feasible in a traditional laboratory setting. So I started the Visual Cognition Online Laboratory. I am getting respectable traffic after one week, but it's going to take a while before I am getting 1,000 participants per week, which is what I need.

Most experiments, I should say, are surveys. What this grad student and I are doing is putting up actual perception experiments, which are always done in the lab. Most researchers believe you need strong controls in timing, display, etc., in order to do perception experiments. For some, this is true, but there are many you can do online given how much bandwidth there is now. Also, if you have enough subjects, that extra noise will wash out.

If you are interested in trying out one of my experiments, they typically take 5 minutes or less.


Global Warming: NASA photo of Earth taken from Apollo 17
Global Warming: NASA photo of Earth taken from Apollo 17

Have you been following the comments in Cari's Buzz Blog post about Gore's movie, An Inconvenient Truth. I find it interesting that very intelligent people can look at the same data ("truths") and yet totally disagree as to what to accept as reality?

A Global Warming Skeptic Challenge

George Musser on the Scientific American Blog has been moderating a discussion about global warming titled Are You a Global Warming Skeptic? Part IV He started the discussion March 6, 2006 with this statement:

In the comments field, explain which aspects of climate change you don't accept (e.g. you might not think Earth is warming at all, you might not think the warming is due to greenhouse gases, you might not think that the gases are produced by humans, or you might not think warming will cause trouble in the future), what exactly has led you to this conclusion, and -- most important -- what it would take to convince you otherwise. Let's get everything out into the open, so that we can have a real discussion.

The discussion is presented in four parts with hundreds of comments. I am recommending this thread because Musser first listens to, then presents a summary of the skeptics' arguments. I find the fairest way to make up my mind on an issue is to thoroughly understand both sides of the argument. Musser explains how scientists crunch the various data to answer difficult questions. He uses the analogy of examining fingerprints during a crime scene investigation.

Climatologists have maps and time series showing how a boatload of climate variables -- mean temperature, temperature ranges, air pressure, precipitation, and so on -- vary in time and in space, horizontally across the surface and vertically through the atmosphere. These data sets are a gold mine for resolving ambiguity, because the different forcings leave distinct fingerprints. Such patterns make it possible to tease out their relative contributions. Over the years, researchers have considered ever more variables besides temperature and ever more forcings besides greenhouse gases. They have merged spatial and temporal patterns, looked at regional as well as global scales, and developed more sophisticated mathematical tools.

View from the Crime Scene

"Fingerprints" included solar variability, volcanic eruptions, greenhouse gasses, ozone, aeosols, and various generic effects. When climatologists run the fingerprinting analysis for different historical epochs, they find that temperature fluctuations prior to the Industrial Revolution were driven primarily by solar and volcanic forcings. In the early 20th century, natural and anthropogenic forcings seem to contribute equally. From midcentury onwards, greenhouse gases rule(temporal pattern). Since 1979, when continuous satellites observations began, the surface and troposphere have warmed and the stratosphere has cooled(vertical pattern). Pretty much the entire surface has gotten warmer, high latitudes more than lower ones(horizontal pattern). All the oceans have warmed; there isn't the zero-sum game of warming and cooling you'd expect from natural variability(energy variability).

Musser concludes,

"unless I'm missing something, it seems to me that the case for anthropogenic warming is pretty strong...Based on the knowledge we have so far, however, I have to call 'em as I see 'em."

To appreciate the use of critical thinking and scientific method I recommend wading through the four installments of "Are you a global warming skeptic?" Part I (673 words); Part II (2219 words); Part III (2617 words); Part IV (3516 words); and an Appendix