ONS Friendly Labs chemistry

Following up on some discussions with Jeremiah Faith and Cameron Neylon, I thought it might be useful to set up a place on the NodalPoint wiki to list PI policies on Open Notebook Science (and other forms of Open Science more broadly) in their labs.

Cameron quickly followed up with a Dabble application to do something similar.

I think there is a need for such "match-up services". Students and postdocs who feel strongly about sharing their research openly would certainly prefer working in laboratories where that type of thing is tolerated, if not outrightly encouraged or required.

Similarly, PI's who feel strongly about Open Science would be thrilled to find students and postdocs sharing their mindset.

Check out (and contribute) to the Nodalpoint and Dabble initiatives if you can.

Experimental Uncertainty Principlem chemistry

Most of us are familiar with the mantra of how science progresses:

A hypothesis can never be completely proved by any finite set of experiments but it can be falsified by a single result.

In mathematical proofs, clear cut algorithms can usually be applied to prove unequivocally the falsehood of a theorem (notwithstanding Godel's incompleteness theorems :)

But in real research in the physical sciences, that is not exactly how scientists process reports of experimental results. And an important reason is the way results are reported.

Lets pick an example from the Open Access Beilstein Journal of Organic Chemistry.

Here is the full description of the experiment from the supplementary materials page:

To a solution of 5a (196 mg, 0.433 mmol) in CH2Cl2 (1.8 mL) was added p-toluenesulfonic acid (19 mg, 0.11 mmol). After stirring for 0.5 hours at 0 oC, the mixture was concentrated under reduced pressure and purified by flash chromatography on silica gel (eluent: ethyl acetate: P. E. = 1: 3) to provide 7 (152 mg, 100%) as a colorless oil.

I have omitted the characterization information. Lets assume for the moment that it is completely correct.

The question is : if 10 chemists follow this procedure as described, will they get 100% yield of pure product?

I think that it is quite possible that the results will vary wildly, including many complete failures. Here is why:

1) The reaction is carried out at 0 deg C for 30 minutes but the conditions of the work-up are completely unspecified. We don't know the pressure, the temperature of the bath or the duration of the solvent evaporation. The temperature of the rotovap bath will vary wildly from lab to lab, depending on vacuum pressure and personal preference. This is key because the conditions of the work-up (warmer and more concentrated) are much harsher than the reported reaction condition. My guess is that when this gets indexed in a database the reaction conditions will be further stripped of detail and likely end up as 0 C, 30 min.

2) The chromatography step does not specify how much silica to use, the dimensions of the column, the number of fractions, the TLC images of the fractions, the amount of solvent used to load the reaction mixture, etc. It may even be the case that the ratio of solvents was changed over the course of the chromatography - in a situation like this some would use a good solvent like methylene chloride to load the mixture then chase it with a solvent mixture containing a lower ethyl acetate/petroleum ether ratio.

3) A 100% isolated yield after chromatography means that not a single milligram was lost during transfer to the column and that all fractions containing the product were very pure. Ethyl acetate is notorious for increasing apparent product yields because it is sometimes difficult to remove on the vacuum pump. I would like to see the NMRs of the fractions.

This last point also brings up the issue of what the researcher does when confronted with an apparent 101% yield - since this is not chemically plausible it cannot be reported as such. Does the researcher state an assumption that there is a bit of extra solvent and slice off a milligram in the report? We can't tell from the information given in journals.

I want to make it clear that I am not picking on the authors for reporting in this way. Within the current norms of the organic chemistry community, this is an acceptable way to report laboratory procedures in peer reviewed journals.

Of course all (or most) of these details should have been recorded in the laboratory notebook. I understand that initially protocols in papers were abbreviated to save on space. But now with unlimited online supplementary materials associated with papers, researchers could scan their notebooks and all associated documents. But that is not required by the chemistry journals that I know and I have not seen it done.

Keep in mind that this is not new work - researchers already have (or should have) all of this as a routine part of doing research. This is one big advantage of Open Notebook Science - very little extra effort required. (Cameron Neylon also has a very nice recent summary of his thoughts on this.)

Any chemist will tell you (if they are honest) that there is almost always a mistake, however small in every experiment. By everyone agreeing to report experiments in a highly abbreviated form, it makes it convenient to get done more quickly and get that all important paper out the door. Do you completely start over an experiment because you measure 101% apparent yield? Or do you realize that you just don't have time and take a "shortcut" of some type to get that paper out.

All of this would go away if we came clean about our experiments - the good, bad and the ugly. Lets stop pretending that we did the reaction EXACTLY as stated in published abbreviated protocol and we might start to get out of this quagmire.

We don't have to change the way we abbreviate experiments - just link to the relevant pages in the laboratory notebook in the supplementary sections of papers.

As chemists try to make sense of the physical world and process results from other researchers, they have to evaluate the meaning of experiments published like this. Instead of processing the information algorithmically, they apply fuzzy logic: more weight is given to results with more proof.

With the limited information provided in this particular experimental description, I would expect that it is possible to get this reaction to work in good yield but I would not question the fundamental laws of nature if some chemists report that it fails completely. If I had access to the laboratory notebook and all raw data, including how the reaction was monitored, I would weigh the evidence of each report quite differently.

The more information one has about an experiment the more confidence one can place in the results. But it would never be possible to have complete confidence in any result, no matter how much information is provided. And because providing more information costs more in terms of time and money, a balance has to be struck.

We might call this the experimental uncertainty principle:

All experimental results are uncertain to some degree. Uncertainty can be reduced with more information but then fewer experiments can be performed with the same resources.

For example, an experiment like EXP064 provides extensive links to monitoring runs after each step in the reaction and provides evidence of the purity of the starting materials. By contrast EXP134 records 4 parallel reactions with only photographs as results. The purpose of the first experiment was to understand the Ugi reaction, while the second aims to quickly identify Ugi reagents that lead to easily purified products. When these reactions get compared, the second carries far less weight than the first - but we only know that by looking at the details in the notebook.

If we expect autonomous agents to contribute to the process of doing science (for example formulating and testing hypotheses), information has to be tagged in such a way that it incorporates a measure of uncertainty.

I suspect that it will be easier in many cases (like organic chemistry) to simply redo the experiment under known conditions rather than attempt to get hold of the original notebook.

Swarthmore Talk on Open Notebook Science chemistry

On Tuesday November 27, 2007 I had the pleasure of speaking at Swarthmore on our UsefulChem project and Open Notebook Science more generally.

Liz Evans and Cheryl Grood from the Swarthmore Sigma Xi Chapter did a wonderful job in rounding up people to have discussions both before and after my talk at dinner. This gave us an opportunity to share teaching experiences with new technologies (blogs, wikis, Second Life, etc.) - something I didn't really get into too deeply during my talk.

The timing was also quite fortunate because I was able to discuss an important new result from our lab (EXP148) obtained just a few days ago. (More on this shortly in a separate blog post).

I had some very thought-provoking conversations with both students and faculty. One of the recurring questions was what format Open Notebook Science would take in various scientific fields. Some disciplines, like mathematics, don't have formal laboratory notebooks like synthetic organic chemistry. But there are still ways of reporting daily progress.

The Rosania Lab Open Notebook Science Wiki chemistry

I recently reported on a new collaborator who agreed to work with us in the open on modelling subcellular drug transport.

I am very pleased to report that Gus Rosania has now created an entire wiki (1CellPK) for his lab to use as an open notebook. From the home page of the wiki:

Open Notebook Science is ideally suited for community-wide collaborative research projects involving mathematical modeling and computer simulation work, as it allows researchers to document model development in a step-by-step fashion, then link model prediction to experiments that test the model, and in turn, use feedback from experiments to evolve the model. By making our laboratory notebooks public, the evolutionary process of a model can be followed in its totality by the interested reader. Researchers from laboratories around the world can now follow the progress of our research day-to-day, borrow models at various stages of development, comment or advice on model developments, discuss experiments, ask questions, provide feedback, or otherwise contribute to the progress of science in any manner possible.

How's that for a Christmas present to the Open Science community?

Back from Science Bloging 2008 chemistry

Like last year, the North Carolina Science Blogging conference was a hit.

I moderated a session on public scientific data with Xan Gregg. Both of our talks were recorded and available here. (I used SciVee this time to store the screencast and was even able to use their supplementary document option to store the mp3 that Feedburner properly processed in the podcast feed.)

We spent about half the session with presenting then there was an active discussion. (Hopefully someone got some decent audio on that part and I'll post a link here if possible.) The usual issues of scalability, findability, fundability, scooping, academic validation and government policy came up. No resolution on any of these of course but that's not the point of the gathering. Some new contacts were made and maybe that will lead to some progress on the Open Science front.

Speaking of new contacts, I was quite pleased to meet Moshe Pritsker from JoVE. He said that his camera people would come to my lab to record some experiments. Any students in the UsefulChem lab who would like to get involved - lets discuss it.

Having the chance to touch base with friends like Bill Hooker, Deepak Singh and Antony Williams was certainly a bonus.

I also enjoyed Hemai Parthasarathy's session in the morning on Open Science. She did a great job in moderating the discussion, which was really a brainstorming session on how the scientific publication process could and should evolve.

Bora has a comprehensive list of pre and post-conference blogging, pics and videos.

Open Science Session at PSB chemistry

Shirley Wu is organizing a session at the Pacific Symposium on Biocomputing (PSB) 2009 in Hawaii.

"Open Science: tools, approaches, and implications”

She is looking for submissions, support and feedback:

So if you would like to support this proposal and are willing to commit to participating should it get accepted, please send me an email to that effect (with as many details of your anticipated participation as you can provide at this time), and I will include all the emails as "supplementary material" next Friday. Please also disseminate this call on your own blogs if you can. Many thanks in advance!

Open Notebook Science News - Barton and Rosania chemistry

Michael Barton has posted a brief essay on Open Notebook Science on his research web site:

As you might expect from the name, Open Notebook Science (ONS) has similarities with Open Source Software. The clearest likeness between the two, is the belief that by sharing and collaborating, more can be achieved than through secrecy and competition. An open approach to software development is proven to be successful: the greatest achievement is the development, and increasing adoption of the Linux operating system. On this foundation other applications like the Apache web server, MySQL database, and the PHP scripting language have been built, and the combination of the four is the engine running many websites, including this one. If ONS can enjoy a fraction of the success open software does, then science can only benefit.

He also discusses ONS on the February 2008 edition of Bio::Blogs.

In terms of an amazing example of recent ONS implementation, take a look at Gus Rosania's 1CellPK wiki. He currently has 9 of his group members with notebooks and he is trying to make the activities in his lab as transparent as possible. Obviously this involves experiments but also group meetings and his meetings with colleagues. He has also been providing detailed descriptions, including background and literature reviews, of his group's projects.

Of particular interest to my group is the description of our collaboration on new anti-malarial agents. Since we can track their activities and they can track ours in close to real time, it will be interesting to see if we can crack open all the black boxes of collaboration.

I have heard the objection many times that there is not enough time for researchers to read each other's lab notebooks. That's absolutely true but that is not an effective way to use these resources. The point is to spend little time skimming content and as much time as required drilling down to details when a relevant post is discovered. With Wikispaces one can also just subscribe by email or RSS feed to edits of a particular page. I would expect the UsefulChem group members to at least subscribe to the malaria project page I mentioned above.

Gus is also looking at displaying experimental results in Second Life and has been doing actual experiments on the physics of Second Life. See his blog for the chronicles of that adventure.

Open Medicine Editorial on Open Science chemistry

The Journal "Open Medicine" has published a very thoughtful editorial on "Open science, open access and open source software at Open Medicine" by Sally Murray, Stephen Choi, John Hoey, Claire Kendall, James Maskalyk and Anita Palepu.

Not only are they writing about it but they want to get their hands dirty as well:

Open Medicine is an open access journal because we believe that free and timely access to research results allows scientific knowledge to be used by all those who need it, not just those who can afford expensive journal subscriptions or user fees for individual articles. But is access to the final polished version of research enough? Could we do more to en­courage the collaborative reuse and reanalysis of existing data, or the verification of analyses? Could we move from open access to open science?

Expanding the UsefulChem Collaboration to Teaching Labs chemistry

A few weeks ago I received a very interesting email from Brent Friesen at Dominican University. He mused:

I am trying to put together a bridge between the type of opensource research you are instigating and the traditional Sophomore Organic Chemistry lab. There are over 4,000 college and universities in the United States - all of them teach Sophomore Organic Chemistry lab. How can we harness this resource? .....

SOC labs must fulfill 4 criteria:
1) inexpensive reagents and equipment
2) fit into the time constraints of 1 3-hour period per week.
3) Must be a robust reaction with fairly stable products. It doesn’t have to be “foolproof” but that helps.
4) Compatible with equipment, glassware, procedures that student know how to use and do.

Bottom Line: I am definitely interested in developing collaborative projects, especially if they can be performed as part of a Sophomore Organic Chemistry laboratory curriculum.

This is extremely encouraging news for open scientific collaborations and I am very impressed with Brent's initiative! It certainly is more work for him compared to maintaining the status quo.

Kevin Owens and I have discussed this possibility for some time now and he is willing to contribute by carrying out mass spectrometry if required.

Brent and I further discussed the applicability of the Ugi reaction that we perform in my lab because of its simplicity - 4 components are mixed together in methanol at room temperature and a Ugi product often precipitates within days, requiring only filtering to isolate. (see EXP150 for a good example)

However, one major limitation of the Ugi reaction for a teaching lab is the terrible stench of most isonitriles, one of the four key components. One way around this is to use isonitriles that don't stink, such as TOSMIC.

So as a starting point, we would like to start with a Ugi reaction which involves TOSMIC and has lead to a precipitate in our lab. There is only one example so far: 171H.



According to the lab notebook, all starting materials dissolved easily and a precipitate appeared after 2 days. The precipitate has not yet been isolated and characterized. Hopefully it will prove to be pure Ugi product, as other similar Ugi reactions have done.

It turns out that many of the top ranking compounds from Rajarshi's falcipain-2 docking run (V2) contain TOSMIC as the isonitrile. Phil Rosenthal at UCSF is still up for testing compounds for anti-malarial activity. Wouldn't it be a testament to the power of open collaborative science if a decent anti-malarial lead was uncovered through the routine teaching of undergraduate organic chemistry labs?

At the very least I'll bet it would be rewarding for the students involved.

Brent has placed the orders from Sigma-Aldrich and is moving full steam ahead. He recently wrote to me:

You know, I'm ready to dance and you are the only dance partner who seems to be ready and willing. Let's figure out a way to adapt the Ugi reaction to Sophomore Organic Chemistry laboratory and give it a try!

I would like to plan it for the week of April 7 and the following week...
Does not give us much time, but it can be done.

Keep track of Brent's activities on his blog

Attila Csordas writing his thesis on a blog chemistry

Attila is writing his thesis openly and is welcoming comments:

From now on I start every “thesis live” post with the standard introduction: In the live thesis building blogxperiment I edit (digest, compile, write, rewrite, delete) my ongoing doctoral thesis in blog posts and put the parts together on thesis live. The title: The physiologic role of stem cells in tissues with different regenerative potential

I am not aiming any perfection, my focus is clearly on getting things (the PhD) done here. Anyway, I found the idea of “writing” a complete, lengthy and formal thesis outdated and inefficient (after all, scientists should conduct nice experiments and publish their results in short, inforich and accessible research papers in order to share it ASAP with the research community, not in book-length, otherwise unaccessible PDFs) and so I try to keep myself motivated by

- doing this “thesis live” series as an open science experiment and getting useful feedback from my fellow scientists and readers

- trying to include as many systemic, whole body level material into it that could be relevant for systemic regmed approaches

- reminding myself every day that without a PhD it is hard to move further in science officially (that’s the least motivating factor though as it is official)

Cell Article on Open Drug Discovery chemistry

Seema Singh wrote a review "India Takes an Open Source Approach to Drug Discovery" which just appeared in Cell: Volume 133, Issue 2, 18 April 2008, Pages 201-203. (The doi doesn't work yet but try this link in the meantime). You'll need a subscription to view it, an increasingly familiar irony of much of the Open Science discussion these days.

UsefulChem and our collaborators got a nice mention:
A related initiative is UsefulChem (http://usefulchem.wikispaces.com/), set up by Drexel University chemist Jean-Claude Bradley. Bradley has pioneered Open Notebook Science in which lab notebooks and raw research data are posted on the web for anyone to see and respond to (http://usefulchem.wikispaces.com/All+Reactions). As for success, Bradley says, “Probably the best example of a positive outcome from UsefulChem is finding two compounds that are somewhat active against malaria [in vitro],” blocking the activity of falcipain-2, a Plasmodium falciparum cysteine protease. “This demonstrates that a team of researchers can work together in the open—Rajarshi Guha from Indiana University did the docking calculations, my group at Drexel did the syntheses and Phil Rosenthal's group at UCSF did the testing.”