Challenging Conventional Wisdom

Questioning widely-held assumptions about tech culture, education, and health has been the basis for several popular posts.

AI Beyond Elite Institutions

Machine learning isn’t just for those at billion dollar companies. These posts highlight unconventional practitioners and offer practical guidance for people in varied domains.

Debunking AI Hype & Holding Tech Accountable

The narratives about AI put forth by major tech companies are often misleading about what is necessary, what values matter, and what types of harms can result.

Immunology & Science

A new world and a powerful idea

You may wonder why so many of these health issues are on the rise, when viruses are nothing new. Our world has changed drastically in recent decades compared to most of human evolution. We live in a hyperconnected age of global mega-cities and record numbers of large international flights now. We spend our time indoors in crowded, poorly ventilated buildings. These factors have allowed viruses to travel faster and farther than ever before.

The misleadingly named “Hygiene Hypothesis” is often used to justify the misconception that all microbes are good for us. However, this theory is more accurately reframed as the “Old friends hypothesis”: humans co-evolved with friendly bacteria and some parasites. Viruses are not our friends, but rather enemies. We did not co-evolve with these crowd infections of mass travel, mega-cities, and indoor confines.

The idea that viruses are contributing so much to human suffering and long-term disease is powerful. It will transform how we approach medicine, health, and aging, if we let it. This revelation is one of the key reasons that I decided to make a mid-life career pivot, stepping back from fulfiling work in AI to return to graduate school in Microbiology-Immunology, a journey I have been chronicling here on my blog.

I hope to spend the next few decades applying my machine learning skills to problems at the intersection of infections, multi-omic data sets, the microbiome, and chronic disease. Below, I will share some of what has captured my attention and upended my old views on disease and medicine.

Viruses have many ways to wreak havoc

Viruses have evolved to evade, outmatch, commandeer, and otherwise hurt our immune systems. Here is an incomplete and overlapping list of ways that viruses can harm us:

The Paradigm Shift

For most viruses, people focus on just a few weeks of initial symptoms. This is the wrong way to think about infections. Viral meningitis or EBV increases your risk of Alzheimer’s or dementia, 5-15 years later. Chicken pox (varicella zoster virus) can reactivate decades afterwards as shingles, which itself then leads to increased risk of stroke for at least the following year. We need to radically change how we think about viruses.

There is much we still don’t know about the immune system. Early during the covid pandemic, many experts made definitive statements about the risks of covid, assuming that those who didn’t die in the first few weeks must be completely fine. However, perturbations from infections that initially seem minor can have far-reaching, long-lasting, and time-delayed impacts. There is a ton that is still unknown.

Trying to figure out how viruses hijack cell processes, alter microbiomes, and dysregulate the immune system are complex questions. Researching these areas with curiosity, determination, and an open mind will reveal a lot.

Reasons for Hope

It can be gloomy to think about all the damage viruses can cause. The good news is that we don’t have to resign ourselves to these outcomes. Facing the disturbing reality that many viruses are worse than we thought is just the first step towards coming up with creative new solutions. There are some bright, curious, and determined people focused on these problems, although we need even more hands and brains to get involved.

The breadth and depth of the harms caused by viruses can focus biomedical research in new directions. Most viruses do not have effective anti-viral treatments. This creates a huge need. Scientific inquiry can fail catastrophically when those closest to the problem are not included. Patient-led research gives me hope, because it is centered on the expertise of those closest to the problem. I am also optimistic about the use of AI for discovering new drugs and designing immune therapies.

On the prevention side, reducing how frequently people get sick will have a big impact. Different viruses spread in different ways. In recent years, we have learned that many infections are airborne. Healthy indoor air is a human right, like access to clean drinking water. The UN recently held a high-level event focused on the right to clean air. There are many measures we can take to reduce transmission of airborne diseases, such as improved ventilation, air purification, and far-UVC technologies. Parliament houses, venues for elites, and barns for pigs have already received these air quality upgrades. We need children in schools, employees in workplaces, and patients in hospitals to get the same protections. Hopefully, we are on the cusp of a clean air revolution, with more people and organizations recognizing that healthy indoor air is essential.

N95 masks offer an immediate way to significantly reduce how often you get sick. Thankfully, the N95s available currently are more comfortable and more effective than the surgical or cloth masks that many of us wore back in 2020.

Conclusion

Viruses can harm our cardiac health and cognition, and increase our chances of cancer. If this revelation is fully realized, it will change how the field of medicine operates, priorities in research funding, and public policy on everything from indoor air quality standards to paid sick leave and school attendance. I believe we are on the threshold of what could be a drastic shift in better understanding, preventing, and treating viruses, thus unlocking longer and healthier lives.

Related posts you may also be interested in:

• 5 Devious Tricks Pathogens Use Against Us
• Viruses are weirder, worse, & more preventable than you realise
• Viruses: The Silent Triggers of Autoimmune and Neurodegenerative Diseases
• Your Immune System is Not a Muscle

If you enjoy my posts, please subscribe to be notified of new posts via email:

Introduction

[00:01]: I’ll be speaking about the opportunities and the risk of applying AI in medicine] and some of some of what’s currently happening in the field. As a brief bit of background about me, I earned my PhD in mathematics at Duke University and worked in the tech industry as a software developer and data scientist. Then in 2016, together with Jeremy Howard, I founded Fast AI. Our goal with Fast AI was to make deep learning more accessible and easier to use for people from a variety of different domain expertise areas. Our work was featured in The Economist, Forbes, MIT Tech Review, and elsewhere. And we’ve had millions of students take our courses and use our software library.

I became increasingly focused on the harms that machine learning was causing as well and I wrote three book chapters on data ethics as well as becoming the founding director of the Center for Applied Data Ethics at the University of San Francisco. More recently, I’ve become really fascinated with applications in microbiology and immunology. I went back to school and earned a master’s in the area because I believe domain expertise is so important and I’m now working on a PhD related to autoimmunity.

Promise in Medicine

[01:35]: AI is being applied to a lot of areas in medicine which hold a lot of opportunity and seem quite promising. One area is for drug discovery and drug design. The space of potential drugs out there is huge and so it can be very helpful to have machine learning to help iterate through and filter down which candidates may be the most promising for further study. A second big area where AI is being applied in medicine is to reading medical images, such as detecting Parkinson’s disease or strokes from images of the back of the eye.

A third area is taking a sequence of amino acids. Amino acids are the building blocks of proteins and so here on the left, this is hemoglobin written as its sequence of amino acids, and on the right is its 3D structure. So these two things are the same (they are both hemoglobin), but the 3D structure has has more information. The computer program AlphaFold, which is based on deep learning, made international headlines by smashing all previous records on how accurately it could predict structures. The developers of AlphaFold were awarded a Nobel Prize in Chemistry last year for their work. Protein folding has many, many applications. One that particularly interests me is to try to figure out what a T cell will bind to, which has applications to cancer vaccines.

What is Machine Learning?

[04:18]: Let me step back for a moment. I have shared some areas that are quite promising to think about of AI and medicine. But what what is AI? What is machine learning? AI is a pretty pretty wide umbrella, and I’m going to focus on deep learning, which is a subfield of machine learning. Let’s go all the way back to the 1950s, when IBM researcher Arthur Samuel coined the term machine learning.

A rules-based approach, which is not using machine learning, involves handcrafting relevant rules. And if you didn’t want to use machine learning, you could teach a computer to play checkers by programming the rules of checkers into the computer. Arthur Samuel had a different idea, which is that he didn’t want to teach a computer to play checkers, he wanted to teach a computer to learn to play checkers from previous data. Arthur wrote that instead of specifying in “minute and exact detail” what a computer can do, that he would rather have it learn from data. To apply this to this problem I mentioned earlier of protein folding, rules-based approaches often had scores about hydrogen bonds or different biochemical properties that are that are well known, whereas approaches such as AlphaFold rely on previous protein structures.

Machine learning algorithms LEARN to match inputs to outputs. And so for this reason, it’s very important to think about what your inputs are and what your outputs are. This is crucial. A lot depends on the input data. I want you all to keep this in mind when people are trying to sell you something because there’s a lot of snake oil out there, as well as often people may overhype discoveries, so you need to really think about the nature and quality of the input data and if what’s being claimed is even plausible.

Another key moment in the history of machine learning and of deep learning in particular comes from the late 1980s when Yann LeCun and a team developed a deep learning algorithm for the US Postal Service. Deep learning refers to multi-layered neural networks, a family of algorithms, and it involves a lot of matrix math. They used this algorithm to identify handwritten zip codes because the US Postal Service wanted a computer program that could sort letters. It is very useful to automatically sort letters by zip code. However, people have messy handwriting and tere are a bunch of different ways you can write each digit. Yann LeCun used multi-layered neural networks or deep learning to do this. And now when we hear about breakthroughs in AI in the news, typically they’re using deep learning.

Why now?

[07:35]: And so even though this dates back decades, it’s really only since 2012 that we’ve reached this popular moment in deep learning where it’s being applied much more broadly. It’s achieving unprecedented success on a number a number of tasks. And there are three key components of why this is happening now:

1. One is the existence of large enough data sets to train these algorithms. ImageNet is a key example of a data set of images that was curated for a competition.
2. Compute power, in the form of GPUs. GPU stands for graphic graphics processing unit and it’s what’s used to render video games. The video gaming industry really pushed the computer development along that has been useful for deep learning as well because it’s the same same types of calculations.
3. Algorithmic advances have made it more feasible to to train and use deep learning. Pictured below is the algorithm behind AlexNet, which won the 2012 ImageNet competition.

I was living in San Francisco in 2012 and saw that this seemed like a very promising and exciting area. However, there were a lot of barriers to entry at the time. You had to have done your PhD with a small handful of advisors and a lot of the practical knowledge was not being written down. People would publish theoretical math papers based on it, but it was hard to hard to learn practical details to implement it for yourself if you were outside a few elite institutions.

fast.ai

[09:33]: So in 2016, Jeremy Howard and I started Fast AI to try to address this problem. We really wanted to get experts from all sorts of domains able to use this technology and to focus on the practical side of how to implement this in code. Most people that took our course, I’ll sometimes refer to them as students, but they were working professionals doing this in their part-time, wanting to apply deep learning to the field they were already working in or to pivot their careers. Many went on to get jobs at top top tech companies, to start their own startups, or to work in research. There are a wide range of applications and a wide range of different fields where they’ve applied deep learning.

Our focus was quite unusual in that we were interested in people with limited resources, which is most of the world. The focus of big tech companies is often on experiments that cost $50 million worth of computer time, which almost nobody can do. We were interested in what you can do if you have very little compute power and limited resources and are working in unusual domains that are outside the focus of the the tech industry. This can actually be an advantage. Resource constraints can drive creativity.

In fact, in 2018 a team of Fast AI students beat teams from Google and Intel that had access to way more resources than we did. And the reason behind this is that our constraints forced more creative solutions. I think there’s often a misconception of if you’re trying to do something that’s more accessible or reaching people from non-traditional backgrounds that it must be watered down or that it’s not state-of-the-art. This win of Fast AI over Google and Intel in a competition hosted by Stanford was covered in the MIT Tech Review and The Verge. I’m really proud of our students for that. And I love the idea of getting people from different domains using deep learning because you know about problems that nobody else knows about. And that’s that’s really important.

A question I have been asked many times over the years is, “isn’t this dangerous to make AI accessible to more people?” or “Don’t you worry about it falling into the wrong hands?” And while it’s not implicit in these questions, there’s often an assumption that as long as only billion-dollar elite corporations have access to this, we’ll all be safe, that these are the more ethical stewards of this technology. And I really disagree with that. In fact, I think it’s quite dangerous to have a very homogeneous and elite group in the tech industry creating creating things that impact the entire world. Having greater diversity and more backgrounds is important and very powerful.

ML centralizes power

[13:39]: That said, there is a lot that can and is going wrong, and people that are being harmed. And in many cases these harms can be caused by people from from elite companies or institutions. I spent many years studying and writing about the harms, looking at different case studies, and seeing patterns of key risks in machine learning. A lot of harms fall under this property that machine learning often has the effect of centralizing power. And it does this in several ways.

One is that it can be used at massive scale and cheaply. And that’s often the appeal; it can be used for for cost cutting endeavors, but those often involve a centralization of power. It’s often implemented with no system for recourse and no way to identify or correct mistakes. It’s adding another node of complexity to systems which can be used to evade responsibility. I’ll give examples of all of these in a moment. As systems become more complex, there’s another finger to point to and say, “well, it’s the algorithm’s fault”, which then if nobody feels responsible that doesn’t lead to to systems with good outcomes. It can create feedback loops where it causes an outcome and then amplifies it. I’ll speak about that in more detail in a moment. Sometimes people will say, “yes, machine learning is biased, but so are humans”. But there are published cases where machine learning amplifies biases from humans making them even larger.

Real world harms

[15:34]: A case study that I return to often comes from an algorithm that’s used in many US states to determine home healthcare benefits. And when it was implemented in one state, there was a bug in the code. This software error incorrectly cut care for people with cerebral palsy. And so people like this woman, Tammy Dobs, who’s featured here, had drastic and devastating cuts in her care that really impacted her quality of life to suddenly lose this care. And she she was given no explanation. She was like, “Why has this happened?” And they’re just like, “Well, that’s what that’s what the algorithm said.” Although she didn’t just need an explanatio, what she needed was actionable recourse. She needed a way to get this reverted and to receive her care again.

Eventually there was a lengthy court case and they discover the software bug. The patients got their care reinstated. But that is that is not an ideal process at all that they had to go all this time without care. This is actually a common issue: systems are implemented with no way to identify and address mistakes. I also want to point out that it was patients with cerebral palsy who noticed the air first. And that the people who are directly impacted noticed the error and they had no way to surface that information to anyone that would listen. More recently, there has been investigations into how Medicare Advantage is using AI algorithms to cut needed care.

This is not unique to the US. I live in Australia and we have had a scandal with Robodebt. This software was implemented by the government to determine if people had been overpaid welfare benefits and then automatically issue them debts that they had to pay back. And the way Robodebt was implemented was later found to be unlawful and used incorrect calculations. It was also set up where the burden of proof was almost impossibly high of what people needed to try to appeal. This ruined lives, including leading to suicides. Robodebt happened at a massive scale. The Australian government went from issuing 20,000 debts per year to 20,000 debts per week. That’s a 50x scale up. There are also a lot of examples from across the EU as well. Human Rights Watch has written a thorough report on this, How the EU’s Flawed Artificial Intelligence Regulation Endangers the Social Safety Net.

Feedback Loops

[19:28]: Feedback loops are another mechanism of centralization of power. Police departments use software to recommend how many police officers they should send to different neighborhoods of you know, we think there’s going to be more crime in X neighborhood and less crime in Y neighborhood, so let’s send more police to X. However, algorithms don’t just make predictions, they help determine outcomes. And if you send more police to a neighborhood, they are likely to make more arrests in that neighborhood than a neighborhood where there are no police. That data gets fed back into the algorithm. And so the algorithm might say, “Wow, you’re making a lot of arrests in neighborhood X. Send even more police there.” And you start creating outcomes and and amplifying them into runaway loops.

This same dynamic of feedback loops shows up in recommendation systems. YouTube recommends videos for you that will often start autoplaying. The YouTube algorithm learned that showing people conspiracy theories often gets them to watch even more conspiracy theories. “Wow, you liked this conspiracy theory. Here’s some more!” Because if people get hooked on conspiracy theories, they want to watch more. And this is true of many recommendation systems across different social media platforms and media platforms.

To recap, we’ve seen this centralizing effect. Machine learning centralizes power because it can:

• be used at massive scale, cheaply
• be implemented with no system for recourse & no way to identify mistakes
• be used to evade responsibility
• create feedback loops
• amplify, not just encode, bias

The risks of machine learning mirror existing shortcomings in medical system. Disempowerment is already a problem in medicine, and I really worry about how AI could make it worse. And so stepping stepping away from the machine learning for a moment, I want to talk about how how this problem shows up in medicine.

Failures of the Medical System

[22:01]: Dr. Ilene Ruhoy is a neurologist. She’s an MD PhD and she developed a 7 cm brain tumor. She was experiencing a variety of disturbing neurological symptoms. She knew something was wrong, but when she went to the doctor, and she went to multiple doctors, she recounts, “I was told I knew too much, that I was working too hard, that I was stressed out, that I was anxious.” They were dismissing her symptoms. And this went on for months. Eventually, eventually she did get the MRI, although her tumor was likely much larger at that point than if it had been diagnosed earlier, and got got rushed into emergency surgery. However, this is not not a one-off case.

There are studies on this. Almost 1 in 3 patients with brain tumors had to visit doctors at least five times before they received an accurate diagnosis. It can be exciting to hear about AI that can can read MRIs accurately, but that’s not going to help patients whose doctors won’t take their symptoms seriously enough to order an MRI in the first place. This is not unique to brain tumors. Gender and racial bias are common in medicine. This is well documented and impacts everything from cancer diagnosis delays to how much pain medication is given. As I said before, machine learning is all about matching input data to outputs. Keep in mind this input data is very much going to be shaped by the patients who are misdiagnosed, the patients who were incorrectly dismissed or didn’t have their symptoms taken seriously.

A Deeper Problem than Bias

[24:06]: I believe that the problem is actually much deeper than bias. And bias is huge, but fundamentally, we’re talking about the relationship between patient expertise and medical authority here. Knowledge is often seen as a one directional flow from doctor to patient, without recognition of patient knowledge. The forms that patient expertise take can really vary. Patient expertise can be an umbrella term to include everything from the lived experience of knowing what is going on in your body to also including the patients that are out there reading lots of medical papers and keeping up on the literature. There are patients that are even publishing medical medical papers. And I’ll give a few examples of that in a moment. A lot of this comes back to the question of whose expertise was or wasn’t included in the design and construction of these these systems. And this is an issue in machine learning as well, even for non-medical applications, about whose expertise is not included. Machine learning systems can fail catastrophically when expertise from those closest to the problem is not included.

A Path Forward

[25:37]: I want to end with some examples that give me hope. One of the best events I’ve attended in the past several years was the Participatory Approaches to Machine Learning workshop from ICML 2020. It is all online, so you can access the talks and papers. The organizers of this workshop were interested in seeking more cooperative, more democratic, more participatory approaches for how machine learning systems can can be designed and operated. We really need to consider how all stakeholders, including the people who will be most impacted, can and should be involved in in the creation and running of ML systems.

Pratyusha Kalluri, who’s an AI researcher, wrote an excellent article titled, “Don’t ask if AI is good or fair. Ask how it shifts power.” Who is going to have more or less power after after particular AI systems are implemented? That’s what really poses poses a risk for exploitation.

On the medical side, something that really gives me hope is to see efforts of patient-led research. Here is a neat paper that came out just six months ago, where the author of the paper has long COVID and she also has a doctorate in pharmacology. She surveyed 4,000 patients about over 150 different treatments that they’ve tried, and together with her co-authors did clustering of different subtypes of symptoms. She brought a deep knowledge that comes from being an informed and active patient that is plugged into patient communities. I love to see this type of research. There’s also a Patient-Led Research Collaborative that is publishing innovative work. The way the risks of machine learning mirror existing shortcomings in medicine and the potential negative negative synergy of how those could combine scares me. I look towards ideas of participatory approaches to machine learning and to patient-led research as sources of hope of how we might be able to to do this in more constructive ways.

So with that, if you have any thoughts or questions, I hope you’ll join me in the comment section below. Thank you.

Related Reading:

• AI and Power: The Ethical Challenges of Automation, Centralization, and Scale - a deeper dive into how AI centralizes power
• Gaps and Risks of AI in the Life Sciences
• AI and Immunology

You can subscribe to be notified of new blog posts by submitting your email below:

The Problem of Determining Enzyme Function

Enzymes are what catalyze reactions, so they are crucial for making things happen in living organisms. Enzyme Commission (EC) numbers provide a hierarchical classification system for thousands of different functions. Given a sequence of amino acids (the building blocks of all proteins, including enzymes), can you predict what the EC number (and thus, the function) is? This seems like a problem that is custom-made for machine learning, with clearly defined inputs and outputs. Moreover, there is a rich dataset available, with over 22 million enzymes and their EC numbers listed in the online database UniProt.

An Approach with Transformers (AI model)

A research paper used a transformer deep learning model to predict the functions of enzymes with previously unknown functions. It seemed like a good paper! The authors used a reasonable, well-regarded neural network architecture (two transformer encoders, two convolutional layers, and a linear layer) that had been adopted from BERT. They looked at regions with high attention to confirm that these were biologically significant, which suggests that the model had learned underlying meaning and provided interpretability. They used a standard training, validation, and test split on a dataset with millions of entries. The researchers then applied the model to a dataset where no “ground truth” was known to make ~450 novel predictions. For these novel predictions, they randomly selected three to test in vitro and confirmed that the predictions were accurate.

The Errors

The Transformer model in the Nature Communications paper made hundreds of “novel” predictions that are almost certainly erroneous. The paper had followed a standard methodology of evaluating performance on a held-out test set, and did quite well on that (although later investigation suggests there may have been data leakage). The results claimed for enzymes where no ground truth is known were full of errors.

For instance, the gene E. coli YjhQ was predicted to be a mycothiol synthase, but mycothiol is not synthesized by E. coli at all! The gene yciO, which evolved from the gene TsaC, had already been shown a decade earlier in vivo to not have the same function as TsaC, yet the Nature Communications paper concluded it did have the same function.

Of the 450 “novel” results given in the paper, 135 of these results were not novel at all; they were already listed in the online database UniProt. Another 148 showed unreasonably high levels of repetition, with the same very specific enzyme functions reappearing up to 12 times for genes of E. coli, which biologically implausible.

The Microbiology Detective

How did these errors come to light? After the model had been trained, validated, and evaluated on a dataset involving millions of entries, it was used to make ~450 novel predictions, and three of these were tested in vitro. It just so happens that one of the enzymes selected for in vitro testing, yciO, had already been studied extensively over a decade earlier by Dr. de Crécy-Lagard. When Dr. de Crécy-Lagard read that deep learning had predicted that yciO had the same function of another gene, TsaC, she knew from her long years in the lab that this was incorrect. Her previous research had shown that the TsaC gene is essential in E. coli even if yciO is present in the same genome and even when yciO gene is overexpressed. Moreover, the yciO activity reported by Kim et al. is more than four orders of magnitude (i.e. 10,000 times) weaker than that of TsaC. All this suggests that yciO does NOT serve the same key function as TsaC.

YciO and TsaC do have structural similarities, and YciO evolved from an ancestor of TsaC. Decades of research on protein and enzyme evolution have shown that new functions often evolve via duplication of an existing gene, followed by diversification of its function. This poses a common pitfall in determining enzyme function, because the genes will have many similarities with the ones they duplicated and then diversified from.

Thus, looking at structural similarities is only one type of evidence for considering enzyme function. It is also crucial to look at other types of evidence, such as neighborhood context of the genes, substrate docking, gene co-occurrence in metabolic pathways, and other features of the enzymes.

Hundreds of Likely Erroneous Results

Spotting this one error inspired de Crécy-Lagard and her co-authors to take a closer look at all of the enzymes found to have novel results in the Kim, et al, paper. They found that 135 of these results were already listed in the online database used to build the training set and thus not actually novel. An additional 148 of the results contained a very high level of repetition, with the same highly specific functions reappearing up to 12 times. Biases, data imbalance, lack of relevant features, architectural limitations, or poor uncertainty calibration can all lead models to “force” the most common labels from the training data.

Other examples were proven wrong via biological context or a literature search. For instance, the gene YjhQ was predicted to be a mycothiol synthase but mycothiol is not synthesized by E. coli. YrhB was predicted to synthesize a particular compound, which was already predicted to be synthesized by the enzyme QueD. A form of E. coli with a QueD mutant was unable to synthesize the compound, showing that this is not in fact the function of YrhB.

Rethinking Enzyme Classification and “True Unknowns”

Identifying enzyme function actually consists of two quite different problems which are commonly conflated:

• propagating known function labels to enzymes in the same functional family
• discovering truly unknown functions

The authors of the second paper observe, “By design, supervised ML-models cannot be used to predict the function of true unknowns.” While machine learning can be useful for propagating known functions to additional enzymes, there are many types of errors that can occur: including failing to propagate labels when they should, propagating labels when they should not, curation mistakes, and experimental mistakes. Unfortunately, erroneous functions are being entered into key online databases such as UniProt, and this incorrect data may be further propagated if it is used to train prediction models. This is a problem that increases over time.

Need for Domain Expertise

It is not news that AI work will be more highly rewarded and supported than work that closely inspects the underlying data and integrates deep domain knowledge. The aptly titled “Everyone Wants to do the Model Work, not the Data Work” paper involving dozens of machine learning practitioners working on high-stakes AI projects and found that inadequate-application domain expertise was one of a few key causes of catastrophic failures.

These papers also serve as a reminder of how challenging (or even impossible) it can be to evaluate AI claims in work outside our own area of expertise. I am not a domain expert in the enzyme functions of E. coli. And for most deep learning papers I read, domain experts have not gone through the results with a fine-tooth comb inspecting the quality of the output. How many other seemingly-impressive papers would not stand up to scrutiny? The work of checking hundreds of enzyme predictions is less glamorous than the work of building the AI model that generated them, yet it is even more important. How can we better incentivize this type of error-checking research?

At a time when funding is being slashed, I believe we should be doing the opposite and investing even more into a range of scientific and biomedical research, from a variety of angles. And we need to push back on an incentive system that is disproportionately focused on flashy AI solutions at the expense of quality results.

Related Reading:

• The problem with metrics is a big problem for AI
• Gaps and Risks of AI in the Life Sciences
• “AI will cure cancer” misunderstands both AI and medicine

You can subscribe to be notified of new blog posts by submitting your email below:

What DNA can’t tell us

First, DNA can’t answer many questions about how cells and organisms work in practice. A neuron in the brain has the same DNA as a liver cell, yet the two have completely different functions. This is because different segments of DNA are turned off or on in different cells. To understand how cells are actually working, you need to know about proteins and RNA (RNA is the intermediary which translates DNA into protein). Proteins are what build the structure of cells, catalyze chemical reactions within the cell, and allow communication between cells. Healthy and unhealthy cells in the same organism will usually have the same DNA. For instance, if some regions of the intestines are experiencing an IBD (irritable bowel disease) flare and others aren’t, they would all have the same DNA, yet likely different RNA and protein levels.

A second big problem is that many key sequencing techniques destroy spatial information. You essentially may have to put tissue or cells into a blender in order to get rich information about DNA or RNA sequences. While this data is informative, it turns out that locations of cells within a piece of tissue, and locations of regions within a cell, are also very important! Again, considering the case in which some regions of the intestine are inflamed due to IBD, yet others aren’t, mixing them all together in a blender will lose or distort useful information. Look at the difference between crypts in a healthy segment of the colon (on the left) compared to inflamed crypts (on the right).

Recently, we have seen a rise in breakthroughs that allow us to obtain data about location. Spatial techniques are a necessary and exciting step beyond DNA sequencing. The power of spatial information showed up as a major theme at a conference I attended last year, and spatial techniques have been recognized by Nature Methods as “Method of the year” twice in the last 5 years.

A Few Major Areas of Innovation

What is genetic sequencing anyway? There are a number of different types of sequencing that have been invented in the last 30 years. Walking through a brief history will illustrate what these technologies are, and what they can and cannot do. To make it concrete, let’s look at the example of how they have been applied to cancer treatment.

Sanger Sequencing: This is an older technology dating back to the 1970s, and which was the main way of sequencing DNA up until 2005. Sanger sequencing was one of the methods used in the mid-90s to identify the genes BRCA1 and BRCA2 as key genetic risk factors for breast cancer. The process of discovering BRCA1/BRCA2 involved scientists slowly zeroing in on their chromosomal locations over a period of years. Several other cancer genes were discovered during this time period as well. While Sanger sequencing is effective on smaller amounts of DNA, it can be quite slow to deal with larger volumes. It took over 10 years to sequence the first copy of the human genome using Sanger Sequencing. It is still used today as a simple and reliable way to test for known mutations (such as BRCA1/BRCA2) or for smaller tasks.

High-Throughput Sequencing: New technology released in the mid-2000s allowed DNA to be cut into lots of short pieces and for millions of pieces to be sequenced in parallel at once. This approach, called high-throughput sequencing, was significantly faster than Sanger sequencing. High-throughput sequencing has many applications, including to cancer treatment, by making it cheaper and faster to sequence DNA to identify particular mutations which can influence treatment decisions. Method of the Year | Nature Methods

Long-read sequencing: High-throughput sequencing has the advantage of high accuracy, but the downside of short sequence lengths. In the 2010s, technologies were released with the opposite set of strengths and weaknesses. Long-read sequencing provides the advantage of long sequence lengths, although the downside of lower accuracy. To compare, high-throughput sequencing uses DNA strands that are a few hundred base pairs long, whereas long-read sequencing uses DNA strands that are tens of thousands base pairs long. Both technologies have different strengths and are widely used today.

Single-Cell Sequencing: High-throughput sequencing involves sequencing the DNA of many cells at once, but sometimes it is useful to sequence individual cells. In a tumor, different cells can have different mutations. It is possible that a small subset of the mutations may drive metastasis (the spread of cancer to other areas) or resistance to treatment. Identifying these driver mutations can guide treatment decisions, since particular driver mutations can predict the effectiveness of various drugs. Key mutations may be drowned out in the average if you sequence the entire tumor. This is one reason why it is useful to be able to sequence single cells, and not just obtain the average of many cells. Single-cell sequencing was selected as Nature Methods method of the year in 2013.

Multi-Omics: The study of DNA is genomics. DNA alone gives us an incomplete picture of an organism. Epigenomics can provide information about which regions of DNA are active or silenced. To understand how different cells function, as well as cells in different states of disease or health, you also need to know about their RNA and proteins. This data is contained in the field of transcriptomics (transcripts are strands of RNA transcribed from DNA) and proteomics (the proteins in a cell). Metabolomics looks at small molecules (such as sugars, amino acids, and vitamins) within the body and exposomics includes all sorts of environmental exposures. Collectively, these fields are known as -omics or multi-omics. It is valuable to combine multiple types of -omics together for richer sources of information, since each has different strengths, limitations, and insights to offer. Multi-Omics is an exciting area that draws on lots of data, with applications to cancer, infectious disease and immunology.

Spatial: Spatial information lets us see all the variation within a section of the body– such as a segment of the intestines, the liver, or a cancerous tumor. This variation can often be significant for understanding disease and treatment prognosis. To better understand why spatial techniques are useful, let us dive into some background about cancer.

Tumors aren’t just lumps of bad cells

Cancer is defined as excess cell division. I used to think that tumors were just clumps of “bad cells”, where “good” and “bad” were binary states. This is incorrect. Tumors are not uniform, and within the category of “bad” there is a great deal of variation and heterogeneity. Different cells within a tumor may have different mutations from one another. And our immune systems sometimes build complex defense structures within tumors in attempts to more effectively fight them. How close a cancerous cell is to one of these immune structures impacts how likely the body is to destroy the cancerous cell.

Immunologist Dr. Angela Ferguson, who studies head and neck cancers, describes tumors as having a “physical landscape”. She has shown how the organization and structure within a tumor can predict and guide treatment outcomes. Her work found that cancer progression is “landscape-dependent”, where landscape refers to the locations of immune cells and structures within a tumor. It is not enough to study cancer cells in isolation. We need to understand their layout. Methods that effectively put tumor cells into a blender in order to sequence them, disrupting their spatial information, are insufficient on their own. Mapping that spatial information can hold the keys for more effective treatment.

Single cell sequencing approaches allow a greater number of genes to be measured, whereas spatial approaches measure fewer genes but also provide location information. Combining these two approaches can prove powerful.

Programming Libraries Applied to Spatial -Omics

We are living at a time when multi-omics, spatial information, and user-friendly programming libraries are converging for easier exploration and discovery. For instance, below is an image I created using the common Python programming libraries pandas and matplotlib of data the NIH has shared about a rare liver disease. The image shows a slice of liver tissue, with gene expression overlaid in a color scale ranging from purple (low expression) to yellow (highest expression).

The NIH dataset contain images of slices of liver and expression of many different genes, from both healthy patients and those with a rare liver disease. https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE240429. The plot shows expression of albumin, a protein which helps transport other molecules around the body, overlaid on top of the liver, illustrating which regions produce more or less of this protein. Data science tools are invaluable for transforming and plotting such data.

In addition to being able to use standard Python libraries (such as Pandas and Matplotlib) for visualing this data, there are many specialist libraries as well. The Scverse (Sc = single cell) includes a number of Python libraries focused on single cell analysis.

There are also popular R libraries for single cell analysis and multi-omics, such as Harmony, Seurat, and mixOmics.

With all of these tools and technologies, it is an exciting time to be working at the intersection of data science and microbiology. The causes of most diseases are complex and multi-factorial, although new approaches of spatial multi-omics provide unprecedented types of useful information.

Related Reading:

• What AI can tell us about microscope slides
• Gaps and Risks of AI in the Life Sciences
• AI and Immunology

You can subscribe to be notified of new blog posts by submitting your email below:

Missing Data Sets

A decade ago, Mimi Onuoha coined the concept of Missing Data Sets. She uses this term specifically to refer to “blank spots that exist in spaces that are otherwise data-saturated.” So it’s not just that this data does not exist, but that it ought to exist.

Originally writing in 2016, Onuoha gives the example of how “traditionally there has been little history of standardized and rigorous data collected about police brutality… The group who would make the most sense to monitor this issue—the law enforcement agents who create the data set in the first place—have no incentive to actually gather such data, which could prove incriminating.”

Missing Datasets are not just a USA-based phenomenon. In early 2024, the UK government announced that it would stop collecting and publishing statistics about homeless people’s deaths, even as rising evictions and a housing shortage were making the issue more urgent than ever. Public outcry led the government to reconsider.

As another example, wastewater data provides a reliable indicator of covid rates, even in the absence of testing. The health department in my home state (Queensland, Australia) used to monitor covid wastewater data, but stopped doing so in 2022, even as several other states in Australia have continued to collect and share such data. This was a political decision. The lack of this data for Queensland from late 2022 onwards registers as a blank space compared to other countries and other Australian states, and compared to Queensland’s own record in 2020-2022. There have been multiple covid waves since 2022. This is missing data.

Onuoha explains several reasons that data may be missing. Those who could collect it lack incentives to do so, or in some cases may even want to hide or obscure it. Another possible reason is that the data in question may be of a type that resists straight-forward quantification or doesn’t fit with our current modes of collection. Her work also reminds us that data won’t solve all problems, and that in some cases missing data can function as protection for disadvantaged groups.

Decades of work, not instant magic

Some people mistakenly believe AI can just be sprinkled on a problem to achieve a magic solution, without understanding the many necessary underlying ingredients. The example of AlphaFold, whose creators won a Nobel Prize in Chemistry for their work, disproves these misconceptions. AlphaFold can predict the 3D structures of proteins with unprecedented accuracy. While clever algorithmic innovations were pivotal to AlphaFold, its success would not have been possible without a large, high-quality dataset that thousands of researchers contributed to for decades.

The construction of the dataset that made AlphaFold possible began a full 50 years earlier. The Protein Data Bank (PDB) was announced in 1971 as a database to store information about the 3D structures of proteins. The files were originally shared and distributed using magnetic tape data storage. Initially, just a handful of entries were added to the database each year. By 1980, there were a total of 69 entries present in the database. However, that number grew exponentially, reaching nearly 200,000 entries by 2022. Access to the data was updated from magnetic tapes to the world wide web. In the mid-1990s, a bi-annual competition was created, encouraging teams to write computer program to predict 3D protein structures. The existence and quality of the database and competition are what led to the record-shattering creation of AlphaFold.

AlphaFold wouldn’t have been possible without the rich data of the PDB. Machine learning engineer Abhishaike Mahajan writes, “In my opinion, what Alphafold2 pulled off — applying a clever model to a large body of pre-existing data to revolutionize a field — is something that will be extremely hard to replicate. Why? Because we’re almost out of that pre-existing data. If we had enough, sure, throw ML at it and call it a day, just as Alphafold2 did. But we don’t have that luxury anymore.”

Mahajan argues that “We are running out of data,” and that the next big innovations of AI applied to biology will require new wet-lab techniques to collect larger quantities of different biological data types than currently possible.

AI projects begin at the wrong starting point

Many machine learning projects begin with an existing dataset, and ask “what can we do with this data?” However, this ignores all the data that we don’t have and ends up overlooking many important questions. The harder question is, “what are the biggest questions in your area, and what data would be useful to answering them?”

In some cases, technical limitations prevent the collection of more, better, or different data. In the case of PDB and CASP, creating those datasets required large teams working for decades to produce, curate, and organize data.

One thing that is holding back the application of AI to medicine is lack of the right data. It is not just that medical data may be scattered or hard to access; in many cases, the most interesting variables aren’t being measured or collected at all. Scores of useful and high impact data are not being gathered. For example, widespread medical biases often lead doctors to incorrectly dismiss the pain or physiological symptoms of patients as “anxiety” or “drug-seeking” behavior. In these cases, doctors may fail to record relevant observations or run tests that could gather valuable data. Promises about what AI can achieve with electronic health records must be tempered with the awareness that the data within is too often biased, incorrect, or missing.

As another example, many people have observed that an infection was the trigger for onset of Type 1 Diabetes in themselves or their children, yet they often don’t know which type of virus caused that infection. Determining causality around this requires large longitudinal studies, which are expensive and difficult to run.

AI is great at finding patterns in existing data. However, AI will not be able to solve this problem of missing and erroneous underlying data.

Asking the right questions

The most fundamental machine learning question, prior to the questions I mentioned above, is “what type of society do we want to live in?”

When school districts implemented a computer program to automatically fire supposedly low-performing teachers, there were numerous issues with the underlying algorithm and data. One key type of data used as input was student test scores, which had often been altered due to cheating scandals by school administrators. The algorithms were also based on the dubious assumption that changes in standardized test scores were a reasonable proxy for teacher quality.

Test scores don’t capture a teacher’s creativity, care for their students, or ingenuity. They don’t capture the challenges their particular students are facing, or how those students may grow and flourish apart from test scores.

This is an example of an algorithm where there was low quality data, the wrong data, and faulty assumptions. However, even if it had been implemented differently, the whole premise would still be a terrible idea. Many of us (myself included) find the idea of a society where we use computerized algorithms to fire teachers to be dystopian. The answer to “How can we best use AI to fire teachers?” is that we shouldn’t.

Negative spaces

The decisions around which, where, and how data is collected, and who it is shared with, all exert a strong but often overlooked influence on the shape of scientific research and algorithm creation. As I dive deeper into the intersection of microbiology and machine learning, I am seeking to understand not just the data that I’m shown, but also the factors shaping its collection and the negative spaces of what is missing.

We are in a disturbing era, in which relevant health and safety data that we previously had access to are actively being cut off. Issues of bias and politics that dismiss the viewpoints of marginalized groups were already bad and seem to be worsening.

While promises of AI improving healthcare may seem appealing, systems that ignore patient expertise risk building on faulty premises and causing more harm than good.

Further Reading

• Lessons from archives: strategies for collecting sociocultural data in machine learning by Eun Seo Jo and Timnit Gebru, on lessons machine learning practitioners could learn from the library sciences
• On Missing Data Sets by Mimi Onuoha
• The end of forgetting: Strategic agency beyond the panopticon by Jonah Bossewitch and Aram Sinnreich, on how information flow relates to power dynamics
• “AI will cure cancer” misunderstands both AI and medicine by Rachel Thomas
• Avoiding Data Disasters by Rachel Thomas

You can subscribe to be notified of new blog posts by submitting your email below:

CPath foundation models

There is a powerful idea about how to make more accurate CPath models. Rather than train a model on a single type of tissue and a single task (e.g. identifying cancer in breast tissue), train a model on images of tissue from many different organs (breasts, lymph nodes, lungs, prostate, heart,…) and on multiple different tasks (recognizing cancer, determining the stage and subtype of the cancer, segmenting cells, and predicting treatment outcomes). Patterns learned from one dataset or one task are likely to generalize to others.

Such models are known as CPath foundation models. In general, a foundation model is a machine learning model which is trained on a sufficiently diverse large dataset which can then be adapted for a range of downstream tasks. This idea is commonly used in the area of language models such as Chat-GPT and Claude.ai. Language foundation models are trained on many types of language tasks and intended to generalize across different corpuses of text (e.g. wikipedia, reddit posts, academic papers, online conversations, news articles, and more). ImageNet models trained to recognize a huge variety of different pictures often serve as foundation models for images. The success of foundation models within the areas of language and more general images is a key reason why we might expect pathology foundation models to be useful too.

Two notable CPath foundation models were released in 2024: Prov-GigaPath and UNI. Both models achieved state-of-the-art performance on dozens of pathology tasks (although they were not directly compared to one another). Another relevant paper (from Kaiko.ai) studied the impact of dataset size and model size on CPath model performance.

Learning the Vocab

Medicine is full of jargon and specialized vocabulary. Pathology refers to the study of disease. It is a broad field, and can include everything from dissecting dead bodies to analyzing blood samples. One key focus of computational pathology is analyzing and interpreting whole slide images (WSIs) and in some cases combined with accompanying meta-data about a patient. Whole slide images refers to the complete microscope slide, although in many cases the region of interest (such as particular cancerous or inflamed cells) may be much smaller, just occupying a subset of the slide.

Machine learning (ML) is a subfield of Artificial intelligence (AI) which involves learning from past data, and is increasingly being used with great success in pathology. The focus of most computational pathology ML models is on images of tissue, on microscope slides. That is what we will focus on in this post as well.

So Many Tasks!

There are many different benchmarks that CPath models can be tested on. These involve numerous datasets: related to different areas of the body, with different sizes, and with different purposes. They also involve a variety of tasks, including binary classification, image segmentation, and outcome prediction. Prov-GigaPath attained state-of-the-art performance on 25 out of the 26 tasks it was evaluated on and UNI attained state-of-the-art performance on 34 different tasks. Here I will give examples of just 3 of these tasks.

We need more data

One key challenge in the area of CPath foundation models is gathering enough training data. The Cancer Genome Atlas (TCGA) was an ambitious project launched in 2006 by the National Cancer Institute in the USA. Over a 12 year period, samples were collected from over 11,000 patients of 33 different cancer types, and all this data was made publicly available. While this is a rich dataset and a useful resource, all 3 papers we’ve looked at concluded that TCGA is not large enough for effective foundation models. In addition to limited data size, TCGA also has limited diversity, consisting mostly of slides from the primary site of cancer, but not metastasized cancers or different types of tissues.

Researchers at Kaiko.ai tested the impact of scaling both the size of their model and the size of the training dataset. While they found limited need to scale model size beyond a certain point, they found that larger datasets continued to lead to increased performance. They concluded that TCGA was likely not large enough and shared their plans to build a larger training set, and are now partnering with cancer centers across Europe to create a dataset for their model.

The researchers behind two other CPath foundation models reached the same conclusion about data set size, and gathered massive datasets to train their models. This required partnering with healthcare centers. Prov-GigaPath, a model created by Microsoft Research and Providence Genomics involved data from 30,000 patients across 28 cancer centers (which are part of Providence Healthcare company). UNI, a cPath model created by a team at Harvard, MIT, and the Broad Institute, involved the creation of the Mass-100K: a dataset with over 100K whole slide images across 20 tissue types collected from Mass General Hospital, Brigham & Women’s Hospital, and Genotype-Tissue Expression (GTEx) consortium.

These partnerships and curation of training datasets are currently a crucial component of building CPath foundation models. Curating datasets carefully poses many challenges as well. Combining data from different sources, which often use different protocols for how slides are sampled and prepared, can introduce significant biases.

Different scales

CPath foundation models face the difficulty of capturing both local patterns (that show up in a small tile within a slide) and global patterns across the whole slide. Many tiny tiles are found within a slide.

Some models, such as the Hierarchical Image Pyramid Transformer (from several of the same authors as UNI), use hierarchical approaches to deal with these multiple scales.

Other models, such as Prov-GigaPath, treat the tiles as tokens, encoding both the tiles and the slide as a whole as model inputs. Prov-GigaPath uses both a slide encoder and a tile encoder to take into account these two different scales.

In pathology clinics, diagnosis and treatment decisions are often made at the patient level, whereas CPath models are often highly focused on regions of interest. Accommodating the multiple relevant scales (small tiles, whole slides, and patient-level) for pathology is a consideration that CPath models need to balance.

Going Forward

It is still early in the world of CPath and there are many growth opportunities, including the continued need for large and diverse datasets, ways to further optimize model training, tasks which have previously received less focus, and the difficulties of integrating models into clinical work. As the authors of the kaiko.ai paper wrote, “We are still at the very beginning of developing a truly foundational pathology foundation model.” It is a hopeful sign that these models achieve state-of-the-art results on dozens of benchmarks, but it still remains to be seen when and how they will be used in clinical settings.

Related Reading:

• The Most Common and Useful Neural Nets
• Using AI to Discover New Antibiotics
• AI and Immunology

You can subscribe to be notified of new blog posts by submitting your email below:

Using AI to predict what T cells will bind to

T cells are one of the most important cell types of our immune systems. Figuring out how to predict what a T cell will bind to (meaning what cells it can recognize as bad and coordinate attacks against) would be a vital medical breakthrough.

• Decoding T cells with AI
• The Most Common and Useful Neural Nets
• AI’s Quest to Predict T Cell Binding– The Holy Grail of Immunology

Mapping immune cell communication networks

Immune cells talk to each other through complex networks. NLP, math, and immunology are useful in studying these networks.

• Applying AI to Immune Cell Networks
• How Immune Cells Communicate

Discovering new antibiotics

Bacterial resistance to existing antibiotics is an urgent threat. AI is being used to search for new antibiotics.

• Using AI to Discover New Antibiotics

Ethical risks of AI applied to immunology

The enthusiasm about AI in medicine is failing to grapple with realities of the system. Recognizing gaps in how AI is applied to scientific research can prevent potential shortcomings and risks.

• Gaps and Risks of AI in the Life Sciences
• “AI will cure cancer” misunderstands both AI and medicine

You can subscribe to be notified of new blog posts by submitting your email below:

1. Commandeering the cell’s own machinery

The year is 2063. An insect-like robot with a swinging gait and clamp-like feet marches along a beam in a space colony, transporting the large cargo suspended behind it.

Ooops– what I meant to say is that a microscopic motor protein in one of our cells moves a bag of proteins along the cytoskeleton, away from or towards the nucleus. This is not a metaphor; we actually have motor proteins that are moving complexes within our cells. The above gif is scientifically accurate, not a fantastical invention.

I used to think that cells were just sacks of liquid, with various items such as the mitochondria or nucleus floating haphazardly around inside. I couldn’t have been more wrong. Just as beams, drywall, and supporting columns are necessary to organize and support the architecture of a building, so too do our cells have an intricate organization arranged by various filaments and microtubules.

Tiny motor proteins that look like robots are used to transport items around. But just like the space colony mentioned above, these robots can be commandeered by enemy invaders. Unfortunately, the virus which plays a role in Alzheimer’s Disease is able to use these motor proteins for its own goals.

Alert! Alert! Herpes Simplex Virus-1 (HSV-1) has breached the cell! When HSV-1 invades a human cell, it commandeers our motor proteins to transport the viral capsids (packets containing the viral DNA) to the nucleus. The sooner the viral capsids arrive at the nucleus, the sooner the virus can begin replicating, printing copies of itself to invade the neighboring space colonies, err, I mean neighboring cells. HSV-1 has turned our cell’s own machinery against us to improve its speed and efficiency in infection.

HSV-1 is a virus responsible for causing fever blisters on the mouth. It can live latently in neurons, activating or reactivating after decades. A growing body of research is linking HSV-1 with a major role in Alzheimer’s Disease. The ability of HSV-1 to use human motor proteins to speed its path to the nucleus for replication is just one of the many devious tricks that different viruses and bacteria have developed.

2. Blocking signals from T cells

T cells are a keystone of the human immune system. T cells can rearrange their genes, allowing for the potential recognition of billions of different invaders. And yet they are outmatched by a single-celled organism (T. cruzi) carried in the feces of so-called “kissing bugs” (triatomine bugs). A kissing bug bites you in your sleep. In many cases, you may not even notice, and experience no symptoms in the coming weeks. T. cruzi is a species of trypanasome, a tiny one-celled parasite, and causes Chagas disease. They live in the feces of the bugs can enter the bite wound and travel through your bloodstream. They have a preference for making a home inside the muscles of your heart, the lining of your digestive tract, or the nervous system. T. cruzi can hang out patiently for 10-30 years, but are then ready to wreak havoc. Up to one third of patients experience heart problems caused by destruction of cardiac tissues. One in 10 experience neurological and digestive problems. Some die sudden deaths.

Why haven’t your T cells protected you? One of T. cruzi’s devious tricks is that it is able to block the key signal which coordinates the T cell response (a messenger known as IL-2). IL-2 is crucial for activating T cells. With its messaging blocked by these tiny parasites, T cells don’t activate and respond to the threat, and T. cruzi is able to survive in humans, later causing great destruction. While there is medication available that will work if addressed early, remember, most people have no symptoms initially. The best protection against T. cruzi is to avoid it in the first place, which typically requires access to a home that keeps out insects.

3. Making a home in B cells

In addition to T cells, the other type of adaptive immune cells that can rearrange their genes are B cells. B cells produce antibodies, small proteins that bind to a target, marking it for destruction by other immune cells. Through a process of hypermutation, B cells are able to create carefully custom-tuned antibodies to target any enemy they want. The flexibility and nearly infinite customizability of this defense are powerful.

Given the powers of B cells, you might expect that invaders would want to stay away from B cells. Yet Epstein-Barr Virus (EBV) and measles virus actively seek out B cells. EBV is able to set up a life-long home in memory B cells, hanging out for decades. While EBV is best known for causing mono, it can also cause multiple sclerosis and cancer. In multiple sclerosis, the immune system attacks the nervous system, damaging neurons and causing plaques in the brain. If the viral antigens present in EBV look similar to neurons, this could confuse the immune system and lead to autoimmune attack.

Another virus, measles, plays an amnesia trick on memory B cells, causing them to forget their memories of other invaders that the B cells had previously learned to recognize. This immune amnesia is why children who have been infected with measles are significantly more likely to die of other illnesses later, compared to children who have never had measles. The measles vaccine drastically reduces the risk of non-measles related deaths by letting the immune system retain the information it has learned.

4. Thriving in acid

T cells and B cells aren’t the only immune cells targeted by particular pathogens. Macrophage literally means “big eater”. These large cells are a useful component of the immune system, because they are ready to eat your enemies. Macrophages contain bags of acid to acidify and ingest invaders. They share the digested remnants of their foes with other immune cells to activate them and coordinate the immune response. Sounds fierce, right? And yet a number of bacteria have developed strategies to survive inside them.

“No! No! Don’t dump me in acid!” cries the bacteria C. burnetii insincerely, “Anything but that!” In fact, that is exactly what C. burnetii (the bacteria that causes Q fever) wants. It thrives in acid. Acidic environments are where it can replicate, making more and more copies of itself. C. burnetii prospers inside the the acidic pouches in macrophages.

C. burnetii is not the only type of bacteria that has figured out strategies for living inside the “Big Eaters” that are meant to protect us. Tuberculosis is a bacteria that spreads when infected people cough, talk, or even just breathe. Approximately 1.2 million people die each year of tuberculosis, almost double the number of people who die of HIV. Tuberculosis that is resistant to multiple antibiotics is an urgently growing problem. And unfortunately, our macrophages are unable to eliminate it. When macrophages take a big bite of an invader, this leads to the invader initially being inside a pouch called a phagosome, within the macrophage. Usually, the next step is for the phagosome (containing the invader) to merge with a pouch of acid (called the lysosome). However, tuberculosis is able to prevent that next step from happening. Tuberculosis loves to just kick back and relax within the phagosome. It can remain latent for years or even decades, but can reactivate if the immune system becomes suppressed.

5. When we put the brakes on translation, viruses cut the brake lines

Pathogens don’t just target immune cells, but also more general processes found in all our cells. A central action is the process by which DNA is transcribed into RNA, which is then translated into proteins. Proteins are the building blocks for all sorts of important materials within our cells, and proteins allow communication throughout the body. The process of producing protein from RNA is known as “translation”. Viruses love to take over our protein translation machinery for their own purposes.

In my molecular cell biology course, the professor recently taught about a method our cells have for halting protein translation when they have been infected by a virus (for my fellow immunology nerds, we were learning about PKR phosphorylation of eIF2α– but you don’t need to know any details of it for this blog post). It is a bit like our cells stepping on the brakes to stop protein translation.

That is encouraging that we can block those pesky viruses from producing proteins! Good job, human cells!

Then, on the following slide, the professor shared a paper that covers 31 different viruses that sabotage this safety mechanism (via 9 categories of actions) so that they can use the cells’ machinery to produce their own viral proteins regardless. Viruses have effectively cut the brake lines, so they can continue barreling along. And so it goes. Each time I learn about an intricate and ingenuous aspect of the human immune system, the following page seems to contain a clever way that pathogens circumvent or subvert it.

Viewing pathogens with appropriate respect gives us a path forward

There is hubris in focusing solely on the impressive coordination of the human immune system, without considering the methods that pathogens have developed for counterattacks. This information is not just an academic point; it has practical impact. Unthinking repetition of the soundbite “the immune system is amazing” can lull people into a false sense of confidence that disease prevention is unnecessary. While it can be discouraging to think about the methods bacteria and viruses have for causing harm and the surprising long-term consequences of some infections, the good news is that many infections are preventable.

Mosquito nets, houses that keep out insects, sewage systems, clean drinking water, air purifiers, N95s in public indoor spaces, and good ventilation are all effective at reducing risks of infection. In particular, research from the past few years has illustrated that cleaning up indoor air quality would likely stop transmission of numerous pathogens.

I still marvel at the intricacy of the human immune system, but alongside that marvel, I keep a commensurate respect for the serious deviousness of many pathogens.

Related Reading:

• Our immune systems evolved for a different world (that didn’t involve 100,000 global flights per day)
• Surprises of the Microbiome
• Viruses: The Silent Triggers of Autoimmune and Neurodegenerative Diseases

A False Binary

I often hear statements such as, “I’d rather kids played outside than on a computer.” But these choices aren’t binary! You don’t have to pick just one. My daughter plays outside AND on a computer. She plays in-person sports regularly AND has online hobbies. This year we threw her 2 birthday parties: an in-person one for her local friends AND an online one for her long-distance friends.

The term “screen time” is so broad as to be meaningless, clumping together many disparate activities. A child sitting next to a parent playing a math game together on the computer is screen time. Calling your grandparents on skype is screen time. Collaboratively composing music with your best friend in another country is screen time. Yet, when many people refer to screen time, they often seem to be referring to a child passively watching TV on their own (something which my husband and I almost never let our daughter do, and that she doesn’t seem interested in). With any screen-based activity, it is useful to consider:

• Is it social or solitary?
• Does it involve creating content or consuming content?
• Is it educational or not?

It is not that some of these options are always good or always bad, but rather that we need to have a nuanced view on different combinations of these factors and how much time you may want to spend on each.

Screens are a crucial tool and outlet for her expressing her creativity, and it saddens me to hear an increasing media narrative vilifying screen time for kids.

Two of my daughter’s best friends are kids she has never met in person (because they live in a different state and in a different country), yet she talks with each of them several times per week. She also has local friends that we get together with regularly in-person. The fact that she has several close long-distance friends isn’t surprising, because my husband and I have many close online / long-distance friendships as well. Our friendships aren’t any less real because they are over screens. Opening up the possibility for long-distance friendships has given us the option of finding additional friends we are particularly compatible with.

Increased options

Screen-based learning options make homeschooling more feasible for a wider range of families. Online resources (games, classes, and clubs) can offer opportunities that may not be available locally, can fill in gaps parents may not be qualified to cover, and offer an exciting variety of options. For instance, online courses have allowed my daughter to do the following (which are not available locally for us):

• have tutors/teachers based in 5 different countries, providing diverse experiences
• take science courses including: particle physics for kids, biochem for kids (she has been with the same group of kids for 3 years), and a biology class where a small group of students could share images from their digital microscopes over zoom
• participate in clubs for creating collaborative narratives, fanart, and fanfiction around two of her favorite series of books
• complete mastery-based math games at her own pace, which allowed her and a few friends to end up many years above their grade levels

As I wrote about in a previous post, we homeschool, and this leaves our daughter with far more time for hobbies and socializing than in-person school gave us.

In-person school does not work for everyone

There are several categories of kids whose needs are often not met in traditional schools and frequently benefit from online options:

• Neurodivergent kids, who may deal with overstimulating, distracting, or painful sensory experiences in traditional schools
• Gifted kids, who are not intellectually challenged in traditional schools
• Kids from marginalized cultures, whose cultural heritage may not be reflected at school, or who may experience discrimination
• Kids who are being bullied
• Kids who have particular passions they would like to hyper-focus on (which are either not included in their school’s curriculum, or which they don’t get adequate time for)
• Medically complex kids, including those who may be getting sick so often that they can’t attend school regularly, those for whom an illness could result in hospitalization, or whose health issues make it difficult to attend school in-person

It is an injustice that schools are not inclusive of all children. While individuals pulling their children out of school does not address this injustice (and we should continue to work for safer and more inclusive schools), many families have found it necessary to leave systems that are harming their kids.

Don’t judge screentime by a few bad examples

I would caution against writing off screen time or online learning based on a low quality or poorly designed experience. Maybe your child’s school went online in 2020 with no notice and no support, and they were stuck in a zoom with 25 other kids (this is not a good set-up for anyone!). We have found that with small group online tutoring, my daughter is able to be more social and talkative than she was in a large class at in-person school, and she masters more material in less time. However, even amongst small online groups, occasionally she’ll try a teacher or club that isn’t a good match for her, and in those cases, we discontinue it.

Since screentime is often equated with a way to occupy kids while parents do something else, there are often unrealistic expectations, particularly for the most rewarding screen-based tools (which tend to involve a learning curve and require support from a trusted adult). In many cases handing your child a new educational app and then leaving them alone can lead to frustration. Sitting with them and working through the app together can be relationship-building, prevent them from getting overly discouraged, and help them build confidence as you figure it out together. With age and experience, kids will likely be able to do more on their own.

Inclusion and accessibility

Countries around the world, including the USA, UK, Australia, New Zealand, Japan, Belgium, and Canada are seeing high levels of school absences. Factors for these record levels of absences include more frequent illnesses (such as covid, RSV, and flu) and chronic health issues. In mid-2024, many countries (including the USA and Australia) experienced a particularly high wave of the ongoing covid pandemic. Rates of disability are increasing, and people, including children, are continuing to develop new health conditions after viral infections.

A number of school districts are responding to these waves of illness by placing heightened emphasis on attendance to in-person school, in many cases pressuring or even requiring children and teachers to attend school while sick. In parts of the UK, USA, and New Zealand, policies are being considered (or implemented) to charge parents fines if their children miss school too often. This is counterproductive, as it is hard to focus when unwell, attending school sick increases the likelihood of spreading illness to others, and rest is a key way to reduce risk of post-viral illness. None of these punitive attendance policies address root causes, such as improving school air quality, making sure parents have adequate paid leave to stay home with sick children, or decoupling school funding from attendance.

We are also seeing rising rates of disability amongst working age adults. In the USA, there has been an increase in people with cognitive disabilities. A BBC article notes that in the UK sick people are leaving the workforce at record highs. Online access is a crucial part of accessibility and disability rights. For many disabled people, accessibility improved in 2020 with more remote options for work, medical appointments, conferences, and other events, and there is now a counter-reaction in which these options are being removed.

Every child is different, so what works for my family won’t work for everyone. I am not recommending that all families homeschool, or that nobody should attend in-person school. However, we need to consider innovative approaches to learning and communication, and that includes screen-based approaches.

Hopes for a creative and inclusive society

I want children to discover and develop passions that they enjoy (astronomy, writing, art, music, math, oceanography, chess, there are so many possibilities!) and to be able to connect with others who share their interests. Taking advantage of computer-based and online options opens up a world of possibility. I also want a society in which disabled people are included in events and opportunities, and in which people can rest when they’re sick or preferably avoid illness in the first place. Fewer people commuting to the office is better for the environment. Traditional school is failing a lot of kids. I hope we can consider a range of innovative educational and social approaches, and not exclude an entire category of valuable ways to communicate, create, and learn (e.g. screentime).

Related Posts

• My family’s unlikely homeschooling journey: Prior to 2020, we never expected to homeschool, and now we have committed to it long-term.
• There’s No Such Thing as Not A Math Person: based on a webinar I gave to parents addressing cultural myths about math and how to support your kids in their math education, even if you don’t see yourself as a “math person.”

Missing Data

A recurring problem within data science is that often the data we would be most interested in does not exist or is too difficult to gather. In response, data scientists often instead use imperfect proxies (Thomas and Uminsky 2022), or alter their research questions to make use of the data they have (Passi and Barocas 2019)). While it is understandable for teams to work with what data is available, this can unduly influence the field to neglect areas where data is unavailable or more difficult to collect.

A research project titled “The library of missing data” (Mimi Onouha 2016) inspected examples of datasets that are not collected and of information that we don’t have. What is not measured or recorded can be very revealing, yet it is harder to see an absence. It would be a valuable project to begin with the most important and high-impact open questions in various scientific fields, and then to determine which types of data would need to be collected, what experiments need to be run, and whether AI could prove useful.

While it is understandable that physical and financial constraints shape what types of data can be gathered, it is important to stay aware of how these influence research questions and findings. In some cases, a research area may be ignored due to lack of data. In other cases, results may be biased due to only cetain types of data being included. Missing data can be a type of representation bias, in which not all groups are adequately represented (Suresh and Guttag 2021). For example, databases of T cell receptors are highly biased towards particular genetic alleles and certain viral antigens. This data can still help us reach valuable conclusions, but it would be a mistake to overgeneralize those findings to alleles or antigen types that are not well-represented in the underlying data.

The physical world is complex and volatile

In other fields, machine learning models have sometimes performed well in testing but failed to work as expected when deployed in the real world (Sambasivan et al. 2021). Issues can arise when the context and quality of the training data is not fully understood (Gebru et al. 2021). Biases and errors in datasets will propagate to downstream research. For example, The ImageNet Large Scale Visual Recognition Challenge (ILSVRC) dataset, often called ImageNet-1k, is widely used in image recognition research and has been cited 32,000 times. Over a quarter of the images in the dataset are of wild animals, yet researchers later found that over 12% of these wild animals are labeled incorrectly (Luccioni and Rolnick 2023).

ML systems are trained in clearly defined environments, while the physical world often has complex and volatile underlying phenomena, which are not fully accounted for during training. This can create a brittleness to ML-generated solutions (Sambasivan et al. 2021).

Neglecting Mechanistic Understanding

Language models are great at interpolating between underlying information, but not at creating things that are completely new (Dziri et al. 2023). Brand new theories will still require human ingenuity. Upon receiving the Nobel Prize, biologist Sydney Brenner, warned that increased data does not necessarily lead to deeper understanding, saying “We’re drowning in an ocean, or a sea of data but we are starved of knowledge.” (Sylwan 2002). Geneticist Paul Nurse, another Nobel Prize-winner, echoed these concerns. He wrote about the need for a renewed emphasis on new ideas and theory, alongside data. Nurse proposed that computer scientists need to be embedded alongside biologists to deeply understand the domain problems, “It is through deep familiarity with the biology — not simply a drive to collect more and more data — that important questions will be asked.” (Nurse 2021).

AI is a powerful tool, but it does not replace other ways of knowing about the world, including qualitative research and benchtop quantitative experiments. Machine learning and AI techniques are great at learning patterns in existing data.

Difficulty of Cross-Disciplinary Work

Interdisciplinary work is challenging, as it can be tough to remain on the cutting edge of two separate fields. At one extreme, you can have research based on sound immunology, but using deep learning techniques which are outdated or perhaps were never considered best practice. At the other extreme, you can have research using state-of-the-art deep learning, yet misunderstanding the core mechanisms of immunology or misconstruing the underlying context of datasets.

Meaningful collaboration between those with in-depth immunology knowledge and those with in-depth deep learning knowledge is necessary. Cross-disciplinary work is inherently difficult. Challenges can include differences of communication and work practices between practitioners with different backgrounds; insufficient expertise in one of the fields; and the difficulty of reconciling different approaches and priorities. Moreover, many academic institutions, journals, and other research centers often tend to value specific types of expertise over others. Deep learning and areas of the life sciences such as immunology are each highly complex, jargon-laden, and fast-moving fields. Finding researchers, or even teams, which can keep pace with both is challenging.

Deep learning practitioners often seek to automate systems without understanding what the greatest needs of domain experts are. When this automation leaves gaps, humans are often expected to fill these gaps in ways that are difficult, tedious, or impractical. In contrast, a more fruitful approach may be to take the strengths of both domain experts and deep learning into account from the start to leverage human strengths (Thomas 2019).

The Double-Edged Sword of Competitions

Carefully structured competitions can play a valuable role in motivating research. Well-defined benchmarks make it clear how various approaches compare to one another. The winner of the 2012 ImageNet competition to identify objects in photographs, an algorithm called AlexNet, led to the mainstream popularization of neural networks and set off an avalanche of increased research into AI. The winner of the 2020 CASP challenge to take amino acid sequences as input and output the most likely 3D protein structure was a computer program called AlphaFold, which has revolutionized protein structure prediction. While these have been huge breakthroughs for their respective fields, over-focusing on competitions can lead to other research areas being neglected. In other cases, the models created for benchmark tasks may be inappropriately applied to other tasks for which they are less suited (Paullada et al. 2021).

It takes effort to construct well-structured competitions. Data must be gathered and selected, often through benchtop laboratory experiments, while being cognizant of biases and overall representation within the dataset. It is also crucial to have appropriate splits between the training set and a held-out test set. When data leakage occurs, clues from the test set can be found in the training set, artificially inflating the performance of models (Samala et al. 2020).

Undervaluing data work

Too often the arduous task of curating quality datasets has been neglected, undervalued, and not given the same currency for career advancement as algorithmic work (Jo and Gebru 2020). A research paper which interviewed over 60 machine learning practitioners was titled “Everyone wants to do the model work, not the data work”, drawing on a representative quote from one of the interviewees. The researchers found that data work was systematically devalued, leading to issues which impacted model performance, sometimes with disastrous consequences (Sambasivan et al. 2021)).

Many areas within science lack consistent benchmark datasets. This can result in models that are not consistently reproducible and makes it tough to compare performance across different papers. Organizers of the largest academic AI conference, Neural Information Processing Systems (NeurIPS), recognized that there were too few incentives for researchers to work on datasets and benchmarks. In 2021, they opened a new track of the conference, on equal standing with the original track, to better support and incentivize research on datasets and benchmarks (Vanschoren and Yeung 2021). This is an encouraging model, and I hope to see more efforts to prioritize data curation.

Opportunities for AI in Science

The flip side of these risks and gaps is that they offer opportunities for how we can work to improve the application of AI to science and to aim to avoid potential pitfalls. To ensure that we are making the most of AI possibilities in the life sciences, it would be a good idea to:

• Embed scientists and AI practitioners on closely integrated teams
• Survey the biggest open problems in immunology and determine whether AI could help, and if so, what data is needed (as opposed to always just starting with the data we already have)
• Reward dataset collection, through conference tracks and journal issues focused on it, similar to the model adopted by NeurIPS
• Invest in research that is:
  • Focused on underlying causal mechanisms
  • Values multiple ways of generating knowledge
  • Gathering new types of data
• Develop benchmark datasets and structured competitions where relevant

By keeping a broader perspective on different types of expertise, different methods of research, and the need for interdisciplinary work, we can prevent a myopic application of AI to science that risks neglecting important questions.

Thank you to Jeremy Howard for feedback on earlier drafts of this post.

You can subscribe to be notified of new blog posts by submitting your email below:

The Hygiene Hypothesis

Allergies are a misfiring of the immune system– when it attacks what should be harmless environmental substances, such as pollen or dust. Autoimmunity is a different type of misfiring of the immune system– when it attacks our own cells, whether those are your neurons (multiple sclerosis), your joints (arthritis), your thyroid (Hashimoto’s disease), or your insulin-producing cells (type 1 diabetes). Both allergies and autoimmune diseases have risen dramatically in recent decades.

First proposed in 1989, the Hygiene Hypothesis offers an explanation for this dramatic rise. Certain types of microbes can help modulate our immune systems. By creating a low level of immune activity, they prevent our immune systems from getting bored and confused, which could lead them to attack the wrong target. Four main categories of pathogens that humans deal with are viruses, bacteria, fungi, and parasites. The evidence for pathogens that may be beneficial to the immune system is almost entirely for parasitic worms and friendly (commensal) bacteria. In contrast, many viruses can even trigger the onset of autoimmune diseases or allergies.

The reason for this makes sense: humans co-evolved with parasites and commensal bacteria, going back to when people lived in small hunter-gatherer tribes. For over 90% of human history, we lived as hunter-gatherers, and were exposed to very different types of microbes than we are now in crowded cities or poorly ventilated office buildings. Researchers have pointed out that the moniker Hygiene Hypothesis is misleading, and have proposed a more accurate alternative. The “Old Friends” mechanism describes which microbes we co-evolved with for 300,000 years.

Our old friends

Our bodies are full of peaceful (commensal) bacteria, which can help synthesize vitamins we need, regulate dopamine, and even protect us from infection. Most famously our guts, but also the skin, throat, and bladder, all have distinct microbiomes. Disruption due to antibiotic use, Western diets, and cesarean births has changed our microbiomes, contributing to the rise of autoimmunity and allergies. Not all bacteria are the same– the positive microbes transmitted in a vaginal birth are quite different from harmful bacteria like anthrax or tuberculosis.

Surprisingly, there are a number of studies showing that infections with helminths (parasitic worms) can be beneficial in treating allergies and autoimmune disease, including for multiple sclerosis. A proposed mechanism is that parasites provide a low-level of background activation for the immune system, which prevents excess activation. This in turn can prevent inflammatory and autoimmune disorders. However, helminth infections can also be harmful and are not something you should try at home! Researchers are developing therapies based on proteins derived from helminths.

“Crowd infections” are new in human history

Our “old friends” are organisms we co-evolved with for > 50,000 years. By evolving together, our bodies learned to take advantage of these organisms. Old friends can be contrasted with “crowd infections” which only developed much more recently, since 10,000 BCE, after people began to live in more densely populated townships.

A crowd infection is one that would not last in sparsely distributed, small hunter-gatherer groups, because it either kills people, or offers enough immunity until the small tribe has been infected and there is nowhere else to spread. For instance, if a small tribe of hunter-gatherers caught a strain of influenza, the virus would die out once they had all caught it in short order, with no more hosts to spread to. The ability to circulate through megacities or crisscross the globe through international travel in our current world offers influenza far more opportunities to infect and time to mutate. It is able to continue infecting and reinfecting, decade after decade. Even viruses that don’t mutate much, such as measles, are able to continue infecting, due to dense populations and an ongoing supply of new people being born.

Homo Sapiens first evolved some 300,000 years ago, yet crowd infections are believed to have only developed in the last 12,000 years, a small blip in human history. Humans living in dense cities is a relatively recent development. An even more recent development is that of sealed indoor spaces and frequent international air travel. Many crowd infections, such as measles, mumps, chickenpox, colds, and flu, are airborne, spreading when humans talk and breathe in close contact, with poor ventilation. These infections could not widely spread until the last few hundred years of human history.

When I began studying immunology, something that surprised me is how much of the immune system is focused on fighting parasites. There is an entire branch, including several cell types, devoted to this. It seems like such a mismatch to the modern, industrialized world. “Can I have a few more immune cell types focused on viruses or intracellular bacteria?” I thought, “in exchange for some of these parasite-focused cells that I’m not using??” Our “old friends” are quite different from the crowd infections that plague us now– it would be bizarre to assume that research based on one of these categories will apply to the other!

Old Friends Hypothesis is more descriptive than the Hygiene Hypothesis

Our “old friends” parasitic worms and beneficial microbes are associated with a reduced risk of allergies and autoimmune diseases. No such relationship exists for crowd diseases. In fact, the opposite is true. Crowd diseases contribute to allergies and autoimmune diseases. Comparing the immune system to a muscle that gets stronger with use is overly simplistic and, in many cases, inaccurate. There is huge variety in how various pathogens impact us. Being precise in considering different types of microbes and infections will allow us to better understand human health.

Related reading:

• Gross ways to live long: Parasitic worms as an anti-inflammaging therapy?
• Microbial ‘old friends’, immunoregulation and socioeconomic status
• Unraveling the Hygiene Hypothesis of helminthes and autoimmunity
• Surprises of the Microbiome
• Viruses: The Silent Triggers of Autoimmune and Neurodegenerative Diseases

When T Cell Binding Makes a Big Difference

T cells play a role in autoimmune diseases like Type 1 diabetes (where T cells mistakenly destroy insulin-producing cells in the pancreas) and psoriasis (where T cells attack our own skin). T cells are also crucial to a healthy response to cancer. In cancer, something has gone amiss: cells undergo uncontrolled growth. T cells should put a stop to this, but in many cases T cells have been misleadingly soothed into exhaustion.

An increasing number of cancer therapies are based on T cells, and predicting which T cells are the most promising cancer-fighting candidates is crucial. When all is in harmony, our T cells should keep cancer from starting in the first place by quickly identifying and killing any cells which have gone rogue. T cells do this by recognizing small pieces of protein, called peptides, held out to them on special molecules called MHC. If those proteins are the sign of a good cell gone bad, the T cell can destroy the miscreant cell.

We can help the immune system by hanging up WANTED posters with the faces of outlaw proteins on them. This can let the immune system know who to look out for.

In practice, this is done by pairing a cancer peptide (called a “neoantigen”) together with a danger signal, and injecting them into a patient in order to sound the alarm and spur T cells into action. We need to identify which rogue cancer proteins are the most promising candidates for such a therapy. Neural networks are being used for this task!

One key question is which peptides are the best candidates to use in these therapies. A key criteria is that T cells need to be able to recognize the peptide, which means that MHC molecules need to be able to bind to it. Everyone has different T cells and MHC molecules, since these are created by the most diverse genes in the human genome! The answer must be personalized.

Applying Neural Networks

This seems like a question where neural networks could be helpful. In part 2, we discussed several different architectures (sequences of math operations) that we could choose between. One consideration is whether the input to a problem is fixed length or variable length. Are we identifying pictures of dogs, or reading Anna Karenina? The problem of T cell binding can be (and is!) framed both ways, depending on the team and the approach they choose to take. In general, sequences of amino acids can have a huge variety of lengths. However, T cell receptors and the peptides held out by MHC have lengths within a narrow range. As a result, we could use neural net architectures that have developed for fixed length problems. Or we could take advantage of the engineering and large training sets that already went into creating neural networks for variable length proteins. Strong research is being published of both types, and I am curious to see if there will be a consensus 5 years from now on the best approach.

To Use AlphaFold or Not?

As discussed in part 1, the computer program AlphaFold made headlines in 2020 for its incredible accuracy in taking a 1D sequence of amino acids as input and producing a 3D protein structure as output. It won an international competition for this task by a huge margin, smashing all previous records. Since T cell receptors (TCRs), MHC molecules, and peptides are all proteins, it makes sense that we could modify it for the T cell problem.

AlphaFold (and similar models, such as RosettaFold and ESMFold) are excellent at what they do, which is to take a sequence of amino acids and predict the 3D structure of a protein. However, it is possible that these models are simultaneously both overkill and underkill for our T cell question. AlphaFold can handle much longer amino acid sequences than are found in the T cell binding problem, yet it doesn’t account for T cells having not just one, but a pair of TWO sequences of amino acids (known as the alpha and beta chains). AlphaFold was trained on a massive data set, yet that dataset did not include much in the way of T cell or MHC specific data. Some researchers have chosen to modify AlphaFold, adding additional layers for prediction, or fine-tuning on a more relevant dataset, whereas other teams have chosen to design neural networks that are more directly suited to the task at hand.

Which approach is better? It’s tough to tell. Reading papers within the field can be confusing, as many boast of great results. Since there is an absence of consistent benchmarks and models are often not evaluated for accuracy on new peptides, it can be tough to compare different models.

The Need for a Grand Challenge

You may wonder why we can’t clearly see which neural network for T cell binding is definitively the best. The ImageNet competition produced a clear winner for recognizing objects in images in 2012: AlexNet, a neural net that was head and shoulders better than the other entries. Similarly, AlphaFold dominated the CASP competition in 2020. However, several factors are necessary to create competitions such as ImageNet or CASP, and these factors do not yet exist for the T cell problem.

You need a large, well-curated, and representative dataset. This can fail in numerous ways. While data is central to any machine learning project, all too often, data work has been undervalued and neglected. The paper “Everyone wants to do the model work, not the data work” drew on interviews with 53 machine learning practitioners across multiple countries. Building fancy math models is rewarded with career advancement, not the arduous labor of collecting and curating the underlying datasets that make those models possible. In many cases, those tasked with collecting data had it piled on top of already demanding jobs, with inadequate communication, training, or compensation. This led to erroneous measurements, miscommunications, and failed projects. To give one example, when machines in a robotic medical AI project were recalibrated, the change was not documented, leading to inconsistent data that was impossible to interpret.

Data work is particularly challenging in the area of T cell receptors (TCRs). Current datasets cover just a tiny fraction of all possible TCRs. It is difficult to scale experimental methods, so less than 1 million unique TCR-peptide pairs have been found experimentally. Of the data that exists, the vast majority just contains one of the two amino acid sequences which make up a TCR, even though both sequences are significant.

You also need everyone to agree on what numerical metrics to use in evaluating success, and for those metrics to closely correspond to the value you care about. Great harm can result when those metrics are a poor proxy for the underlying goal. Providing a meaningful numeric score and declaring a winner require the right setup and a lot of effort behind the scenes.

Some researchers have called for a “grand challenge” of inferring T cell receptor binding, similar to the grand challenge in prediction protein folding which led to AlphaFold. Standardizing datasets and setting clear metrics for comparisons could motivate innovation. This is a high-impact area within immunology and medicine and I hope that it is a continued focus of research!

This post is part 3 in a series on using AI to predict T cell binding. Here are part 1 and part 2.

Thank you to Jeremy Howard for feedback on an earlier draft of this post.

You can subscribe to be notified of new blog posts by submitting your email below:

What are Neural Networks?

Neural networks are a type of machine learning algorithm that are currently in the limelight, powering chatbots like ChatGPT and image generators like Midjourney. Neural networks are just math equations, written in computer code. When people hear the terms “neural networks” or “artificial intelligence,” they may picture humanoid robots, but it is more accurate to imagine a scaled-up version of 10th grade math class.

When considering an AI system, two major categories of tasks are:

• Classifying things: These are not predictions in the sense of predicting the future, but rather, guessing the answer to a question. Is this a picture of a chihuahua or a blueberry muffin? Which of these potential drugs would be most likely to target antibiotic resistant bacteria?
• Generating things: Create a picture of Kanye West made out of tiny Captain Picard faces. Generate a new molecule that may work as an antibiotic.

Some problems in medicine and immunology can be framed either as a classification problem OR as a generative problem. For instance, suppose your goal is to produce a new antibiotic to address antimicrobial resistance. Some scientists are trying to classify which compounds, out of hundreds of millions of existing ones, may have the desired properties, whereas others are trying to generate new compounds. Both approaches are useful.

Language Models

You may have heard the term “language model” describing chatbots (such as Chat-GPT or Claude) and automated language translation (such as Google Translate and Skype Translator). One approach to developing sophisticated language capabilities is to write a computer program that predicts the next word in a sentence. Predict the next word and then the next, and you can generate strings of text. This is a language model. Both prediction and generation are used to build language models.

Along the way, the model has learned a number of patterns regarding context, vocabulary, and relationships. Fine tuning lets us use these capabilities for specific problems. For instance, a language model trained on medical chart notes was used to make predictions about which patients were most likely to be readmitted to the hospital. While the model was trained on a prediction task (predicting the next word in a sentence), it can now be used for generation as well when combined with some additional techniques when it is given additional instructions and training from humans.

When language models are trained, they are not always just tasked with predicting the next word. In a sentence, sometimes random words are covered up, and the computer must predict what goes in that spot. It is a fill-in-the-blank assignment.

Through predicting words, the model can learn which other words to focus on. Suppose I told you, “I am so tired that I can barely keep my XXXX open”. You could likely guess that the missing word is “eyes”. To do so, you may have focused on “tired” and “open”. Some of the word positions add little, if anything, to your efforts of deduction. In a similar way, attention-based neural networks learn what to pay attention to, where to focus.

Like the sequence of words in a novel, a sequence of amino acids is its own language, telling the story of a protein. The same techniques and approaches that were developed for natural languages (such as translating between English and French) can also be applied to the biological language of protein creation.

Popular architectures

Neural networks are often created from a few popular architectures, which are particular combinations of mathematical operations. Something that can be confusing is that different architectures can often be combined with one another. Also, most of these architectures can be used for both predictive and generative problems. Here are a few particularly popular ones:

• Multi-Layer Perceptrons (MLPs): the oldest example of a neural network was developed in the 1960s! It is the basis of many systems used today (Rosenblatt 1962).
• Convolutional Neural Networks (CNNs): are like a filter that you scan across a picture, inch by inch, to recognize what is in it. This is useful for programs such as those bird-watcher apps that will tell you what type of bird you spotted, based on the photo you upload (Lecun et al. 1998; Krizhevsky et al. 2012).
• Recursive Neural Networks (RNNs): can take variable-length input (that is, sequences) and store state. Google began using these for translating between languages in 2016 (Hochreiter and Schmidhuber 1997; Wu et al. 2016).
• Transformers: offer a more efficient approach for sequences, which can be any length. These are based on attention, as explained above. These have now replaced RNNs in many applications, including Google Translate, which lets a user enter a word or phrase, or even entire paragraphs, and get back the response in another language (Vaswani et al. 2017).
• Graph Neural Network (GNNs): Allow for geometric relationships in the input to be captured. For example, GNNs can help represent the spatial relationships between different amino acids in a sequence. Principles of GNNs are typically combined with other options in the list above (Scarselli et al. 2009; Micheli 2009).

How do you choose a neural network for your problem? One key consideration is how much the size of your input varies. Suppose you have a collection of photos and are trying to determine which are of cats and which are of dogs. Your photos are likely of similar sizes, and you can make them the same by adding a frame around the smaller ones. Whole families of neural networks have been developed for such problems with fixed-size input.

Now suppose you are developing a neural network that can read books and predict which word will come next. The difference in length between The Hungry Hungry Caterpillar and Les Miserables is huge. For this problem, we need a network that allows variable-size input. RNNs and Transformers allow variable-length input, whereas CNNs take fixed-size input.

The Innovation of AlphaFold

Recall from Part 1 of this series that the computer program AlphaFold revolutionized the task of taking a 1D sequence of amino acids (shown on the left), and transforming it into a 3D molecule (shown on the right).

The text of a book is 1-dimensional. It could be written on a single long ribbon, stretched out, and read as such. However, best making sense of amino acid sequences requires an additional dimension. We want to know not just the long ribbon of a single sequence, but the evolutionary relationships of related sequences it is similar to.

A crucial innovation of AlphaFold is that it expanded attention to 2 dimensions. The 2 dimensions are the sequence of the amino acids, as well as a block of sequences of similar sequences. These similar sequences provide important evolutionary clues of what our protein of interest may be like. Like distant cousins in a family tree, they can give us insights of what may be some common features. This set of related sequences is known as a Multiple Sequence Alignment (MSA) and is a key component in bioinformatics. AlphaFold applies attention to the MSA, learning which parts of related sequences to focus on in trying to decode a protein structure.

Returning to our list of popular architectures, AlphaFold combines aspects of Transformers (in a new form, named the Evoformer, which has 2D attention) and Graph Neural Networks (to represent the spatial relationships between amino acids). Now that we have covered some of the types of neural networks and the structure of AlphaFold, we are ready to combine this with the information about T cells from part 1. Check out part 3 to see how AI is being used to predict T cell binding.

And for understanding the ethical risks of AI systems, you may be interested in reading my previous posts:

• “AI will cure cancer” misunderstands both AI and medicine
• AI and Power: The Ethical Challenges of Automation, Centralization, and Scale
• 11 Short Videos About AI Ethics

Thank you to Jeremy Howard for feedback on earlier drafts of this post.

You can subscribe to be notified of new blog posts by submitting your email below:

T cells, our multi-purpose immune heroes

At their best, T cells destroy viral-infected cells and stop cancer before we ever develop symptoms. When things go awry, T cells can accidentally attack our own cells – for instance Type 1 Diabetes occurs when T cells mistakenly destroy insulin-producing cells in the pancreas and psoriasis is caused by T cells attacking the skin (check out this 5 minute video I made for more information about confused T cells). Or “exhausted” T cells can sit idly by as tumors rapidly grow.

T cell receptors (TCRs) are protein complexes on the outside of a T cell. The way a T cell recognizes a problematic cell is by binding to a small piece of protein, called a peptide. T cells can bind only to peptides that are presented to them on fancy platters– err, I mean on special molecules called MHC molecules, found on the outside of cells. The binding of TCR-peptide-MHC is like a secret handshake. The TCR is very specific about which peptides it can bind to. Most combinations will not work.

Since TCRs are made of protein, and they bind to small pieces of protein, it is helpful to consider some related questions about proteins before we tackle TCR binding in more detail.

Proteins: workhorses of the cell

Proteins make life as we know it possible; they are responsible for the hard work that goes on within our cells. Proteins give our cells shape and structure, and help them stay organized. Most biochemical reactions occur thanks to enzymes, which are entirely made of proteins. And hormones are proteins that send messages around the body.

The 3D shape of a protein is crucial to understanding how it functions within the body and what roles it plays. However, the information we often receive about proteins is not 3D. It is a list of letters – a sequence of amino acids, the building blocks of proteins. These lists can twist and fold, creating spirals, sheets, and other interesting sculptures.

For example, insulin is an important protein messenger that lets our cells know when they should absorb more sugar. Insulin consists of this sequence of 110 amino acids (there are 20 distinct amino acids, and each has a letter representing it):

MALWMRLLPLLALLALWGPDPAAAFVNQHLCGSHLVEALYLVCGERGFFYTPKTRREAEDLQVGQVELGGGPGAGSLQPLALEGSLQKRGIVEQCCTSICSLYQLENYCN

However, to really understand insulin, we need to know what it looks like as a 3D molecule. Its shape and structure helps determine which other molecules will interact with it.

The insulin receptor (which is on the surface of cells and can bind to insulin, receiving the message and sending off an internal signal within the cell) is also a protein. It is a sequence of over 1,000 amino acids and has a far more complicated structure.

A new champion: AlphaFold

This question of converting amino acid sequences (on the left) to 3D structures (on the right) has wide-reaching implications within biology and medicine.

In 1994, a competition was established for researchers to go head-to-head in trying to best answer this question. Leading up to the competition, as scientists discovered new protein structures, using laboratory techniques such as X-ray crystallography or NMR spectroscopy, they kept some of the results secret. Later, these structures would be freely shared with other scientists, but for now, they must be held apart for the challenge. The somewhat dry name, Critical Assessment of Structure Prediction (CASP), belies the importance of this task.

In the competition, each team submits their guesses, generated by computer programs they created. The distance between the guesses and the actual structures is measured in Angstroms, with 1 Angstrom being one just ten-billionth of a meter.

CASP did not make international headlines until 2020. This is when an entry, titled AlphaFold smashed all existing records, and all competitors, by a wide margin. AlphaFold was created by DeepMind, an AI company owned by Google.

Challenges with T cell data

At first glance, it may seem like AlphaFold would give us a solution to our T cell problem. After all, T cell receptors are made of proteins, and they bind to other proteins. Aren’t protein structures a solved problem? Not so fast. The T cell receptor problem poses many different and additional challenges.

There are a few factors that make the question of what a T cell will bind to– known as T cell receptor specificity– distinctive and more complicated than the problem AlphaFold solves. First of all, AlphaFold was created to predict protein structures from a single chain of amino acids. However, TCRs have 2 key chains of amino acids: alpha and beta. Both are important. These alpha and beta chains can be mixed and matched willy-nilly.

Another issue is that we have far less data on TCRs than we do on general proteins. The genes that code for the TCR vary greatly across the population. They are some of the most diverse genes in the entire human genome! This means that the (relatively small) dataset we do have is only a fraction of what exists. AlphaFold is based on machine learning, and machine learning models are highly dependent on the quality, quantity, and nature of the data used to train them.

This post is part 1 in a series on using AI to predict T cell binding. Here are part 2 and part 3.

Thank you to Jeremy Howard for feedback on earlier drafts of this post.

You can subscribe to be notified of new blog posts by submitting your email below:

Microbes are everywhere

When I lived in California there were several years in which we had widespread wildfire smoke that was harmful to breathe, polluted the air, and created an eerie yellow hue. I knew that smoke contains toxins, and I incorrectly assumed that it wouldn’t be a suitable habitat for anything living. Counter-intuitively, smoke is a good habitat for some microbes, as the particulate matter helps block UV rays that might otherwise kill them. Water vapor produced by the combustion of biomass can help prevent microbes from drying out. And convective columns and updraft winds can draw in microbes from long distances.

One study on the California wildfires found that hospitalizations for coccidioidomycosis, a dangerous fungal infection, increased by 20% in the month following smoke exposure. This is a worrying threat given that the climate crisis has led to increasing frequency and intensity of wildfires. There is even a nascent field of pyroaerobiology, studying the aerosolization and transport of microbes by wildfires.

An estimated 5-13 million tons of plastic enter the oceans each year, including run-off from polluted rivers. Floating “islands” of plastic debris, including microplastics less than 5 mm in size, are found in the oceans. These plastics impact over 700 different species of aquatic life, ranging from microscopic phytoplankton to large whales. There are microbes that thrive on microplastics, forming the Plastisphere. Unfortunately, this thin film of bacteria and fungi living on microplastics may make them more appealing to some sea animals, such as loggerhead turtles, to eat. Researchers have found that the Atlantic and Pacific oceans have distinct Plastisphere communities, and that these differences may necessitate different remediation approaches.

Researchers are also investigating how microbes can be used to break down these dangerous plastics. One interesting research study collated data on over 16,000 genes involved in plastic degradation. These genes came from 6,000 microbial species, showing that there are numerous possibilities of potential microbes that may be used for bioremediation. Researchers have even found some microbes can be used to help break down PFAS, often called “forever chemicals”, which are polluting aquatic environments. While it is very difficult to break down the fluorine-carbon bonds found in PFAS, microbes can instead first break chlorine-carbon bonds in the chemicals, which then makes it easier to remove the fluorine atoms.

There is no clear “good” vs. “bad” for microbes

Pseudomonas syringae is a plant pathogen that is a scourge of agricultural crops. It infects almost all economically valuable species of crops and threatens global agriculture production.

But actually it’s not all bad – this same species of bacteria is important to weather precipitation cycles. For temperatures above -40 degrees C, ice formation requires a catalyst, called an ice nucleator. Researchers have discovered that most ice nucleators are biological: tiny microbes that are widely dispersed throughout the atmosphere all over the world, even in Antarctica! And it turns out that Pseudomonas syringae is a common nucleator for catalyzing precipitation. This example highlights the complexity of microbial ecosystems. Efforts to protect plants from this pathogen must take into account the interrelated impact on precipitation.

Another example of a microbe that can’t be clearly classified as “good” or “bad” is Prevotella Copri, found in the human gut microbiome. An aptly titled paper, “The curious case of Prevotella copri” describes”puzzling discrepancies” in how various research studies have associated this species with a range of positive and negative impacts on disease.

The authors share that high abundance of P. copri is associated with non-Western diets, rheumatoid arthritis, Ankylosing spondylitis (a type of inflammatory arthritis), Graves’ orbitopathy (autoimmune condition that can damage vision vision), diabetes mellitus, and cancer. This is particularly surprising, because several of the inflammatory and autoimmune conditions listed are more strongly associated with Western diets. Low abundance of P. copri is associated with Western diets, glucose intolerance, chronic hives, Parkinson’s disease, and food allergies.

It is unclear whether abundant Prevotella copri in the gut is good or bad. The paper notes that P. copri has a diverse set of strains, which differ from the reference strain, and may partly explain the discrepancies. The ways that gut microbes interact with each other, our immune systems, and our genetics can be quite complex, and these factors may mediate how P. copri influences a person’s health.

The Social Microbiome

We often talk about microbiomes at an individual level: the microbes inside the guts of a person (or insect) or the microbes on their skin or in their mouth. However, researchers also study the social microbiome: the metacommunity of all the microbiomes of animals in a social community. Members of a community can influence each other’s microbiomes through their social bonds.

One study sampled the skin microbiome of roller derby players before and after games. The three teams in the study each came from a different city and state. They found that prior to a game, the microbiomes were more similar within a team, and different across teams. However, after competing in a game, which involves a fair amount of physical contact, all of the players’ skin microbiomes were more similar to one another.

Much of the research on the social microbiome has been done on nonhuman primates, where grooming, mating, feeding, and other close contact have been shown to impact the microbiome. Social networks and group membership were found to determine the microbiomes of baboons, and were more predictive than age or gender.

For some animals, microbiomes inside of scent glands are used to communicate group membership. Hyenas typically live in social clans of 40-80 individuals, and these clans are often overlapping in their territories. They secrete fermentative bacteria from their scent glands. This allows for group-specific odors, making it easier for the hyenas to communicate if they are in the same group as one another. One study found that the relative abundance of different types of bacteria was distinctive for different hyena clans.

More to learn

Understanding the microbiome is a crucial piece of better understanding the health of plants, humans, and other animals, as well as our environment. Microbes synthesize vitamins, catalyze precipitation, spread disease or protect from it, can remediate or exacerbate the impacts of pollution, and influence food chains. Only with relatively recent advances in genomic sequencing have we been able to better study and compare the balance of species composing different microbiomes. This is a fascinating and growing area of research.

When the immune system attacks friendly bacteria…

Commensal bacteria, often referred to as “friendly”, are bacteria that live on human skin or in the gut and help us. The human gut microbiome can help synthesize vitamins we need, regulate dopamine, and protect us from infection. However, so-called friendly bacteria can cause grave problems when they cross out of the gut into the bloodstream. Microbes can’t necessarily be classified as “good” or “bad”, and this impact of what happens when microbes cross from the gut to the blood is an example of this.

I created a short 5 minute video to explain these ideas, geared for a general audience:

If “friendly” microbes stray into the bloodstream, our immune system may mount an attack against them by producing antibodies and activating T cells (the two primary approaches of our adaptive immune systems). The small regions on a bacteria recognized by T cells or antibodies are known as epitopes. All bacteria have many epitopes, and genetics helps determine which particular epitopes your immune system is able to recognize. Two different people will likely produce T cells recognizing different epitopes for the same bacteria.

If the epitopes our immune system attacks are too similar to regions on our own cells, autoimmune disease can develop. This means that the friendly bacteria that has leaked into the bloodstream (where it does not belong) “looks” too similar to some of your own cells. For instance, if the bacteria “looks” like your pancreatic cells (which produce insulin), your T cells may begin mistakenly attacking these insulin-producing cells, leading to Type 1 Diabetes.

The microbiome can also play a role in how likely microbes are to leak out of a person’s gut in the first place! Low levels of certain commensal bacteria increases gut permeability. This heightened permeability allowing the microbes to leave the intestines. This process has been linked to progression of Type 1 Diabetes to more advanced stages. Note that commensal bacteria are just one potential trigger for the development of autoimmune diseases. As I discussed in a previous post, viral infections can also trigger Type 1 Diabetes and other autoimmune diseases.

Bacteria travel on fungal highways in the plant microbiome

Plants also have microbiomes. One component of the plant microbiome are fungi that surround the plant roots and the bacteria that travel along these “fungal highways.” The fungi exude sugars that the bacteria use as an energy source along their journeys. The bacteria are able to mineralize organic phosphate, which is beneficial to the fungus and to the plant. Together, the fungus, plant, and bacteria are in symbiotic relationships.

Watch this short 90 second video to see bacteria traveling along a fungal highway in real-time. You won’t believe how fast they are!

For more on fungal highways, here is a 3 minute Science Fridays video which discusess how fungal networks optimize traffic to avoid traffic jams. It’s created by Christian Baker, who does a great job of explaining science in a succinct and engaging way. In this video, he interviews math professor Dr. Marcus Roper of UCLA who studies the fluid dynamics of bacterial travel.

Other researchers are looking at the use of fungal highways to allow pollutant-degrading bacteria to travel, which could help remove pollutants in a broader area as part of bioremediation efforts.

Plant microbiomes have a strong influence on their immune function. They help prevent the colonization and growth of pathogens. Unfortunately, climate change is altering the microbiomes of plants, which can make them more susceptible to disease. There are several mechanisms by which this can happen. Climate change alters the composition of microbes found in soil, which is the first line of defense for plants against pathogens. Plants release chemicals and metabolites into the soil, which are useful for attracting beneficial microbiomes, and climate change can harm plants’ capacity to do this (at least in the same quantities as before). Both plants and microbes may migrate into new areas due to changes in climate, further disrupting previous equilibriums.

Study of the microbiome is both old and very new

My microbiome course began with an in-depth unit covering different methods for sequencing genomes. There have been multiple revolutions in gene sequencing technology in recent decades, and new machines for gene sequencing are released regularly. Our growing understanding of the microbiome would not be possible without this technology to sequence numerous bacterial species in a sample, and in many ways the field is very young, with lots of open questions remaining.

Even though microbiome research has only taken off in the last 20 years with the advent of new sequencing technologies, ideas about the microbiome and its role in health have been around for a long time. Antonie van Leewenhoek used a microscope to compare the microbes in his oral and fecal microbiota in the 1680s.

Élie Metchnikoff, who won the Nobel Prize in Medicine 1908, is best known as the father of cellular immunity. He discovered that there are immune cells that can ingest pathogens. He named them “eating cells” (phagocytes), and we now refer to these immune cells as “big eaters” (macrophages). In addition to discovering cellular immunity, Metchnikoff is considered one of the founders of probiotics. He believed consuming sour milk (yogurt) would be beneficial for the microbiome and he laid the conceptual framework for fecal transplantation to modify the microbiome, over 100 years ago. Questions about whether and how we can alter our microbiomes are still being studied and debated today, as are the roles of genetics, lifestyle, and environment in influencing the microbiome.

Stay tuned…

Please subscribe to my blog to be notified of Part 2, which will describe the microbiomes of nanoplastics and wildfire smoke, as well as some puzzling examples of microbes that are both good and bad.

In the meantime, you may be interested in a previous post of mine that discussed the microbiomes of insects, which can help produce both essential nutrients and defensive toxins to use against predators. Read more in 4 Things I Learned About Bugs.

Submit your email below to be notified of new posts:

Possibilities with Deep Learning

Machine learning uses data to find patterns, as opposed to relying on manually coded rules. Deep learning is a powerful family of machine learning algorithms, responsible for the recent breakthroughs in artificial intelligence. The field of has seen massive improvements in accuracy and speed, and reductions in cost. Key reasons for this progress are the affordability of GPUs (processors used in video gaming which can quickly perform numerical computations in parallel), increases in available data, and research advances in the underlying algorithms (LeCun, Bengio, and Hinton 2015).

Deep learning-assisted approaches to drug discovery offer a potentially faster and cheaper way to identify new antibiotic candidates. It is estimated that there are 10³⁰–10⁶⁰ potential drug-like chemicals; this scale means it is impossible to explore even a fraction of these (Melo, Maasch, and de la Fuente-Nunez 2021). Algorithms can help determine what compounds to prioritize.

Protein Folding Approaches with AI

AlphaFold has revolutionized protein structure prediction (Jumper et al. 2021, my summary here) and there has been interest in using it to aid in antibiotic discovery. Researchers applied AlphaFold to the essential proteins from E. coli (Wong et al. 2022). They then used molecular docking programs to predict if these sites would be able to bind with over 300 different existing drug compounds. When they tested the predictions experimentally in the lab, they found that accuracy was relatively low. One limitation of the approach is that AlphaFold does not differentiate between active vs. inactive conformations. The researchers highlighted the need for better scoring functions to accurately rank the binding poses and for benchmarking datasets to evaluate performance (Wong et al. 2022).

A more recent pre-print (not yet peer-reviewed) offered a potential explanation of AlphaFold’s poor performance on predicting existing drugs (Lyu et al. 2023, news coverage). The authors proposed that minor differences in structure may cause AlphaFold to miss existing drugs, but that it may be identifying equally promising ones. Starting with 2 known protein structures, the researchers used AlphaFold to virtually screen millions of potential drugs and then synthesized a few hundred of the most promising candidates. For comparison, they also followed a more traditional process to screen and synthesize drug candidates. Even though AlphaFold and the traditional process yielded completely different drug candidates, they had similar success rates of how many of the candidates bound to the proteins of interest as predicted (Lyu et al. 2023).

Predicting Antibacterial Activity with AI

Other work has not relied on protein structure prediction software like AlphaFold, but instead created deep learning models directly for the underlying problem. Researchers used training sets of thousands of small molecule drugs to build models predicting which will have antibacterial activity (Stokes et al. 2020; Liu et al. 2023). Message-passing neural networks, a type of deep learning model, allow for iteratively exchanging information between adjacent atoms and bonds. This captures some of the structural information of a protein, without explicitly predicting protein structure. A model was created using a dataset of thousands of repurposed drugs, and then applied to chemical libraries containing over 100 million molecules (Stokes et al. 2020). The output was a ranking of the most promising drug candidates based on predicted antibacterial activity, chemical structure, and availability. This approach predicted that Halicin, which is structurally distinct form existing classes of antibiotics, would be an effective drug. When tested empirically, Halicin demonstrated a broad-spectrum impact against bacteria including M. tuberculosis, carbapenem-resistant Enterobacteriaceae, C. difficile, and multi-resistant A. baumannii (Stokes et al. 2020).

A similar approach was used by the same lab to train a deep learning model to predict antibiotics that would be effective against multi-resistant Acinetobacter baumannii (Liu et al. 2023). The WHO has designated drug resistant A. baumannii as one of the highest priorities for urgent development of new antibiotics. The model discovered a new antibiotic drug, Aubacin, which is structurally distinct from existing classes of antibiotics. In a lab experiment, Aubacin was effective at supressing A. baumannii in a mouse wound. It was also effective when tested against 41 different strains of resistant A. baumannii from the CDC Antibiotic Resistant Isolation bank (Liu et al. 2023).

Data Sources and Tools

AI models are highly dependent on the data used to train them. Training data shapes the accuracy of a model, helps set what possibilities are considered, and determines the context in which it is appropriate to use. One useful data source is the Drug Repurposing Hub (Corsello et al. 2017), which contains a hand-curated collection of thousands of existing drugs, including ones where safety was established in clinical trials, but that never obtained regulatory approval. Applying existing drugs to new diseases other than that which they were originally developed for is a pragmatic approach that can have a rapid and lower-cost impact. Another useful tool is the ZINC15 database, which includes data on small molecules, including their biological activity and chemical properties (Sterling and Irwin 2015). Both the Drug Repurposing Hub and ZINC 15 databases have been used in neural network approaches (Stokes et al. 2020; Liu et al. 2023).

RDKit is an open-source library that can be used to compute molecular features. Some researchers use these computed features to augment the data gathered purely from existing structures and ligands, using one of the above databases together with RDKit-computed features as inputs to their models (Liu et al. 2023).

Risks and Limitations

The research described above of antibiotics discovered through deep learning-assisted processes is still in preliminary stages, with none of those drugs having yet begun clinical trials, much less reached regulatory approval. In other fields, machine learning models have sometimes performed well in testing but failed to work as expected when deployed in the real world (Sambasivan et al. 2021). Issues can arise when the context and quality of the training data is not fully understood (Gebru et al. 2021). ML systems are trained in clearly defined environments, while the physical world often has complex and volatile underlying phenomena, which are not fully accounted for during training. This can create a brittleness in ML-generated solutions. Moreover, there is a conflicting incentive system in which the creators of ML models are rewarded with prestige, but the arduous work used to gather and curate the data used to train those models is often neglected and taken for granted (Sambasivan et al. 2021).

However, given the urgency of the antimicrobial resistance crisis, it is important that we consider a variety of innovative solutions for tackling it. Deep learning can be a powerful tool, and approaches to antibiotic discovery should continue to be pursued.

You can subscribe to be notified of new blog posts by submitting your email below:

How Automated Decision Making is Used

Automated computer systems, often involving AI, are increasingly being used to make decisions that have a big impact on people’s lives: determining who gets jobs, housing, or healthcare. Disturbing patterns are found across numerous countries and a range of systems: there is typically no way to surface or correct errors, and all too often the goal is to increase corporate and government revenues by denying poor people resources they need to survive.

A woman in France had her food benefits reduced by a computer program and can no longer afford enough to eat. She talked to a program officer, who said the cut was due to an error in the computer program, but was unable to change it. She did not have her food benefits reinstated. The system was designed such that the computer is always considered correct, even when humans recognize an error. This woman was not alone; she was one of 60,000 people who went hungry due to these errors.

A man in Australia was told that he had been overpaid welfare benefits and that he was now in debt to the government. The debt was an error, based on an intentionally faulty calculation as part of the computational RoboDebt program. However, the man had no way to contest it. Despondent, he died by suicide. This was not a one-off incident. The Australian government was later found to have wrongly created debts for hundreds of thousands of people.They were putting poor people into debt with a flawed calculation system, ruining lives efficiently at scale. The government had increased the number of poor people it put into debt each week by 50x, compared to before RoboDebt.

A woman in the USA with cerebral palsy needed a health aid to help her get out of bed in the morning, to get her meals, and to complete other basic tasks. Her care was drastically cut due to a computer bug. She was given no explanation and no option for recourse as her quality of life drastically plummeted. Only through a lengthy court case was it finally revealed that many people with cerebral palsy had wrongly lost their care due to a computer error.

Patterns in Automated Decision Making

These examples always flow in the same direction. Professor Alvaro Bedoya, the founding director of the Center on Privacy and Technology at the Georgetown University Law Center, wrote “It is a pattern throughout history that surveillance is used against those considered ‘less than’, against the poor man, the person of color, the immigrant, the heretic. It is used to try to stop marginalized people from achieving power.” The same pattern is found in the role of technology in decision systems.

The goal of many automated decision systems is to increase revenues for governments and private companies. When this is applied to health and medicine, the goal is often achieved by denying poor people food or medical care. People often trust computers to be more accurate than humans, in a bias known as automation bias. A systematic review of 74 research studies found that automation bias exists across a range of fields, including healthcare, exerting a consistent influence. This bias can make it harder for people to recognize errors in automated decision-making. Moreover, implementing mechanisms to identify and correct errors is often seen as an unnecessary expense.

In all of the cases above, the people most impacted (those losing access to needed food or medical care, or unjustly being thrown into debt) recognized the errors in the system earliest. Yet the systems were built with no mechanism for recognizing mistakes, for allowing the participation of those impacted, nor for providing recourse to those harmed. Unfortunately, this can also be the case in medicine.

How the Medical System Operates

An AI algorithm that reads MRIs more accurately would not have helped neurologist Ilene Ruhoy, MD, PhD, when she developed a 7 cm brain tumor. The key obstacle to her treatment was getting fellow neurologists to believe her symptoms and even order an MRI in the first place. “I was told I knew too much, that I was working too hard, that I was stressed out, that I was anxious,” Dr. Ruhoy recounts. Eventually, after her symptoms worsened further, she was able to get an MRI and urgently sent in for a 7 hour surgery. Because of the delay in her diagnosis, her tumor was so large that it could not be completely removed, which has led to it growing back since her first surgery.

Dr. Ruhoy’s experience is sadly common. While Dr. Ruhoy lives in the USA, a study in the UK found that almost 1 in 3 patients with brain tumors had to visit doctors at least 5 times before receiving an accurate diagnosis. Again, MRI-reading AI can not help these patients whose doctors won’t order an MRI in the first place. On average, it takes Lupus patients 7 years to receive a correct diagnosis, and 1 in 3 are initially misdiagnosed with doctors incorrectly claiming mental health issues are the root of their symptoms. Even healthcare workers are often shocked at how quickly they are dismissed and disbelieved once they become patients. For instance, interviews with a dozen healthcare workers revealed that their colleagues shifted to discarding their expertise as soon as they developed Long Covid.

This disregard of patient experience and patient expertise severely limits medical knowledge. It results in delayed diagnoses, misdiagnoses, missing data, and incorrect data. AI is great at finding patterns in existing data. However, AI will not be able to solve this problem of missing and erroneous underlying data. Furthermore, there is a negative feedback loop around lack of medical data for poorly understood diseases: doctors disbelieve patients and dismiss them as anxious or complaining too much, failing to gather data which could help illuminate the disease.

The Wrong Data

Even worse, often research problems are reformulated to shoe-horn inadequate data sources in. Analyzing electronic health record data is cheaper than searching for new causal mechanisms. Medical data is often limited by the categories of billing codes, by what doctors choose to note from a patient’s account, and what tests are ordered. The data are inherently incomplete.

Medical bias is widespread, with studies documenting that doctors give less pain medication to Black patients than to white patients for the same conditions. On average, women have to wait months or years longer than men to get an accurate diagnosis for the same conditions. This impacts the data that is collected and will be used for AI. Multiple research studies have shown that AI not only encodes existing biases, but can also amplify their magnitude. At heart, these biases often pivot on not believing marginalized people about their experiences: not believing when they say that they are in pain nor how they report their symptoms.

On a deeper level, ignoring patient expertise limits what hypotheses are devised, and can slow research progress. These issues will propagate into AI, unless researchers seek ways to include meaningful patient participation. These problems are not unique to any one country or any one type of medical system. Patients across the USA, UK, Australia, and Canada (the 4 countries I am most familiar with) are all well-documented to experience these issues.

Being Honest About the Risks and Opportunities

While AI holds transformative possibilities for medicine, it is important that we are clear-eyed about both the risks and the opportunities. Many talks on Medical AI give the impression that the only thing holding medicine back is a lack of knowledge. The many other factors that influence medical care, including the systematic disregard for patients’ knowledge of their own experiences, are often ignored in these discussions. Ignoring these realities will lead people to design AI for an idealized medical system that does not exist. There is already a clear pattern in which AI is used to centralize power and harm the marginalized. In medicine, this could lead to patients, who are already disempowered and often disregarded, having even less autonomy or voice.

In my view, some of the most promising areas of research are participatory approaches to machine learning and patient-led medical research. The Participatory Approaches to Machine Learning workshop at ICML included a powerful collection of talks and papers on both the need and the opportunities for designing systems with greater participation of those impacted. AI ethics work on topics of contestability (building in ways for participants to contest outputs) and actionable recourse are necessary. For medical research more generally, the Patient Led Research Collaborative (focused on Long Covid) is an encouraging model. I hope that we can see more efforts within medical AI to center patient expertise.

For further reading / watching:

• AI, Medicine, and Bias: Diversifying Your Dataset is Not Enough - my keynote talk at the Stanford AI in Medical Imaging symposium
• Medicine’s Machine Learning Problem - my Boston Review article on medicine and AI
• ‘Everybody was telling me there was nothing wrong’ - BBC article by Maya Dusenberry
• Medicine’s Women Problem - comic by Aubrey Hirsch
• AI and Power: The Ethical Challenges of Automation, Centralization, and Scale - a deeper dive into how AI centralizes power
• How the EU’s Flawed Artificial Intelligence Regulation Endangers the Social Safety Net - Human Rights Watch

Thank you to Jeremy Howard and Krystal South for providing feedback on earlier drafts of this post.

You can subscribe to be notified of new blog posts by submitting your email below:

Immune Cells Coordinate Like a Social Network

Immune cells and the cytokines they use to communicate can be analyzed using the same techniques as are used for social networks. Networks can be represented as matrices, in which each column/row represents a node, and the matrix values represent the presence or strength of connections between them. This type of representation underlies how the network of the internet is represented for search engines such as Google. Matrix math is everywhere! I covered many applications of it in the computational linear algebra course that Jeremy Howard and I designed and taught at University of San Francisco and fast.ai.

The paper ImmProt sampled immune cells from the bloodstream of humans and used proteomics (the large-scale analysis of proteins) to study immune communication networks. The researchers considered 28 cell types in both steady and activated states, and over 10,000 proteins. ImmProt used several classic math techniques to analyze the network, including principal component analysis, supervised clustering, and Lasso regression analysis.

Cytokines bind to receptors on the surface of the cell that is receiving them. Receptors for some cytokines are found on many different types of immune cells (these receptors are said to be “broadly expressed”), while receptors for other cytokines may be expressed on a very limited number of cell types. The researchers found that there were two types of cytokine communication patterns: there were either many types of cells sending and few types receiving for a given cytokine, or there were few types of cells sending and many receiving. This is a directed, asymmetric information exchange.

The authors looked at the number of in-going and out-going communication between various cell types, and determined how much cross-talk there is between different clusters of cell types. In the above diagram, the outer circle lists cell types and the middle circle lists receptor and cytokine gene names. This picture shows that eosinophils and basophils (two cell types that evolved primarily to address parasites) send the cytokine CCL5/3 to a variety of other cells that receive the message using the receptor CCR3. The color of the lines indicates the type of cell sending the cytokine.

While many of the relationships found were well established in existing scientific papers, they also discovered new relationships, and validated two of these new relationships experimentally: one in-going and one out-going. The authors wrote, “The immune system displays an almost infinite diversity in cell types and activation states, and achieving completeness in both dimensions is particularly challenging for human cells.”

AI does not replace other ways of knowing about the world: benchtop research is still crucial

AI is a powerful tool, but it does not replace other ways of knowing about the world, including qualitative research (as I wrote about together with Louisa Bartolo) and benchtop quantitative experiments. Machine learning and AI techniques are great at learning patterns in existing data. However, they are limited by what types of data have been collected so far, and it is still crucial to continue running new laboratory-based experiments. For this reason, the recent Immune Dictionary paper was a valuable contribution to the field. The researchers gathered direct measurements of over >1,400 cytokine-cell type pairings in an in vivo experiment.

This study was significant because it considered all major immune cell types and all major cytokines (studies typically look at only 5 immune cell types) and was conducted in vivo (not in a culture). The researchers injected 86 different cytokines into individual mice and measured the responses of 17 types of immune cells in the mouse lymph nodes. They used single-cell RNA sequencing, which is more direct than considering ligand and receptor expression association data. They released their findings as an “Immune Dictionary” and developed accompanying computer software. This software was then used to identify cytokine networks in tumors after checkpoint blockade therapy (a cancer therapy that helps to reactivate exhausted immune cells).

One key finding was that the responses induced by cytokines are highly cell-specific. Rarer types of immune cells expressed a greater number of cytokine types than more common cell types. One rare cell type, typically not even covered in immunology courses, was found to express the highest number of distinct cytokines, influencing nearly every other cell type. This result suggests that rare immune cell types are crucial for cell-to-cell communication.

Another important finding was that every immune cell type could be polarized into multiple states depending on the combination of cytokines that it received. The two key researchers were surprised at this level of plasticity and complexity in immune cell types, with even the most well-studied cytokines inducing more complex responses than previously expected.

An Ongoing Area of Research

This is an exciting area of research to follow. There is still much work to be done in more thoroughly understanding the intricate and complex ways immune cells coordinate with one another to respond to threats. Immune cell-cytokine networks are a great illustration of the power of interdisciplinary work, since NLP, mathematics, and bench top immunology research all provide important insights into the problem. And this is just one of several important problems in immunology where AI is being applied!

You can subscribe to be notified of new blog posts by submitting your email below: