In the two prior parts of this little series (I and II), we've discussed the way in which unknown, putatively causative entities were invoked to explain their purported consequences, even if the agent itself could not be seen or its essence characterized. Atoms and an all-pervasive ether are examples. In the last two centuries, many scientists followed some of the principles laid down in the prior Enlightenment period, and were intensely empirical, to avoid untrammeled speculation. Others followed long tradition and speculated about the underlying essentials of Nature that could account for the empiricists' observations. Of course, in reality I think most scientists, and even strongly religious people, believed that Nature was law-like: there were universally true underlying causative principles. The idea of empiricism was to escape the unconstrained speculation that was the inheritance even from the classical times (and, of course, from dogmatic religious explanations of Nature). Repeated observation was the key to finding Nature's patterns, which could only be understood indirectly. I'm oversimplifying, but this was largely the situation in 19th and early 20th century physics and it became true of historical sciences like geology, and in biology during the same time.
At these stages in the sciences, free-wheeling speculation was denigrated as delving in metaphysics, because only systematic empiricism--actual data!--could reveal how Nature worked. I've used the term 'metaphysics' because in the post-Enlightenment era it has had and been used in a pejorative sense. On the other hand, if one cannot make generalizations, that is, infer Nature's 'laws', then one cannot really turn retrospective observation into prospective prediction.
By the turn of the century, we had Darwin's attempt at Newtonian law-like invocation of natural selection as a universal force for change in life, and we had Mendel's legacy that said that causative elements, that were dubbed 'genes', underlay the traits of Nature's creatures. But a 'gene' had never actually been 'seen', or directly identified until well into the 20th century. What, after all, was a 'gene'? Some sort of thing? A particle? An action? How could 'it' account for traits as well as their evolution? To many, the gene was a convenient concept that was perhaps casually and schematically useful, but not helpful in any direct way. Much has changed, or at least seems to have changed since then!
Genetics is today considered a mainline science, well beyond the descriptive beetle-collecting style of the 19th century. We now routinely claim to identify life's causative elements as distinct, discrete segments of DNA sequence, and a gene is routinely treated as causing purportedly 'precisely' understandable effects. If raw Big Data empiricism is the Justification du Jour for open-ended mega-funding, the implicit justifying idea is that genomics is predictive the way gravity and relativity and electromagnetism are--if only we had enough data! Only with Big Data can we identify these distinct, discrete causal entities, characterize their individual effects and use that for prediction, based on some implicit theory or law of biological causation. It's real science, not metaphysics!
But even with today's knowledge, how true is that?
The inherent importance of context-dependency and alternative paths
It seems obvious that biological causation is essentially relative in nature: it fundamentally involves context and relationships. Treating genes as individual, discrete causal agents really is a form of metaphysical reification, not least because it clearly ignores what we know about genetics itself. As we saw earlier, today there is no such thing as 'the' gene, much less one we can define as the discrete unit of biological function. Biological function seems inherently about interactions. The gene remains in that sense, to this day, a metaphysical concept--perhaps even in the pejorative sense, because we know better!
We do know what some 'genes' are: sequences coding for protein or mature RNA structure. But we also know that much of DNA has function unrelated to the stereotypical gene. A gene has multiple exons and often differently spliced (among many other things, including antisense RNA post-transcription regulation, and RNA editing), combined with other 'genes' to contribute to some function. A given DNA coding sequence often is used in different contexts in which 'its' function depends on local context-specific combinations with other 'genes'. There are regulatory DNA sequences, sequences related to the packaging and processing of DNA, and much more. And this is just the tip of the current knowledge iceberg; that is, we know there's the rest of the iceberg not yet known to us.
Indeed, regardless of what is said and caveats offered here and there as escape clauses, in practice it is routinely assumed that genes are independent, discrete agents with additive functional effects, even though this additivity is a crude result of applying generic statistical rather than causal models, mostly to whole organisms rather than individual cells or gene products themselves. Our methods of statistical inference are not causal models as a rule but really only indicate whether, more probably than not, in a given kind of sample and context a gene actually 'does' anything to what we've chosen to measure. Yes, Virginia, the gene concept really is to a great extent still metaphysical.
But isn't genomic empiricism enough? Why bother with metaphysics (or whatever less pejorative-sounding term you prefer)? Isn't it enough to identify 'genes', however we do it, and estimate their functions empirically, regardless of what genes actually 'are'? No, not at all. As we noted yesterday, without an underlying theory, we may sometimes be able to make generic statistical 'fits' to retrospective data, but it is obvious, even in some of the clearest supposedly single-gene cases, that we do not have strong bases for extrapolating such findings in direct causal or predictive terms. We may speak as if we know what we're talking about, but those who promise otherwise are sailing as close to the wind as possible.
That genetics today is still rather metaphysical, and rests heavily on fancifully phrased but basically plain empiricism, does not gainsay that fact that we are doing much more than just empiricism, in many areas, and we try to do that even in Big Promise biomedicine. We do know a lot about functions of DNA segments. We are making clear progress in understanding and combatting diseases and so on. But we also know, as a general statement, that even in closely studied contexts, most organisms have alternative pathways to similar outcomes and the same mutation introduced into different backgrounds (in humans, because the causal probabilities vary greatly and are generally low, and in different strains of laboratory animals) often has different effects. We already know from even the strongest kind of genetic effects (e.g., BRCA1 mutations and breast cancer) that extrapolation of future risk from retrospective data-fitting can be grossly inaccurate. So our progress is typically a lot cruder than our claims about it.
An excuse that is implicit and sometimes explicit is that today's Big Data 'precision, personalized' medicine, and much of evolutionary inference, are for the same age-old argument good simply because they are based on facts, on pure empiricism, not resting on any fancy effete intellectual snobs' theorizing: We know genes cause disease (and everything else) and we know natural selection causes our traits. And those in Darwinian medicine know that everything can be explained by the 'force' of natural selection. So just let us collect Big Data and invoke these 'theories' superficially as justification, and mint our predictions!
But--could it be that the empiricists are right, despite not realizing why? Could it be that the idea that there is an underlying theory or law-like causal reality, of which Big Data empiricism provides only imperfect reflections, really is, in many ways, only a hope, but not a reality?
Or is life essentially empirical--without a continuous underlying causal fabric?
What if Einstein's dream of a True Nature, that doesn't play dice with causation, was a nightmare. In biology, in particular, could it be that there isn't a single underlying, much less smooth and deterministic, natural law? Maybe there isn't any causal element of the sort being invoked by terms like 'gene'. If an essential aspect of life is its lack of law-like replicability, the living world may be essentially metaphysical in the usual sense of there being no 'true' laws or causative particles as such. Perhaps better stated, the natural laws of life may essentially be that life does not following any particular law, but is determined by universally unique local ad hoc conditions. Life is, after all, the product of evolution and if our ideas about evolution are correct, it is a process of diversification rather than unity, of local ad hoc conditions rather than universal ones.
To the extent this is the reality, ideas like genes may be largely metaphysical in the common sense of the term. Empiricism may in fact be the best way to see what's going on. This isn't much solace, however, because if that's the case then promises of accurate predictability from existing data may be culpably misleading, even false in the sense that a proper understanding of life would be that such predictions won't work to a knowable extent.
I personally think that a major problem is our reliance on statistical analysis and its significance criteria, that we can easily apply but that have at best only very indirect relationship to any underlying causal fabric, and that 'indirect' means largely unknowably indirect. Statistics in this situation is essentially about probabilistic comparisons, and has little or often no basis in causal theory, that is, in the reason for observed differences. Statistics work very well for inference when properly distributed factors, such as measurement errors, are laid upon some properly framed theoretically expected result. But when we have no theory and must rely on internal comparisons and data fitting, as between cases and controls, then we often have no way to know what part of our results has to do with sampling etc. and where any underlying natural laws, might be in the empirical mix--if such laws even exist.
Given this situation, the promise of 'precision' can be seen starkly as a marketing ploy rather than knowledgeable science. It's a distraction to the public but also to the science itself, and that is the worst thing that can happen to legitimate science. For example, if we can't really predict based on any serious-level theory, we can't tell how erroneous future predictions will be relative to existing retrospective data-fitting so we can't, largely even in principle, know how much this Big Data romance will approximate any real risk truths, because true risks (of some disease or phenotype) may not exist as such or may depend on things, like environmental exposures and behavior, that cannot be known empirically (and perhaps not even in theory), again, even in principle.
Rethinking is necessary, but in our current System of careerism and funding, we're not really even trying to lay out a playing field that will stimulate the required innovation in thought. Big Data advocates sometimes openly, without any sense of embarrassment, say that serendipity will lead those with Big Data actually to find something important. But deep insight may not be stimulated as long as we aren't even aware that we're eschewing theory basically in favor of pure extrapolated empiricism--and that we have scant theory even to build on.
There are those of us who feel that a lot more attention and new kinds of thinking need to be paid to the deeper question of how living Nature 'is' rather than very shaky empiricism that is easy, if costly, to implement but whose implications are hard to evaluate. Again, based on current understanding, it is quite plausible that life, based on evolution which is in turn based on difference rather than replicability, simply is not a phenomenon that obeys natural law in the way oxygen atoms, gravity, and even particle entanglement do.
To the extent that is the case, we are still in a metaphysical age, and there may be no way out of it.
genetics etiketine sahip kayıtlar gösteriliyor. Tüm kayıtları göster
genetics etiketine sahip kayıtlar gösteriliyor. Tüm kayıtları göster
Is genetics still metaphysical? Part II. Is that wrong?
What is the role of theory vs empiricism in science? How do these distinctions apply to genetics?
Yesterday, we discussed some of the history of contesting views on the subject. Much of the division occurred before there was systematically theoretical biology. In particular, when creationism, or divine creative acts rather than strictly material processes, was the main explanation for life and its diversity, the issues were contended in the burgeoning physical sciences, with its dramatic technological advances, and experimental settings, and where mathematics was a well-established part of the science and its measurement aspects.
Around the turn of the 20th century, Darwinian evolution was an hypothesis that not even all the leading biologists could accept. Inheritance was fundamental to any evolutionary view, and inherited somethings seemed obviously to be responsible for the development of organisms from single cells (fertilized eggs). Mendel had shown examples of discretely inherited traits, but not all traits were like that. Ideas about what the inherited units were (Darwin called them gemmules, Mendel called them Elements, and hereafter I'll use the modern term 'genes') were simply guesses (or just words). They were stand-ins for what was assumed to exist, but in the absence of their direct identification they were, essentially, only metaphysical or hypothetical constructs.
The cloak of identity had serious implications. For example, evolution is about inherited variation, but genes as known in Darwin's time and most of the later 19th century didn't seem to change over generations, except perhaps due to grotesquely nonviable effects called 'mutations'. How could these 'genes', whatever they were, be related to evolution, which is inherently about change and relative positive effects leading to selection among organisms that carried them?
Many critics thought the gene was just a metaphysical concept, that is, used for something imagined, that could not in a serious way be related to the empirical facts about inherited traits. The data were real, but the alleged causal agent, the 'gene', was an unseen construct, yet there was a lot of dogma about genes. Many felt that the life sciences should stick to what could be empirically shown, and shy away from metaphysical speculation. As we saw yesterday, this contention between empiricism and theory was a serious part of the debate about fundamental physics at the time.
That was more than a century ago, however, and today almost everyone, including authors of textbooks and most biologists themselves, asserts that we definitely do know what a gene is, in great detail, and it is of course as real as rain and there's nothing 'metaphysical' about it. To claim that genes are just imagined entities whose existential reality cannot be shown would today be held to be not just ignorant, but downright moronic. After all, we spend billions of dollars each year studying genes and what they do! We churn out a tsunami of papers about genes and their properties, and we are promised genetically based 'precision' medicine, and many other genetic miracles besides, that will be based on identifying 'genes for' traits and diseases, that is enumerable individual genes that cause almost any trait of interest, be it physical, developmental, or behavioral. That's why we're plowing full budget ahead to collect all sorts of Big Data in genetics and related areas. If we know what a gene is then the bigger the data the better, no?
Or could it be that much of this is marketing that invokes essentially metaphysical entities to cover what, despite good PR to the contrary, remains just empiricism? And if it is just empiricism, why the 'just'? Isn't it good that, whatever genes 'are', if we can measure them in some way we can predict what they do and live to ripe old ages with nary a health problem? Can't we in fact make do with what is largely pure empiricism, without being distracted by any underlying law of biological causation, or the true nature of these causative entities--and deliver the miraculous promises? The answer might be a definitive no!
The metaphysical aspects of genes, still today
In essence, genes are not things, they are not always discrete DNA sequence entities with discrete functions, and they are not independently separable causative agents. Instead, even the term 'gene' remains a vague, generically defined one. We went through decades in the 20th century believing that a gene was a distinct bit of DNA sequence, carrying protein code. But it is not so simple. Indeed, it is not simple at all.
It is now recognized by those who want to pay attention to reality, that the concept of the 'gene' is still very problematic, and to the extent that assertions are made about 'genes' they are metaphysical assertions, no matter how clothed in the rhetoric of empiricism they may be. For example, many DNA regions code for functional RNA rather than protein. Much DNA function has to do with expression of these coding regions. Many coding regions are used in different ways (for example, different exon splicing) in different circumstances. Some DNA regions act only when they are chemically modified by non-DNA molecules (and gene expression works exclusively in that way). Some of 'our' DNA is in microbes that are colonizing us. And 'traits' as we measure them are the result of many--often hundreds or more--DNA elements, and of interactions among cells. Each cell's DNA is different at least in some details from that of its neighbors (due to somatic mutation, etc.). And then there is 'the' environment! This is central to our biological state but typically not accurately measurable.
Some discussion about these issues can be seen in a report of a conference on the gene concept in 2011 at the Santa Fe Institute. Even earlier, in 2007 when it seemed we had really learned about genomes, hardly suspecting how much more there was (and is) still to be learned, a review in Genome Research was defined in an almost useless way as follows:
Or could there perhaps be another way to think about this, in which genetics as currently understood remains largely metaphysical, that genetics is real but we simply don't yet have an adequate way of thinking that will unite empiricism to some underlying global reality, some theory in the proper scientific sense?
Tomorrow we'll address the possibility that genetics is inherently metaphysical in that there isn't any tractably useful universal natural law out there to be discovered.
Yesterday, we discussed some of the history of contesting views on the subject. Much of the division occurred before there was systematically theoretical biology. In particular, when creationism, or divine creative acts rather than strictly material processes, was the main explanation for life and its diversity, the issues were contended in the burgeoning physical sciences, with its dramatic technological advances, and experimental settings, and where mathematics was a well-established part of the science and its measurement aspects.
Around the turn of the 20th century, Darwinian evolution was an hypothesis that not even all the leading biologists could accept. Inheritance was fundamental to any evolutionary view, and inherited somethings seemed obviously to be responsible for the development of organisms from single cells (fertilized eggs). Mendel had shown examples of discretely inherited traits, but not all traits were like that. Ideas about what the inherited units were (Darwin called them gemmules, Mendel called them Elements, and hereafter I'll use the modern term 'genes') were simply guesses (or just words). They were stand-ins for what was assumed to exist, but in the absence of their direct identification they were, essentially, only metaphysical or hypothetical constructs.
The cloak of identity had serious implications. For example, evolution is about inherited variation, but genes as known in Darwin's time and most of the later 19th century didn't seem to change over generations, except perhaps due to grotesquely nonviable effects called 'mutations'. How could these 'genes', whatever they were, be related to evolution, which is inherently about change and relative positive effects leading to selection among organisms that carried them?
Many critics thought the gene was just a metaphysical concept, that is, used for something imagined, that could not in a serious way be related to the empirical facts about inherited traits. The data were real, but the alleged causal agent, the 'gene', was an unseen construct, yet there was a lot of dogma about genes. Many felt that the life sciences should stick to what could be empirically shown, and shy away from metaphysical speculation. As we saw yesterday, this contention between empiricism and theory was a serious part of the debate about fundamental physics at the time.
That was more than a century ago, however, and today almost everyone, including authors of textbooks and most biologists themselves, asserts that we definitely do know what a gene is, in great detail, and it is of course as real as rain and there's nothing 'metaphysical' about it. To claim that genes are just imagined entities whose existential reality cannot be shown would today be held to be not just ignorant, but downright moronic. After all, we spend billions of dollars each year studying genes and what they do! We churn out a tsunami of papers about genes and their properties, and we are promised genetically based 'precision' medicine, and many other genetic miracles besides, that will be based on identifying 'genes for' traits and diseases, that is enumerable individual genes that cause almost any trait of interest, be it physical, developmental, or behavioral. That's why we're plowing full budget ahead to collect all sorts of Big Data in genetics and related areas. If we know what a gene is then the bigger the data the better, no?
Or could it be that much of this is marketing that invokes essentially metaphysical entities to cover what, despite good PR to the contrary, remains just empiricism? And if it is just empiricism, why the 'just'? Isn't it good that, whatever genes 'are', if we can measure them in some way we can predict what they do and live to ripe old ages with nary a health problem? Can't we in fact make do with what is largely pure empiricism, without being distracted by any underlying law of biological causation, or the true nature of these causative entities--and deliver the miraculous promises? The answer might be a definitive no!
The metaphysical aspects of genes, still today
In essence, genes are not things, they are not always discrete DNA sequence entities with discrete functions, and they are not independently separable causative agents. Instead, even the term 'gene' remains a vague, generically defined one. We went through decades in the 20th century believing that a gene was a distinct bit of DNA sequence, carrying protein code. But it is not so simple. Indeed, it is not simple at all.
It is now recognized by those who want to pay attention to reality, that the concept of the 'gene' is still very problematic, and to the extent that assertions are made about 'genes' they are metaphysical assertions, no matter how clothed in the rhetoric of empiricism they may be. For example, many DNA regions code for functional RNA rather than protein. Much DNA function has to do with expression of these coding regions. Many coding regions are used in different ways (for example, different exon splicing) in different circumstances. Some DNA regions act only when they are chemically modified by non-DNA molecules (and gene expression works exclusively in that way). Some of 'our' DNA is in microbes that are colonizing us. And 'traits' as we measure them are the result of many--often hundreds or more--DNA elements, and of interactions among cells. Each cell's DNA is different at least in some details from that of its neighbors (due to somatic mutation, etc.). And then there is 'the' environment! This is central to our biological state but typically not accurately measurable.
Some discussion about these issues can be seen in a report of a conference on the gene concept in 2011 at the Santa Fe Institute. Even earlier, in 2007 when it seemed we had really learned about genomes, hardly suspecting how much more there was (and is) still to be learned, a review in Genome Research was defined in an almost useless way as follows:
Finally, we propose a tentative update to the definition of a gene: A gene is a union of genomic sequences encoding a coherent set of potentially overlapping functional products. Our definition sidesteps the complexities of regulation and transcription by removing the former altogether from the definition and arguing that final, functional gene products (rather than intermediate transcripts) should be used to group together entities associated with a single gene. It also manifests how integral the concept of biological function is in defining genes.
Really?! Is that a definition or an academically couched but empty kicking of the can down the road while seeming to be knowledgeable and authoritative? Or is it simply so empty as to be risible?
There are many now who advocate a 'Third Way' that in a rather generic sense of advocating less dogma and more integrative and indeed innovative or integrative approaches. But even this doesn't say what the Third Way actually is, though one thing for sure is that it's every Third Way member's favorite way of coopting the concept of biological causation as his or her own. I'm being cynical, and I'm associated with the Third Way myself and believe that serious rethinking about biological causation and evolution is in order, but that doesn't seem to be too unfair a way to characterize the Third Way's characterization of mainline genome-centered or perhaps genome-obsessed thinking. At least, it acknowledges that we don't just have 'genes' and 'environment', but that biological causality is based fundamentally on interactions of many different kinds.
DNA is basically an inert molecule on its own
In genetic terminology, DNA is basically an inert molecule. That is, whatever you want to call genes act in a context-specific way, and this goes beyond what is known as cis interactions among local DNA elements (like regulatory sequences flanking coding sequences) along a given strand. Instead, genetic function is largely a trans phenomenon, requiring interaction among many or even countless other parts of DNA on the different chromosomes in the cell. And often if not typically, nothing happens until the coded product--RNA or protein--itself is modified by or interacts with other compounds in the cell (and responds to external things the cell detects).
Beyond even that complexity provides comparable evolutionary or physiological complexity. There are many, perhaps often also countless alternative biological pathways to essentially the same empirical result (say, height or blood pressure or intelligence). These causally equivalent combinations, if we can even use the term 'causal', are many and un-enumerated, and perhaps un-enumerable. The alternatives may be biochemically different, but if it they confer essentially no difference in terms of natural selection, they are evolutionarily as well as physiologically equivalent. Indeed, the fact is that every cell, and hence every organism is different in regard to the 'causal' bases of traits. We may be able to define and hence measure some result, such as blood pressure or reproductive fitness; but to speak of causes as if they are individually distinct or discrete entities is still essentially being metaphysical. Yet, for various sociocultural and economic reasons, we seem unwilling to acknowledge this.
Beyond even that complexity provides comparable evolutionary or physiological complexity. There are many, perhaps often also countless alternative biological pathways to essentially the same empirical result (say, height or blood pressure or intelligence). These causally equivalent combinations, if we can even use the term 'causal', are many and un-enumerated, and perhaps un-enumerable. The alternatives may be biochemically different, but if it they confer essentially no difference in terms of natural selection, they are evolutionarily as well as physiologically equivalent. Indeed, the fact is that every cell, and hence every organism is different in regard to the 'causal' bases of traits. We may be able to define and hence measure some result, such as blood pressure or reproductive fitness; but to speak of causes as if they are individually distinct or discrete entities is still essentially being metaphysical. Yet, for various sociocultural and economic reasons, we seem unwilling to acknowledge this.
You might object by saying that in fact most geneticists, from Francis Collins down to the peons who plead for his funding support, are being essentially empirical and not indulging in theory. Yes, they drop words like 'gene' and 'epigenome' and 'microbiome' or 'network' or 'system', but this are on or over the edge of metaphysics (speculative guessing). Many who feed at the NIH (and NSF) trough might proudly proclaim that they are in fact not dealing with airy-fairy theory, but simply delivering empirical and hence practical, useful results. They do genomewide mapping because, or even proudly declaring, they have no causative theory for this disease or that behavioral trait. Usually, however, they confound statistical significance with formal theory, even if they don't so declare explicitly.
For example, most studies of genotypes and genetic variation relative to traits like disease, are based on internal comparisons (cases vs control, tall vs short, smart vs not-smart, criminal vs non-criminal, addictive vs sober, etc.). They don't rest on any sort of theory except that they do implicitly identify entities like 'genes'. Often this is so metaphysical as to be rather useless, but it is only right to acknowledge that these results are occasionally supported by finding an indicated 'gene' (DNA sequence element), whose manipulation or variation can be shown to have molecular function relevant to the trait, at least under some experimental conditions. But this causative involvement is usually quite statistical, providing only weak causative effects, rather than in any clear sense deterministic. We are enabled by this largely pure empiricism to argue that the association we saw in our retrospective study is what we'll see prospectively as causation in the future. And we now know enough to know that when it seems to work it is (as, indeed, in Mendel's own time) it's only the simplest tip of the causative iceberg.
We are tempted to believe, and to suggest, that this 'gene' (or genetic variant, an even cruder attempt at identifying a causative element) will be predictive of, say, a future disease at least in some above-average sense. That is, even if we don't know the exact amount of associated risk. But even that is not always the case: the associated risks are usually small and data-specific and often vary hugely from study to study, over time, or among populations. That means, for example, that people--typically by far most people--carrying the risk variant will not get the associated disease! It may often do nothing when put into, say, a transgenic mouse. The reason has to be context, but we usually have scant idea about those contexts (even when they are environmental, where the story is very similar). That is a profound but far under-appreciated (or under-acknowledged) fact with very widespread empirical support!
We are tempted to believe, and to suggest, that this 'gene' (or genetic variant, an even cruder attempt at identifying a causative element) will be predictive of, say, a future disease at least in some above-average sense. That is, even if we don't know the exact amount of associated risk. But even that is not always the case: the associated risks are usually small and data-specific and often vary hugely from study to study, over time, or among populations. That means, for example, that people--typically by far most people--carrying the risk variant will not get the associated disease! It may often do nothing when put into, say, a transgenic mouse. The reason has to be context, but we usually have scant idea about those contexts (even when they are environmental, where the story is very similar). That is a profound but far under-appreciated (or under-acknowledged) fact with very widespread empirical support!
Indeed, the defense of pure empiricism is one of convenience, funding-wise among other reasons; but perhaps with today's knowledge all we can do if we are wedded to Big Data science and public promises of 'precision' genomic prediction. When or if we have a proper theory, a generalization about Nature, we can not only test our empirical data agains the theory's predictions, but also use the theory to predict new, future outcomes with a convincing level of, yes, precision. Prediction is our goal and the promises (and, notably, research funding) rest on prediction, not just description. So, as Einstein (and Darwin) felt, an underlying theory of Nature makes data make sense. Without it we are just making hopeful guesses. Anyone who thinks we have such a theory based on all the public rhetoric by scientists is, like most of the scientists themselves, confusing empiricism with theory, and description with understanding. Those who are thoughtful know very well that they are doing this, but can't confess it publicly. Retired people (like me) are often less inhibited!
Or could there perhaps be another way to think about this, in which genetics as currently understood remains largely metaphysical, that genetics is real but we simply don't yet have an adequate way of thinking that will unite empiricism to some underlying global reality, some theory in the proper scientific sense?
Tomorrow we'll address the possibility that genetics is inherently metaphysical in that there isn't any tractably useful universal natural law out there to be discovered.
Is genetics still metaphysical? Part I. Some general history.
In very broad terms, modern science has had debates about two basic kinds of approaches to understanding the world. To over-simplify, they are the empirical and the theoretical approaches. Some argue that we can know only what we can detect with our sensory systems (and machines to extend them), but we can never know what general causal principles account for those data, or even if such real, true principles exist. Others view science's essential job as not just accumulating collections of data, which are necessarily imperfect, but to use such observations to build a picture of the true, or perfect underlying regularity--the 'laws' of Nature.
In the former case we just have to make measurements and try to show the ways in which comparable situations lead to comparable outcomes. In the latter, we want what we call 'theory', that is, perfect generalizations that tell us how a given situation will turn out, and what the causal reasons are. The standard assumption of the physical sciences is that Nature is, indeed, universally law-like. Variables like the gravitational constant and the speed of light really are universally, precisely constant.
These are age-old differences, often 'just' philosophical, but they're quite important. Comparably important are the still-unanswered question as to whether any phenomena in Nature is irreducibly probabilistic rather than deterministic, or whether probabilistic aspects of Nature really just reflect our imperfect sampling and measurement. This is the important distinction between epistemology--how we know things, and ontology--how things really are. Can we ever tell the difference?
Empiricism is in some ways the easy part. We just go out and make measurements and let them accumulate so we can generalize about them. That's a lot of slogging to get the data, but all you have to do is be systematic and careful. Don't give me airy generalizations, just the facts, please!
In other ways, theory is the easy part. All you have to do is sit in your armchair, as the proverbial denigratory model has it, and make up something that sounds exotic (or even mathematically intricate) and claim you know how Nature 'is'. Data are imperfect, so don't bother me about that! There are long traditions in both kinds of approach, and to a great extent it's only been the past few hundred years in which there has been melding of these two basic approaches.
Often, theory hypothesizes some fundamental objects whose properties and actions can only be seen indirectly, as they are manifest in measurable phenomena. Here there is a delicate boundary between what is essentially 'metaphysical' as opposed to real. Many object to the use of metaphysical concepts and claims as being essentially untestable, and argue that only empiricism is real and should be taken seriously. In the 19th and early 20th centuries, as technology revealed more and more about unseen Nature, things that were not yet seen directly but that could be hypothesized and assigned to things we could measure, we taken as true by some but denigrated as metaphysical by pure empiricists.
These distinctions were never that clear, in my view (even if they provided jobs for philosophers to write about). Empiricism is retrospective but understanding requires some sorts of predictability, which is prospective. If we cannot reliably generalize, if the same conditions don't always lead to the same result, how can the observing the former lead us to the latter? Predictive power is largely what we want out of science, even if it's just to confirm our understanding of Nature's laws.
Until fairly recently, these issues have mainly been housed in the physical sciences, but since Linnaeus' time, but especially after Darwin and Wallace, the issues have applied to biology as well.
In this brief series we'll try to explore whether or how we can think of biology as the result of such universal laws or whether all we can do is make observations and rough causal generations about them. What is the place for strong causal theory in biology, or are empiricism and very general notions of process enough?
An example from the early prime era in modern science is the 'atom'. Matter was conceived as being composed of these unseen particles, that accounted for the weight and properties of chemicals, and whose movement accounted for the weight, temperature, and pressure in gases. Similar kinds of issues related to electromagnetism: what 'was' it?
An important late 19th-early 20th century example had to do with the existence of 'ether' as the medium through which electromagnetic radiation moved. Ether could not be seen or felt but wavelike radiation had to be waves in something, didn't it? Late-century tests failed to find it (e.g., the famous Michelson-Morely experiment). In well-known interchanges at the time, figures like Ernst Mach, Albert Einstein and Max Planck thought about and debated whether there was a 'real' underlying general 'fabric' of Nature or whether specific empirical data simply showed us enough, and trying to delve deeper was dealing in metaphysics. Many felt that was simply not justified--measurement or empiricism was what science could hope for. On the other hand some, like Einstein, were convinced that Nature had a universal, and real underlying reality of which measurements were reflections. He felt that theory, and in this case mathematics, could reveal or even 'intuit' Nature's underlying fabric. An interesting article by Amanda Gefter in Nautilus science magazine deals with some of this history, with useful references.
So what about biology?
Biology had been largely a descriptive or even theological field before it became a modern science. But then came Darwin and his idea of evolution. He viewed natural selection as a kind of Newtonian universal force. Was it a type of explanation fitted simply around the empirical data that had been collected by Naturalists, or did it constitute some form of universal theory of life as Darwin asserted? Selection as a force had to work through some 'medium' or elements of inheritance. His causal elements ('gemmules') were (like Lamarck's before him) entirely invented to 'fit' what was being observed about the evolution of diversity. Indeed, he modeled natural selection itself after intentional agricultural selection because the latter could be demonstrated by human intent, while the former was generally far too slow to observe directly. But there had to be some 'units' of inheritance for it to work, so he essentially invented them out of thin air. Even in the early 20th century, 'genes' (as they became known) were largely hypothesized units for whose physical nature--or even reality--there was only indirect empirical evidence.
Assuming these discrete causal particles could enable the force, natural selection, to work on adaptive change was much like assuming that electromagnetic radiation needed ether to do its job. Since differential reproductive success is observable, one can always define it to be the result of selection and to assume some gene(s) to be responsible. The test for relative success is, after all, only a statistical one with subjective decision-making criteria (like significance level) in empirical data. In that sense, natural selection is a very metaphysical notion because after the fact we can always empirically observe what has succeeded over time, or what functions have evolved, and call that the result of selection. Such an explanation can hardly be falsified. What is the reality of the underlying force, that Darwin likened to gravity? Since it is always dependent on changing local conditions, what sort of a 'law' is it anyway? And if it's basically metaphysical, should we reject it?
Mendelian genetics as metaphysics
If selection is a process, like gravity, it had to work on objects. Because individual organisms are temporary (they all die), the objects in question had to be transmitted from parent to offspring. That transmission was also found, by Mendel's experiment, to be a regular kind of process. Mendel's causative 'elements', that we now call 'genes', appeared in his carefully chosen pea experiments to be transmitted as discrete things. They fit the discretely causative world of the energized new field of atomic chemistry (see my Evolutionary Anthropology article on Mendel), with its idea that a chemical is made up of a particular kind of atom (thought by some to be multiples of hydrogen at the time), and Mendel's statistical tests showed a reasonably good fit to that discrete-unit worldview (indeed accusations that he or his assistants cheated may reflect his acceptance of discrete underlying but unseen and hence metaphysical, elements). But what were these genes? In what serious sense did they exist as things rather than just an imaginary but essentially unconstrained variables conjured up to account for actual observations--of some sorts of inheritance, that of discretely varying traits--whose actual nature was entirely inaccessible?
These questions became very important in the debate about how evolution worked, since evolution required inheritance of favored states. But what Mendelian analysis, the only 'genetic' analysis available at the time, showed was that the causal genes' effects did not change, and they only were shown to fit discretely varying traits, not the quantitative traits of Darwinian evolution. For these reasons even many mainline evolutionary biologists felt that genes, whatever they were, couldn't account for evolution after all. Maybe geneticists were indulging in metaphysics.
This was similar to the situation that engaged Einstein, Ernst Mach, and others about physics, but when it came to biology, the difference between empiricism and metaphysics became, literally, quite lethal! The tragic impact of Profim Lysenko in the Soviet Union was due to a direct rejection by the scientific power structure that he established based on promises of rapid adaptation in plants, for example to the long, frozen Soviet winters, without adaptive 'genes' having to arise by evolution's slow pace. As I summarized in another Ev. Anth article, it was in part the alleged 'metaphysical' nature of 'genes' in the early 20th century that Lysenko used to reject what most of us would call real science, and put in place an agricultural regime that failed, with mortally disastrous consequences. Along the way, Lysenko with Stalin's help purge many skilled Soviet geneticists, leading many of them to tragic ends. The mass starvation of the era of Lysenkoist agriculture in the USSR may in part have been the result of this view of theoretical science (of course, Lysenko had his own theory, which basically didn't work as it was as much wishful thinking as science).
But how wrong was it to think of genes as metaphysical concepts at the time? Mendel had showed inheritance patterns that seemed to behave, statistically, as if they were caused by specific particles. But he knew many if not most traits did not follow the same pattern. Darwin knew of Mendel's work (and he of Darwin's), but neither thought the other's theories were relevant to his own interests.
But in the first part of the 20th century, the great experimental geneticist TH Morgan used Mendelian ideas in careful breeding experiments to locate 'genes' relative to each other on chromosomes. Even he was an empiricist and avowedly didn't really deal with what genes 'were', just how their causal agency was arranged.
Mendel's work also provided a research experimental approach that led via Morgan and others to the discovery of DNA and its protein coding sequences. We call those sequences 'genes' and research has documented what they are and how they work in great detail. In that sense, and despite early vague guesses about their nature, for most of a century one could assert that genes were in fact quite real, not metaphysical, entities at all. Not only that, but genes were the causal basis of biological traits and their evolution!
But things have turned out not to be so simple or straightforward. Our concept of 'the gene' is in rather great flux, in some ways each instance needing its own ad hoc treatment. Is a regulatory element a 'gene', for example, or a modified epigenetic bit of DNA? Is the 'gene' as still often taught in textbooks still in fact largely a metaphysical concept whose stereotypical properties are convenient but not nearly as informative as is the commonly presented view, even in the scientific literature?
Are we still resting on empiricism, invoking genetic and evolutionary theory as a cover but, often without realizing it, fishing for an adequate underlying theory of biological causation, that would correspond to the seamless reality Einstein (and Darwin, for that matter) felt characterized Nature? Is the gene, like Procrustes, being surgically adapted after the fact, to fit our desired tidy definition? Is claiming a theory on which genetic-based predictions can be 'precise' a false if self-comforting claim, as a marketing tool by NIH, when in fact we don't have the kind of true underlying theory of life that Einstein dreamed of for physics and the cosmos?
We'll deal with that in our next posts.
In the former case we just have to make measurements and try to show the ways in which comparable situations lead to comparable outcomes. In the latter, we want what we call 'theory', that is, perfect generalizations that tell us how a given situation will turn out, and what the causal reasons are. The standard assumption of the physical sciences is that Nature is, indeed, universally law-like. Variables like the gravitational constant and the speed of light really are universally, precisely constant.
These are age-old differences, often 'just' philosophical, but they're quite important. Comparably important are the still-unanswered question as to whether any phenomena in Nature is irreducibly probabilistic rather than deterministic, or whether probabilistic aspects of Nature really just reflect our imperfect sampling and measurement. This is the important distinction between epistemology--how we know things, and ontology--how things really are. Can we ever tell the difference?
Empiricism is in some ways the easy part. We just go out and make measurements and let them accumulate so we can generalize about them. That's a lot of slogging to get the data, but all you have to do is be systematic and careful. Don't give me airy generalizations, just the facts, please!
In other ways, theory is the easy part. All you have to do is sit in your armchair, as the proverbial denigratory model has it, and make up something that sounds exotic (or even mathematically intricate) and claim you know how Nature 'is'. Data are imperfect, so don't bother me about that! There are long traditions in both kinds of approach, and to a great extent it's only been the past few hundred years in which there has been melding of these two basic approaches.
Often, theory hypothesizes some fundamental objects whose properties and actions can only be seen indirectly, as they are manifest in measurable phenomena. Here there is a delicate boundary between what is essentially 'metaphysical' as opposed to real. Many object to the use of metaphysical concepts and claims as being essentially untestable, and argue that only empiricism is real and should be taken seriously. In the 19th and early 20th centuries, as technology revealed more and more about unseen Nature, things that were not yet seen directly but that could be hypothesized and assigned to things we could measure, we taken as true by some but denigrated as metaphysical by pure empiricists.
These distinctions were never that clear, in my view (even if they provided jobs for philosophers to write about). Empiricism is retrospective but understanding requires some sorts of predictability, which is prospective. If we cannot reliably generalize, if the same conditions don't always lead to the same result, how can the observing the former lead us to the latter? Predictive power is largely what we want out of science, even if it's just to confirm our understanding of Nature's laws.
Until fairly recently, these issues have mainly been housed in the physical sciences, but since Linnaeus' time, but especially after Darwin and Wallace, the issues have applied to biology as well.
In this brief series we'll try to explore whether or how we can think of biology as the result of such universal laws or whether all we can do is make observations and rough causal generations about them. What is the place for strong causal theory in biology, or are empiricism and very general notions of process enough?
An example from the early prime era in modern science is the 'atom'. Matter was conceived as being composed of these unseen particles, that accounted for the weight and properties of chemicals, and whose movement accounted for the weight, temperature, and pressure in gases. Similar kinds of issues related to electromagnetism: what 'was' it?
An important late 19th-early 20th century example had to do with the existence of 'ether' as the medium through which electromagnetic radiation moved. Ether could not be seen or felt but wavelike radiation had to be waves in something, didn't it? Late-century tests failed to find it (e.g., the famous Michelson-Morely experiment). In well-known interchanges at the time, figures like Ernst Mach, Albert Einstein and Max Planck thought about and debated whether there was a 'real' underlying general 'fabric' of Nature or whether specific empirical data simply showed us enough, and trying to delve deeper was dealing in metaphysics. Many felt that was simply not justified--measurement or empiricism was what science could hope for. On the other hand some, like Einstein, were convinced that Nature had a universal, and real underlying reality of which measurements were reflections. He felt that theory, and in this case mathematics, could reveal or even 'intuit' Nature's underlying fabric. An interesting article by Amanda Gefter in Nautilus science magazine deals with some of this history, with useful references.
So what about biology?
Biology had been largely a descriptive or even theological field before it became a modern science. But then came Darwin and his idea of evolution. He viewed natural selection as a kind of Newtonian universal force. Was it a type of explanation fitted simply around the empirical data that had been collected by Naturalists, or did it constitute some form of universal theory of life as Darwin asserted? Selection as a force had to work through some 'medium' or elements of inheritance. His causal elements ('gemmules') were (like Lamarck's before him) entirely invented to 'fit' what was being observed about the evolution of diversity. Indeed, he modeled natural selection itself after intentional agricultural selection because the latter could be demonstrated by human intent, while the former was generally far too slow to observe directly. But there had to be some 'units' of inheritance for it to work, so he essentially invented them out of thin air. Even in the early 20th century, 'genes' (as they became known) were largely hypothesized units for whose physical nature--or even reality--there was only indirect empirical evidence.
Assuming these discrete causal particles could enable the force, natural selection, to work on adaptive change was much like assuming that electromagnetic radiation needed ether to do its job. Since differential reproductive success is observable, one can always define it to be the result of selection and to assume some gene(s) to be responsible. The test for relative success is, after all, only a statistical one with subjective decision-making criteria (like significance level) in empirical data. In that sense, natural selection is a very metaphysical notion because after the fact we can always empirically observe what has succeeded over time, or what functions have evolved, and call that the result of selection. Such an explanation can hardly be falsified. What is the reality of the underlying force, that Darwin likened to gravity? Since it is always dependent on changing local conditions, what sort of a 'law' is it anyway? And if it's basically metaphysical, should we reject it?
Mendelian genetics as metaphysics
If selection is a process, like gravity, it had to work on objects. Because individual organisms are temporary (they all die), the objects in question had to be transmitted from parent to offspring. That transmission was also found, by Mendel's experiment, to be a regular kind of process. Mendel's causative 'elements', that we now call 'genes', appeared in his carefully chosen pea experiments to be transmitted as discrete things. They fit the discretely causative world of the energized new field of atomic chemistry (see my Evolutionary Anthropology article on Mendel), with its idea that a chemical is made up of a particular kind of atom (thought by some to be multiples of hydrogen at the time), and Mendel's statistical tests showed a reasonably good fit to that discrete-unit worldview (indeed accusations that he or his assistants cheated may reflect his acceptance of discrete underlying but unseen and hence metaphysical, elements). But what were these genes? In what serious sense did they exist as things rather than just an imaginary but essentially unconstrained variables conjured up to account for actual observations--of some sorts of inheritance, that of discretely varying traits--whose actual nature was entirely inaccessible?
These questions became very important in the debate about how evolution worked, since evolution required inheritance of favored states. But what Mendelian analysis, the only 'genetic' analysis available at the time, showed was that the causal genes' effects did not change, and they only were shown to fit discretely varying traits, not the quantitative traits of Darwinian evolution. For these reasons even many mainline evolutionary biologists felt that genes, whatever they were, couldn't account for evolution after all. Maybe geneticists were indulging in metaphysics.
This was similar to the situation that engaged Einstein, Ernst Mach, and others about physics, but when it came to biology, the difference between empiricism and metaphysics became, literally, quite lethal! The tragic impact of Profim Lysenko in the Soviet Union was due to a direct rejection by the scientific power structure that he established based on promises of rapid adaptation in plants, for example to the long, frozen Soviet winters, without adaptive 'genes' having to arise by evolution's slow pace. As I summarized in another Ev. Anth article, it was in part the alleged 'metaphysical' nature of 'genes' in the early 20th century that Lysenko used to reject what most of us would call real science, and put in place an agricultural regime that failed, with mortally disastrous consequences. Along the way, Lysenko with Stalin's help purge many skilled Soviet geneticists, leading many of them to tragic ends. The mass starvation of the era of Lysenkoist agriculture in the USSR may in part have been the result of this view of theoretical science (of course, Lysenko had his own theory, which basically didn't work as it was as much wishful thinking as science).
But how wrong was it to think of genes as metaphysical concepts at the time? Mendel had showed inheritance patterns that seemed to behave, statistically, as if they were caused by specific particles. But he knew many if not most traits did not follow the same pattern. Darwin knew of Mendel's work (and he of Darwin's), but neither thought the other's theories were relevant to his own interests.
But in the first part of the 20th century, the great experimental geneticist TH Morgan used Mendelian ideas in careful breeding experiments to locate 'genes' relative to each other on chromosomes. Even he was an empiricist and avowedly didn't really deal with what genes 'were', just how their causal agency was arranged.
Mendel's work also provided a research experimental approach that led via Morgan and others to the discovery of DNA and its protein coding sequences. We call those sequences 'genes' and research has documented what they are and how they work in great detail. In that sense, and despite early vague guesses about their nature, for most of a century one could assert that genes were in fact quite real, not metaphysical, entities at all. Not only that, but genes were the causal basis of biological traits and their evolution!
But things have turned out not to be so simple or straightforward. Our concept of 'the gene' is in rather great flux, in some ways each instance needing its own ad hoc treatment. Is a regulatory element a 'gene', for example, or a modified epigenetic bit of DNA? Is the 'gene' as still often taught in textbooks still in fact largely a metaphysical concept whose stereotypical properties are convenient but not nearly as informative as is the commonly presented view, even in the scientific literature?
Are we still resting on empiricism, invoking genetic and evolutionary theory as a cover but, often without realizing it, fishing for an adequate underlying theory of biological causation, that would correspond to the seamless reality Einstein (and Darwin, for that matter) felt characterized Nature? Is the gene, like Procrustes, being surgically adapted after the fact, to fit our desired tidy definition? Is claiming a theory on which genetic-based predictions can be 'precise' a false if self-comforting claim, as a marketing tool by NIH, when in fact we don't have the kind of true underlying theory of life that Einstein dreamed of for physics and the cosmos?
We'll deal with that in our next posts.
Genomic causation....or not
By Ken Weiss and Anne Buchanan
The Big Story in the latest Nature ("A radical revision of human genetics: Why many ‘deadly’ gene mutations are turning out to be harmless," by Erika Check Hayden) is that genes thought to be clearly causal of important diseases aren't always (the link is to the People magazine-like cover article in that issue.) This is a follow-up on an August Nature paper describing the database from which the results discussed in this week's Nature are drawn. The apparent mismatch between a gene variant and a trait can be, according to the paper, the result of technical error, a mis-call by a given piece of software, or due to the assumption that the identification of a given mutation in affected but not healthy individuals means the causal mutation has been found, without experimentally confirming the finding--which itself can be tricky for reasons we'll discuss. Insufficient documentation of 'normal' sequence variation has meant that the frequency of so-called causal mutations hasn't been available for comparative purposes. Again we'll mention below what 'insufficient' might mean, if anything.
People in general and researchers in particular need to be more than dismissively aware of these issues, but the conclusion that we still need to focus on single genes as causal of most disease, that is, do MuchMoreOfTheSame, which is an implication of the discussion, is not so obviously justified. We'll begin with our usual contrarian statement that the idea here is being overhyped as if it were new, but we know that except for its details it clearly is not, for reasons we'll also explain. That is important because presenting it as a major finding, and still focusing on single genes as being truly causal vs mistakenly identified, ignores what we think the deeper message needs to be.
The data come from a mega-project known as ExAC, a consortium of researchers sharing DNA sequences to document genetic variation and further understand disease causation, and now including data from approximately 60,000 individuals (in itself, rather small compared to the need for purpose). The data are primarily exome sequences, that is, from protein-coding regions of the human genome, not from whole genome sequences, again a major issue. We have no reason at all to critique the original paper itself, which is large, sophisticated, and carefully analyzed as far as we can tell; but the excess claims about its novelty are we think very much hyperbolized, and that needs to be explained.
Some of the obvious complicating issues
We know that a gene generally does not act alone. DNA in itself is basically inert. We've been and continue to be misled by examples of gene causation in which context and interactions don't really matter much, but that leads us still to cling to these as though they are the rule. This reinforces the yearning for causal simplicity and tractability. Essentially even this ExAC story, or its public announcements, doesn't properly acknowledge causal context and complexity because it is critiquing some simplistic single-gene inferences, and assuming that the problems are methodological rather than conceptual.
There are many aspects of causal context that complicate the picture, that are not new and we're not making them up, but which the Bigger-than-Ever Data pleas don't address:
Well, yes, we're always critical, but this new finding isn't really a surprise
To some readers we are too often critical, and at least some of us have to confess to a contrarian nature. But here is why we say that these new findings, like so many that are by the grocery checkout in Nature, Science, and People magazines, while seemingly quite true, should not be treated as a surprise or a threat to what we've already known--nor a justification of just doing more, or much more of the same.
Gregor Mendel studied fully penetrant (deterministic) causation. That is what we now know to be 'genes', in which the presence of the causal allele (in 2-allele systems) always caused the trait (green vs yellow peas, etc.; the same is true of recessive as dominant traits, given the appropriate genotype). But this is generally wrong, save at best for the exceptions such as those that Mendel himself knowingly and carefully chose to study. But even this was not so clear! Mendel has been accused of 'cheating' by ignoring inconsistent results. This may have been data fudging, but it is at least as likely to have been reacting to what we have known for a century as 'incomplete penetrance'. (Ken wrote on this a number of years ago in one of his Evolutionary Anthropology columns.) For whatever reason--and see below--the presence of a 'dominant' gene or 'recessive' homozyosity at a 'causal' gene doesn't always lead to the trait.
In most of the 20th century the probabilistic nature of real-world as opposed to textbook Mendelism has been completely known and accepted. The reasons for incomplete penetrance were not known and indeed we had no way to know them as a rule. Various explanations were offered, but the statistical nature of the inferences (estimates of penetrance probability, for example) were common practice and textbook standards. Even the original authors acknowledge incomplete penetrance, but this essentially shows that what the ExAC consortium is reporting are details but nothing fundamentally new nor surprising. Clinicians or investigators acting as if a variant were always causal should be blamed for gross oversimplification, and so should hyperbolic news media.
Recent advances such as genomewide association studies (GWAS) in various forms have used stringent statistical criteria to minimize false discovery. This has led to mapped 'hits' that satisfied those criteria only accounting for a fraction of estimated overall genomic causation. This was legitimate in that it didn't leave us swamped with hundreds of very weak or very rare false positive genome locations. But even the acceptable, statistically safest genome sites showed typically small individual effects and risks far below 1.0. They were not 'dominant' in the usual sense. That means that people with the 'causal' allele don't always, and in fact do not usually, have the trait. This has been the finding for quantitative traits like stature and qualitative ones like presence of diabetes, heart attack-related events, psychiatric disorders and essentially all traits studied by GWAS. It is not exactly what the ExAC data were looking at, but it is highly relevant and is the relevant basic biological principle.
This does not necessarily mean that the target gene is not important for the disease trait, which seems to be one of the inferences headlined in the news splashes. This is treated as a striking or even fundamental new finding, but it is nothing of that sort. Indeed, the genes in question may not be falsely identified, but may very well contribute to risk in some people under some conditions at some age and in some environments. The ExAC results don't really address this because (for example) to determine when a gene variant is a risk variant one would have to identify all the causes of 'incomplete penetrance' in every sample, but there are multiple explanations for incomplete penetrance, including the list of 1 - 13 above as well as methodological issues such as those pointed out by the ExAC project paper itself.
In addition, there may be 'protective' variants in the other regions of the genome (that is, the trait may need the contribution of many different genome regions), and working that out would typically involve "hyper astronomical" combinations of effects using unachievable, not to mention uninterpretable, sample sizes--from which one would have to estimate risk effects of almost uncountable numbers of sequence variants. If there were, say, 100 other contributing genes, each with their own variant genotypes including regulatory variants, the number of combinations of backgrounds one would have to sort through to see how they affected the 'falsely' identified gene is effectively uncountable.
Even the most clearly causal genes such as variants of BRCA1 and breast cancer have penetrance far less than 1.0 in recent data (here referring to lifetime risk; risk at earlier ages is very far from 1.0). The risk, though clearly serious, depends on cohort, environmental and other mainly unknown factors. Nobody doubts the role of BRCA1 but it is not in itself causal. For example, it appears to be a mutation repair gene, but if no (or not enough) cancer-related mutations arise in the breast cells in a woman carrying a high-risk BRCA1 allele, she will not get breast cancer as a result of that gene's malfunction.
There are many other examples of mapping that identified genes that even if strongly and truly associated with a test trait have very far from complete penetrance. A mutation in HFE and hemochromatosis comes to mind: in studies of some Europeans, a particular mutation seemed always to be present, but if the gene itself were tested in a general data base, rather than just in affected people, it had little or no causal effect. This seems to be the sort of thing the ExAC report is finding.
The generic reason is again that genes, essentially all genes, work only in their context. That context includes 'environment', which refers to all the other genes and cells in the body and the external or 'lifestyle' factors, and also age and sex as well. There is no obvious way to identify, evaluate or measure the effects of all possibly relevant lifestyle effects, and since these change, retrospective evaluation has unknown bearing on future risk (the same can be said of genomic variants for the same reason). How could these even be sampled adequately?
Likewise, volumes of long-existing experimental and highly focused results tell the same tale. Transgenic mice, for example, in which the same mutation is introduced into their 'same' gene as in humans, very often show little or no, or only strain-specific effects. This is true in other experimental organisms. The lesson, and it's by far not a new or very recent one, is that genomic context is vitally important, that is, it is person-specific genomic backgrounds of a target gene that affect the latter's effect strength--and vice versa: that is, the same is true for each of these other genes. That is why to such an extent we have long noted the legerdemain being foist on the research and public communities by the advocates of Big Data statistical testing. Certainly methodological errors are also a problem, as the Nature piece describes, but they aren't the only problem.
So if someone reports some cases of a trait that seem too often to involve a given gene, such as the Nature piece seems generally to be about, but searches of unaffected people also occasionally find the same mutations in such genes (especially when only exomes are considered), then we are told that this is a surprise. It is, to be sure, important to know, but it is just as important to know that essentially the same information has long been available to us in many forms. It is not a surprise--even if it doesn't tell us where to go in search of genetic, much less genomic, causation.
Sorry, though it's important knowledge, it's not 'radical' nor dependent on these data!
The idea being suggested is that (surprise, surprise!) we need much more data to make this point or to find these surprisingly harmless mutations. That is simply a misleading assertion, or attempted justification, though it has become the intentional industry standard closing argument.
It is of course very possible that we're missing some aspects of the studies and interpretations that are being touted, but we don't think that changes the basic points being made here. They're consistent with the new findings but show that for many very good reasons this is what we knew was generally the case, that 'Mendelian' traits were the exception that led to a century of genetic discovery but only because it focused attention on what was then doable (while, not widely recognized by human geneticists, in parallel, agricultural genetics of polygenic traits showed what was more typical).
But now, if things are being recognized as being contextual much more deeply than in Francis' Collins money-strategy-based Big Data dreams, or 'precision' promises, and our inferential (statistical) criteria are properly under siege, we'll repeat our oft-stated mantra: deeply different, reformed understanding is needed, and a turn to research investment focused on basic science rather than exhaustive surveys, and on those many traits whose causal basis really is strong enough that it doesn't really require this deeper knowledge. In a sense, if you need massive data to find an effect, then that effect is usually very rare and/or very weak.
And by the way, the same must be true for normal traits, like stature, intelligence, and so on, for which we're besieged with genome-mapping assertions, and this must also apply to ideas about gene-specific responses to natural selection in evolution. Responses to environment (diet etc.) manifestly have the same problem. It is not just a strange finding of exome mapping studies for disease. Likewise, 'normal' study subjects now being asked for in huge numbers may get the target trait later on in their lives, except for traits basically present early in life. One can't doubt that misattributing the cause of such traits is an important problem, but we need to think of better solutions that Big Big Data, because not confirming a gene doesn't help, or finding that 'the' gene is only 'the' gene in some genomic or environmental backgrounds is the proverbial and historically frustrating needle in the haystack search. So the story's advocated huge samples of 'normals' (random individuals) cannot really address the causal issue definitively (except to show what we know, that there's a big problem to be solved). Selected family data may--may--help identify a gene that really is causal, but even they have some of the same sorts of problems. And may apply only to that family.
The ExAC study is focused on severe diseases, which is somewhat like Mendel's selective approach, because it is quite obvious that complex diseases are complex. It is plausible that severe, especially early onset diseases are genetically tractable, but it is not obvious that ever more data will answer the challenge. And, ironically, the ExAC study has removed just such diseases from their consideration! So they're intentionally showing what is well known, that we're in needle in haystacks territory, even when someone has reported big needles.
Finally, we have to add that these points have been made by various authors for many years, often based on principles that did not require mega-studies to show. Put another way, we had reason to expect what we're seeing, and years of studies supported that expectation. This doesn't even consider the deep problems about statistical inference that are being widely noted and the deeply entrenched nature of that approach's conceptual and even material invested interests (see this week's Aeon essay, e.g.). It's time to change, but doing so would involve deeply revising how resources are used--of course one of our common themes here on the MT--and that is a matter almost entirely of political economy, not science. That is, it's as much about feeding the science industry as it is about medicine and public health. And that is why it's mainly about business as usual rather than real reform.
The Big Story in the latest Nature ("A radical revision of human genetics: Why many ‘deadly’ gene mutations are turning out to be harmless," by Erika Check Hayden) is that genes thought to be clearly causal of important diseases aren't always (the link is to the People magazine-like cover article in that issue.) This is a follow-up on an August Nature paper describing the database from which the results discussed in this week's Nature are drawn. The apparent mismatch between a gene variant and a trait can be, according to the paper, the result of technical error, a mis-call by a given piece of software, or due to the assumption that the identification of a given mutation in affected but not healthy individuals means the causal mutation has been found, without experimentally confirming the finding--which itself can be tricky for reasons we'll discuss. Insufficient documentation of 'normal' sequence variation has meant that the frequency of so-called causal mutations hasn't been available for comparative purposes. Again we'll mention below what 'insufficient' might mean, if anything.
People in general and researchers in particular need to be more than dismissively aware of these issues, but the conclusion that we still need to focus on single genes as causal of most disease, that is, do MuchMoreOfTheSame, which is an implication of the discussion, is not so obviously justified. We'll begin with our usual contrarian statement that the idea here is being overhyped as if it were new, but we know that except for its details it clearly is not, for reasons we'll also explain. That is important because presenting it as a major finding, and still focusing on single genes as being truly causal vs mistakenly identified, ignores what we think the deeper message needs to be.
The data come from a mega-project known as ExAC, a consortium of researchers sharing DNA sequences to document genetic variation and further understand disease causation, and now including data from approximately 60,000 individuals (in itself, rather small compared to the need for purpose). The data are primarily exome sequences, that is, from protein-coding regions of the human genome, not from whole genome sequences, again a major issue. We have no reason at all to critique the original paper itself, which is large, sophisticated, and carefully analyzed as far as we can tell; but the excess claims about its novelty are we think very much hyperbolized, and that needs to be explained.
Some of the obvious complicating issues
We know that a gene generally does not act alone. DNA in itself is basically inert. We've been and continue to be misled by examples of gene causation in which context and interactions don't really matter much, but that leads us still to cling to these as though they are the rule. This reinforces the yearning for causal simplicity and tractability. Essentially even this ExAC story, or its public announcements, doesn't properly acknowledge causal context and complexity because it is critiquing some simplistic single-gene inferences, and assuming that the problems are methodological rather than conceptual.
There are many aspects of causal context that complicate the picture, that are not new and we're not making them up, but which the Bigger-than-Ever Data pleas don't address:
1. Current data are from blood-samples and that may not reflect the true constitutive genome because of early somatic mutation, and this will vary among study subjects,
2. Life-long exposure to local somatic mutation is not considered nor measured,
3. Epigenetic changes, especially local tissue-specific ones, are not included,
4. Environmental factors are not considered, and indeed would be hard to consider,
5. Non-Europeans, and even many Europeans are barely included, if at all, though this is beginning to be addressed,
6. Regulatory variation, which GWAS has convincingly shown is much more important to most traits than coding variation, is not included. Exome data have been treated naively by many investigators as if that is what is important, and exome-only data have been used a major excuse for Great Big Grants that can't find what we know is probably far more important,
7. Non-coding regions, non-regulatory RNA regions are not included in exome-only data,
8. A mutation may be causal in one context but not in others, in one family or population and not others, rendering the determination that it's a false discovery difficult,
9. Single gene analysis is still the basis of the new 'revelations', that is, the idea being hinted at that the 'causal' gene isn't really causal....but one implicit notion is that it was misidentified, which is perhaps sometimes true but probably not always so,
10. The new reports are presented in the news, at least, as if the gene is being exonerated of its putative ill effects. But that may not be the case, because if the regulatory regions near the mutated gene have no or little activity, the 'bad' gene may simply not be being expressed. Its coding sequence could falsely be assumed to be harmless,
11. Many aspects of this kind of work are dependent on statistical assumptions and subjective cutoff values, a problem recently being openly recognized,
12. Bigger studies introduce all sorts of statistical 'noise', which can make something appear causal or can weaken its actual apparent cause. Phenotypes can be measured in many ways, but we know very well that this can be changeable and subjective (and phenotypes are not very detailed in the initial ExAC database),
13. Early reports of strong genetic findings have well known upward bias in effect size, the finder's curse that later work fails to confirm.
Well, yes, we're always critical, but this new finding isn't really a surprise
To some readers we are too often critical, and at least some of us have to confess to a contrarian nature. But here is why we say that these new findings, like so many that are by the grocery checkout in Nature, Science, and People magazines, while seemingly quite true, should not be treated as a surprise or a threat to what we've already known--nor a justification of just doing more, or much more of the same.
Gregor Mendel studied fully penetrant (deterministic) causation. That is what we now know to be 'genes', in which the presence of the causal allele (in 2-allele systems) always caused the trait (green vs yellow peas, etc.; the same is true of recessive as dominant traits, given the appropriate genotype). But this is generally wrong, save at best for the exceptions such as those that Mendel himself knowingly and carefully chose to study. But even this was not so clear! Mendel has been accused of 'cheating' by ignoring inconsistent results. This may have been data fudging, but it is at least as likely to have been reacting to what we have known for a century as 'incomplete penetrance'. (Ken wrote on this a number of years ago in one of his Evolutionary Anthropology columns.) For whatever reason--and see below--the presence of a 'dominant' gene or 'recessive' homozyosity at a 'causal' gene doesn't always lead to the trait.
In most of the 20th century the probabilistic nature of real-world as opposed to textbook Mendelism has been completely known and accepted. The reasons for incomplete penetrance were not known and indeed we had no way to know them as a rule. Various explanations were offered, but the statistical nature of the inferences (estimates of penetrance probability, for example) were common practice and textbook standards. Even the original authors acknowledge incomplete penetrance, but this essentially shows that what the ExAC consortium is reporting are details but nothing fundamentally new nor surprising. Clinicians or investigators acting as if a variant were always causal should be blamed for gross oversimplification, and so should hyperbolic news media.
Recent advances such as genomewide association studies (GWAS) in various forms have used stringent statistical criteria to minimize false discovery. This has led to mapped 'hits' that satisfied those criteria only accounting for a fraction of estimated overall genomic causation. This was legitimate in that it didn't leave us swamped with hundreds of very weak or very rare false positive genome locations. But even the acceptable, statistically safest genome sites showed typically small individual effects and risks far below 1.0. They were not 'dominant' in the usual sense. That means that people with the 'causal' allele don't always, and in fact do not usually, have the trait. This has been the finding for quantitative traits like stature and qualitative ones like presence of diabetes, heart attack-related events, psychiatric disorders and essentially all traits studied by GWAS. It is not exactly what the ExAC data were looking at, but it is highly relevant and is the relevant basic biological principle.
This does not necessarily mean that the target gene is not important for the disease trait, which seems to be one of the inferences headlined in the news splashes. This is treated as a striking or even fundamental new finding, but it is nothing of that sort. Indeed, the genes in question may not be falsely identified, but may very well contribute to risk in some people under some conditions at some age and in some environments. The ExAC results don't really address this because (for example) to determine when a gene variant is a risk variant one would have to identify all the causes of 'incomplete penetrance' in every sample, but there are multiple explanations for incomplete penetrance, including the list of 1 - 13 above as well as methodological issues such as those pointed out by the ExAC project paper itself.
In addition, there may be 'protective' variants in the other regions of the genome (that is, the trait may need the contribution of many different genome regions), and working that out would typically involve "hyper astronomical" combinations of effects using unachievable, not to mention uninterpretable, sample sizes--from which one would have to estimate risk effects of almost uncountable numbers of sequence variants. If there were, say, 100 other contributing genes, each with their own variant genotypes including regulatory variants, the number of combinations of backgrounds one would have to sort through to see how they affected the 'falsely' identified gene is effectively uncountable.
Even the most clearly causal genes such as variants of BRCA1 and breast cancer have penetrance far less than 1.0 in recent data (here referring to lifetime risk; risk at earlier ages is very far from 1.0). The risk, though clearly serious, depends on cohort, environmental and other mainly unknown factors. Nobody doubts the role of BRCA1 but it is not in itself causal. For example, it appears to be a mutation repair gene, but if no (or not enough) cancer-related mutations arise in the breast cells in a woman carrying a high-risk BRCA1 allele, she will not get breast cancer as a result of that gene's malfunction.
There are many other examples of mapping that identified genes that even if strongly and truly associated with a test trait have very far from complete penetrance. A mutation in HFE and hemochromatosis comes to mind: in studies of some Europeans, a particular mutation seemed always to be present, but if the gene itself were tested in a general data base, rather than just in affected people, it had little or no causal effect. This seems to be the sort of thing the ExAC report is finding.
The generic reason is again that genes, essentially all genes, work only in their context. That context includes 'environment', which refers to all the other genes and cells in the body and the external or 'lifestyle' factors, and also age and sex as well. There is no obvious way to identify, evaluate or measure the effects of all possibly relevant lifestyle effects, and since these change, retrospective evaluation has unknown bearing on future risk (the same can be said of genomic variants for the same reason). How could these even be sampled adequately?
Likewise, volumes of long-existing experimental and highly focused results tell the same tale. Transgenic mice, for example, in which the same mutation is introduced into their 'same' gene as in humans, very often show little or no, or only strain-specific effects. This is true in other experimental organisms. The lesson, and it's by far not a new or very recent one, is that genomic context is vitally important, that is, it is person-specific genomic backgrounds of a target gene that affect the latter's effect strength--and vice versa: that is, the same is true for each of these other genes. That is why to such an extent we have long noted the legerdemain being foist on the research and public communities by the advocates of Big Data statistical testing. Certainly methodological errors are also a problem, as the Nature piece describes, but they aren't the only problem.
So if someone reports some cases of a trait that seem too often to involve a given gene, such as the Nature piece seems generally to be about, but searches of unaffected people also occasionally find the same mutations in such genes (especially when only exomes are considered), then we are told that this is a surprise. It is, to be sure, important to know, but it is just as important to know that essentially the same information has long been available to us in many forms. It is not a surprise--even if it doesn't tell us where to go in search of genetic, much less genomic, causation.
Sorry, though it's important knowledge, it's not 'radical' nor dependent on these data!
The idea being suggested is that (surprise, surprise!) we need much more data to make this point or to find these surprisingly harmless mutations. That is simply a misleading assertion, or attempted justification, though it has become the intentional industry standard closing argument.
It is of course very possible that we're missing some aspects of the studies and interpretations that are being touted, but we don't think that changes the basic points being made here. They're consistent with the new findings but show that for many very good reasons this is what we knew was generally the case, that 'Mendelian' traits were the exception that led to a century of genetic discovery but only because it focused attention on what was then doable (while, not widely recognized by human geneticists, in parallel, agricultural genetics of polygenic traits showed what was more typical).
But now, if things are being recognized as being contextual much more deeply than in Francis' Collins money-strategy-based Big Data dreams, or 'precision' promises, and our inferential (statistical) criteria are properly under siege, we'll repeat our oft-stated mantra: deeply different, reformed understanding is needed, and a turn to research investment focused on basic science rather than exhaustive surveys, and on those many traits whose causal basis really is strong enough that it doesn't really require this deeper knowledge. In a sense, if you need massive data to find an effect, then that effect is usually very rare and/or very weak.
And by the way, the same must be true for normal traits, like stature, intelligence, and so on, for which we're besieged with genome-mapping assertions, and this must also apply to ideas about gene-specific responses to natural selection in evolution. Responses to environment (diet etc.) manifestly have the same problem. It is not just a strange finding of exome mapping studies for disease. Likewise, 'normal' study subjects now being asked for in huge numbers may get the target trait later on in their lives, except for traits basically present early in life. One can't doubt that misattributing the cause of such traits is an important problem, but we need to think of better solutions that Big Big Data, because not confirming a gene doesn't help, or finding that 'the' gene is only 'the' gene in some genomic or environmental backgrounds is the proverbial and historically frustrating needle in the haystack search. So the story's advocated huge samples of 'normals' (random individuals) cannot really address the causal issue definitively (except to show what we know, that there's a big problem to be solved). Selected family data may--may--help identify a gene that really is causal, but even they have some of the same sorts of problems. And may apply only to that family.
The ExAC study is focused on severe diseases, which is somewhat like Mendel's selective approach, because it is quite obvious that complex diseases are complex. It is plausible that severe, especially early onset diseases are genetically tractable, but it is not obvious that ever more data will answer the challenge. And, ironically, the ExAC study has removed just such diseases from their consideration! So they're intentionally showing what is well known, that we're in needle in haystacks territory, even when someone has reported big needles.
Finally, we have to add that these points have been made by various authors for many years, often based on principles that did not require mega-studies to show. Put another way, we had reason to expect what we're seeing, and years of studies supported that expectation. This doesn't even consider the deep problems about statistical inference that are being widely noted and the deeply entrenched nature of that approach's conceptual and even material invested interests (see this week's Aeon essay, e.g.). It's time to change, but doing so would involve deeply revising how resources are used--of course one of our common themes here on the MT--and that is a matter almost entirely of political economy, not science. That is, it's as much about feeding the science industry as it is about medicine and public health. And that is why it's mainly about business as usual rather than real reform.
Genes: convenient tokens of our time
My post today, perhaps typically cranky, was triggered by an essay at Aeon about the influence that the film Still Alice has had on thinking about Alzheimer's Disease (AD). As the piece puts it, AD is presented in the film as a genetic disease with a simply predictable doom-like known genetic cause. The authors argue that the movie is more than entertainment. It's a portrayal that raises an important ethical issue, because it is very misleading to leave the impression that AD is a predictable genetic disease. That's because a clear genetic causation, and thus the simple 'we can test for it' representation, applies only to a small fraction of AD. The film badly misrepresents the overall reality of this awful form of the disease (a good treatment of Alzheimer's disease and its history is Margaret Lock's thoughtful The Alzheimer Conundrum, 2013, Princeton Press).
While focusing on AD, the Aeon piece makes strong statements about our obsession with genes, in ways that we think can be readily generalized. In a nutshell, genes have become the convenient tokens of our time.
Symboling is a key to making us 'human'
If there is any one thing that most distinguishes our human species from others, it may be the use of language as a symbolic way to perceive the world and communicate to others. Symboling has long been said by anthropologists to be an important key to our evolution and the development of culture, itself based on language.
Symbol and metaphor are used not just to represent the world and to communicate about it, but also to sort out our social structure and our relationships with each other and the world. Language is largely the manipulation or invocation of symbols. In a species that understands future events and generalities, like death and sex, in abstract terms, the symbols of language can be reassuring or starkly threatening. We can use them to soothe ourselves or to manipulate others, and they can also be used in the societal dance around who has power, influence, and resources.
Symbols represent a perception of reality, but a symbol is not in itself reality. It is our filter, on or around which we base our interactions and even our material lives. And, science is as thoroughly influenced by symbols as any other human endeavor.
Science is, like religion, a part of our culture that purports to lead us to understand and generalize about the world, but because science is itself a cultural endeavor, it is also part and parcel of the hierarchy and empire building we do in general, part of a cultural machinery that includes self-promotion, and mutually reinforcing service industries including news media, and even scientific journals themselves.
The current or even growing pressures to maintain factory-like 'productivity' in terms of grants coming in and papers going out is largely at odds with the fundamental purpose of science (as opposed to 'technology'). Unlike designing a better product, in the important, leading-edge areas of science, we don't know where we're going. That is indeed the reason that it is science. Exploring the unknown is what really good science is about. That's not naturally an assembly-line process, because the latter depends on using known facts. However, our society is increasingly forcing science to be like a factory, with a rather short-term kind of fiscal accountability.
Our culture, like any culture, creates symbols to use as tokens as we go about our lives. Tokens are reassuring or explanatory symbols, and we naturally use them in the manipulations for various resources that culture is often about. Nowadays, a central token is the gene.
Genes as symbols
Genes are proffered as the irrefutable ubiquitous cause of things, the salvation, the explanation, in ways rather similar to the way God and miracles are proffered by religion. Genes conveniently lead to manipulation by technology, and technology sells in our industrial culture. Genes are specific rather than vague, are enumerable, can be seen as real core 'data' to explain the world. Genes are widely used as ultimate blameworthy causes, responsible for disease which comes to be defined as what happens when genes go 'wrong'. Being literally unseen, like angels, genes can take on an aura of pervasive power and mystery. The incantation by scientists is that if we can only be enabled to find them we can even cure them (with CRISPR or some other promised panacea), exorcising their evil. All of this invocation of fundamental causal tokens is particulate enough to be marketable for grants and research proposals, great for publishing in journals and for news media to gawk at in wonder. Genes provide impressively mysterious tokens for scientists to promise almost to create miracles by manipulating. Genes stand for life's Book of Truth, much as sacred texts have traditionally done and, for many, still do.
Genes provide fundamental symbolic tokens in theories of life--its essence, its evolution, of human behavior, of good and evil traits, of atoms of causation from which everything follows. They lurk in the background, responsible for all good and evil. So in our age in human history, it is not surprising that reports of finding genes 'for' this or that have unbelievable explanatory panache. It's not a trivial aspect of this symbolic role that people (including scientists) have to take others' word for what they claim as insights.
This token does, of course, have underlying reality
We're in the age of science, so that it is only to be expected that we'll have tokens relevant to this endeavor. That we have our symbols around which to build other aspects of our culture doesn't mean that the biology of genes is being made up out of whole cloth. Unlike religion, where things can be 'verified' only by claims of communication with God, genes can of course, at least in principle, be checked and claims tested. Genes obviously do have major and fundamental roles in life. If that isn't true, we are really misperceiving fundamentals of our existence. So, even when complexities of causation are daunting, we can claim and blame what we want on genes and in a sense be correct at least at some level. That enhances and endorses the token value of genes.
Genes do have great sticking power. The Aeon piece about AD is just one of countless daily examples. A fraction of cases of AD are so closely associated with the presence of some known variants in a couple of genes, that true causation--whatever the mechanism--seems an entirely plausible explanation. Likewise, there are hundreds or thousands of disorders that seem clearly to be inherited and as the result of malfunction of one or two specific genes. The cultural extension of this in our society that we are stressing here is the extension of these clearly causative findings to the idea that causation can be enumerated in convenient ways mainly by peoples' inherited genomes and that other aspects of biological causation are often treated as being rather superficial or incidental. That in a sense is typical of deeply held cultural icons or tokens.
The problem with genes as tokens is that they are invoked generally or generically in the competition for cultural resources, material and symbolic. Personally, we think there are issues, genetic issues in fact, that deserve greater investment, rather than just the easier to invoke bigger-is-better approach. They include a much more intense attack on those many traits that we already know without any serious doubt are tractably genetic--due to one or only a couple of genes, and therefore which real genetic therapy might treat or prevent effectively. By contrast, most traits even if they are affected by genetic variation as all traits must be, are predominantly due to environmental or chance causative factors. We have ways to avoid many diseases that don't require genetic approaches, but as vague entities they're perfect subjects for invoking the gene token, and policy in the industrial world clearly shows this.
Some progress does of course occur because of genetically-based research, but the promise far outpaces the reality of genetic cures. But genes are the material tokens that keep the motor running far beyond the actual level of progress. They effectively reflect our time--our molecular, computer, technological culture imagery, our love of scale, size and the material grandeur they generate.
Every culture, every generation has its tokens and belief systems. Genes are among ours. They're never perfect. People seek hope, and what velvet robes and gilded cathedrals and mosques provide for many, whereas the humming laboratories do for a growing number of others.
Tokens, symbols and metaphors: they drive much of what people do, even in science.
While focusing on AD, the Aeon piece makes strong statements about our obsession with genes, in ways that we think can be readily generalized. In a nutshell, genes have become the convenient tokens of our time.
Symboling is a key to making us 'human'
If there is any one thing that most distinguishes our human species from others, it may be the use of language as a symbolic way to perceive the world and communicate to others. Symboling has long been said by anthropologists to be an important key to our evolution and the development of culture, itself based on language.
Symbol and metaphor are used not just to represent the world and to communicate about it, but also to sort out our social structure and our relationships with each other and the world. Language is largely the manipulation or invocation of symbols. In a species that understands future events and generalities, like death and sex, in abstract terms, the symbols of language can be reassuring or starkly threatening. We can use them to soothe ourselves or to manipulate others, and they can also be used in the societal dance around who has power, influence, and resources.
Symbols represent a perception of reality, but a symbol is not in itself reality. It is our filter, on or around which we base our interactions and even our material lives. And, science is as thoroughly influenced by symbols as any other human endeavor.
Science is, like religion, a part of our culture that purports to lead us to understand and generalize about the world, but because science is itself a cultural endeavor, it is also part and parcel of the hierarchy and empire building we do in general, part of a cultural machinery that includes self-promotion, and mutually reinforcing service industries including news media, and even scientific journals themselves.
The current or even growing pressures to maintain factory-like 'productivity' in terms of grants coming in and papers going out is largely at odds with the fundamental purpose of science (as opposed to 'technology'). Unlike designing a better product, in the important, leading-edge areas of science, we don't know where we're going. That is indeed the reason that it is science. Exploring the unknown is what really good science is about. That's not naturally an assembly-line process, because the latter depends on using known facts. However, our society is increasingly forcing science to be like a factory, with a rather short-term kind of fiscal accountability.
Our culture, like any culture, creates symbols to use as tokens as we go about our lives. Tokens are reassuring or explanatory symbols, and we naturally use them in the manipulations for various resources that culture is often about. Nowadays, a central token is the gene.
 |
| DNA; Wikipedia |
Genes as symbols
Genes are proffered as the irrefutable ubiquitous cause of things, the salvation, the explanation, in ways rather similar to the way God and miracles are proffered by religion. Genes conveniently lead to manipulation by technology, and technology sells in our industrial culture. Genes are specific rather than vague, are enumerable, can be seen as real core 'data' to explain the world. Genes are widely used as ultimate blameworthy causes, responsible for disease which comes to be defined as what happens when genes go 'wrong'. Being literally unseen, like angels, genes can take on an aura of pervasive power and mystery. The incantation by scientists is that if we can only be enabled to find them we can even cure them (with CRISPR or some other promised panacea), exorcising their evil. All of this invocation of fundamental causal tokens is particulate enough to be marketable for grants and research proposals, great for publishing in journals and for news media to gawk at in wonder. Genes provide impressively mysterious tokens for scientists to promise almost to create miracles by manipulating. Genes stand for life's Book of Truth, much as sacred texts have traditionally done and, for many, still do.
Genes provide fundamental symbolic tokens in theories of life--its essence, its evolution, of human behavior, of good and evil traits, of atoms of causation from which everything follows. They lurk in the background, responsible for all good and evil. So in our age in human history, it is not surprising that reports of finding genes 'for' this or that have unbelievable explanatory panache. It's not a trivial aspect of this symbolic role that people (including scientists) have to take others' word for what they claim as insights.
This token does, of course, have underlying reality
We're in the age of science, so that it is only to be expected that we'll have tokens relevant to this endeavor. That we have our symbols around which to build other aspects of our culture doesn't mean that the biology of genes is being made up out of whole cloth. Unlike religion, where things can be 'verified' only by claims of communication with God, genes can of course, at least in principle, be checked and claims tested. Genes obviously do have major and fundamental roles in life. If that isn't true, we are really misperceiving fundamentals of our existence. So, even when complexities of causation are daunting, we can claim and blame what we want on genes and in a sense be correct at least at some level. That enhances and endorses the token value of genes.
Genes do have great sticking power. The Aeon piece about AD is just one of countless daily examples. A fraction of cases of AD are so closely associated with the presence of some known variants in a couple of genes, that true causation--whatever the mechanism--seems an entirely plausible explanation. Likewise, there are hundreds or thousands of disorders that seem clearly to be inherited and as the result of malfunction of one or two specific genes. The cultural extension of this in our society that we are stressing here is the extension of these clearly causative findings to the idea that causation can be enumerated in convenient ways mainly by peoples' inherited genomes and that other aspects of biological causation are often treated as being rather superficial or incidental. That in a sense is typical of deeply held cultural icons or tokens.
The problem with genes as tokens is that they are invoked generally or generically in the competition for cultural resources, material and symbolic. Personally, we think there are issues, genetic issues in fact, that deserve greater investment, rather than just the easier to invoke bigger-is-better approach. They include a much more intense attack on those many traits that we already know without any serious doubt are tractably genetic--due to one or only a couple of genes, and therefore which real genetic therapy might treat or prevent effectively. By contrast, most traits even if they are affected by genetic variation as all traits must be, are predominantly due to environmental or chance causative factors. We have ways to avoid many diseases that don't require genetic approaches, but as vague entities they're perfect subjects for invoking the gene token, and policy in the industrial world clearly shows this.
Some progress does of course occur because of genetically-based research, but the promise far outpaces the reality of genetic cures. But genes are the material tokens that keep the motor running far beyond the actual level of progress. They effectively reflect our time--our molecular, computer, technological culture imagery, our love of scale, size and the material grandeur they generate.
Every culture, every generation has its tokens and belief systems. Genes are among ours. They're never perfect. People seek hope, and what velvet robes and gilded cathedrals and mosques provide for many, whereas the humming laboratories do for a growing number of others.
Tokens, symbols and metaphors: they drive much of what people do, even in science.
Statistical Reform.....or Safe-harbor Treadmill Science?
We have recently commented on the flap in statistics circles about the misleading use of significance test results (p-values) rather than a more complete and forthright presentation of the nature of the results and their importance (three posts, starting here). There has been a lot of criticism of what boils down to misrepresentative headlines publicizing what are in essence very minor results. The American Statistical Association recently published a statement about this, urging clearer presentation of results. But one may ask about this and the practice in general. Our recent set of posts discussed the science. But what about the science politics in all of this?
The ASA is a trade organization whose job it is, in essence, to advance the cause and use of statistical approaches in science. The statistics industry is not a trivial one. There are many companies who make and market statistical analytic software. Then there are the statisticians themselves and their departments and jobs. So one has to ask is the ASA statement and the other hand-wringing sincere and profound or, or to what extent, is this a vested interest protecting its interests? Is it a matter of finding a safe harbor in a storm?
Statistical analysis can be very appropriate and sophisticated in science, but it is also easily mis- or over-applied. Without it, it's fair to say that many academic and applied fields would be in deep trouble; sociopolitical sciences and many biomedical sciences as well fall into this category. Without statistical methods to compare and contrast sampled groups, these areas rest on rather weak theory. Statistical 'significance' can be used to mask what is really low level informativeness or low importance under a patina of very high quantitative sophistication. Causation is the object of science, but statistical methods too often do little more than describe some particular sample.
When a problem arises, as here, there are several possible reactions. One is to stop and realize that it's time for deeper thinking: that current theory, methods, or approaches are not adequately addressing the questions that are being asked. Another reaction is to do public hand-wringing and say that what this shows is that our samples have been too small, or our presentations not clear enough, and we'll now reform.
But if the effects being found are, as is the case in this controversy, typically very weak and hence not very important to society, then the enterprise and the promised reform seem rather hollow. The reform statements have had almost no component that suggests that re-thinking is what's in order. In that sense, what's going on is a stalling tactic, a circling of wagons, or perhaps worse, a manufactured excuse to demand even larger budgets and longer-term studies, that is to demand more--much more--of the same.
The treadmill problem
If that is what happens, it will keep scientists and software outfits and so on, on the same treadmill they've been on, that has led to the problem. It will also be contrary to good science. Good science should be forced by its 'negative' results, to re-think its questions. This is, in general, how major discoveries and theoretical transformations have occurred. But with the corporatization of academic professions, both commercial and in the sense of trade-unions, we have an inertial factor that may actually impede real progress. Of course, those dependent on the business will vigorously resist or resent such a suggestion. That's normal and can be expected, but it won't help unless a spirited attack on the problems at hand goes beyond more-of-the-same.
Is it going to simulate real new thinking, or mainly just strategized thinking for grants and so on?
So is the public worrying about this a holding action or a strategy? Or will we see real rather than just symbolic, pro forma, reform? The likelihood is not, based on the way things work these days.
There is a real bind here. Everyone depends on the treadmill and keeping it in operation. The labs need their funding and publication treadmills, because staff need jobs and professors need tenure and nice salaries. But if by far most findings in this arena are weak at best, then what journals will want to publish them? They have to publish something and keep their treadmill going. What news media will want to trumpet them, to feed their treadmill? How will professors keep their jobs or research-gear outfits sell their wares?
There is fault here, but it's widespread, a kind of silent conspiracy and not everyone is even aware of it. It's been built up gradually over the past few decades, like the frog in slowly heating water who does't realize he's about to be boiled alive. We wear the chains we've forged in our careers. It's not just a costly matter, and one of understandable careerism. It's a threat to the integrity of the enterprise itself.
We have known many researchers who have said they have to be committed to a genetic point of view because that's what you have to do to get funded, to keep your lab going, to get papers in the major journals or have a prominent influential career. One person applying for a gene mapping study to find even lesser genomic factors than the few that were already well-established said, when it was suggested that rather than find still more genes, perhaps the known genes might now be investigated instead, "But, mapping is what I do!". Many a conversation I've heard is a quiet boasting about applying for funding for work that's already been done, so one can try something else (that's not being proposed for reviewers to judge).
If this sort of 'soft' dishonesty is part of the game (and if you think it's 'soft'), and yet science depends centrally on honesty, why do we think we can trust what's in the journals? How many seriously negating details are not reported, or buried in huge 'supplemental' files, or not visible because of intricate data manipulation? Gaming the system undermines the very core of science: its integrity. Laughing about gaming the system adds insult to injury. But gaming the system is being taught to graduate students early in their careers (it's called 'grantsmanship').
We have personally encountered this sort of attitude, expressed only in private of course, again and again in the last couple of decades during which big studies and genetic studies have become the standard operating mode in universities, especially biomedical science (it's rife in other areas like space research, too, of course).
There's no bitter personal axe being ground here. I've retired, had plenty of funding through the laboratory years, our work was published and recognized. The problem is of science not personal. The challenge to understand genetics, development, causation and so forth is manifestly not an easy one, or these issues would not have arisen.
It's only human, perhaps, given that the last couple of generations of scientists systematically built up an inflated research community, and the industries that serve it, much of which depends on research grant funding, largely at the public trough, with jobs and labs at stake. The members of the profession know this, but are perhaps too deeply immersed to do anything major to change it, unless some sort of crisis forces that upon us. People well-heeled in the system don't like these thoughts being expressed, but all but the proverbial 1%-ers, cruising along just fine in elite schools with political clout and resources, know there's a problem and know they dare not say too much about it.
The statistical issues are not the cause. The problem is a combination of the complexity of biological organisms as they have evolved, and the simplicity of human desires to understand (and not to get disease). We are pressured not just to understand, but to translate that into dramatically better public and individual health. Sometimes it works very well, but we naturally press the boundaries, as science should. But in our current system we can't afford to be patient. So, we're on a treadmill, but it's largely a treadmill of our own making.
The ASA is a trade organization whose job it is, in essence, to advance the cause and use of statistical approaches in science. The statistics industry is not a trivial one. There are many companies who make and market statistical analytic software. Then there are the statisticians themselves and their departments and jobs. So one has to ask is the ASA statement and the other hand-wringing sincere and profound or, or to what extent, is this a vested interest protecting its interests? Is it a matter of finding a safe harbor in a storm?
Statistical analysis can be very appropriate and sophisticated in science, but it is also easily mis- or over-applied. Without it, it's fair to say that many academic and applied fields would be in deep trouble; sociopolitical sciences and many biomedical sciences as well fall into this category. Without statistical methods to compare and contrast sampled groups, these areas rest on rather weak theory. Statistical 'significance' can be used to mask what is really low level informativeness or low importance under a patina of very high quantitative sophistication. Causation is the object of science, but statistical methods too often do little more than describe some particular sample.
When a problem arises, as here, there are several possible reactions. One is to stop and realize that it's time for deeper thinking: that current theory, methods, or approaches are not adequately addressing the questions that are being asked. Another reaction is to do public hand-wringing and say that what this shows is that our samples have been too small, or our presentations not clear enough, and we'll now reform.
But if the effects being found are, as is the case in this controversy, typically very weak and hence not very important to society, then the enterprise and the promised reform seem rather hollow. The reform statements have had almost no component that suggests that re-thinking is what's in order. In that sense, what's going on is a stalling tactic, a circling of wagons, or perhaps worse, a manufactured excuse to demand even larger budgets and longer-term studies, that is to demand more--much more--of the same.
The treadmill problem
If that is what happens, it will keep scientists and software outfits and so on, on the same treadmill they've been on, that has led to the problem. It will also be contrary to good science. Good science should be forced by its 'negative' results, to re-think its questions. This is, in general, how major discoveries and theoretical transformations have occurred. But with the corporatization of academic professions, both commercial and in the sense of trade-unions, we have an inertial factor that may actually impede real progress. Of course, those dependent on the business will vigorously resist or resent such a suggestion. That's normal and can be expected, but it won't help unless a spirited attack on the problems at hand goes beyond more-of-the-same.
Is it going to simulate real new thinking, or mainly just strategized thinking for grants and so on?
So is the public worrying about this a holding action or a strategy? Or will we see real rather than just symbolic, pro forma, reform? The likelihood is not, based on the way things work these days.
There is a real bind here. Everyone depends on the treadmill and keeping it in operation. The labs need their funding and publication treadmills, because staff need jobs and professors need tenure and nice salaries. But if by far most findings in this arena are weak at best, then what journals will want to publish them? They have to publish something and keep their treadmill going. What news media will want to trumpet them, to feed their treadmill? How will professors keep their jobs or research-gear outfits sell their wares?
There is fault here, but it's widespread, a kind of silent conspiracy and not everyone is even aware of it. It's been built up gradually over the past few decades, like the frog in slowly heating water who does't realize he's about to be boiled alive. We wear the chains we've forged in our careers. It's not just a costly matter, and one of understandable careerism. It's a threat to the integrity of the enterprise itself.
If this sort of 'soft' dishonesty is part of the game (and if you think it's 'soft'), and yet science depends centrally on honesty, why do we think we can trust what's in the journals? How many seriously negating details are not reported, or buried in huge 'supplemental' files, or not visible because of intricate data manipulation? Gaming the system undermines the very core of science: its integrity. Laughing about gaming the system adds insult to injury. But gaming the system is being taught to graduate students early in their careers (it's called 'grantsmanship').
We have personally encountered this sort of attitude, expressed only in private of course, again and again in the last couple of decades during which big studies and genetic studies have become the standard operating mode in universities, especially biomedical science (it's rife in other areas like space research, too, of course).
It's only human, perhaps, given that the last couple of generations of scientists systematically built up an inflated research community, and the industries that serve it, much of which depends on research grant funding, largely at the public trough, with jobs and labs at stake. The members of the profession know this, but are perhaps too deeply immersed to do anything major to change it, unless some sort of crisis forces that upon us. People well-heeled in the system don't like these thoughts being expressed, but all but the proverbial 1%-ers, cruising along just fine in elite schools with political clout and resources, know there's a problem and know they dare not say too much about it.
The statistical issues are not the cause. The problem is a combination of the complexity of biological organisms as they have evolved, and the simplicity of human desires to understand (and not to get disease). We are pressured not just to understand, but to translate that into dramatically better public and individual health. Sometimes it works very well, but we naturally press the boundaries, as science should. But in our current system we can't afford to be patient. So, we're on a treadmill, but it's largely a treadmill of our own making.
Playing the Big Fiddle while Rome burns?
We've seemed to have forgotten the trust-busting era that was necessary to control monopolistic acquisition of resources. That was over a century ago, and now we're again allowing already huge companies to merge and coalesce. It's rationalized in various ways, naturally, by those on the gain. It's the spirit and the power structure of our times, for whatever reason. Maybe that explains why the same thing is happening in science as universities coo over their adoption of 'the business model'.
We're inundated in jargonized ways of advertising to co-opt research resources, with our 'omics' and 'Big Data' labeling. Like it or not, this is how the system is working in our media and self-promotional age. One is tempted to say that, as with old Nero, it may take a catastrophic fire to force us to change. Unfortunately, that imagery is apparently quite wrong. There were no fiddles in Nero's time, and if he did anything about the fire it was to help sponsor various relief efforts for those harmed by it. But whatever imagery you want, our current obsession with scaling up to find more and more that explains less and less is obvious. Every generation has its resource competition games, always labeled as for some greater good, and this is how our particular game is played. But there is a fire starting, and at least some have begun smelling the smoke.
The smolder threatens to become an urgent fire, truly, and not just as a branding exercise. It is a problem recognized not just by nay-saying cranks like us who object to how money is being burnt to support fiddling with more-of-the-same-not-much-new research. It is an area where a major application of funds could have enormously positive impact on millions of people, and where causation seems to be quite tractable and understandable enough that you could even find it with a slide rule.
We refer to the serious, perhaps acute, problem with antibiotic resistance. Different bugs are being discovered to be major threats, or to have evolved to become so, both for us and for the plants and animals who sacrifice their lives to feed us. Normal evolutionary dynamics, complemented with our agricultural practices, our population density and movement, and perhaps other aspects of our changing of local ecologies, is opening space for the spread of new or newly resistant pathogens.
This is a legitimate and perhaps imminent threat of a potentially catastrophic scale. Such language is not an exercise in self-promotional rhetoric by those warning us of the problem. There is plenty of evidence that epidemic or even potentially pandemic shadows loom. Ebola, zika, MRSA, persistent evolving malaria, and more should make the point and we have history to show that epidemic catastrophes can be very real indeed.
Addressing this problem rather than a lot of the wheel-spinning, money-burning activities now afoot in the medical sciences would be where properly constrained research warrants public investment. The problem involves the ecology of the pathogens, our vulnerabilities as hosts, weaknesses in the current science, and problems in the economics of such things as antibacterial drugs or vaccinations. These problems are tractable, with potentially huge benefit.
For a quick discussion, here is a link to a program by the statistical watchdog BBC Radio program MoreOrLess on antibiotic resistance Of course there are many other papers and discussions as well. We're caught between urgently increasing need, and the logistics, ecology, and economics that threaten to make the problem resistant to any easy fixes.
There's plenty of productive science that can be done that is targeted to individual causes that merit our attention, and for which technical solutions of the kind humans are so good at might be possible. We shouldn't wait to take antibiotic resistance seriously, but clearing away the logjam of resource commitments in genetic and epidemiological research to large weakly statistical efforts well into diminishing returns, or research based on rosy promises where we know there are few flowers, will not be easy...but we are in danger of fiddling around detecting risk factors with ever-decreasing effect sizes until the fire spreads to our doorsteps.
We're inundated in jargonized ways of advertising to co-opt research resources, with our 'omics' and 'Big Data' labeling. Like it or not, this is how the system is working in our media and self-promotional age. One is tempted to say that, as with old Nero, it may take a catastrophic fire to force us to change. Unfortunately, that imagery is apparently quite wrong. There were no fiddles in Nero's time, and if he did anything about the fire it was to help sponsor various relief efforts for those harmed by it. But whatever imagery you want, our current obsession with scaling up to find more and more that explains less and less is obvious. Every generation has its resource competition games, always labeled as for some greater good, and this is how our particular game is played. But there is a fire starting, and at least some have begun smelling the smoke.
 |
| Nero plucks away. Sourcc: Wikipedia images, public domain |
We refer to the serious, perhaps acute, problem with antibiotic resistance. Different bugs are being discovered to be major threats, or to have evolved to become so, both for us and for the plants and animals who sacrifice their lives to feed us. Normal evolutionary dynamics, complemented with our agricultural practices, our population density and movement, and perhaps other aspects of our changing of local ecologies, is opening space for the spread of new or newly resistant pathogens.
This is a legitimate and perhaps imminent threat of a potentially catastrophic scale. Such language is not an exercise in self-promotional rhetoric by those warning us of the problem. There is plenty of evidence that epidemic or even potentially pandemic shadows loom. Ebola, zika, MRSA, persistent evolving malaria, and more should make the point and we have history to show that epidemic catastrophes can be very real indeed.
Addressing this problem rather than a lot of the wheel-spinning, money-burning activities now afoot in the medical sciences would be where properly constrained research warrants public investment. The problem involves the ecology of the pathogens, our vulnerabilities as hosts, weaknesses in the current science, and problems in the economics of such things as antibacterial drugs or vaccinations. These problems are tractable, with potentially huge benefit.
For a quick discussion, here is a link to a program by the statistical watchdog BBC Radio program MoreOrLess on antibiotic resistance Of course there are many other papers and discussions as well. We're caught between urgently increasing need, and the logistics, ecology, and economics that threaten to make the problem resistant to any easy fixes.
There's plenty of productive science that can be done that is targeted to individual causes that merit our attention, and for which technical solutions of the kind humans are so good at might be possible. We shouldn't wait to take antibiotic resistance seriously, but clearing away the logjam of resource commitments in genetic and epidemiological research to large weakly statistical efforts well into diminishing returns, or research based on rosy promises where we know there are few flowers, will not be easy...but we are in danger of fiddling around detecting risk factors with ever-decreasing effect sizes until the fire spreads to our doorsteps.
The statistics of Promissory Science. Part II: The problem may be much deeper than acknowledged
Yesterday, I discussed current issues related to statistical studies of things like genetic or other disease risk factors. Recent discussion has criticized the misuse of statistical methods, including a statement on p-values by the American Statistical Association. As many have said, the over-reliance on p-values can give a misleading sense that significance means importance of a tested risk factor. Many touted claims are not replicated in subsequent studies, and analysis has shown this may preferentially apply to the 'major' journals. Critics have suggested that p-values not be reported at all, or only if other information like confidence intervals (CIs) and risk factor effect sizes be included (I would say prominently included). Strict adherence will likely undermine what even expensive major studies can claim to have found, and it will become clear that many purported genetic, dietary, etc., risk factors are trivial, unimportant, or largely uninformative.
However, today I want to go farther, and question whether even making these correctives doesn't go far enough, and would perhaps serve as a convenient smokescreen for far more serious implications of the same issue. There is reason to believe the problem with statistical studies is more fundamental and broad than has been acknowledged.
Is reporting p-values really the problem?
Yesterday I said that statistical inference is only as good as the correspondence between the mathematical assumptions of the methods and what is being tested in the real world. I think the issues at stake rest on a deep disparity between them. Worse, we don't and often cannot know which assumptions are violated, or how seriously. We can make guesses and do all auxiliary tests and the like, but as decades of experience in the social, behavioral, biomedical, epidemiological, and even evolutionary and ecological worlds show us, we typically have no serious way to check these things.
The problem is not just that significance is not the same as importance. A somewhat different problem with standard p-value cutoff criteria is that many of the studies in question involve many test variables, such as complex epidemiological investigations based on long questionnaires, or genomewide association studies (GWAS) of disease. Normally, p=0.05 means that by chance one test in 20 will seem to be significant, even if there's nothing causal going on in the data (e.g., if no genetic variant actually contributes to the trait). If you do hundreds or even many thousands of 0.05 tests (e.g., of sequence variants across the genome), even if some of the variables really are causative, you'll get so many false positive results that follow-up will be impossible. A standard way to avoid that is to correct for multiple testing by using only p-values that would be achieved by chance only once in 20 times of doing a whole multivariable (e.g., whole genome) scan. That is a good, conservative approach, but means that to avoid a litter of weak, false positives, you only claim those 'hits' that pass that standard.
You know you're only accounting for a fraction of the truly causal elements you're searching for, but they're the litter of weakly associated variables that you're willing to ignore to identify the mostly likely true ones. This is good conservative science, but if your problem is to understand the beach, you are forced to ignore all the sand, though you know it's there. The beach cannot really be understood by noting its few detectable big stones.
But even this sensible play-it-conservative strategy has deeper problems.
How 'accurate' are even these preferred estimates?
The metrics like CIs and effect sizes that critics are properly insisting be (clearly) presented along with or instead of p-values face exactly the same issues as the p-value: the degree to which what is modeled fits the underlying mathematical assumptions on which test statistics rest.
To illustrate this point, the Pythagorean Theorem in plane geometry applies exactly and universally to right triangles. But in the real world there are no right triangles! There are approximations to right triangles, and the value of the Theorem is that the more carefully we construct our triangle the closer the square of the hypotenuse is to the sum of the squares of the other sides. If your result doesn't fit, then you know something is wrong and you have ideas of what to check (e.g., you might be on a curved surface).
In our statistical study case, knowing an estimated effect size and how unusual it is seems to be meaningful, but we should ask how accurate these estimates are. But that question often has almost no testable meaning: accurate relative to what? If we were testing a truth derived from a rigorous causal theory, we could ask by how many decimal places our answers differ from that truth. We could replicate samples and increase accuracy, because the signal to noise ratio would systematically improve. Were that to fail, we would know something was amiss, in our theory or our instrumentation, and have ideas how to find out what that was. But we are far, indeed unknowably far, from that situation. That is because we don't have such an externally derived theory, no analog to the Pythagorean Theorem, in important areas where statistical study techniques are being used.
In the absence of adequate theory, we have to concoct a kind of data that rests almost entirely on internal comparison to reveal whether 'something' of interest (often that we don't or cannot specify) is going on. We compare data such as cases vs controls, which forces us to make statistical assumptions such as that, other than (say) exposure to coffee, our sample of diseased vs normal subjects differ only in their coffee consumption, or that the distribution of other variation in unmeasured variables is random with regard to coffee consumption among our cases and controls subjects. This is one reason, for example, that even statistically significant correlation does not imply causation or importance. The underlying, often unstated assumptions are often impossible to evaluate. The same problem relates to replicability: for example, in genetics, you can't assume that some other population is the same as the population you first studied. Failure to replicate in this situation does not undermine a first positive study. For example, a result of a genetic study in Finland cannot be replicated properly elsewhere because there's only one Finland! Even another study sample within Finland won't necessarily replicate the original sample. In my opinion, the need for internally based comparison is the core problem, and a major reason why theory-poor fields often do so poorly.
The problem is subtle
When we compare cases and controls and insist on a study-wide 5% significance level to avoid a slew of false-positive associations, we know we're being conservative as described above, but at least those variables that do pass the adjusted test criterion are really causal with their effect strengths accurately estimated. Right? No!
When you do gobs of tests, some very weak causal factor may by good luck pass your test. But of those many contributing causal factors, the estimated effect size of the lucky one that passes the conservative test is something of a fluke. The estimated effect size may well be inflated, as experience in follow-up studies often or even typically shows.
In this sense it's not just p-values that are the problem, and providing ancillary values like CIs and effect sizes in study reports is something of a false pretense of openness, because all of these values are vulnerable to similar problems. The promise to require these other data is a stopgap, or even a strategy to avoid adequate scrutiny of the statistical inference enterprise itself.
It is nobody's fault if we don't have adequate theory. The fault, dear Brutus, is in ourselves, for using Promissory Science, and feigning far deeper knowledge than we actually have. We do that rather than come clean about the seriousness of the problems. Perhaps we are reaching a point where the let-down from over-claiming is so common that the secret can't be kept in the bag, and the paying public may get restless. Leaking out a few bits of recognition and promising reform is very different from letting all it all out and facing the problem bluntly and directly. The core problem is not whether a reported association is strong or meaningful, but, more importantly, that we don't know or know how to know.
This can be seen in a different way. If all studies including negative ones were reported in the literature, then it would be only right that the major journals should carry those findings that are most likely true, positive, and important. That's the actionable knowledge we want, and a top journal is where the most important results should appear. But the first occurrence of a finding, even if it turns out later to be a lucky fluke, is after all a new finding! So shouldn't investigators report it, even though lots of other similar studies haven't yet been done? That could take many years or, as in the example of Finnish studies, be impossible. We should expect negative results should be far more numerous and less interesting in themselves, if we just tested every variable we could think of willy-nilly, but in fact we usually have at least some reason to look, so it is far from clear what fraction of negative results would undermine the traditional way of doing business. Should we wait for years before publishing anything? That's not realistic.
If the big-name journals are still seen as the place to publish, and their every press conference and issue announcement is covered by the splashy press, why should they change? Investigators may feel that if they don't stretch things to get into these journals, or just publish negative results, they'll be thought to have wasted their time or done poorly designed studies. Besides normal human vanity, the risk is that they will not be able to get grants or tenure. That feeling is the fault of the research, reputation, university, and granting systems, not the investigator. Everyone knows the game we're playing. As it is, investigators and their labs have champagne celebrations when they get a paper in one of these journals, like winning a yacht race, which is a reflection of what one could call the bourgeois nature of the profession these days.
How serious is the problem? Is it appropriate to characterize what's going on as fraud, hoax, or silent conspiracy? Probably in some senses yes; at least there is certainly culpability among those who do understand the epistemological nature of statistics and their application. Plow ahead anyway is not a legitimate response to fundamental problems.
When reality is closely enough approximated by statistical assumptions, causation can be identified, and we don't need to worry about the details. Many biomedical and genetic, and probably even some sociological problems are like that. The methods work very well in those cases. But this doesn't gainsay the accusation that there is widespread over-claiming taking place and that the problem is a deep lack of sufficient theoretical understanding of our fields of interest, and a rush to do more of the same year after year.
It's all understandable, but it needs fixing. To be properly addressed, an entrenched problem requires more criticism even than this one has been getting recently. Until better approaches come along, we will continue wasting a lot of money in the rather socialistic support of research establishments that keep on doing science that has well-known problems.
Or maybe the problem isn't the statistics, after all?
The world really does, after all, seem to involve causation and at its basis seems to be law-like. There is truth to be discovered. We know this because when causation is simple or strong enough to be really important, anyone can find it, so to speak, without big samples or costly gear and software. Under those conditions, numerous details that modify the effect are minor by comparison to the major signals. Hundreds or even thousands of clear, mainly single-gene based disorders are known, for example. What is needed is remediation, hard-core engineering to do something about the known causation.
However, these are not the areas where the p-value and related problems have arisen. That happens when very large and SASsy studies seem to be needed, and the reason is that there causal factors are weak and/or so complex. Along with trying to root out misrepresentation and failure to report the truth adequately, we should ask whether, perhaps, the results showing frustrating complexity are correct.
Maybe there is not a need for better theory after all. In a sense the defining aspect of life is that it evolves not by the application of external forces as in physics, but by internal comparison--which is just what survey methods assess. Life is the result of billions of years of differential reproduction, by chance and various forms of selection--that is, continual relative comparison by local natural circumstances. 'Differential' is the key word here. It is the relative success among peers today that determines the genomes and their effects that will be here tomorrow. In a way, in effect and if often unwittingly and for lack of better ideas, that's just the sort of comparison made in statistical studies.
From that point of view, the problem is that we don't want to face up to the resulting truth, which is that a plethora of changeable, individually trivial causal factors is what we find because that's what exists. That we don't like that, don't report it cleanly, and want strong individual causation is our problem, not Nature's.
Is reporting p-values really the problem?
Yesterday I said that statistical inference is only as good as the correspondence between the mathematical assumptions of the methods and what is being tested in the real world. I think the issues at stake rest on a deep disparity between them. Worse, we don't and often cannot know which assumptions are violated, or how seriously. We can make guesses and do all auxiliary tests and the like, but as decades of experience in the social, behavioral, biomedical, epidemiological, and even evolutionary and ecological worlds show us, we typically have no serious way to check these things.
The problem is not just that significance is not the same as importance. A somewhat different problem with standard p-value cutoff criteria is that many of the studies in question involve many test variables, such as complex epidemiological investigations based on long questionnaires, or genomewide association studies (GWAS) of disease. Normally, p=0.05 means that by chance one test in 20 will seem to be significant, even if there's nothing causal going on in the data (e.g., if no genetic variant actually contributes to the trait). If you do hundreds or even many thousands of 0.05 tests (e.g., of sequence variants across the genome), even if some of the variables really are causative, you'll get so many false positive results that follow-up will be impossible. A standard way to avoid that is to correct for multiple testing by using only p-values that would be achieved by chance only once in 20 times of doing a whole multivariable (e.g., whole genome) scan. That is a good, conservative approach, but means that to avoid a litter of weak, false positives, you only claim those 'hits' that pass that standard.
You know you're only accounting for a fraction of the truly causal elements you're searching for, but they're the litter of weakly associated variables that you're willing to ignore to identify the mostly likely true ones. This is good conservative science, but if your problem is to understand the beach, you are forced to ignore all the sand, though you know it's there. The beach cannot really be understood by noting its few detectable big stones.
 |
| Sandy beach; Wikipedia, Lewis Clark |
But even this sensible play-it-conservative strategy has deeper problems.
How 'accurate' are even these preferred estimates?
The metrics like CIs and effect sizes that critics are properly insisting be (clearly) presented along with or instead of p-values face exactly the same issues as the p-value: the degree to which what is modeled fits the underlying mathematical assumptions on which test statistics rest.
To illustrate this point, the Pythagorean Theorem in plane geometry applies exactly and universally to right triangles. But in the real world there are no right triangles! There are approximations to right triangles, and the value of the Theorem is that the more carefully we construct our triangle the closer the square of the hypotenuse is to the sum of the squares of the other sides. If your result doesn't fit, then you know something is wrong and you have ideas of what to check (e.g., you might be on a curved surface).
 |
| Right triangle; Wikipedia |
In our statistical study case, knowing an estimated effect size and how unusual it is seems to be meaningful, but we should ask how accurate these estimates are. But that question often has almost no testable meaning: accurate relative to what? If we were testing a truth derived from a rigorous causal theory, we could ask by how many decimal places our answers differ from that truth. We could replicate samples and increase accuracy, because the signal to noise ratio would systematically improve. Were that to fail, we would know something was amiss, in our theory or our instrumentation, and have ideas how to find out what that was. But we are far, indeed unknowably far, from that situation. That is because we don't have such an externally derived theory, no analog to the Pythagorean Theorem, in important areas where statistical study techniques are being used.
In the absence of adequate theory, we have to concoct a kind of data that rests almost entirely on internal comparison to reveal whether 'something' of interest (often that we don't or cannot specify) is going on. We compare data such as cases vs controls, which forces us to make statistical assumptions such as that, other than (say) exposure to coffee, our sample of diseased vs normal subjects differ only in their coffee consumption, or that the distribution of other variation in unmeasured variables is random with regard to coffee consumption among our cases and controls subjects. This is one reason, for example, that even statistically significant correlation does not imply causation or importance. The underlying, often unstated assumptions are often impossible to evaluate. The same problem relates to replicability: for example, in genetics, you can't assume that some other population is the same as the population you first studied. Failure to replicate in this situation does not undermine a first positive study. For example, a result of a genetic study in Finland cannot be replicated properly elsewhere because there's only one Finland! Even another study sample within Finland won't necessarily replicate the original sample. In my opinion, the need for internally based comparison is the core problem, and a major reason why theory-poor fields often do so poorly.
The problem is subtle
When we compare cases and controls and insist on a study-wide 5% significance level to avoid a slew of false-positive associations, we know we're being conservative as described above, but at least those variables that do pass the adjusted test criterion are really causal with their effect strengths accurately estimated. Right? No!
When you do gobs of tests, some very weak causal factor may by good luck pass your test. But of those many contributing causal factors, the estimated effect size of the lucky one that passes the conservative test is something of a fluke. The estimated effect size may well be inflated, as experience in follow-up studies often or even typically shows.
In this sense it's not just p-values that are the problem, and providing ancillary values like CIs and effect sizes in study reports is something of a false pretense of openness, because all of these values are vulnerable to similar problems. The promise to require these other data is a stopgap, or even a strategy to avoid adequate scrutiny of the statistical inference enterprise itself.
It is nobody's fault if we don't have adequate theory. The fault, dear Brutus, is in ourselves, for using Promissory Science, and feigning far deeper knowledge than we actually have. We do that rather than come clean about the seriousness of the problems. Perhaps we are reaching a point where the let-down from over-claiming is so common that the secret can't be kept in the bag, and the paying public may get restless. Leaking out a few bits of recognition and promising reform is very different from letting all it all out and facing the problem bluntly and directly. The core problem is not whether a reported association is strong or meaningful, but, more importantly, that we don't know or know how to know.
This can be seen in a different way. If all studies including negative ones were reported in the literature, then it would be only right that the major journals should carry those findings that are most likely true, positive, and important. That's the actionable knowledge we want, and a top journal is where the most important results should appear. But the first occurrence of a finding, even if it turns out later to be a lucky fluke, is after all a new finding! So shouldn't investigators report it, even though lots of other similar studies haven't yet been done? That could take many years or, as in the example of Finnish studies, be impossible. We should expect negative results should be far more numerous and less interesting in themselves, if we just tested every variable we could think of willy-nilly, but in fact we usually have at least some reason to look, so it is far from clear what fraction of negative results would undermine the traditional way of doing business. Should we wait for years before publishing anything? That's not realistic.
If the big-name journals are still seen as the place to publish, and their every press conference and issue announcement is covered by the splashy press, why should they change? Investigators may feel that if they don't stretch things to get into these journals, or just publish negative results, they'll be thought to have wasted their time or done poorly designed studies. Besides normal human vanity, the risk is that they will not be able to get grants or tenure. That feeling is the fault of the research, reputation, university, and granting systems, not the investigator. Everyone knows the game we're playing. As it is, investigators and their labs have champagne celebrations when they get a paper in one of these journals, like winning a yacht race, which is a reflection of what one could call the bourgeois nature of the profession these days.
How serious is the problem? Is it appropriate to characterize what's going on as fraud, hoax, or silent conspiracy? Probably in some senses yes; at least there is certainly culpability among those who do understand the epistemological nature of statistics and their application. Plow ahead anyway is not a legitimate response to fundamental problems.
When reality is closely enough approximated by statistical assumptions, causation can be identified, and we don't need to worry about the details. Many biomedical and genetic, and probably even some sociological problems are like that. The methods work very well in those cases. But this doesn't gainsay the accusation that there is widespread over-claiming taking place and that the problem is a deep lack of sufficient theoretical understanding of our fields of interest, and a rush to do more of the same year after year.
It's all understandable, but it needs fixing. To be properly addressed, an entrenched problem requires more criticism even than this one has been getting recently. Until better approaches come along, we will continue wasting a lot of money in the rather socialistic support of research establishments that keep on doing science that has well-known problems.
Or maybe the problem isn't the statistics, after all?
The world really does, after all, seem to involve causation and at its basis seems to be law-like. There is truth to be discovered. We know this because when causation is simple or strong enough to be really important, anyone can find it, so to speak, without big samples or costly gear and software. Under those conditions, numerous details that modify the effect are minor by comparison to the major signals. Hundreds or even thousands of clear, mainly single-gene based disorders are known, for example. What is needed is remediation, hard-core engineering to do something about the known causation.
However, these are not the areas where the p-value and related problems have arisen. That happens when very large and SASsy studies seem to be needed, and the reason is that there causal factors are weak and/or so complex. Along with trying to root out misrepresentation and failure to report the truth adequately, we should ask whether, perhaps, the results showing frustrating complexity are correct.
Maybe there is not a need for better theory after all. In a sense the defining aspect of life is that it evolves not by the application of external forces as in physics, but by internal comparison--which is just what survey methods assess. Life is the result of billions of years of differential reproduction, by chance and various forms of selection--that is, continual relative comparison by local natural circumstances. 'Differential' is the key word here. It is the relative success among peers today that determines the genomes and their effects that will be here tomorrow. In a way, in effect and if often unwittingly and for lack of better ideas, that's just the sort of comparison made in statistical studies.
From that point of view, the problem is that we don't want to face up to the resulting truth, which is that a plethora of changeable, individually trivial causal factors is what we find because that's what exists. That we don't like that, don't report it cleanly, and want strong individual causation is our problem, not Nature's.
Kaydol:
Kayıtlar (Atom)
Rare Disease Day and the promises of personalized medicine
O ur daughter Ellen wrote the post that I republish below 3 years ago, and we've reposted it in commemoration of Rare Disease Day, Febru...
-
Pakistan dizileri Hint dizilerinden farklı. Onlar gibi coşkulu olmuyor genelde. Bu yüzden yarım bıraktıklarım hayli fazla. Ama bu dizi ... -
O ur daughter Ellen wrote the post that I republish below 3 years ago, and we've reposted it in commemoration of Rare Disease Day, Febru... -
W e discussed a Japanese pachinko machine in an earlier post , a pinball machine, as an example of the difference between randomness and det...