Cartwright on theory and experiment in science

Nancy Cartwright’s A Philosopher Looks at Science is a new treatment of some of the longstanding themes of her work.  It is written in her characteristically agreeable style and full of insights.  The book is devoted to criticizing three widespread but erroneous assumptions about science: that science is essentially just theory plus experiment; that in some sense everything science tells us is reducible to physics; and that science reveals that everything that happens, including human action, is determined by the laws of physics.  In this post I’ll discuss what she says about the first of these claims, which is the subject of the first and longest chapter in the book.  I may devote a later post to the other claims.

The supposition that science amounts to theory plus experiment is, Cartwright observes, widespread among laymen, scientists, and philosophers alike.  The mathematically expressible kind of scientific theory, familiar from modern physics and enshrined in equations like F = ma, is taken to be the gold standard.  From such equations, it is thought, specific observable consequences are predicted, and the point of experimentation is to test these predictions.  And that’s basically it.  Except, as Cartwright shows, that isn’t it, not by a long shot.  In addition to theory and experimentation, there are models, narratives, diagrams, illustrations, concrete applications, and so on.  None of these is reducible to theory or experiment, and neither are they any less essential to the practice and content of science.  And when we take account of them, both science and the world it describes are seen to be far more complicated than the common conception of science and its results implies.

Theoretical concepts

Cartwright begins her analysis by noting that any theory is expressed in concepts, and that science aims for concepts with content that is both unambiguous and empirical.  As all philosophers of science know, it turns out to be very difficult to come up with a general account of how this is achieved.  Cartwright summarizes the familiar difficulties.  First of all, explicit definitions of theoretical terms are obviously of limited help when the definition is itself couched in yet further theoretical terms.  At some point we need to arrive at terms with clear empirical content.  But exactly how does that work?

Operationalism held that the solution was to define a theoretical concept in terms of some operation by which the scientist could measure the empirical phenomenon captured by the concept.  But there are several reasons why this won’t work.  For one thing, it entails reductionist analyses that we can independently know to be false.  Cartwright offers the example of behaviorism, which was an application of operationalism to psychology.  The behaviorist would define anger, for example, in terms of the observable behavior on the basis of which we would attribute anger to someone. 

Note that the implication of operationalism here is not just that we can know someone is angry by way of observing his behavior.  It is that there is nothing more to anger than the behavior.  Now, one problem with this claim is that it simply isn’t true.  A person could be angry without exhibiting the usual behavioral signs of anger, and could also exhibit those signs without actually being angry.  Hence anger is something more than the behavior.  Another problem is that it turns out even apart from that to be impossible entirely to analyze anger or any other mental state in entirely behavioral terms.  Suppose we say that “John is angry” means “John is disposed to raise his voice, frown, stomp his feet, etc.”  The trouble is that this sentence will be true only if John does not desire to hide his feelings.  But if we add a reference to the absence of this desire to our definition, we’ve now got a further mentalistic concept – desire – that needs to be given a behaviorist analysis.  And it turns out that to carry out such an analysis, we need to make reference to yet further mental states, with those now needing a behaviorist analysis, and so on ad infinitum.  Hence the operationalist analysis cannot actually be carried out. 

A second problem with operationalism is that it has the false implication that there cannot be different empirical tests for the same concept.  For again, operationalism holds that there is nothing more to a concept than the operation by which we test its application.  Hence, if we have two different tests, we must be dealing with two different concepts.  But that’s absurd.  Take, for example, the concept of being round.  I can test whether something is round either by looking at it or by feeling it, and obviously it is one and the same concept I am applying in both cases.

A third problem, as Cartwright emphasizes, is that in actual scientific practice it often takes a lot of hard work and argumentation to show that a certain empirical test plausibly measures the reality captured by some scientific concept.  That could not be the case if there were nothing more to the concept than the empirical test.  It follows that there is more to theoretical concepts than what is captured by such tests, in which case operationalism is false.

Logical empiricism, as Cartwright notes, was another failed attempt to solve the problem.  The “logical” component of logical empiricism had to do with its application of modern formal logic to the formulation of scientific theories, e.g. as axiomatic systems from which theorems could be deduced.  The “empiricism” component had to do with the idea that the claims of a theory could be verified by observation.  Here too there are several problems. 

For one thing, exactly what counts as an observation?  Only what can be perceived by the naked eye (or the naked ear, the naked nose, etc.)?  Or do observations made using instruments count?  Furthermore, what exactly is it that we are observing – mind-independent physical objects, or sense data?  And are all scientific claims really verifiable in this way in the first place?  (See pp. 139-51 of my book Aristotle’s Revenge: The Metaphysical Foundations of Physical and Biological Science for detailed discussion of the intractable problems facing verificationism.)

It turns out that, just as the content of theoretical concepts outstrips what can be captured in an operational definition, so too, more generally, does it outstrip what is observable.  The content of concepts is given instead by the axioms of the theory in which they are embedded.  But the problem now, as Cartwright notes, is that such axioms are never sufficient to determine exactly what it is in the empirical world a theory is about.  Consider, again, the equation F = ma.  Considered just by itself, it tells us nothing more than that one quantity is equal to the product of two others.  And as Cartwright observes, this is true not only of the force, mass, and acceleration of a material object, but also of the area of a rectangle with respect to the length of its sides.  There is nothing in the equation itself that tells us which of these is its subject matter.  Of course, we could add further items to our set of axioms, such as Newton’s law of universal gravitation.  But no matter how many we add, there will always be alternative possible interpretations. 

(This issue is closely related to the epistemic structural realist thesis that physical theories reveal to us only the abstract structure of the physical world and not its intrinsic nature.  See chapter 3 of Aristotle’s Revenge for detailed discussion.)

Of course, in practice, scientists and the laymen who are familiar with their work don’t worry about such problems.  The reason is that, for one thing, when encountering an equation like F = ma, most people have at least in the backs of their minds the ordinary language usage of terms like “force,” “mass,” and “acceleration,” and thus naturally interpret the variables in light of them, even if they know that the variables aren’t meant to correspond exactly to our commonsense notions.  For another thing, they often apply the equation as a tool for carrying out very practical tasks, such as figuring out the speed of a ball hit by a tennis player (to borrow an example of Cartwright’s). 

But all of this comes from outside the theory itself, at least if we take the mathematics alone to be what is essential to theory as such.  Moreover, as Cartwright emphasizes, this utility of theory in practical applications does not entail that the world really is exactly the way the abstract theory represents it as being.  (She gives the well-known example of phlogiston theory, which was very useful predictively and technologically despite the fact that it turns out that there is no such thing as phlogiston.) 

I would emphasize a further point.  It is commonly assumed that scientific theory gives us a richer and more accurate representation of the world than common sense does, and indeed ought to replace the commonsense description of phenomena.  But as Cartwright’s argument indicates, this is the reverse of the truth.  For one thing, scientific theory in fact cannot even be given a determinate interpretation without some connection to the ordinary linguistic usage from which its concepts ultimately derive, and the concrete applications to which theory is put.  For another thing, what theory describes are really only abstract features of the world of common experience rather than that world in all its rich complexity.  That doesn’t necessarily entail that scientific theory should be given an instrumentalist rather than realist interpretation.  But it does support the epistemic structural realist view that, while what theory describes is really there in nature, it is very far from capturing everything that is there in nature.  (See Aristotle’s Revenge for detailed exposition and defense of this view.)

Beyond theory

Now, because of the way the actual applications of a theory often unconsciously determine how we interpret it, we can be blind to how much work is being done by the application and how little by the theory considered in isolation.  In particular, when we consider a theory in isolation, just in terms of its mathematical formulation, its concepts can seem very precise.  But a concrete application of the theory may nevertheless involve an interpretation of those concepts that is not so precise.  Yet it may retain its utility nonetheless, and retain it precisely because the concepts are being applied in a way that goes beyond the content of the theory itself.

The consequence of this is that scientists often end up supposing that precision is possible where really it is not.  Or, because a concept’s application may be susceptible of precision in one, limited domain, scientists can fallaciously suppose that it must be equally capable of precision when extended beyond that domain.  This is, Cartwright argues, especially likely in social science.  She gives as an example the notion of probability.  When we consider simple examples like pulling cards from a fair deck, the probabilities of various possible outcomes can be determined with precision.  But it simply doesn’t follow that we can meaningfully assign probabilities to events in general, and Cartwright thinks there are good reasons to suppose that this is not in fact possible. 

In particular, she notes that probabilities are determined relative to what Ian Hacking calls “chance set-ups.”  These are circumstances where both the possible outcomes and the processes that might lead to them can be fully specified, and where there are probabilities built into the situation at the start from which the probabilities we wish to calculate fall out logically.  Again, pulling cards from a fair deck would be an example.  But much of what happens in nature does not amount to a chance set-up in this sense.  For example, in the real world (as opposed to what Cartwright calls the “small world” representations that social scientists make use of) there often simply isn’t one relatively simple and fixed set of variables that might influence possible outcomes.

For this reason, Cartwright judges that much of what is said by social scientists about “effect sizes” when evaluating alternative policy proposals is poorly founded.  (Cartwright doesn’t mention the relevance of all this to arguments for various pandemic policies, criminal justice reforms, “equity-conscious” educational proposals, and other currently trendy issues, but it is obvious.  I leave the specifics as homework.)

In any event, in natural science and social science alike, Cartwright argues, theory is only ever brought to bear on the world by way of various intermediaries.  First of all, there are the idealized models by which we bring abstractions like the laws of physics to bear on concrete reality.  For example, when we apply Newton’s laws to the solar system, we do so by modeling the latter (in terms of a system of point masses orbiting a larger point mass, and so on).  In this way, our application of abstractions is mediated by further abstractions.  There are also the concrete narratives by which all of these abstractions are made intelligible.  (Think of the way that, in order to understand even a simple system like the solar system, we still have crudely to visualize large objects moving through space over time around other large objects; that in order to understand the implications of special relativity, we tell stories about twins traveling on rocket ships; and so on.)  Cartwright notes that diagrams, graphs, and illustrations also deeply influence how we interpret and apply theory.  Nor are these various intermediaries dispensable.  We simply couldn’t understand or make use of theories without them.

Finally, experimentation too, Cartwright argues, is a much more complex affair than is implied by the common notion that “science = theory + experiment.”  Experiment is often treated as if its only point is to test theory.  But that is not the case.  Sometimes experimentation is carried out even in the absence of any well-worked out theory, in an exploratory way that aims simply to see what will happen under various circumstances.  Sometimes experimentation creates new phenomena that would otherwise not be observed – where, precisely because they have not otherwise been observed, no theory yet exists to account for them.  Sometimes experimentation reconstitutes phenomena in the sense of deeply altering our understanding of them, even in the absence of theoretical considerations.  And in all these cases, experimentation, like theory, depends on fixing the content of concepts, on models, and so on.

“Science ain’t an exact science”

I’ve mentioned already one of the implications I see in Cartwright’s discussion, viz. support for an epistemic structural realist interpretation of modern physics.  Here’s another.  It is a commonplace of modern philosophy of science that theory is underdetermined by empirical evidence.  What that means is that for any body of empirical evidence, there are always alternative possible theories that are incompatible with each other but consistent with that evidence.  That does not entail that all theories are equally good, but only that considerations independent of both theory and empirical evidence are ultimately necessary in order to choose between theories.  Philosophers of science like Thomas Kuhn and Paul Feyerabend have also shown how extra-scientific considerations (of a philosophical sort, for example) play a crucial role in determining the outcome of scientific investigation.

The considerations raised by Cartwright greatly reinforce these judgments.  In particular, they reinforce the underdetermination of theory by evidence insofar as it isn’t just alternative theories that are compatible with the same empirical evidence.  There are also the alternative possible models, narratives, diagrams, etc. which mediate between theory and evidence.  And as with theories, so too with models, narratives, diagrams, etc., philosophical considerations no less than empirical ones can influence our judgments about what is within the range of respectable options, what is plausible all things considered, and so forth.

By no means does this entail that science is not a rational enterprise, any more than philosophy is not a rational enterprise.  What it does entail, though, is that the boundary between science and philosophy is much less sharp than is commonly supposed.  As I have argued at length elsewhere (including in Aristotle’s Revenge), much of what is today assumed to be “scientific” – the refusal to countenance irreducibly teleological explanations, the primary/secondary quality distinction, and so on – are really just contentious philosophical assumptions masquerading as empirical results.  And it is not possible to do science without making philosophical assumptions of some kind, which are bound to be controversial.

To borrow a line from the movie 12 Monkeys, “science ain’t an exact science.”  To be sure, there is an exactness in its purely mathematical aspects, but that is precisely because mathematical representations simply leave out all aspects of reality that don’t fit that exact mode of representation – which turns out to be quite a lot.  There are not only more things in heaven and earth than are dreamt of by scientists, there is more to science itself than is dreamt of by them.

Related reading:

Dupré on the ideologizing of science

Scientism: America’s State Religion

The particle collection that fancied itself a physicist

Fallacies physicists fall for

One Long Circular Argument

Science and Scientism

Blinded by Scientism

Rosenberg roundup