## Observations

Observations

- Observations are likely our most reliable source of claims. However, we know even that observations can go wrong. Knowing the limitations and occasional failures of observations will help us avoid those failures.
- Observations come in at least two kinds

- Particular occurent observations: a claim about some particular thing observed right now. "This table is red."
- Generalizations from observation

Particular and Occurrent Observations

- Particular observations are observations of a single event or individual thing.
- Occurrent observations are observations made right here, right now.
- Particular and occurrent observations are usually our most reliable sources of claims.
- However, particular and occurrent claims are susceptible to a number of problems.

- Our observations can be influenced by our
expectations. (Rosenthal and Fode 1966, 1963; Rosenthal and Lawson 1964).- We attend to things in our environment, and remember them, if they are particularly vivid -- for example, if they are emotionally salient. Similarly, there is some evidence that our observations can be influenced by our
emotionsso that we are more likely to interpret those observations in a way consistent with our emotions. This a phenomenon called "emotional congruence in perception" (Neidenthal $$$).- Particular remembered claims (that is, particular but not occurrent) can be inaccurate. Memories can form in an inaccurate way, or can be revised over time inaccurately. This is true even for vivid, flashbulb memories. ($$$)
- Generally, we must recognize that particular observations are strong but fallible evidence for a claim.

Controlling for Bias in Particular Occurrent Observations

- One way to avoid potential problems with particular occurrent observations, particularly a bias arising from expectations, is to control for the expectations of the observer.
- This is what is done in double blind experiments. Such experiments are constructed typically in such a way that the individuals making observations do not know which observations have the relevant feature.
- For example, suppose that we wanted to test the effects of substance X on some rats.

- We would construct the experiment so that half of our rats were fed substance X, and the other half would be given a placebo.
- The actual feeding of the rats would be done in such a way that the experimenter who will record the observations will not know which rats are fed which substance. We could do this, for example, but having the feeding done by experimenter A, or by having the food prepared by experimenter A, when another experimenter B will make the observations.
- Experimenter A and B should not talk about the experiment. Ideally, they should not even meet.
- Experimenter B makes observations not knowing which rats received which food.
- Ideally, experimenter A (or some other experimenter) interprets the results of B without knowing which rats are which.
- Here expectation bias cannot directly effect observations because the observer is not sure which of the things it is observing have the cause about which she has some expectations.

Generalized Observations

- Often we need to make general claims based on a finite number of observations. This kind of reasoning -- generalizing from some sample of observations to conclude something about an entire population of events or things -- is sometimes called inductive reasoning.
- When we generalize from particular to general in this way, we need to ensure, to the best of our ability, that our sample observations are
representativeof the population.- If our population is not homogenous regarding the feature we are measuring, we can try to ensure that a sample is representative by ensuring that it is
random.- A sample is random if every potential observation on our population had an equally likely chance of being in the sample of observations.
- Getting a random sample of observations is not easy.
- Many populations will also have natural variation. If we are measuring the height of people, for example, there is significant variation in the heights of individuals. But this means that a small, even if random, sample has a chance of being quite different than the mean (average) in the population.
- We try to ensure that variation in the population is controlled for by getting a sample that is
sufficiently large.- The size of a sufficiently large sample is a function of how accurate we hope to be, the size of the overall population, and the expected variability of the population. That calculation is beyond our goals here, but in general we should at least be aware that:

- Generally, the larger the sample size, the more accurate our generalization is likely to be.
- Generalizations have a
margin of errorthat is a function of the size of the sample, the size of the population, and the variability in the population.- The margin of error is the expected likely error in our generalization, given our sample size and our knowledge of the population.

Common Reasoning Mistakes with Generalizations

- Forgetting that larger samples are more representative of the population (and that thus smaller samples are more variable).
- Failing to recognize when samples are not random.
- Over-valuing anecdotes, and not recognizing that these represent a small, even irrelevantly small, sample. Vivid or concrete anecdotes have an especially strong influence on people's generalizations. Vivid information can overwhelm uninteresting information in people's perceptions.
- Resisting revision of first generalizations.

Generalizations to Probabilities from Observation

- One form of generalization from observations is the derivation of a probability from past events. In most cases this is a simple derivation: if we observe that n% of our sample has some property P, then assuming our sample is random and sufficiently large, we assume that observations in the future will have property P n% of the time.
- Example: we observe in a random sample of oysters that 1% of them have a pearl. For any given oyster that we may later encounter, we conclude there is about a 1% chance that there is a pearl in it.
- Three problems commonly occur for this kind of generalization; these echo concerns about other forms of generalizations from observations discussed above.

- Availability. Individuals tend to confuse the availability of something with its frequency. That is, if something is available to them, they may assume that it is more frequent than it actually is. (This is akin to generalizing from one's own, non-random, sample.)
- Vividness. As noted above, people tend to notice and to recall more vivid experiences. This can lead people to overestimate the probability of some event.
- Gamblers fallacy. This common mistake arises when someone confuses the probability of two or more outcomes, taken as a whole, with the probability of an single outcome after some other outcomes of that kind have preceeded it.

- An example of the gambler's fallacy. Above, we assumed that 1% of oysters had pearls. Consider the following two cases:
The odds of finding a pearl in an oyster are 1%, we assumed. You should then expect it to be very likely to find a pearl in a sample of 100 of them. However, if you have opened 99 and found no pearl, the odds of a pearl in the last oyster is 1%. That is, you must not confuse the case of many many oysters with each single one. The gambler's fallacy occurs if Jones assumes she is now due for a pearl, and expects that it is very likely the last oyster has a pearl.

- Jones has two oysters, and wants to know what the odds of finding a pearl in each is.
- Jones has 100 oysters, opens 99 of them and finds no pearls inside, and now wants to know what the odds of finding a pearl in the 100th oyster are.
- The gamblers fallacy has led many a gambler to poor reasoning, such as concluding that if they have been losing for a long time they are due to win.