One of the commonplaces of scientific analysis is that it must be objective. Unfortunately, this is commonplace is often misinterpreted.

In everyday usage objectivity is often taken to mean impartiality. In research and analysis, though, true impartiality is hard to find. People have stakes in the results of their and others' research and analysis (keeping the paycheques coming, for example, or promoting one's career). The type of objectivity which is promoted in research and analysis is intended as a check on partiality.

In research and analysis objectivity simply means reproducibility. An objective test, for example, is one which produces the same scores regardless of who scores it (so the Rorschach test would not be an objective test). Similarly, decisionmaking has been made objective through the use of tests of statistical significance. These tests stipulate exactly what type of evidence is required before a researcher can decide that two groups are different or similar.

In general any method – including so-called qualitative methods and projective tests like the Rorschach – can be made objective by applying the relevant statistical method. One way to make qualitative methods objective is to use multiple raters and assess their agreement with a formula like the Spearman-Brown prediction formula.

Openness is another important aspect of objectivity. Research reports should provide enough information to allow anyone to test your findings by reproducing your study (or replicating it, as research jargon has it). Openness even extends to the sharing of data sets, as long as it is possible to maintain any necessary confidentiality. Sharing data can be an especially important safeguard when an analytical method requiring the exercise of individual judgment (multiple linear regression, for example) has been used.