South African J. Psychology, 1979, 9, 104- 107.

Is the dogmatism scale irreversible?



J.J. Ray

School of Sociology, University of New South Wales, Australia

A review of various attempts to produce balanced versions of the Rokeach Dogmatism Scale ('D' scale) shows that the highest positive-negative correlation obtained on a general population sample was -0.32. This was obtained with the Ray (1974a) 'BD' Scale Mark II. An attempt to improve on this figure was made by administering only the best eight items of this scale to a community sample of 87 Sydney people. The positive-negative correlation obtained was -0.37. It was pointed out that Rorer's (1965) work showed only that there was no such thing as a general tendency to acquiesce. It did not rule out acquiescence as a response to ambiguity (Peabody 1966). Balanced scales are therefore still necessary. It was concluded that the 'D' Scale may be inherently and unacceptably ambiguous.

J.J. Ray
School of Sociology, University of New South Wales, P.O. Box 1, Kensington, New South Wales, Australia.

Accepted August 1979

The title of this paper is a deliberate allusion to the influential article by Christie, Havel and Seidenberg (1956) wherein the history of attempts to balance the 'F' Scale up to that time was reviewed and the conclusion drawn that the `F' Scale in fact was irreversible. Subsequent research has shown that conclusion to be in fact far too pessimistic (Lee & Warr 1969; Ray 1972a), but it will nonetheless be the contention of the present paper that what turned out not to be true of the 'F' Scale may be much more true of the 'D' Scale.

The essence of the approach that led to a successful balancing of the 'F' Scale appears first to have been suggested by Byrne and Bounds (1964). It was simply that reversed items should be selected not on any a priori criterion but rather on the empirical criterion of whether they did in fact correlate in the expected way with positive items.

Although not as extensive a literature as that concerned with the 'F' Scale, there is nonetheless also a history of attempts to balance the Rokeach (1960) 'D' Scale. Peabody (1961), Haiman and Duns (1964) and Stanley and Martin (1964) all used the a priori method of item selection that had failed with the 'F' Scale and found that it also failed with the 'D' Scale. Ray (1970) however used empirical methods of item selection and produced a balanced scale which showed correlations between its two types of item of up to -0.71. This might seem the sort of result that one could hardly cavil at but unfortunately this 'D' Scale version was constructed using second-year psychology students as subjects and it was reported in the same article that when the scale was applied to a more representative sample in the community the correlation dropped to only - 0.22. This was, of course, precisely the sort of result that had led the 'F' Scale to be viewed as "irreversible". Kirton (1977) has also confirmed that this balanced 'D' Scale does not work with a general population sample.

An obvious question, then, is: `Does this balanced "D" Scale at least work with other student samples?. Ray (1974a) reported a replication study wherein the correlation between the two halves was -0.40. One interesting aspect of this result however is to be found in Table 1 of Ray & Martin (1974). It is there shown that when the sample used to get this overall result is broken up into first, second and third year psychology students the correlations as respectively -0.34, -0.37 and -0.51. Evidently, then, the balancing is only really satisfactory with the more sophisticated respondents.

A further question that does then obviously arise is what the correlation would be like with students who had no background in academic psychology at all? Rigby (1978 pers. comm.) examined this using 155 students at the South Australian Institute of Technology. He found a correlation of -0.077. Even for most students, then, the Ray (1970) Balanced Dogmatism scale may not be satisfactory.

This does, however, leave one remaining claimant for the title of an adequate balanced `D' Scale. In addition to the replication study mentioned above, Ray (1974a) also reported two other attempts to balance the 'D' Scale. The Mark II version showed correlations between its two halves of -0.32 on a general population sample and -0.34 on a first-year psychology student sample. The Mark III version showed a correlation of -0.27 on a general population sample. In a further replication not previously reported, the Mark II Scale was administered to 117 Teacher's College students. The pos.-neg. correlation there observed was -0.38. While this correlation of the Mark II Scale is not therefore high, it does at least seem to be fairly stable.

It was therefore decided to use the Mark II Scale as the starting-point for a fourth attempt to construct a balanced 'D' Scale by empirical methods.

Method

It was felt that there might be in the Mark II 'BD' Scale at least some positive and negative items that did show a high correlation between themselves. It was decided therefore to select a small sub-set of the Mk II items which had shown the highest item-total correlation in previous item analyses. Ten such items were selected, five negative and five positive. Since the most general of the usual internal reliability coefficients (Cronbach's (1951) 'alpha') may be calculated as average inter-item correlation weighted by number of items (Ray 1972b), this procedure might seem bound to reduce the reliability of the scale by virtue of reducing the number of items. Since it was however hoped that the procedure would increase the pos.-neg. correlation, it seemed possible that a rise in the other element in the weighting (average inter-item correlation) might substantially offset this effect and leave a scale with still acceptable reliability. Selecting only 10 items, then, was designed to maximize the pos.-neg. correlation while still leaving some variety in item content.

The 10 selected items were administered together with items of other scales to a community sample of 87 Sydney people. This sample was gathered by asking students to give the questionnaire to people they knew under the constraint that the people selected were not to be students and were preferably to be in manual occupations. The sample so gathered turned out to show no significant differences in demographic characteristics from other samples gathered in the Sydney area by more usual random door-to-door methods. The present sample may then be regarded as an adequate quota sample of the Sydney area.

Results

On item analysis, the ten-item scale showed two negative items which had virtually no correlation at all with the scale total. This again shows the difficulty of obtaining successful 'D' reversals. When these were deleted, the remaining eight-item scale (hereafter to be referred to as the Mark IV BD Scale) showed an internal reliability of 0.53 and a pos.-neg. correlation of -0.373. Again, then, a high pos.-neg. correlation could not be produced.

Interpretation of this result may be assisted if it is placed in the context of results obtained with the Ray (1972a) 'Balanced F Scale'. As reported in Ray (1972a) this scale showed an initial pos.-neg. correlation on a community sample of - 0.71, a pos.-neg. correlation on students of -0.53 and a pos.-neg. correlation on a random doorstep sample of -0.56. In other results not previously reported, a random cluster sample of the Sydney area (N = 95) gave a pos.-neg. correlation of -0.65 and a sample of 47 University of Sydney first-year psychology students gave a pos.-neg. correlation of -0.70. A 14 item short form of the BF Scale was also included in a national mail sample by a commercial polling organization, giving a pos.-neg. correlation of -0.50 and a reliability of 0.80 with an N of 200. Given the present results, it must appear that the F Scale is 'much more reversible' than the `D' Scale.

The 'expected value' of the pos.-neg. correlation (rPN) for an adequate balanced scale is not inherently obvious but one approach would be to compare the obtained value with the correlation to be expected from a random split of the scale into two halves. How does the particular split giving rise to rPN, compare with an average split into any two halves? An r for such an average split can in fact be rather easily obtained by 'de-correcting' the reliability coefficient alpha' with the Spearman-Brown formula applied in reverse, i.e. r = alpha/2-alpha.

Applying this formula to the alphas for the balanced 'F' and 'D' (Mk. II) Scales on their norming samples (in fact the same sample) gives rs of 0.76 and 0.71. This compares with observed values for rPN of 0.71 and 0.32 respectively. Clearly the correlation between positive and negative items of the 'BF' Scale is typical of the correlation between any set of items in that scale but this is not at all true of the Mark II 'BD' Scale.

Applying the same formula to the present Mk IV `BD' Scale gives a value of 0.36 for average r. While this is very close to the observed value (0.37) of rPN, the convergence occurs only because the level of alpha is on this occasion unacceptably low. In addition, the structure they do have could reflect the fact that most of them would have some use in conventional discussions about the tenability of religion. They are listed below with reversed items marked 'R'.

1. Do you agree with the saying: `Eat, drink and be merry for to-morrow we may die'? R

2. The `one true faith' is a myth. R

3. Man on his own is a helpless and miserable creature.

4. Do you think it is possible to really live without believing in any great cause? R

5. Of all the different philosophies that exist in the world, there is probably only one which is correct.

6. When it comes to differences of opinion in religion, we must be careful not to compromise with those who believe differently from the way we do.

7. It is annoying to listen to a speaker or teacher who seems unable to make up his mind about what he really believes.

8. It is only when a person devotes himself to an ideal or cause that life becomes meaningful.

Another way in which a balanced scale can be evaluated is in terms of its reliability as a measure of acquiescence. If the two halves of opposite meaning are scored as if they were alike (i.e. without reversals), the total score becomes an index almost entirely of acquiescence with item meaning controlled for. The more meaningful the items (i.e. the greater the oppositeness in the way they are responded to) the less internally consistent the scale so scored should be. The Mk II `BD' Scale so scored generally shows internal reliabilities (`alpha') of around 0.50 - indicating that substantial acquiescence is generated by its items. Is this also true of the Mk IV version in the present study?

With 10 items scored without reversals, the coefficient 'alpha' reliability observed was 0.00. This indicates that the initial form of the Mk IV Scale was a successful selection of acquiescence-free items.

In several ways, then, the characteristics of the Mk IV Scale were a vindication of the thinking that led to its creation. These characteristics were achieved, however, only by a severe sacrifice in reliability and by abandoning the original goal of perfect equality between the number of positive and negative items.

Discussion

A tempting response to the above findings may well be 'So what?'. Has not Rorer (1965) shown that acquiescent set or style does not matter anyhow? If so, whether or not balancing is achieved is also immaterial. We could conclude as Kirton (1977) does that one might as well go on using the original 'D' Scale.

To do this would however be negligent indeed. Rorer showed that there was no such thing as acquiescent style by pointing out that different measures of acquiescence generally show very little correlation. What he overlooked, however, and what Peabody pointed out in his reply to Rorer (Peabody 1966), was the possibility that acquiescence might be a response. It might be elicited as a non-meaningful response to particular items with particular subjects. In Peabody's terms, it might be a characteristic response to ambiguous items. If the items are not as meaningful to the subject as they are to the scale constructor, the subject may simply say `Yes' to anything.

Thus it must come as no surprise that the Wilson C-Scale which normally shows pos.-neg. correlations of around -0.70 (Wilson 1974), might on some occasions show very different characteristics. Ray (1971, 1972c), for instance, shows that when administered to Army conscripts and students, the C-Scale shows on some occasions both much reduced reliability and a correlation between its two halves may even be in the wrong direction altogether. That this is due to acquiescence is shown by the reliability of the C-Scale scored as a measure of acquiescence (i.e. without reversals). On the sample of 110 conscripts mentioned in Ray (1972c) this was 0.61. The pos.-neg. correlation on the same sample was -0.199 (See Ray 1979). Thus while acquiescence may not be general between scales, it may show up as a consistent and distorting tendency within a particular scale on a particular occasion. The only way this possibility can ever be examined is by having a balanced scale to start with.

In this light, the failure of so many attempts to balance the 'D' Scale may suggest that the 'D' Scale is ambiguous - not only for some samples on some occasions but for almost all samples on all occasions. Even empirical methods may be incapable of selecting reversed items of recognizably opposite meaning. Such items just may not exist in general. The only explanation for this would surely be that the positive items meant very little in the first place. What the original 'D' Scale has been measuring all along then is probably little more than the tendency to acquiesce when faced with an ambiguous task. This may be of some use and interest in itself but it is certainly quite different from what Rokeach thought he was measuring.

In general, then, the indicated conclusion from the results so far would appear to be that the 'D' Scale should be used in future only with extreme caution.

As some qualification to this rather severe conclusion, however, it could perhaps be argued that the validation available for the 'D' Scale as a measure of dogmatism is impressive and that although the characteristic pos.-neg. correlation of just above -0.30 for the Mk II 'BD' Scale is not high, it is nonetheless highly significant statistically. Some ability to elicit meaningful responses has been demonstrated for 'D' Scale items. If this argument seems impressive, it might be noted that any balanced scale, no matter how low the correlation between its halves, does control out the effect of acquiescence. The Mk II 'BD' Scale could at least perform this service for prospective 'D' Scale users. The low pos.-neg. correlation is, in other words, a reflection on the scale's validity rather than on its adequacy at controlling out the effects of acquiescence.

References

Byrne, D. & Bounds, C. The reversal of F Scale Items. Psychol. Rep. 1964, 14, 216.

Christie, R., Havel, J. & Seidenberg, B. Is the `F' Scale irreversible? J. Abnorm. Psychol. 1956, 56, 141 - 148.

Cronbach, L.J. Coefficient alpha and the internal structure of tests. Psvchometrika 1951, 61, 297-334.

Haiman, F.S. & Duns, D.F. Validations in communicative behaviour of attitude scale measures of dogmatism. J. Soc. Psychol. 1964, 64, 287-297.

Kirton, M.J. Ray's balanced dogmatism scale re-examined. Brit. J. Soc. & Clin. Psychol. 1977, 16, 97-98.

Lee, R.E. & Warr, P.B. The development and standardization of a balanced 'F' Scale. J. Genetic Psychology 1969, 81, 109-129.

Peabody, D. Attitude content and agreement set in scales of authoritarianism, dogmatism, anti-semitism and economic conservatism. J. Abnorm. & Soc. Psychol. 1961, 63, 1 - 11.

Peabody, D. Authoritarianism scales and response bias. Psychol. Bull. 1966, 65, 11 - 23.

Ray, J.J. (1970) The development and validation of a balanced Dogmatism scale. Australian Journal of Psychology, 22, 253-260.

Ray, J.J. (1971) "A new measure of conservatism" -- Its limitations. British Journal of Social & Clinical Psychology, 10, 79-80.

Ray, J.J. (1972) A new balanced F scale -- And its relation to social class. Australian Psychologist 7, 155-166.

Ray, J.J. A new reliability maximization procedure for Likert scales. Australian J. Psychology, 1972b, 7, 40-46.

Ray, J.J. (1972) Are conservatism scales irreversible? British J. Social & Clinical Psychology 11, 346-352.

Ray, J.J. (1974) Balanced Dogmatism scales. Australian Journal of Psychology 26, 9-14.

Ray, J.J. & Martin, J. (1974) How desirable is dogmatism? Australian & New Zealand J. Sociology, 10, 143 - 144.

Ray, J.J. & Pratt, G.J. (1979) Is the influence of acquiescence on "catchphrase" type attitude scale items not so mythical after all? Australian Journal of Psychology 31, 73-78.

Rokeach, M. The open and closed mind. New York: Basic books, 1960.

Rorer, L. The great response style myth. Psychol. Bull. 1965, 63, 129

Stanley, G. & Martin, J. How sincere is the dogmatist? Psychol. Review, 1964, 71, 331-333.

Wilson, G. Evaluation of the conservatism scale: A reply to Ray. New Zealand Psychologist, 1974, 3,27.



POST-PUBLICATION ADDENDUM

Replication is one of the cornerstones of science. A new research result will normally require replication by later researchers before the truth and accuracy of the observation concerned is generally accepted. If a result is to be replicated, however, careful specification of the original research procedure is important.

In questionnaire research it has been my observation that the results are fairly robust as to questionnaire format. It is the content of the question that matters rather than how the question is presented (But see here and here). It is nonetheless obviously desirable for an attempted replication to follow the original procedure as closely as possible so I have given here samples of how I presented my questionnaires in most of the research I did. On all occasions, respondents were asked to circle a number to indicate their response.

FINIS

Australian Journal of Psychology Vol. 22, No. 3, 1970, pp. 253-260.

(With three post-publication addenda following the original article)

THE DEVELOPMENT AND VALIDATION OF A

BALANCED DOGMATISM SCALE




JOHN J. RAY

Macquarie University

Abstract

A "balanced" Dogmatism scale was produced by writing entirely new items to express sentiments opposed to those accepted by the archetypical Dogmatic. These items are not reversals of old ones. The scale of 36 items showed a reliability of .91 on the standardization (student) sample and .78 on an adult community sample. Validity was demonstrated for the scale as a whole and for the two halves separately.

INTRODUCTION

Since the publication of the California 'F' scale (Adorno et al., 1950) two major criticisms have been levelled against it as a measure of authoritarianism. 1. That it is ideology specific and slanted so as to cause conservatives automatically to obtain high scores; 2. That its all-positive wording causes acquiescence and authoritarianism to be confounded.

The first of these criticisms was largely met by the Rokeach (1960) dogmatism scale and Rokeach (1967), claims that his scale is sufficiently free from response bias to render a balanced version unnecessary. Although this may be true to some extent, it does seem desirable to lay the doubts about "the acquiescent personality" completely to rest where-ever it is possible to do so. The voluminous literature on the one-way wording of the 'F' scale is sufficient to inspire in every user of scales the desire to preclude from the beginning a "response set" interpretation of his own results.

Four attempts at writing balanced `D' scales have been made-by Peabody (1961), Haiman and Duns (1964), Stanley and Martin (1964) and Haiman (1964). The first three above all employed the method of taking an existing item and attempting to re-write it so as to reverse its meaning. All three were unsuccessful. Haiman (1964) however, did produce a reliable balanced scale. An important part of his method was to write some entirely new items rather than rely on reversals of existing ones. Most authors have been reluctant to do this, but the failure of the reversed item method seems to make it necessary. The only other alternative is the forced choice method (Berkowitz & Wolkon, 1964) but the basic assumption of this method is that the alternatives offered are opposites -- and it is precisely this that the failure of reversed item scales has shown not to be the case.

Haiman's thirty-item scale contained five `F' scale items, five reversed `F' scale items, five 'D' scale items, five reversed 'D' scale items plus five positive and five negative items written by himself. Unfortunately the ten items he himself contributed seem largely indistinguishable from what would normally be considered conservatism items. Haiman does state that it was his aim to write items having a specific political reference. We are told that these ten items make a disproportionate contribution to the scale's homogeneity, i.e. what this group of items measures is closest to what the scale as a whole measures.

It will be seen that Haiman's scale has irretrievably blurred the important distinctions between authoritarianism, conservatism and dogmatism that the original authors (Rokeach, 1960; Adorno et al., 1950) wished to make. For this reason, the labelling of his scale as measuring "open-mindedness" does seem rather arbitrary. Haiman appears to have put back the ideological specificity that Rokeach was at pains to take out.

Nevertheless, there is much to be said for Haiman's bold attempt to write new items and in this paper a scale will be presented wherein all negative items are new in content. In order to meet Rokeach's original aim of reducing ideology specificity, all positive items will be drawn solely from existing Dogmatism scales. Unlike Haiman's scale, no `F' scale or conservatism items will be included.

METHOD

Reliability Study

As a preliminary step, the forty item 'D' scale of Rokeach and the nine item Australian revision by Anderson & Western (1967) were administered to a group of 114 first-year psychology students at the University of Sydney. These results were combined and used to item-analyse the original scale and thus produce the most reliable form possible of the 'D' scale before further work with it was undertaken. The scale proved to be most reliable (.82) when all but 25 items were dropped. This was an improvement over the .78 reliability observed for the 49 item scale.

In the next step, the concept of Dogmatism and its potential opposites were discussed with a group of third year students in a course on attitudes conducted at Macquarie University. The opposite of dogmatism was agreed to be "open-mindedness" and a total of fifty-nine items was written by the students to tap this concept. By obtaining the items in this way it was hoped that some initial content validity could be guaranteed. These items were then included with the twenty-five 'D' scale items (mentioned above) in a questionnaire administered to forty-six second-year day and evening students at Macquarie University. Responses to each item were scored from 5 (strongly agree) to 1 (strongly disagree). The negative items were then divided into three sets, of twenty, twenty and nineteen items each and successively combined with the set of twenty-five `D' items and the usual item-total correlations computed. This procedure allowed new items to be selected with some view to their correlation with one another as well as their correlation with the original items. From these three versions of the balanced scale the twenty-four most successful negative items were chosen and combined with the twenty-four strongest `D' scale items ("strength" being assessed from correlations with the total score on the 'D' items alone). This `scale' was then reduced to a balanced thirty-six item length by dropping the items showing a low correlation with total scores on the combined scale.

Validation Study

Since the 'D' scale is taken as an already well-established instrument for which we are interested to provide a balanced version, it might seem that the only validation necessary is the information that the negative items correlate as well with the positive items as the positive items themselves correlate with the scale total. This, however, is not completely true. The whole reason for constructing a balanced scale was the claim that the `D' scale might really be measuring acquiescent tendency, and if this is so the empirical correlates with the 'D' scale heretofore observed might have been due to the characteristics of acquiescent people rather than of dogmatic people.

In other words, it should now be possible to examine for the first time empirical correlates of Dogmatism with the influence of acquiescence as an artifact removed.

The study undertaken was of the "criterion groups" type. Ideally, this type of study requires that the two groups differ on no other characteristic than that purportedly measured by the scale. For Dogmatism, such groups are very hard to find. The two groups eventually decided on were active Methodists and active Humanists. It was argued that both of these would have an equivalent high level of education and both would have a strong commitment to humanitarian ideals. The only difference would be the Methodists' adherence to a more or less clearly defined body of dogma. It was also decided to take the relatively unusual step for this type of study of going outside the University for subjects. It is a rather stringent test of a scale to administer it to a population for which it was not standardized but it was desired to eliminate the criticism (which can so often be levelled at studies in this area) that the relationships observed represent a psychology of university students rather than of the general population.

Through local Methodist church study groups and local Humanist meetings, responses were obtained from 120 people, sixty-three Methodists and fifty-seven Humanists. From each, measurements were obtained on the following seven variables: 1. Occupational Status-scored 1 to 7 where a high score indicates low status: 2. Education-scored 1 to 4 where a high score indicates high educational level: 3. Political Preference-scored 5 to 1 where a high score indicates conservatism of choice: 4. Belief scored 1 for Methodists and 2 for Humanists: 5. Dogmatism total score -- scored 180 to 36: 6. Dogmatism positive score -- scored 90 to 18: 7. Dogmatism negative score-scored 90 to 18. For all three 'D' scale scores, a high score indicates a highly dogmatic response.

Yeasaying

To answer questions about the empirical relationship between acquiescence and dogmatism (Frandsen, 1967) the Methodist-Humanist sample was divided into yeasayers (n = 80), naysayers (n = 4) and "normals" (n = 36). The basis for this classification was that 20 or more responses of "agree" or "Disagree" (out of 36) allocated the respondent to "yeasayers" or "naysayers" whilst "normals" were the residue. The means for all three groups were calculated on three "scales"-the first being formed of the eighteen Rokeach items, the second of the 18 balancing items and the third of the total thirty-six items.

RESULTS

The reliability obtained on the norming sample was .91. It should be noted that the scale is naturally balanced at this length, i.e. identical criteria for item selection were applied to both positive and negative items. The very high reliability obtained was in fact required, being a corollary of the other two requirements that only the strongest `D' scale items be used and that the negative items be equally as strong as the positive items. Such a high initial reliability is also important because the original norming maximizes both "error" and "true" inter-item correlation. Therefore some drop is always experienced on re-administration to another group of a freshly constructed scale. The items of the new scale are presented in Table 1.


TABLE 1

The Items of the Balanced `D' Scale. (The last 18 are negative items.)

1. Man on his own is a helpless and miserable creature.
2. There is so much to be done and so little time to do it in.
3. It is better to be a dead hero than a live coward.
4. A man who does not believe in some great cause has not really lived.
5. It is only when a person devotes himself to an ideal or cause that life becomes meaningful.
6. A person who gets enthusiastic about too many causes is likely to be a pretty "wishy-washy" sort of person.
7. When it comes to differences of opinion in religion we must be careful not to compromise with those who believe differently from the way we do.
8. A group which tolerates too much difference of opinion among its own members cannot exist for long.
9. There are two kinds of people in this world: those who are for the truth and those who are against the truth.
10. A person who thinks primarily of his own happiness is beneath contempt.
11. Most of the ideas which get printed nowadays aren't worth the paper they are printed on.
12. In this complicated world of ours the only way we can know what's going on is to rely on leaders or experts who can be trusted.
13. Most people just don't know what's good for them.
14. Of all the different philosphies that exist in the world there is probably one which is more in accord with reality than any of the others.
15. The person who is extremely tolerant of widely different and even conflicting viewpoints probably has few opinions of his own.
16. It is annoying to listen to a speaker or teacher who seems unable to make up his mind about what he really believes.
17. For most questions there is only one right answer once a person is able to get all the facts.
18. Although many details still remain to be worked out, we now have definite answers to most practical problems in life.
19. It's unfortunate that we have censorship in this country.
20. Live and let live is a good motto in life.
21. It doesn't matter much what religion a person follows.
22. There is good in everyone.
23. There is something to be appreciated in all forms of art.
24. I usually try to keep a fairly open mind on most issues.
25. It is possible that there are many facets to the "truth".
26. It is usually a help to get a new slant on an old problem.
27. There are few decisions which don't bear some re-consideration.
28. Criticism can perhaps be useful if it results in a reconciliation of opposing views.
29. It is usually unwise to indulge in generalizations since individual circumstances often alter cases.
30. Many problems have more than one acceptable solution.
31. What may be all right for one person isn't necessarily right for his neighbour.
32. The Churches don't give enough weight to individual conscience.
33. It is impossible to generalize about members of other races and cultures.
34. People cannot be expected to stick to the same opinions month after month.
35. Even intelligent people often change their opinions after hearing others' views. 36. People cannot be blamed for some inconsistencies in what they think.

The correlation between the positive and negative halves was .71.

The correlations observed with validity criteria etc. are given in Table 2. The reliability of the total scale for the second sample was .78. The reliability of the negative items considered as a separate scale was .64. For the positive items it was .80. This latter statistic indicates that the original `D' scale items function very similarly on both university and non-university samples.


TABLE 2

Intercorrelation of Seven Variables on 120 Subjects

........................Educ......Polit.....Belief......'D' tot........'D' pos........'D' neg.

Status.............. -.65....... .21....... .06.......... .24............. .29............. .03
Education..................... -.25....... .09......... -.33............ -.37............ -.09
Political Pref............................. -.53.......... .34............. .29............. .24
Belief........................................................ -.51............ -.40............ -.41
'D' total.......................................................................... .88............. .66
'D' positive........................................................................................ .22


Using a contrast-wise error-rate approach, all correlations above .16 are significant at the < .05 level.


Note from Table 2, that as planned, the Methodists and Humanists were empirically not differentiated by degree of education (r = .09) or occupational status (r = .06). Why these two variables are related to scores on the "positive" scale but not to scores on the "negative" scale is unclear but it is in any case not disturbing to the goals set by the study. To the extent to which we might expect dogmatism to be equally distributed throughout the population it is in fact a positive feature.

Results of the yeasaying study appear in Table 3. As the "naysayer" group turned out to be so small, it was felt that the only comparisons of interest were those between the "yeasayers" and the "normals". The yeasayers had significantly higher scores on the Rokeach items and the total scale but significantly lower scores on the balancing items (Rokeach items -- t (114) = 9.28; Balancing items t (114) = 2.42; Total scale t (114) = 4.34). This reduced difference for the balancing items and the reversed direction appears to be at least partly a scoring artifact. It is of course the difference on the total scale which is free of artifact and of most interest here.


TABLE 3 Mean Scores and S. D.s on Three Dogmatism "Scales" for Naysayers, Yeasayers and "Normals" Among the Methodist-Humanist Sample

Group................ Rokeach items.........Balancing items...........Total Scale

Normals...............43.36 (5.74) ..............41.58 (5.89)................84.94 (10.95)
Yeasayers............56.36 (9.08)...............38.62 (6.60)................94.98 (13.46)
Naysayers............36.00 (0.70)...............45.25 (3.34)................81.25 ( 3.70)




DISCUSSION

The reliability on the student sample was all that might be desired of a scale. Given this high initial level, the drop observed in the second study was still not such as to render the reliability unsatisfactory.

As a validation experiment, the second study was successful in that the correlation between Belief and Dogmatism was .51, which is significant and in the predicted direction. The significant correlation with political preference is contrary to Rokeach's expectation but consonant with his findings (Rokeach, 1956). It is in any case a feature common to both negative and positive halves of the scale. It is not unparalleled in other published studies (Direnzo, 1968; Kirtley & Harkness, 1969) and Rokeach (1960) has claimed that such a feature is not inconsistent with the validity of a dogmatism measure. Dogmatism is a variable of great interest in its own right (Rokeach and Fruchter, 1956) and the fact that it does empirically turn out to correlate weakly with right wing ideology is a fact about the world (albeit a fact disappointing to the hope that dogmatism might turn out to be the non-ideological component of authoritarianism). The charge by Simons (1968) that the relationship between dogmatism and conservatism is artifactual rests on the not surprising finding that dogmatism items are rated by student judges to be more characteristic of right wing ideology. This finding was one we might have expected from the correlations between dogmatism and ideology already observed among students and can just as well be explained by saying that the students have accurate perceptions of what goes on in the world as by saying that the dogmatism items have a built-in right wing bias. The drop in reliability (although the reliability observed here is in fact identical with that for the all-positive scale quoted earlier) was of course somewhat disappointing. This drop was almost entirely contributed by the negative items, as can be seen in the correlation of only .22 between the two halves of the scale. It still remains true, however, that what is measured by the scale is the intersection of what is measured by the negative and positive halves. The scale here is unlike the balanced 'F' scales that have been produced in that the reliability of these scales is normally very low (Berkowitz & Wolkon, 1964). In any case, this low relationship of the two halves is only true for a population unlike that for which the scale was designed. On the university student sample, the two halves correlated .71. This compares with .17 on Stanley and Martin's (1964) scale (student sample) and a range between .21 and .11 on Peabody's (1961) scale (student sample). What appears to have happened is that for non-university subjects this study has replaced the weaker half of the `D' scale by an equally weak but negative set of items. This does not affect the validity of the instrument as the correlation with belief is approximately the same for each half of the scale.

In interpreting the results of the acquiescence study it must be borne in mind that the aim adopted was to remove any systematic or artifactual influence of acquiescent response set on the final total score. The stronger aim of presenting a scale with which acquiescent tendency does not correlate significantly as a matter of empirical fact was deemed to be inappropriate here. It would in fact be to create another artifact -- one directly opposite to that originally attacked. Because the published literature does offer opposing findings on the relationship of acquiescence (yeasaying) to open-mindedness (Frandsen, 1967), results bearing on this question were presented here. As it turns out, when the acquiescence artifact in Dogmatism measurement was removed by use of the new scale, Dogmatic people are shown still to be more acquiescent (t (114) = 4.34). Users of this scale may therefore wish to partial out yeasaying score in experimental applications. This has not been possible heretofore because, on a unidirectional scale, the yeasaying score and the scale score are essentially identical.

CONCLUSION

A balanced and valid 'D' scale has been constructed with reliability between .91 (for the norming sample of students) and .78 (for an adult population sample). A validity coefficient of .51 was obtained.


REFERENCES

ADORNO, T. W., FRENKEL-BRUNSWIK, E., LEVINSON, D. J. & L SANFORD, R. N. The Authoritarian Personality, New York: Harper & Row, 1950.

ANDERSON, D. S. BC WESTERN, J. S. An inventory to measure students' attitudes, St. Lucia, Brisbane: University of Queensland Press, 1967.

BERKOWITZ, N. H. & WOLKON, G. H. A forced choice form of the 'F' scale-free of acquiescent response set. Sociometry, 1964, 27, 54-65.

DIRENZO, G. J. Dogmatism and presidential preferences in the 1964 elections, Psychological Reports, 1968, 22, 1197-1202.

FRANDSEN, K. D. Haiman's revised open-mindedness scale: A comparative study of response patterns, Speech Monographs, 1967, 34, 389391.

HAIMAN, F. S. A revised scale for the measurement of open-mindedness, Speech Monographs, 1964, 31, 97102.

HAIMAN, F. S. & DUNS, D. F. Validations in communicative behaviour of attitude-scale measures of dogmatism. Journal of Social Psychology, 1964, 64, 287-297.

KIRTLEY, D. & HARKNESS, R. Some personality and attitude correlates of dogmatism. Psychological Reports, 1969, 24, 851-854.

PEABODY, D. Attitude content and agreement set in scales of Authoritarianism, Dogmatism, Anti-Semitism and Economic Conservatism. Journal of Abnormal and Social Psychology, 1961, 63, 1-11.

ROKEACH, M. Political and religious dogmatism: An alternative to the authoritarian personality. Psychological Monographs, 1956, 70, Whole no. 425.

ROKEACH, M. The open and closed mind, New York: Basic Books, 1960.

ROKEACH, M. Authoritarianism scales and response bias: Comment on Peabody's paper. Psychological Bulletin, 1967, 67, 349-355.

ROKEACH, M. & FRUCHTER, B. A factorial study of dogmatism and related concepts. Journal of Abnormal and Social Psychology, 1956, 53, 356-360.

SIMONS, H. W. Dogmatism scales and leftist bias, Speech Monographs, 1968, 35, 149-153.

STANLEY, G. & MARTIN, J. How sincere is the Dogmatist? Psychological Review, 1964, 71, 331-333.

J. J. Ray, School of Behavioural Sciences, Macquarie University, North Ryde, N.S.W, 2113.

(Manuscript received 24 March, 1970)


POST-PUBLICATION ADDENDA

1. The pos/neg correlations from the second study above have proved more typical in subsequent applications of the scale. See:

Kirton, M.J. Ray's balanced dogmatism scale re-examined. Brit. J. Soc. & Clin. Psychol. 1977, 16, 97-98.

Ray, J.J. (1979) Is the Dogmatism scale irreversible?
South African Journal of Psychology 9, 104-107.

2. Subsequent articles germane to the matters discussed above are as follows:

Ray, J.J. (1972) A new balanced F scale -- And its relation to social class. Australian Psychologist 7, 155-166.

Ray, J.J. (1972) Are conservatism scales irreversible? British J. Social & Clinical Psychology 11, 346-352.

Ray, J.J. (1974) Balanced Dogmatism scales. Australian Journal of Psychology 26, 9-14.

Ray, J.J. (1979) A short balanced F scale. Journal of Social Psychology, 109, 309-310.

Ray, J.J. (1980) Acquiescence and the Wilson Conservatism scale. Personality & Individual Differences, 1, 303-305.

Ray, J.J. & Pratt, G.J. (1979) Is the influence of acquiescence on "catchphrase" type attitude scale items not so mythical after all? Australian Journal of Psychology 31, 73-78.

Ray, J.J. (Unpublished) ACQUIESCENT RESPONSE TENDENCY: An update and some data on the invalidity of the Dogmatism scale

3. SCALE FORMAT: Replication is one of the cornerstones of science. A new research result will normally require replication by later researchers before the truth and accuracy of the observation concerned is generally accepted. If a result is to be replicated, however, careful specification of the original research procedure is important.

In questionnaire research it has been my observation that the results are fairly robust as to questionnaire format. It is the content of the question that matters rather than how the question is presented (But see here). It is nonetheless obviously desirable for an attempted replication to follow the original procedure as closely as possible so I have given here samples of how I presented my questionnaires in most of the research I did. On all occasions, respondents were asked to circle a number to indicate their response.

FINIS

Australian Journal of Psychology Vol. 26, No. 1, 1974, pp. 9-14

(With a post-publication addendum following the original article)

BALANCED DOGMATISM SCALES







JOHN J. RAY

University of New South Wales

Balancing of one-way worded scales such as the D scale has been sought by several authors. So far only the Ray (1970) BD scale has appeared successful. It is here examined whether the 1970 success can be replicated and also whether a new scale can be produced which will be suitable for use with community samples -- not only with students. The suitability of the 1970 "Mark I" version of the BD scale for use with students was confirmed and a new "Mark II" BD scale was produced which was suitable for general use. An attempt to construct an eclectic scale using the best items of Mark I and Mark II was not successful.

The problem of acquiescent set has inspired many attempts to produce balanced versions of the F (Adorno et al., 1950) and 'D' (Rokeach, 1960) scales. See, for instance, Peabody (1961), Stanley & Martin (1964) and Haiman (1964). The poor success that for long attended such efforts inspired some writers to conclude that the scales concerned were "irreversible" (Christie, Havel & Seidenberg, 1956). Ray (1970 & 1972b) has, however, shown that this is not so -- by producing balanced Dogmatism (BD) and balanced F (BF) scales that showed both satisfactory reliability and high correlations between their oppositely-worded halves.

Unlike the BF scale mentioned above, however, the BD scale was normed entirely on a university student sample. This does raise the question of its suitability for use on non-university samples. Ray (1970) does report one application of the scale to a non-university sample where the reliability obtained did drop substantially and where the correlation between the two halves of the scale dropped to a level little better than that found by previous authors.

This raises the possibility that the results obtained with the university sample reported on in Ray (1970) were in some sense "freak" results and, as such, not likely to be replicated. There is in any case a clear need to see whether it is at all possible to construct a BD scale which is suitable for general, as well as student, use. It is proposed to examine both these questions in the present paper.

Initially, it must be reiterated that there is almost always a drop in reliability recorded when a new scale is administered to samples other than its own norming sample. This is because an item analysis contains no way of distinguishing "error" correlation (due to sampling variability) from "true" correlation and must in fact maximize the inter-item correlation in the group of items to be chosen, regardless of the origin of that correlation. In a subsequent administration of the scale so constructed, however, the error component of the correlation has of course changed and correlation from this source is no longer maximized by the set of items used. The major impact of this phenomenon, then, is that one always seeks some replication before considering a freshly constructed scale suitable for regular use --the hope always being that although some drop in reliability is to be expected, it will not be great.

STUDY I

In this study, the 1970 BD scale in its original format was administered by Dr J. Martin of Macquarie University to students in first, second and third year psychology courses. The results were also collated and prepared for computer anaysis by Dr Martin. Having someone other than the original author test the scale hopefully meant that "Rosenthal effects" due to experimenter expectations were reduced. In spite of the 1970 results, Dr Martin was quite sceptical about the possibility of a balanced 'D' scale.

The total of 177 first, second and third year students were sampled roughly in proportion to their numbers in the overall university popluation. They filled out the scale in class time.

Analysis of these results revealed a correlation of .40 between the two halves of the scale and a reliability ("alpha") of .81. As expected, this was less than the original (1970) findings of .71 and .91 respectively but still leaves a quite satisfactory scale. It might also be noted that some of this drop could be due to the fact that the second sample was different from the first in being much more widely based. It may then be regarded as confirmed that the Ray (1970) BD scale is suitable for use with students. The original results were not entirely "freak" findings. This gives some encouragement to the search for a balanced Dogmatism scale that will be applicable to the general population.

STUDY II

One characteristic of the above-mentioned BD scale that might be unsatisfactory to some users was that its negative items were entirely new constructions (new in both direction of wording and in content) -- not merely reversals of Rokeach originals. Since Ray (1972b) has shown that a balanced F scale using items that were merely reversals of the originals could be produced, it seems also possible that a BD scale of this sort might be produced. The scale reported on below is of this type. More importantly, however, the present study is an attempt to build a new BD scale from scratch using the responses of a community (as distinct from a student) sample. The second study of Ray (1970) had shown that the original BD scale was unsuitable for such a sample.

The data for this study was in fact gathered at the same time as the data used for the production of the balanced F scale. It is being presented here separately for the sake of thematic unity. Full details of the sampling etc. are to be found, therefore, in Ray (1972b) but it may briefly be said that the sample was a heterogeneous one obtained partly by house to house calls and partly by enlisting evening student volunteers. The evening students are of course an older and more varied group than the day students used on previous occasions. Total n was 120. The corpus of proposed negative items was placed before the 40 Rokeach original items in the questionnaire. For most positive items, there were at least two candidate-reversals -- many of which were drawn from the work of Peabody (1961) and Stanley & Martin (1964 ). A lengthy preamble was used (see Ray, 1972b for the wording) to disarm criticism among the respondents of the rather strange-sounding questionnaire that resulted.

Analysis of the data was carried out by correlating each negative item with the Rokeach scale. The 40 highest correlating items were then selected and combined with the Rokeach items to produce a new 80 item balanced scale. Each of these items was then correlated with the total score on the same scale and the 20 strongest negative and the 20 strongest positive items selected. The 40 items resulting did, however, show some content overlap in that on some occasions two reversals of the same original item had survived the selection process. Where repetitiouness of this sort was obvious, it was eliminated by deleting the least highly correlating item of such pairs and replacing it with another negative item from the 80 item scale -- i.e. by the 41st highest correlating item and so on.

The 40 items finally selected then formed a perfectly balanced scale with a reliability of .83 and a correlation between the oppositely-worded halves of .32. For the purposes of replication, this scale was then administered, in its new reduced format, to the Introductory Psychology class at Macquarie University (n = 180). A reliability of .84 and a correlation between the halves of .34 were observed -- a finding representing extremely close replication of the results obtained on the norming sample. This new BD scale was then named the "BD scale, Mark II". See Table 1 for the items.

TABLE 1

The first 20 items are reverse scored. Five response for each item (SA, A, ?, D, SD).

1. If we are going to have free speech we must defend the right to be heard of even those we disagree with.
2. Man is master of his own fate and captain of his destiny.
3. If people in one's own group are always disagreeing among themselves that is probably a rather healthy sign.
4. There is no such thing as "the Truth".
5. We must find happiness in the present because no one can predict what the future will be like.
6. No one has a "mission in life" that he must accomplish no matter what.
7. Eat, drink and be merry -- for tomorrow we may die.
8. The "one true faith" is a myth.
9. The way to happiness is to get involved in the things going on about you.
10. There is never one right answer for any question.
11. Man has within himself the power to control his destiny.
12. In general most people show consideration for others.
13. It is not worth sacrificing your life to became a hero.
14. It's possible to really live without believing in any great cause.
15. Life can be meaningful without devotion to ideals or causes.
16. All of the philosophies which exist in this world have some truth in them and probably not one is totally correct.
17. In these present days everyone should look to their own happiness.
18. It is never necessary to be on guard against ideas no matter where they may originate.
19. Truth is so elusive that no one can say when he has it.
20. I think none the worse of a person for being concerned chiefly with pleasure.
21. Man on his own is a helpless and miserable creature.
22. Fundamentally, the world we live in is a pretty lonesome place.
23. Most people just don't give a "damn" for others.
24. I'd like it if I could find someone who would tell me how to solve my personal problems.
25. It is only natural for a person to be rather fearful of the future.
26. A person who thinks primarily of his own happiness is beneath contempt.
27. Unfortunately, a good many people with whom I have discussed important social and moral problems don't really understand what's going on.
28. Most people just don't know what's good for them.
29. The main thing in life is for a person to want to do something important.
30. If given the chance I would do something of great benefit to the world.
31. A man who does not believe in some great cause has not really lived.
32. It is only when a person devotes himself to an ideal or cause that life becomes meaningful.
33. Of all the different philosophies which exist in this world there is probably only one which is correct.
34. A person who gets enthusiastic about too many causes is likely to be a pretty "wishy-washy" sort of person.
35. To compromise with our political opponents is dangerous because it usually leads to the betrayal of our own side.
36. When it comes to differences of opinion in religion we must be careful not to compromise with those who believe differently from the way we do.
37. In times like these, a person must be pretty selfish if he considers primarily his own happiness.
38. The worst crime a person could commit is to attack publicly the people who believe in the same thing he does.
39. It is annoying to listen to a speaker or teacher who seems unable to make up his mind about what he really believes.
40. For most questions there is only one right answer once a person is able to get all the facts.


On neither sample, however, did this Mark II scale show a correlation between its halves that could be described as high. It did seem at least theoretically possible that better results could be obtained. For this reason a yet further attempt was made to achieve such results.

STUDY III

In this study a more eclectic approach to item selection was reverted to. In this study, the best (highest correlating) negative items from both previous scales (based on analyses of the data obtained from the community samples) were combined with the 20 positive items of the Mark II scale. Whatever the outcome of this study, it seemed a necessary one to examine the possibility that the somewhat disappointing results obtained with the Mark II scale might have been due to the restriction placed on it of using as negative items only reversals of Rokeach originals.

The sample used to test this third BD scale is again more fully described in Ray (1972b) but briefly it was obtained by sampling blocks in the Sydney metropolitan area and then attempting to interview one person from each house in the block. The final n was 118. With this data, the correlation of each item with the total score on the combined scale was found and the highest correlating negative items selected. A 34 item scale (17 negative and 17 positive items) was found to be the most reliable -- with an alpha of .82. The correlation between the two halves was only .27.

The Mark III scale is then no advance over the Mark II scale. Its construction was important in showing, however, that reversals of Rokeach items are not inferior to negative items of entirely new construction. In fact, they appear, at least with general population samples, to be superior to entirely new items.

The Mark II scale is then the scale most suitable for general use. Even with student samples, it is little inferior to the Mark I scale reported in Ray (1970). The correlation between its two halves (.32) is not ideal but does at least offer something to the research worker who wants to use general population sampling and, while so doing, eliminate the effects of response set. People who intend to do research only with students, however, would be best advised to use the existing Mark I scale.

In evaluating whether the correlation between the halves of the Mark II scale is satisfactory or not, it is extremely relevant to examine the findings reported in Ray (1972a). It is there shown that scales outside the authoritarianism/dogmatism area also show correlations between their halves which can be quite low. Low correlations between the halves of balanced scales are a general problem. Set in the context of the findings reported in Ray (1972a), in fact, the Mark II BD scale shows up as quite satisfactory -- as quite within the normally to be expected range.

The reason for this generally low correlation is not, of course, far to seek. Acquiescent set alone would be sufficient to explain it. Acquiescent set causes different items to be responded to as if they were similar. Items are agreed to regardless of their content. Thus acquiescent set will be causing a positive and a negative item to be responded to as if they were similar while their explicit content will be causing them to be responded to as if they were opposed. The outcome of the two opposing forces is a tendency towards orthogonality -- which is in fact what we usually observe.

Balanced scales then may reduce acquiescence but they do not of course eliminate it. What they do is to ensure that acquiescers are classified along with the indifferent or the genuinely non-polarized. They are not classified or confounded with genuine high scorers on the attribute purportedly being measured.

One could, of course, argue that the present results merely confirm that balanced scales in any field are not really possible. Correlations of .3 to .4 are just "too low" in some absolute sense. In evaluating such a claim, one must be clear about the purpose for which such a correlation is "too low". It is most certainly not "too low" to justify a claim that acquiescence artifact has been eliminated. In fact, any balanced scale, no matter what the correlation between its halves, eliminates acquiescence artifact completely! Any balanced scale ensures that acquiescers do not automatically get high scores regardless of any other property of that scale.

Why then might one require high correlations between the two halves of such a scale? As an indication of validity. We design all our items to measure one thing. If two subsets of our items fail to correlate, how can we have any confidence that either measures what we think it measures? The two halves of the Mark II BD scale do, however, correlate highly significantly, so some demonstration of validity in this sense has been accomplished. Since we can also assume that these results are affected by acquiescence, this validity demonstration must also be regarded as representing only some sort of minimum. Because of the masking effect of acquiescence, not all the validity of the scale can be demonstrated simply by using a single correlation coefficient.

One remaining question is the extent to which the Mark II scale covers the same content area as the Rokeach original. Does its content cover the same aspects that Rokeach originally had in mind (differentiation, time perspective etc.)? Obviously not. Several of Rokeach's items did not survive either in their original or in a reversed form. Nonetheless, the correlation observed between the Mark II scale and Rokeach's 40 item original unbalanced scale was .694 on the second sample mentioned above. Considering that scores on the Rokeach original were confounded with acquiescence, this is unequivocal evidence that there is a strong common component in what the two scales measure.

CONCLUSION

It has been confirmed that the Mark I BD scale is suitable and satisfactory for use with students and shown that the Mark II BD scale is suitable for use with general population samples. If an all-round acquiescence-free scale is required, the Mark II scale may be used.

REFERENCES

ADORNO, T. W., FRENKEL-BRUNSWIK, ELSE, LEVINSON, D. J. & SANFORD,
R. N. The authoritarian personality. N.Y.: Harper, 1950.

CHRISTIE, R., HAVEL, JOAN, & SEIDENBERG, B. Is the 'F' scale irreversible? Journal of Abnormal and Social Psychology, 1956, 56, 141-158.

HAIMAN, F. S. A revised scale for the measurement of open-mindedness. Speech Monographs, 1964, 31, 97-102.

PEABODY, D. Attitude content and agreement set in scales of authoritarianism dogmatism, anti-Semitism and economic conservatism. Journal of Abnormal and Social Psychology, 1961, 63, 1-11.

RAY, J.J. (1970) The development and validation of a balanced Dogmatism scale. Australian Journal of Psychology, 22, 253-260.

RAY, J.J. (1972) Are conservatism scales irreversible? British J. Social & Clinical Psychology 11, 346-352. (a)

RAY, J.J. (1972) A new balanced F scale -- And its relation to social class. Australian Psychologist 7, 155-166. (b)

ROKEACH, M. The open and closed mind. N.Y.: Basic Books, 1960.

STANLEY, G. & MARTIN, J. How sincere is the dogmatist? Psychological Review, 1964, 71, 331-333.


Manuscript received 17 May 1973.

John J. Ray, School of Sociology, University of New South Wales, P.O. Box 1, Kensington, N.S.W., 2033.

POST-PUBLICATION ADDENDUM

Subsequent articles germane to the matters discussed above are:

Ray, J.J. (1979) A short balanced F scale. Journal of Social Psychology, 109, 309-310.

Ray, J.J. (1979) Is the Dogmatism scale irreversible?
South African Journal of Psychology 9, 104-107.

Ray, J.J. (Unpublished) ACQUIESCENT RESPONSE TENDENCY: An update and some data on the invalidity of the Dogmatism scale



SCALE FORMAT: Replication is one of the cornerstones of science. A new research result will normally require replication by later researchers before the truth and accuracy of the observation concerned is generally accepted. If a result is to be replicated, however, careful specification of the original research procedure is important.

In questionnaire research it has been my observation that the results are fairly robust as to questionnaire format. It is the content of the question that matters rather than how the question is presented (But see here). It is nonetheless obviously desirable for an attempted replication to follow the original procedure as closely as possible so I have given here samples of how I presented my questionnaires in most of the research I did. On all occasions, respondents were asked to circle a number to indicate their response.


FINIS

European J. Social Psychology, 1973, 3 (3), 221-132.

DOGMATISM IN RELATION TO SUB-TYPES OF

CONSERVATISM: Some Australian data



JOHN J. RAY

University of New South Wales

Abstract

An examination of the relation between authoritarianism and conservatism is made using newly developed balanced forms of the 'D' and 'F' scales together with scales to measure political, social, moral and economic conservatism. Neither 'BD' nor 'BF scales predicted voting preference. The 'BD' scale was significantly positively related to the political, social and moral conservatism scales but was non-significantly negatively related to economic conservatism. It was concluded that both the 'BD' and 'BF scales are equally good measures of general authoritarianism among supporters of Australian political parties and that while it is in general true that dogmatic people tend to be ideologically conservative, an exception must be made for economic conservatism. This exception is seen to be inferable from the theory of 'working class authoritarianism' advanced by Lipset (1960).

Costin (1971) has recently raised anew the controversy over the political polarization of the Rokeach Dogmatism scale (see Simons, 1968; Kirtley and Harkness, 1969). Although the measure of conservatism he used was an interesting one, there are three major problems that limit the acceptability of his findings. These are: 1) His use of student subjects; 2) his assumption that conservatism is a unitary variable; 3) his use of the Dogmatism scale in its original, unbalanced form. It is well-known that results obtained with students cannot often be generalized to the population at large, and the relation between conservatism in different fields is known to be quite variable (Kerr, 1955; Anderson and Western, 1967). Lipset, in particular, has proposed a fundamental distinction between conservatism in economic and non-economic fields (Lipset, 1960). In his test of Lipset's theory, O'Kane (1970) found a correlation of only .29 between these two forms of conservatism. The use of one-way-worded scales also leaves open the question whether it might not be the characteristics of the acquiescent person that are being measured rather than the characteristics of the true Dogmatist (Peabody, 1961, 1966). The existence of the Ray (1970) balanced Dogmatism scale makes unnecessary the continued reliance on the unbalanced form.

Method

The present study, then, represents an attempt to provide more defensible data on the questions that Costin (1971) raises. General population sampling will be used; separate scales to measure economic, social, moral and political conservatism will be used; and Dogmatism will be measured by a further development of Ray's (1970) balanced Dogmatism scale. The items of the four conservatism scales have already been given in Ray (1971).

Sample

The sample used in this study was one especially designed to enable a test of Lipset's (1960) theory that the working class is more conservative than the middle and upper classes. The study of Dogmatism reported here is essentially a by-product of that larger study. The sample was obtained therefore by selecting city blocks in the Sydney metropolitan area for extreme position on the Congalton (1969) 'prestige ranking of Sydney suburbs'. Extreme low prestige and extreme high prestige suburbs were selected, and blocks from these randomly selected from the map with a pin and blindfold. An attempt was then made to interview at least one person from every household in the block. The sample so obtained numbered 118. Since this sample is being used to find out whether there is in general a relationship -- not to provide precise estimates of parameters -- the fact that it is not a perfectly random one is little disadvantage. It is in any event much more heterogeneous than previous samples.

Scales

The items of the modified balanced Dogmatism scale are given in the Appendix. The modifications were based on a pretest of a large body of reversed 'D' scale items correlated against scores on the original. This pretest was carried out on a heterogeneous sample including people contacted door-to-door. The original BD (Balanced Dogmatism -- Ray, 1970) scale was normed on students only, and it was desired to produce a second form suitable for general population use. The BD scale as used in the present research, then, was composed of items drawn both from the pretest mentioned above and from the original 1970 study. An attempt was made to combine the strongest items from the pretest with the strongest items from the 1970 study.

Also included in the questionnaire was the balanced 'F' scale or 'BF' scale. For details of this scale see Ray (1972b). On its norming sample (identical with the 'pretest' sample mentioned above) this scale had shown a reliability of .87 and a correlation between its positive and negative halves of -.71. All positive items were original F scale items and all negative items were reversals of original F scale items. It is thus unlike the 'balanced F scale' (sic) produced by Lee and Warr (1969). The Lee and Warr scale contains only five items traceable to the F scale original.

Results

The correlations resulting from the present study are set out in Table 1. It will be evident that the correlates of the BD and BF scales are in general quite similar - though the BF scale shows notably higher correlations with political and social conservatism. The results also provide ample warrant for the breakdown of conservatism by content area. Dogmatism is highly positively related to social conservatism but negatively (though insignificantly) related to economic conservatism. Conservatism on issues of sexual morality seems to be the form of conservatism that has the highest relation to Dogmatism. Forming something of a contrast with the scales of conservatism are the results obtained for political party preference. In accordance with the Australian party system, this was scored: D.L.P. - 5; L.C.P. - 4; No preference - 3; A.L.P. - 2; Communist -1. This scoring system has been used in several previous Australian studies (e.g. Ray, 1970; 1971, 1972) though, strictly speaking, only ordinal properties can be claimed for it. It represents a continuum from Right to Left across the Australian political spectrum. The correlations so obtained show that neither Dogmatism nor Authoritarianism were related to conservatism of political party choice.


Table 1. Correlations between the BD and BF scales and selected conservatism measures. n = 118. An r above .180 is significant at the .05 level, two-tailed. 'Sex' scored: Male = 1; Female =2.

......................................................2.........3........4.........5...........6........7............8........9
...................................................Sex.....Vote.....Polit...Soc......Ec.....Mor.......BD.....BF

1. Age........................................-.018... -.086... .074... .183... .157.. .352.. .123.. .218
2. Sex.................................................... .057... .204... .068... .206.. .005..-.021.. .031
3. Voting preference........................................ .312... .194... .393... .121.. .072.. .097
4. Political conservatism scale................................... .528.. .400... .311.. .204.. .519
5. Social conservatism scale................................................ .242... .422.. .406.. .717
6. Economic conservatism scale..................................................... .100..-.126.. .102
7. Moral conservatism scale......................................................................... .572.. .580
8. Balanced Dogmatism......................................................................................... .617


Discussion

It has been shown here that in one sense both the BF and BD scales measure 'general authoritarianism'. Both sorts of authoritarianism are equally likely to be found in either of the major political parties. Considering Rokeach's intentions, this result is not particularly surprising for the Dogmatism scale, but it might perhaps require some comment for the 'F' scale.

For a start, both scales showed satisfactory internal consistency. The correlations between the two halves of the BF scale and the BD scale were respectively -.560 and -.273. Coefficient alpha reliabilities were respectively .86 and .82. The correlation between the two halves of the BD scale is perhaps a little low but it fulfils what Ray (1972a) has argued are satisfactory criteria for any balanced scale - i.e., it is both negative in direction and significant. The present results cannot therefore be said to be due to unsatisfactory scales. In fact, the correlations were computed for the two halves of the BF scale treated as separate scales, and these showed that the 14 original F items had an even lower relation to political preference (r = .033) than did the balanced scale. We must conclude, therefore, that we have a case here where the often weak relation to the F scale of political preference has dropped to the point of insignificance. The correlation observed here does however conform to Brown's (1965) generalization that the correlation between the F scale and political preference is always 'at least positive'. The BF negative items with scoring reversed correlated .152 with political choice. This is still insignificant but it does seem to indicate greater discriminating power for the negative items. As reversals they are perhaps less cliche and thus require greater thought from the respondents. This effect is even more marked when we examine the two halves of the BD scale separately. The positive items correlated -.125 with political choice but the negative items (again reverse scored) correlated .291! This latter correlation is significant and does perhaps indicate that, when the items are really attended to, Dogmatism is related to political choice as well as ideological conservatism. We thus have the ironical circumstance of the F scale being a better measure of 'general authoritarianism' than the 'D' scale! One suggested explanation for this that may be worthy of consideration is that the items of the F scale are probably by now severely dated and are simply less socially relevant now than they were in the 1940s. The conservative voter of the 1970s probably simply sees these items as 'old-fashioned'. Age did correlate significantly positively with BF score (r = .218). It should also be remembered that party choice is determined by many things other than general ideology. Traditional loyalties, class-identification and self-interest may all play a part. Indeed Lipset (1960) suggests that class self-interest may generally cause the working-class subject to vote contrary to what his ideology would dictate. The separation out of economic conservatism suggested by Lipset's theory certainly did seem to pay off in that this form of conservatism alone showed a negative relation to Dogmatism.

It seems, then, that the only conclusion we can possibly draw is that the Dogmatist is conservative in some things but not in all.

APPENDIX

The scales used in this study. Responses are from '5' (strongly agree) to '1' (strongly disagree) for each item. Items marked 'R' are scored '1' to '5'.

Political conservatism

1. The danger of communist infiltration into the union movement is great and the government should take whatever steps are necessary to ensure that it does not become widespread.
2. An occupation by a foreign power is better than war. R.
3. Because many of the minor political parties merely confuse national issues, all political parties except the two major ones should be abolished.
4. Australia will not win respect in Asia by building up armed forces. R.
5. In taking part in any form of world organisation, this country should make certain that none of its independence and power is lost.
6. In disarmament negotiations the West should take the initiative by making concessions since such a procedure could produce concessions from the Soviet block. R.
7. Australia should seek more ties with Asia. R.
8. Patriotism and loyalty to one's country are more important than one's intellectual convictions and should have precedence over them.
9. A standing army of 100,000 men or over is necessary for our national defense at all times.
10. 'My country right or wrong' is a saying which expresses a fundamentally dangerous attitude. R.
11. In some ways dictatorships are desirable.
12. Australia should withdraw its troops from Vietnam immediately. R.
13. Conscription should be abolished. R.
14. International communism is the main danger to Australia today.


Economic conservatism

1. A free dental service should be provided by the government. R.
2. Private enterprise is always the most efficient system.
3. Capitalism is immoral because it exploits the worker by failing to give him full value for his productive labour. R.
4. The nationalization of the great industries is likely to lead to inefficiency, bureaucracy and stagnation.
5. Ultimately, private property should be abolished and complete socialism introduced. R.
6. Production and trade should be free from government interference.
7. The principle of free competition is a natural law which should govern our business system without governmental interference.
8. The growth of large corporations makes government regulation of business necessary. R.
9. The government should take over all industries. R.
10. Trade unions should have much more voice in deciding government policies. R. 11. For men to do their best, there must be the possibility of unlimited profit.
12. Poverty is chiefly a result of injustice in the distribution of wealth. R.
13. The government should not attempt to limit profits.
14. Large incomes should be taxed more than they are now. R.
15. Men would not do their best if government owned all industry.
16. On the whole, our economic system is just and wise.
17. When a rich man dies, most of his property should go to the state. R.
18. The incomes of most people are a fair measure of their contribution to human welfare.
19. A man should strike in order to secure greater returns to labour. R.
20. A man should be allowed to keep as large an income as he can get.
21. Money should be taken from the rich and given to the poor during hard times. R.

Moral conservatism

1. Most decent men have a right to expect that they will marry a virgin.
2. I think girls should remain virgins until they marry.
3. Men and women have the right to find out whether they are sexually suited before marriage (e.g. by trial marriage). R.
4. Pre-marital pregnancy is nothing to be ashamed of or to hide providing the couple is in love and later gets married. R.
5. It is best not to try to prohibit erotic and obscene literature and pictures by law but rather to leave people free to follow their judgments and tastes in such matters. R.
6. Marriage is a sacred covenant which should be broken only under the most drastic circumstances.
7. Sex relations except in marriage are always wrong.
8. Petting and necking between single people is unwise.
9. The pill should be made freely available to schoolchildren and students, even though they do not expect to marry soon. R.
10. Abortion should be legalized. R.

Social conservatism

1. The Japanese are a very productive people and should be allowed to settle in Australia. R.
2. Most migrants from Southern Europe become good citizens. R.
3. Nearly all Jews are money hungry.
4. The English-speaking countries have reached a higher state of civilization that any other country in the world and as a consequence have a culture which is superior to any other.
5. Over 10 per cent of the population is incapable of democratic participation in government by reason of their lack of inherited abilities.
6. Crime could be greatly reduced if we restricted migration to British people only.
7. Our treatment of criminals is too harsh; we should try to cure them, not punish them. R.
8. All men are equal. R.
9. Treason and murder should be punishable by death.
10. The death penalty for crime is barbaric and should be abolished. R.
11. Certain religious sects whose beliefs do not permit them to salute the flag should either be forced to conform or else be abolished.
12. There will always be superior and inferior races in the world, and in the interests of all concerned it is best that the superior continue to dominate the inferior.
13. In the national interest, private schools should either be abolished or restricted in their teachings so that the control of education is largely in the hands of the federal government.
14. We should have complete freedom of speech even for those who criticize the law. R.
15. Allowing educated Asians to immigrate benefits Australian society. R.
16. The white Australia policy is a good policy because it helps to keep Australia white.
17. We must be careful not to let too many Asians into the country or they'll take over the place.
18. People should be allowed to hold demonstrations in the streets without police interference. R.

The revised BD (Balanced Dogmatism) scale

1. It is a sign of maturity to be tolerant of widely different of even conflicting viewpoints. (Reverse)
2. To know the truth about what is going on we cannot rely simply on experts or leaders. (Reverse)
3. People who do something just because their religion commands it are blind. (Reverse)
4. The churches don't give enough weight to individual conscience. (Reverse)
5. It is possible that there are many facets to the 'truth'. (Reverse)
6. Many problems have more than one acceptable solution. (Reverse)
7. It's unfortunate that we have censorship in this country. (Reverse)
8. It doesn't matter much what religion a person follows. (Reverse)
9. A person who thinks primarily of his own happiness is beneath contempt. (Reverse)
10. Most people just don't know what's good for them.
11. The main thing in life is for a person to want to do something important.
12. If given the chance I would do something of great benefit to the world.
13. A man who does not believe in some great cause has not really lived.
14. It is only when a person devotes himself to an ideal or cause that life becomes meaningful.
15. Of all the different philosophies which exist in this world there is probably only one which is correct.
16. A person who gets enthusiastic about too many causes is likely to be a pretty 'wishy-washy' sort of person.
17. To compromise with our political opponents is dangerous because it usually leads to the betrayal of our own side.
18. When it comes to differences of opinion in religion we must be careful not to compromise with those who believe differently from the way we do.
19. In times like these, a person must be pretty selfish if he considers primarily his own happiness.
20. The worst crime a person could commit is to attack publicly the people who believe in the same thing he does.
21. It's possible to really live without believing in any great cause. (Reverse)
22. Life can be meaningful without devotion to ideals or causes. (Reverse)
23. All of the philosophies which exist in this world have some truth in them and probably not one is totally correct. (Reverse)
24. Man on his own is a helpless and miserable creature.
25. Most people just don't give a 'damn' for others.
26. I'd like it if I could find someone who would tell me how to solve my personal problems.
27. It is only natural for a person to be rather fearful of the future.
28. If we are going to have free speech we must defend the right to be heard of even those we disagree with. (Reverse)
29. If people in one's own group are always disagreeing among themselves that is probably a rather healthy sign. (Reverse)
30. There is no such thing as 'the Truth'. (Reverse)
31. The 'one true faith' is a myth. (Reverse)
32. The way to happiness is to get involved in the things going on about you. (Reverse)
33. There is never one right answer for any question. (Reverse)
34. For most questions there is only one right answer once a person is able to get all the facts.

The BF scale

1. The rebellious ideas of young people are often a constructive source of change for the better. R.
2. Homosexuality between consenting adults may be distasteful but it should not be regarded as a crime. R.
3. Many of the radical ideas of today will become the accepted beliefs and practices of tomorrow. R.
4. Familiarity does not breed contempt. R,
5. People who want to whip or imprison sex criminals are themselves sick. R.
6. What a youth needs most is to be free to make up his own mind, to be flexible and to work and fight for what he considers right personally, even though it might not be best for his family and country. R.
7. Many good people honestly could never bring themselves round to feeling much love, gratitude or respect for their parents. R.
8. You need to get out and rub elbows with all kinds of people to get new ideas and broaden your understanding of life. R.
9. There are times when it is necessary to probe into even the most personal and private matters. R.
10. It's all right for people to raise questions about even the most personal and private matters. R.
11. Insults to our honour are not always important enough to bother about. R.
12. Sex crimes such as rape and attacks on children are signs of mental illness; such people belong in hospitals rather than in prison. R.
13. Most honest people admit to themselves that they have sometimes hated their parents. R.
14. Disobedience to the government is sometimes justified. R.
15. If people would talk less and work more, everybody would be better off.
16. The business man and the manufacturer are much more important to society than the artist and the professor.
17. Every person should have complete faith in some supernatural power whose decisions he obeys without question.
18. No sane, normal, decent person would ever think of hurting a close friend or relative.
19. Young people sometimes get rebellious ideas, but as they grow up they ought to get over them and settle down.
20. An insult to our honour should always be punished.
21. Sex crimes, such as rape and attacks on children, deserve more than mere imprisonment; such criminals ought to be publicly whipped or worse.
22. Obedience and respect for authority are the most important virtues children should learn.
23. Most of our social problems would be solved if we could somehow get rid of the immoral, crooked and feeble-minded people.
24. People can divide into two distinct classes: The weak and the strong.
25. What the youth needs most is strict discipline, rugged determination and the will to work and fight for family and country.
26. Familiarity breeds contempt.
27. There is hardly anything lower than a person who does not feel great love, gratitude and respect for his parents.
28. Homosexuals are hardly better than criminals and ought to be severely punished.

REFERENCES

Anderson, D. S., and Western, J. S. (1967) An inventory to measure students' attitudes. U. Queensland Papers, 1, 175206.

Brown, R. (1965) Social psychology. N.Y., Free Press, 1965.

Congalton, A. A. (1969) Prestige and status in Australia. Melbourne, Cheshire.

Costin, F. (1971) Dogmatism and conservatism: An empirical follow-up of Rokeach's findings. Educ. psych. Meas., 31, 1007-1010.

Kerr, W. A. (1955) Tulane factors of liberalism-conservatism (manual). Chicago, Psychometric affiliates.

Kirtley, D., and Harkness, R. (1969) Some personality and attitude correlates of Dogmatism. Psych. Rep., 24, 851-854.

Lee, R. E., and Warr, P. B. (1969) The development and standardization of a balanced 'F' scale. J. gen. Psychol., 81, 109129.

Lipset, S. M. (1960) Political Man. N.Y., Doubleday.

O'Kane, J. M. (1970) Economic and non-economic liberalism, upward mobility potential, and Catholic working class youth. Social Forces, 48, 499-506.

Peabody, D. (1961) Attitude content and agreement set in scales of authoritarianism, Dogmatism, anti-Semitism and economic conservatism. J. Abn. soc. Psychol., 63, 1-11.

Peabody, D. (1966) Authoritarianism scales and response bias. Psychol. Bull., 65, 11-23.

Ray, J.J. (1970) The development and validation of a balanced Dogmatism scale. Australian Journal of Psychology, 22, 253-260.

Ray, J.J. (1971) An "Attitude to Authority" scale. Australian Psychologist, 6, 31-50.

Ray, J.J.(1972) Acceptance of aggression and Australian voting preference. Australian Quarterly 44, 64-70.

Ray, J.J. (1972a) Are conservatism scales irreversible? British J. Social & Clinical Psychology 11, 346-352.

Ray, J.J. (1972b) A new balanced F scale -- And its relation to social class. Australian Psychologist 7, 155-166.

Simons, H. W. (1968) Dogmatism scales and Leftist bias. Speech Monographs, -35, 149-153.



POST-PUBLICATION ADDENDUM

Replication is one of the cornerstones of science. A new research result will normally require replication by later researchers before the truth and accuracy of the observation concerned is generally accepted. If a result is to be replicated, however, careful specification of the original research procedure is important.

In questionnaire research it has been my observation that the results are fairly robust as to questionnaire format. It is the content of the question that matters rather than how the question is presented (But see here and here). It is nonetheless obviously desirable for an attempted replication to follow the original procedure as closely as possible so I have given here samples of how I presented my questionnaires in most of the research I did. On all occasions, respondents were asked to circle a number to indicate their response.

FINIS

Journal of Clinical Psychology, July, 1984, Vol, 40, No. 4

ALIENATION, DOGMATISM AND ACQUIESCENCE




JOHN J. RAY

University of New South Wales, Australia

Abstract

On a random sample of 118 Australians, dogmatism and alienation were found not to correlate when acquiescence was controlled for. Scales of cognitive complexity and alienation may correlate only because of a shared radicalism artifact.

Sexton (1983) finds a relationship between dogmatism and alienation, but neglects the problem of acquiescence with the one-way-worded Dogmatism scale (Ray, 1979). Her results simply may show that alienated people are prone to agree with vague statements.

This can be tested because there was included in the battery administered to a random sample of 118 Australians described by Ray (1974, Study III) not only a balanced Dogmatism scale, but also a balanced Alienation scale (Ray, 1982).

The positively and negatively worded halves of the Dogmatism scale correlated -.27. Because the anti-Dogmatic items were written and pretested to be as opposite as possible to the original items, this low correlation between them strongly suggests a powerful tendency toward meaningless acquiescence in the data.

RESULTS

The reliabilities (alpha) of the two balanced scales were .75 (Alienation) and .82 (Dogmatism). Using only the positively-worded halves of both scales, dogmatism and alienation were found to correlate.461 (p <.01). Using the full balanced forms of the two scales, however, the correlation was only .104 (ns). The relationship described by Sexton is then replicable, but vanishes when controls for acquiescence are applied. All correlations with the one-way-worded Dogmatism scale are therefore of doubtful meaning.

In also correlating cognitive complexity scores from the OPI with alienation scores, Sexton ignores the possibility of another artifact -- the heavy contamination of both scales by liberalism /conservatism. Despite their different titles, from their item content both scales could be seen largely as alternative scales of radicalism. For instance, "Politically, I am something of a radical" is an item that supposedly measures complexity. Sexton's interpretation of her results is then very overblown.

REFERENCES


RAY, J.J. (1974) Balanced Dogmatism scales. Australian Journal of Psychology 26, 9-14.

RAY, J.J. (1979) Is the acquiescent response style not so mythical after all? Some results from a successful balanced F scale.
Journal of Personality Assessment 43, 638-643.

RAY, J.J. (1982) Towards a definitive alienation scale. J. Psychology, 112, 67-70.

SEXTON, M. E. (1983). Alienation, dogmatism and related personality characteristics. Journal of Clinical Psychology, 39, 80-86.


FINIS