Church Attendance, Problems of Measurement, and Interpreting Indicators: A Study of Religious Practice in the United States,

Transcription

1 JOURNAL for the SCIENTIFIC STUDY of RELIGION Church Attendance, Problems of Measurement, and Interpreting Indicators: A Study of Religious Practice in the United States, MAURIZIO ROSSI Department of Education Studies University of Bologna ETTORE SCAPPINI Department of Education Studies University of Bologna Church attendance is usually measured in surveys by asking a direct question about frequency of churchgoing over a preset period of time, which is typically a year. Different studies have cast doubt over the validity of this indicator as it tends to overestimate actual attendance to a significant degree. The aim of this article is to compare data on church attendance provided by two different types of research conducted in the United States between 1975 and 2010: survey data (GSS) and data obtained from time use surveys (ATUS). This comparison has three main objectives: (1) to confirm the hypothesis that survey data tend to overestimate actual attendance; (2) to show that this overestimation is not constant over time and space, but tends to vary in an erratic and unpredictable way; and (3) to demonstrate that data provided by time use surveys are more reliable than the frequencies of churchgoing provided by traditional surveys when the objective is to identify trends in religiosity in a population. Keywords: presence at church, church attendance, measured density, calculated density, bias in self-reported surveys, time use surveys. INTRODUCTION Church attendance is the most important and widely used measure to estimate the level of religious practice in a population. A central aspect of the recent debate on advancing knowledge about religiosity focuses on the different problems of measurement using questionnaires and daily diaries. The aim of this study is to clarify the meaning of the different indicators of church attendance that can be obtained from the two survey tools and assess their validity by using the United States as a specific case study. As we shall see, church attendance can take on two different empirical forms: frequency of churchgoing over a given period of time, which is more common and is usually measured using questionnaires, or presence at church on one or more Sundays, which is used more rarely and is mainly measured through the completion of daily diaries. 1 Although the validity of frequency indicators has been under discussion for some time, there is still no unanimous consensuson the matter; some researchers advocate using them (Caplow 1998; Hout and Fischer 2002; Hout and Greeley 1998), while others highlight their dubious level Note: This article is an equal collaboration between these authors. Names are listed alphabetically. Acknowledgments: The authors would like to thank the three reviewers for their helpful comments, which made an important contribution to the aim of making the issues discussed in the article clearer. Correspondence should be addressed to Ettore Scappini, Department of Education Studies, University of Bologna, Via Filippo Re 6, Bologna, Italy. ettore.scappini@unibo.it 1 In the following pages we will consider the dichotomy between questionnaires and diaries to be comparable to the dichotomy between measures of frequency and measures of presence. Strictly speaking the two do not coincide: sometimes questionnaires adopt questions about presence at church on a particular Sunday. However, this practice is far less widespread than questions about frequency. Journal for the Scientific Study of Religion (2014) 53(2): C 2014 The Society for the Scientific Study of Religion

2 250 JOURNAL FOR THE SCIENTIFIC STUDY OF RELIGION of reliability due to significant overestimation (Chaves and Cavendish 1994; Hadaway, Marler, and Chaves 1993, 1998; Marcum 1999; Marler and Hadaway 1999; Presser and Stinson 1998; and more recently, Hadaway and Marler 2005; Presser and Chaves 2007; Rossi and Scappini 2010, 2012). Despite the doubts that have arisen, frequency is still the most commonly used indicator. This is also because of the wide availability of surveys featuring this type of question. The uncertainty surrounding this form of measurement and variations between different surveys and research institutes have led to contrasting opinions about what has happened in the recent past in the United States. For example, whereas some claim that the level of religious participation has largely remained constant or decreased slightly (Caplow 1998; Fischer and Hout 2006; Hout and Fischer 2002; Hout and Greeley 1987; Stark 1999), others maintain that it has dropped considerably and therefore that American society is undergoing a process of secularisation (Bruce 2011; Chaves 2011; Chaves and Cavendish 1994; Hadaway and Marler 2005; Hadaway, Marler, and Chaves 1993; Marcum 1999; Marler and Hadaway 1999; Murray 2012; Presser and Stinson 1998). By contrast, there are still relatively few empirical studies that measure religious participation through presence at church, even though, as we shall see, this class of indicator is less vulnerable to certain typical forms of bias than frequency indicators. Our study will take shape over three stages. In the first of these, we will attempt to clarify the meaning of presence at church, which we will call density. While converting data obtained from frequency indicators into density measures is a simple process and a familiar feature of literature dealing with religious practice, it is less well known that these two indicators actually have extremely different meanings. In the second stage we will identify potential sources of error when using both frequency indicators and daily diaries. We will show how these different sources of error characterize questionnaires and diaries in different ways, and will demonstrate that frequency indicators are more prone to bias. Finally, in the third part we will use synchronic and diachronic comparisons to show the high degree of error associated with presence at church calculated using frequency indicators and, above all, the strong lack of uniformity in this error over space and time. In other words, we will highlight that data obtained from a questionnaire can easily mislead researchers in their understanding of the spread of church attendance and the progress of the trend over time. APPROACHES TO MEASURING CHURCH ATTENDANCE: FREQUENCY VERSUS DENSITY As we have seen, the range of indicators used to measure church attendance can be classified into two main groups, depending on whether they are based on data collected from a questionnaire or a daily diary. A questionnaire can be used to determine the frequency of churchgoing (generally approximate) of each individual on Sunday over a given period of time, which is usually a year. 2 Ideally, if n is the number of Sundays in the period in question generally n = 52 the n + 1 values f x can be calculated, each of which shows the number of people that attend church x times, with x ranging from 0 to 52. Each f x /N ratio, in which N is the size of the population, provides the attendance rate for every single value or group of values of x. This frequency may also assume 2 In general, to respect the precept one must attend church on Sunday. However, particularly for Catholic congregations it is often possible to go to Mass on Saturday evening, typically between 5 and 6 p.m. For this reason the period of reference, which for the sake of brevity here and hereafter we will call Sunday, runs from 2 p.m. on Saturday afternoon to Sunday evening. Furthermore, with regard to diaries we will always only consider presence at church on Sunday for each individual, ignoring possible multiple frequencies on the same day.

3 CHURCH ATTENDANCE IN THE UNITED STATES 251 the form of a cumulative rate, thereby indicating the proportion of people who attend church at least X times: F(X) = 52 f x/n, x X x=x Table 1 shows an example of the latter use: the cumulative frequency of attendance for interviewees who went to church on Sunday at least once a month between 1975 and The daily diary is a completely different method of acquiring data on individual behavior and is mainly adopted in surveys designed to study daily time use. Here follows a simplified description referring exclusively to religious practice. After identifying the total number of subjects who will keep a diary (N), we can create a series of subsamples, each of which consists of N/52 individuals. There are 52 subsamples, one for each Sunday in a year. The subjects in the first subsample will be asked to complete a diary about all activities carried out on the first Sunday of the year, while members of the second subsample will complete the diary about the second Sunday of the year and so on. We will extract information from each diary about presence or absence at the relevant Sunday church service. In this way we will obtain N pieces of data about presence/absence covering all 52 Sundays in the year. A simple tool for arranging these data is a matrix consisting of N/52 rows and 52 columns. It is established that the box in the first row and first column will be used to record the presence/absence (x = 1,0) of the first interviewee on the first Sunday of the year, the box in the second column on the same row will be used to record the same interviewee s presence/absence on the second Sunday of the year and so on. We will now define the ratio P m (X) = N/52 i=1 52 j=1 x i, j/n showing the relationship between positive events and possible events (the suffix m shows that the index P(X)ismeasured directly). We can consider the index P m (X) to be a measure of the density of the participation events, meaning precisely the degree to which the matrix, or space of events, is filled by positive values. It follows that measured density is only a ratio between positive events and possible events; it does not refer in any way to individuals and their characteristic attendance rate (Rossi 2008; Rossi and Scappini 2012). We will now develop this point in greater depth by considering the following hypothetical example. Imagine that we take 52 samples of size n (1,000) from a given population. Each sample is assigned one of the 52 Sundays in the year and keeps a diary for that day. Let us suppose that 300 individuals from each sample were present at church, while 700 were not. Consequently, over the course of the year in question there will be presences, namely, 15,600 out of a potential 52,000. The density of the participation events will therefore be 30 percent. 3 Can we deduce from this result that 30 percent of the population attends church and 70 percent does not? The answer is no, as a density of 30 percent can result from two contrasting situations or intermediate combinations of them. In the first case, 30 percent attend church every Sunday, while the remaining part of the population never go (we will call this the hypothesis). In the second case, everybody attends church with equal intensity and is present on precisely 15.6 Sundays a year. In other words, each of the 1,000 interviewees for each Sunday attends church on some Sundays during the year. Therefore, the probability of finding a subject at church on the Sunday when the diary is administered is only 15.6/52 or.30. The result obtained is that 300 interviewees will say that they attended church on every Sunday when the diary was 3 Using slightly different terminology, we can also say that the presence rate on every average Sunday is 30 percent, the same as the average presence rate over the whole year. However, we prefer using the term density as it excludes any undue reference to the level of religiosity of the individual members of a population in an unambiguous way.

4 252 JOURNAL FOR THE SCIENTIFIC STUDY OF RELIGION Table 1: Frequency of church attendance: calculated and measured densities by year (percentage) Frequency of Church Attendance (GSS) Density More than Every Nearly 2 3 times Once Several Once Less than Almost once GSS ATUS Ic once a week week every week a month a month times a year a year once a year Never Total a month (N) (a) (b) (a/b) (1,483) (1,487) (1,521) (1,452) (1,835) (1,457) (1,459) (1,474) (1,333) (1,490) (1,563) (2,937) (2,818) (2,784) (2,730) (2,733) (2,793) (4,475) (2,005) (2,033) Sources: GSS Surveys ; AHTUS 1975 and 1993; ATUS 2004, 2006, 2008, and 2010.

5 CHURCH ATTENDANCE IN THE UNITED STATES 253 administered, while the remaining 700 will state that they did not, simply because they attended on other Sundays when the diary was not completed (we will call this the hypothesis). Let us imagine that at time t + n the same population starts to attend church less, with a uniform reduction of, say, 7.5 percent in the annual number of religious services attended by each individual. In the hypothesis, for 30 percent of churchgoers this decline translates into a frequency of 48 services a year, compared to 52 at time t. On each Sunday diaries will only show /52 = 277 individuals present and an annual total of 14,400 with a density of 27.7 percent a drop of 7.5 percent compared to the previous density at time t (30 percent). In the hypothesis, the decline translates into 14.4 services a year for the whole population, compared to 15.6 at time t. On each Sunday, we will record 1, /52 = 277 individuals present. Here too, the annual total will be 14,400 and the density will drop from 30 percent to 27.7 percent (again 7.5 percent). What information would we obtain from the same population at the two different times if we used a measure of frequency? In the hypothesis, at time t 30 percent of churchgoers would clearly give the answer every week. However, time t + n is more problematic: Would they still say every week or would they opt for the technically more precise answer nearly every week? Given that there is only a slight drop (of four fewer Sundays a year), it is more realistic that the vast majority of the 30 percent in question would give the same answer as they did at time t, namely, every week. In the hypothesis, each member of the population has to decide between two options: 2 3 times a month or once a month. In terms of their literal numerical content, the two options correspond to Sundays a year and 12 Sundays a year. At time t each individual attends 15.6 services and is therefore faced with an uncertain choice, given that his or her behavior is covered by the two options but does not fall into either of them. However, we can be sure that whichever option is chosen at time t will be confirmed at time t + n, as one less Sunday a year (from 15.6 to 14.4) does certainly not justify changing from one option to the other. As a general summary we can state that: (1) Density is a poor indicator. A density value does not provide any indication of the distribution of the different levels of attendance in a population at most it establishes a minimum share of the population (30 percent in the example, the same as the density value) when there is maximum attendance among individuals (52 Sundays a year), and a maximum share of 100 percent when the average level of frequency is at a minimum (15.6 Sundays a year). 4 (2) Density provides a precise measure of changes in overall attendance from one period to another if attendance drops by a given percentage value (in terms of average number of Sundays per year), density falls by the same fraction. (3) On the other hand, measures of frequency used in questionnaires, which are forced to adopt a few aggregated answer options, lead to ambiguity and imprecision in determining attendance levels. The example discussed above also shows that they can be completely insensitive to significant variations in the average attendance rate of a population. We will now return to the American context. As Table 1 shows, the density of attendance measured in the United States in 2010 is 23.4 percent, a statistic that is compatible with a variety of situations. For example, it may mean that every American goes to church on average 12.2 ( ) Sundays out of 52 in the year. Alternatively, it could mean that the number of churchgoers is only 23.4 percent, but that these subjects go to church regularly every week. As we have said, 4 The maximum amount is only lower than 100 percent when density is lower than approximately 2 percent. In this case it would be difficult to speak about the degree of religiosity of the population.

6 254 JOURNAL FOR THE SCIENTIFIC STUDY OF RELIGION the index P m (X) only makes sense as a measure of participation in events during the year; it does not directly relate to Americans as individuals, or tell us how many of them are churchgoers and how often they go. We only know that the proportion of churchgoers fluctuates between a minimum of 23.4 percent if they attend diligently (on 52 Sundays out of 52) and a maximum of 100 percent if everybody attends 12.2 Sundays a year. CALCULATED DENSITY VERSUS MEASURED DENSITY Our aim is to make a comparison between surveys based on questionnaires (GSS) and diaries (ATUS). 5 We have already shown that the two types of data are fundamentally different and that density measured from daily diaries does not allow us to make any form of deduction about the attendance frequencies of a population. The example discussed above provides concrete evidence that a density of 23.4 percent is compatible with a wide variety of frequency structures. While it is not possible to convert from density to frequency, in principle the step can be carried out in reverse by using the distribution of frequencies to establish the density of presence. Ideally, if we have detailed knowledge of the distribution of the population over the different possible annual attendance frequencies (from 0 to 52), we will be able to reconstruct the underlying space of events and then determine its density. To do this we just need to add together the number of people and the relative value of attendance frequency x to identify the positive events and then divide the result by the number of possible events, or in formal terms P c (X) = 52 x=0 f x x/ (N 52) 6 Here we can talk about calculated density, P c (X), to distinguish it from measured density obtained through the use of diaries, P m (X). 7 It is clear that if we had exact knowledge of the structure of attendance frequencies by the term exact we mean with maximum detail and without any possible bias as a result of data collection the calculated and measured densities would coincide, or at least tend to converge within the limits of sampling errors. It would be unrealistic, however, to ask interviewees such a detailed question about their annual attendance at church on Sunday. As we know, it is preferable to offer a reduced series of answer options that correspond more or less explicitly to frequency intervals. For example, in the case we have analyzed the question used in the GSS studies How often do you attend religious services? offers nine alternatives: (1) more than once a week, (2) every week, (3) nearly every week, (4) 2 3 times a month, (5) once a month, (6) several times a year, (7) once a year, (8) less than once a year, and (9) never. Therefore, to convert frequency to density we also need to address the problem of identifying a characteristic frequency for each category x used in the formulation of the question and considering the latter to be the actual attendance frequency of each respondent in the category. This characteristic frequency is usually approximately situated at the mean point between the extremes of the period given in the answer option. For the question used in the GSS studies, the frequency values we identified for the different answer options with weights applied (x/52) are as follows: for the first two categories (1) and (2) 52 Sundays out of 52 with a weight of 1.00; 5 The time use surveys do not ask the respondents to specify the religion they practice. However, we should underline that the GSS data between 1998 and 2010 shows that the majority of believers in the United States are Judeo-Christian (about 80 percent) and that only a minority practice other religions (2/ 4 percent) (see also Smith 2002). 6 Like the previous formulae, this one naturally refers to proportions. Instead, the data in the tables are given as percentages. 7 In other words, a suitable conversion operation is needed. Frequency data cannot be directly compared to density data (presence), as Brenner basically did in his study of religious practice (2011a, 2012), and the fraction of the population that does not attend church cannot be identified using daily diaries (Brenner 2011b).

7 CHURCH ATTENDANCE IN THE UNITED STATES 255 for option (3) 44/52 with a weight of.846; for category (4) 30 Sundays out of 52 with a weight of.577; for category (5) 12/52 with a weight of.231; for category (6) 6/52 with a weight of.115; for category (7) 1/52 with a weight of.019; for category (8).5/52 with a weight of.010; and for category (9) 0/52 with a weight of.0. Frequency-density conversion procedures similar to ours have only been used relatively recently (Gershuny 2003:267 68; Pisati 2000; Presser and Chaves 2007; Presser and Stinson 1998; Woodberry 1998). It would be useful at this point to recall the three basic types of risk that are implicit in the mechanical application of such procedures, which are also outlined in Rossi and Scappini (2012). First of all, there is no guarantee that when interviewees give their answers, they will adopt the same interpretation used by the researcher in defining the meaning of the categories, the time intervals, and the characteristic frequency to be used. Second, it presupposes that the same weights (x/52) are adopted for the different social segments of a population (young people/adults, families with/without preteen children, etc.). Finally, with diachronic analysis, weights need to be kept unvaried over time as they are obtained from a literal interpretation. In the absence of any empirical evidence, it is therefore assumed that the average attendance of the subpopulation in each category does not vary at all. In conclusion, the conversion procedure may introduce bias that negatively affects the indicator P c (X). Time Use Surveys (AHTUS and ATUS) THE DATA: AHTUS, ATUS, GSS, AND PSDI Three time use surveys were carried out in 1975, 1985, and and are now managed by the American Time Use Survey (AHTUS) (see Fisher et al. 2006). The first of these (1975) was conducted by the University of Michigan Survey Research Center (SRC). The survey raised numerous problems that need to be underlined. The total sample consisted of a panel study of four waves, where some interviewees were interviewed again in successive quarters, even if this happened on different days of the week. However, nonresponse in the panels is not uniform among churchgoers and nonchurchgoers, and unfortunately we do not have any similar information about panel studies that use daily diaries. Nevertheless, it is plausible to believe that the consequence of nonresponse is a slight overestimate in religious practice. Moreover, there is a relatively reduced level of nonresponse: between the first and second waves of the dataset it was approximately 25 percent, while attrition between the second and third waves was about 8 percent and a further 1 percent of respondents were lost between the third and fourth waves. The periods of administration of diaries were: first wave October December 1975; second wave January March 1976; third wave April June 1976; fourth wave July September Finally, we should point out that the response rate in wave one was 72 percent. The second survey, which was carried out by the same university in 1985, was characterized by a limited sample size, even though it provided more information than previously as the church was included as one of the possible places where activities are carried out. In this case the response rate was 55 percent. The third survey (for the sake of brevity indicated as 1993) was conducted by the University of Maryland SRC from September 1992 to September Here, too, the church was included as one of the possible places where activities are carried out. The response rate was 63 percent. Since 2003, the U.S. Census Bureau has conducted the American Time Use Survey (ATUS 2013). As we will see below, these surveys provide information that is more detailed and more complete than the previous ones. The period of administration of diaries that we analyzed is from January 2003 to December 2010 and the response rates were between 50 percent and 60 percent.

8 256 JOURNAL FOR THE SCIENTIFIC STUDY OF RELIGION GSS and PSID Surveys Since 1972, the University of Chicago s National Opinion Research Center has conducted an annual or biennial General Social Survey (Smith, Marsden, and Hout 2011). These surveys are a well-known element often used in studies on religious behaviour in American society. The period of administration of questionnaires that we analyzed is from 1975 to 2010 and the response varied between 70 percent and 80 percent. The Panel Study of Income Dynamics (PSID) is an annual longitudinal survey that has been conducted since 1968 (Morgan and Smith 1969; PSID 1987). At the survey s inception, the sample included a representative cross-section of the United States, as well as a supplementary sample of families who had previously been interviewed once or twice by the Census Bureau. The PSID provides a wide variety of information about the family, as the information gathered from the survey refers to the circumstances of the family unit as a whole (e.g., type of housing) and individual members the family unit (e.g., age, earnings). Some data are collected about all the individuals in a family unit, but the most extensive data are gathered about the head of the family (who is male in married-couple families and of either sex in other cases). It should be stressed that as the question about church attendance was only addressed to the head of the family in 1968, this is the only subject that has all the necessary information for the purposes of this study. The period of administration of questionnaires that we analyzed is from 1968 to 1987 and the response rate in wave one was approximately 75 percent. Finally, the weights for density calculation of open question postcoding are: more than once a week, every week, once a week, every Sunday (52/52); every few weeks, several times a month, once or twice a month, often (26/52); about once a month, sometimes (12/52); once in a while, a few times a year, not often, seldom (2/52); never (0/52). We should underline that all samples are weighted (the PSID data are weighted by individual respondent weight), but in the table the samples (N) indicated are unweighted. The analysis carried out only includes subjects aged 18 or over. Some Additional Considerations About AHTUS and ATUS Data While there are no particular problems of analysis with regard to the surveys on frequency of churchgoing that we will use (GSS), we need to clarify a few points before we analyze the AHTUS and ATUS data. 8 Time use surveys have only been used for a relatively short time; intensive use only started in the United States and other Western countries in the last three decades of the last century. As always, when new survey tools are first applied, they undergo significant revision processes, which creates a number of problems in terms of comparing different surveys. The main two difficulties relate to procedures for codifying activities and the places where they are carried out. This is because diaries contain subjective descriptions of time use at different moments of the day. It is therefore easy to see the central role played by codification and the choice of categories when classifying the activities carried out and the places where each activity is done. With regard to attendance at religious services, the categories and details used in codification have changed significantly over the years. Originally, the sole objective of time use surveys was to study the time spent by individuals divided into broad categories of activities; more precise analysis only developed gradually over time. In the years 1975, 1985, and 1993 a single category was used to group together not only Attending religious services, but also presence at church for other reasons (singing in choir, leading youth group) and even individual religious practice (praying, meditating, Bible study group [not at church]). It was only from 8 Hereafter we shall use this acronym indifferently for AHTUS and ATUS surveys.

9 CHURCH ATTENDANCE IN THE UNITED STATES onwards that attending a religious service was classified independently, separating it by definition from presence for other reasons and individual practice. With regard to the place where religious activities are carried out, no specific category was provided in 1975, while church was used in 1985 and Since 2003, the broader category place of worship has been used. The definition of presence that we have decided to adopt to calculate density is Attending religious services in a place of worship. The definition is a little restricted, as religious services can also be attended in places not specifically designed for worship. However, excluding the restriction to a place of worship would have created significant overestimates for the years from 1975 to 1993, when the adopted definition of attending religious services was too broad. For the year 1975, in which there is no specific code for places of worship, we used the best approximation available, considering only activities carried out in other places. Although this is a broader category than places of worship, it is the only one that includes them. The data corresponding to this definition can be seen in the second part of Table 2. We felt it would be appropriate to use the first part of the same table to show the densities that would have been obtained if the restriction to a place of worship (or other place for 1975) had not been adopted. The definition used here is thus more restricted than it should be. The distance between the densities in the two sections of Table 2 in the years is an approximate measure of the effects of this restriction. On the other hand, we can assume that the measured density for Sunday services (last row of Table 2) is overestimated; since it is obtained by adding together the attendance figures for Saturday afternoon and Sunday, those who attend church on both days are counted twice. We do not know the size of this phenomenon; it can only be estimated by looking at the percentages for average attendance on weekdays. As Table 2 shows (second part), the figures for presence at religious services on weekday afternoons are 2.2 percent in 1975, 1.2 percent in 1993,.9 percent in 2004, and 1 percent in Therefore, our data feature both underestimates and overestimates. We feel it is more than plausible to suppose that these two phenomena cancel each other out as they are opposites that seem to be equivalent in size. 10 CAUSES OF BIAS AND OVERESTIMATION OF RELIGIOUS PRACTICE At the beginning we mentioned claims that the rate of church attendance measured in surveys is overestimated to a large degree. Researchers have now reached a broad consensus on the causes of such bias. In brief, the factors that generate bias and influence the validity of attendance measures are: (1) the self-selection of the sample, (2) the time of year in which the survey is carried out, and (3) the characteristics of the measuring tool. We will conclude this section by drawing some preliminary conclusions about the varying weight that this bias carries for the two types of research analyzed here. 9 Our adopted hypothesis is that double attendances at church on Saturday afternoon and Sunday can be attributed to subjects who go to church several times a week. In this sense attendance on weekdays constitutes an acceptable estimate of this double attendance. In 1975 the structure of the sample makes it possible to identify the subsample of those who attended church on Saturday and Sunday even if they did so in different weeks. In this case the upper density limit can be estimated at 2.4 percent (N = 776). 10 We should point out that although the data we are using are from a daily diary, the rhythm of religious attendance has a typical weekly cycle. We will therefore assume that this is the cycle that best represents the collective dimension of religious practice. Moreover, this is the only possible comparison between questionnaires and daily diaries as monthly or annual densities can only be obtained from the former, while the procedure cannot be carried out with daily diaries and can only be done on an approximate basis with weekly diaries. Therefore, in our case calculated and measured densities are weekly rather than daily densities (Scappini 2010).

10 258 JOURNAL FOR THE SCIENTIFIC STUDY OF RELIGION Table 2: Density of religious attendance by day of week, part of day, place, and year (percentage) Attending religious services anywhere Weekday Until 2 p.m After 2 p.m (N) (658) (956) (1,327) (1,226) (1,180) (1,261) Saturday Until 2 p.m After 2 p.m (N) (1,589) (940) (3,305) (3,061) (3,016) (3,070) Sunday Until 2 p.m After 2 p.m (N) (1,707) (1,329) (3,377) (3,007) (3,192) (3,304) Service on Sunday (and Saturday after 2 p.m.) Attending religious services in a place of worship Weekday Until 2 p.m After 2 p.m (N) (658) (956) (1,327) (1,226) (1,180) (1,261) Saturday Until 2 p.m After 2 p.m (N) (1,589) (940) (3,305) (3,061) (3,016) (3,070) Sunday Until 2 p.m After 2 p.m (N) (1,707) (1,329) (3,377) (3,007) (3,192) (3,304) Service on Sunday (and Saturday after 2 p.m.) Activity code 49 (1975 and 1993); ( ); place code: all ( ). 2. Activity code 49 (1975 and 1993); ( ); place code: 9 (1975); 7 (1993); 5. ( ). Sources: ATUS 1975, 1993, 2004, 2006, 2008, and Self-Selection of the Sample: Estimating Nonresponse Bias from the PSID Data In sample-based research the self-selection of the sample produces bias in the form of overrepresentation of levels of religious practice. In other words, it is assumed that the majority of individuals who refuse to collaborate are not churchgoers. The higher tendency of churchgoers to take part in social surveys may be connected to their different characteristic lifestyle (Woodberry 1998), a greater willingness to cooperate (Abraham, Helms, and Presser 2009; Brennan and London 2001; Ellison 1992; Morgan 1983), or, more generally, a greater sense of trust in others or altruism on the part of believers, an aspect that was studied, for example, by Fukuyama

11 CHURCH ATTENDANCE IN THE UNITED STATES 259 Table 3: Sample response rates, initial density, and annual percent increases in weekly density of religious practice by year and age (percentage) Individuals aged 18 and over Individuals aged only Initial density Initial density and annual percentage and annual percentage Response rates point increases Response rates point increases N (18+) = 4,969; N (30 59) = 2,682. Source: PSID panel study, (1995:283 et seq.) and Putnam (2000). Unfortunately, it is extremely difficult to determine the exact size of this overrepresentation without access to data regarding the religious practices of those who refuse to be interviewed or cannot be contacted. One form of empirical demonstration, which is by no means conclusive but definitely more effective (Rossi and Scappini 2010), consists of analyzing the effect of nonresponse on the distribution of frequency of churchgoing during the different waves of a panel study (PSID). Table 3 shows that measured density in the years is approximately 4 percent higher than the figure recorded by the initial sample, in which the nonresponse percentage in the panel study came close to the response rates recorded in the ATUS surveys ( 55 percent). To make sure that these results are not related to a change in the composition of the age breakdown of the sample, we can also present density and mortality for the age group, where the increase in density is approximately equivalent. It should be stressed that the data presented here do not take account of the fact that respondents to the panel study are already self-selected at the beginning, as not all sampled subjects agreed to take part in the survey. It is possible that the percentage of nonchurchgoers among those who declined to take part in the panel study is even higher than the figure recorded during its different waves. It should also be noted that as the PSID is only concerned with heads of households, the subjects are therefore mainly men. Nevertheless, we have no reason to think

12 260 JOURNAL FOR THE SCIENTIFIC STUDY OF RELIGION Table 4: Measured density, attending religious services in place of worship by month of diary completion and year (percentage) (N) (N) (N) (N) (N) January 38.2 (425) 47.3 (259) 27.2 (652) 25.9 (540) 27.6 (559) February 45.5 (401) 33.5 (251) 28.4 (396) 29.0 (429) 25.6 (463) March 39.3 (407) 31.9 (251) 34.3 (437) 29.2 (508) 28.3 (557) April 41.2 (406) 35.6 (262) 29.9 (607) 30.7 (520) 29.4 (466) May 39.2 (414) 35.4 (266) 29.1 (462) 25.8 (412) 25.3 (613) June 32.9 (412) 34.6 (273) 24.9 (448) 26.6 (572) 31.3 (512) July 41.1 (407) 35.1 (271) 29.8 (610) 26.7 (500) 24.3 (518) August 34.5 (405) 40.7 (281) 28.5 (458) 25.0 (458) 28.6 (598) September 28.8 (418) 36.8 (284) 32.5 (484) 27.0 (575) 25.4 (471) October 33.5 (464) 27.9 (389) 23.2 (622) 27.3 (453) 27.4 (576) November 30.1 (422) 33.9 (285) 29.7 (499) 26.0 (515) 22.3 (556) December 37.8 (419) 43.7 (271) 27.1 (463) 23.6 (548) 26.2 (456) Min Max Source: ATUS that gender differences in religiosity modify the trend of growth in attendance rates generated by the panel attrition rate. Therefore, although some respondents naturally abandon the panel study during its different waves, they are not equally distributed among churchgoers and nonchurchgoers. Our conclusion is that the self-selection of the sample may produce an overestimate in density of at least 4 percentage points. As measured density in ATUS surveys is considerably lower than the value calculated in the panel study (in 2010 P m = 23.4 percent, while in 1968 P c 45 percent), the value of the overestimate of P m can be established at about 2 percentage points. Similarly, an overestimate of approximately 2 percentage points can also be calculated by using the data in Table 3 and the nonresponse rate in GSS surveys (response rates 75 percent, with P c 40 percent). Quantifying Monthly Fluctuation Although generally overlooked, a second factor that can influence the validity of measures of attendance is the period in which the interview is carried out (Iannaccone and Everton 2004; Olson 2008; Olson and Beckworth 2011; Rossi and Scappini 2010). By using time use surveys, which unlike traditional surveys last for a year, we can highlight variations in attendance on a monthly basis. The size of such variations demonstrates that the choice of the period in which a survey is conducted might dramatically influence its results. Indeed, there are significant variations from month to month that sometimes exceed 15 percentage points (Table 4). We should stress that the ATUS weighting process generated person-level weights that can be used to produce monthly average estimates between 2003 and 2004, whereas since 2005 ATUS weighting has produced quarterly and annual average estimates (ATUS 2013). The correct estimates are therefore those for 2003 and 2004, even though the latter year has relatively restricted subsamples. For these sampling limits we grouped together subsequent years two at a time in an attempt to reduce sample variability. We can see that although there is a slight dip, the variations in density between

13 CHURCH ATTENDANCE IN THE UNITED STATES 261 different months remain considerable even in the period It is certain that such major variations cannot be ignored in surveys on this type of phenomenon. Characteristics of the Measuring Tool The third factor that explains why self-placement on a scale of frequencies diverges so greatly from what can be considered to be a reliable estimate of the actual frequency is the imprecision of the measuring tool. As we shall see shortly, this has the effect of amplifying the typical insurmountable problems of surveys: (1) social desirability (Smith 1998), (2) the tendency to remember behavior that is congruent with personal beliefs more easily and to forget behavior that is divergent, leading churchgoers to overestimate their attendance at church, and (3) the decision to report intentions in the answer rather than actual behaviour (Chaves 2010). In addition, bias is also generated by the inevitably incomplete answer options offered in frequency questions. For example, 2 3 times a month means literally between 24 and 36 Sundays a year, while once a month corresponds to 12 Sundays in numerical terms. There is therefore a gap in the middle, as frequencies of between 13 and 23 Sundays are not covered by the answer options. Similar problems are encountered with the two categories every week, which corresponds to 52 Sundays a year, and nearly every week, which covers an interval ranging from 37 to 51 Sundays a year. We can optimistically assume that all subjects in the category Sundays fell into the category nearly every week. 11 However, if the interviewees really behaved in this way, we would be unlikely to obtain a ratio of approximately 4 to 1 between the two categories (see Table 1: in percent and 6.7 percent; in percent and 4.3 percent). Considering that many factors illness, temporary absence from place of residence, and so on conspire to make the target of 52 Sundays difficult to reach for even the most diligent churchgoers, we would have instead expected an inverted ratio (1 to 4). The open-ended nature of the term nearly means that the definition of the boundary between diligent attendance (52 Sundays) and almost diligent attendance is left to a subjective decision. In other words, if respondents have to lie about the answer as a result of their specific situation, it is likely that they will solve the dilemma by choosing the lie that shows them in a better light (Rossi and Scappini 2012). 12 Causes of Bias and Overestimation: Some Preliminary Considerations To summarize, we have identified three main problems in research on religious practice: the self-selection of the sample, the period in which the interview is conducted, and the data collection method. The first of these is common to both types of survey and the overestimate generated in calculated and measured densities will be more or less equivalent, around 2 percentage points in the years 1975, 1993, and The second problem affects GSS surveys but not ATUS studies, as their sample of interviewees is distributed throughout the course of a year. Unfortunately, we are unable to quantify the size of this bias. The third and main source of distortion which, as 11 We have to underline that the GSS attendance question is an open-ended item. Respondents are not read the response categories, but are simply asked how often they attend. The interviewers, not the respondents, then code their answers into the given categories. Interviewers may also use the categories as probes to clarify respondent answers, but these categories are not initially read off as response options (Smith, Marsden, and Hout 2011). 12 We want to underline that if the higher category is constantly selected in the event of doubt those who go to church nearly every week say that they go every week, while those who go once a month state that they go nearly every week and so on, the difference between GSS calculated density and ATUS measured density is almost completely bridged (Castegnaro and Dalla Zuanna 2006). Conversely, there is good reason to think that those completing a daily diary are not driven to alter their position with regard to church attendance (Juster 1985; Juster and Stafford 1991; Presser and Stinson 1998).

14 262 JOURNAL FOR THE SCIENTIFIC STUDY OF RELIGION we have already claimed, is associated with a certain degree of inaccuracy in self-declarations about attendance naturally only affects GSS surveys. 13 It must be stated that the use of daily diaries leads to a slight overestimate of attendance, generally less than 1 percent, due to the same subject sometimes attending church services on both Saturday and Sunday. On the other hand, this slight overestimate should be compensated for by a definition of religious practice that is limited exclusively to services in church. Overall, we believe that the level of overestimation of measured density (about 2 percentage points) can be considered acceptable. In the next stage of our study we will take the ATUS surveys to be correct and calculate the different levels of bias generated by the GSS surveys, thereby obtaining the measure of the degree of inaccuracy of calculated density in formal terms Ic =Pc(X)/Pm(X) EVIDENCE OF THE EFFECTS OF BIAS: SOME COMPARISONS We have claimed that the use of frequency indicators is problematic as they tend to provide distorted data. It could naturally be argued that this analysis is unnecessary, since both indicators presented here have their advantages and disadvantages. While it is true that measured density is extremely sensitive to variations in attendance, it has the drawback of not providing us with any information about the composition of the population with regard to religious attendance, which is widely considered to be a highly desirable piece of information. Furthermore, the availability of diary-based surveys is somewhat restricted in terms of time and space as they require huge samples and are costly and lengthy to carry out. Only national statistics institutes have sufficient resources to conduct them on a long-term basis. They have only been regular in the United States since On the other hand, although the frequency rates obtained through questionnaires are not reliable, there is an extremely wide range of surveys that contain data on frequency of attendance in terms of both time and space, by virtue of the smaller sample size required, lower costs, and faster execution. Questionnaire data could still have a role to play if the overrepresentation of declared frequency is limited and systematic in nature, or, as we have defined it, a degree of inaccuracy in calculated density is low and constant over time and/or in the different social segments of the population. If the inaccuracy were instead significant and the trend were erratic, we would have to conclude that attempts to measure church attendance by surveying the frequency of churchgoing were doomed to failure, or were even pointless. Documenting the Attendance Trend in the ATUS Data Since 1975 The first comparison between GSS and ATUS surveys highlights a significant inaccuracy with regard to calculated density, which fluctuates over time in an equally significant way. In 1975 the ratio between calculated and measured densities respectively, 43.6 percent and 34.1 percent is I c = In other words, the data obtained from the frequencies recorded in the GSS surveys overestimate the data provided by the ATUS surveys by 28 percent (Figure 1 and Table 1). In 1993, the difference between the two measures of density increases considerably: the value of I c is now 1.65, with the GSS results overestimating the ATUS results by 65 percent. Finally, in the period the difference between the two measures of density stabilizes at a slightly lower but, nevertheless, extremely high value in these cases the value of I c is almost always 13 In this article the term inaccuracy is not used with a negative meaning. It simply shows the discrepancy between calculated and measured densities in a context where measured density is deemed to be reliable.