Willem Saris

Willem Egbert (Wim) Saris (born 8 July 1943) is a Dutch sociologist and Emeritus Professor of Statistics and Methodology, especially known for his work on "Causal modelling in non-experimental research" and measurement errors (for example, MTMM analyses and development of the Survey Quality Predictor (SQP) program).

Biography

Saris was born in Leiden, South Holland, Netherlands, in 1943. He finished his study of Sociology at the Utrecht University in 1968 and earned his PhD from the University of Amsterdam in 1979. He became full professor in political science, especially the methodology of the social sciences in 1983. Till 2001, he was working at the University of Amsterdam. In 1984, he created the Sociometric Research Foundation in order to improve social science research by the application of statistics. In 1998, he became member of the methodology group that facilitated the creation of the European Social Survey (ESS). As a consequence, he was also member of the Central Coordinating Team of the ESS from 2000 until 2012.

In 2001, he moved to Barcelona where he was granted a position as ICREA professor at ESADE. From 2009 to 2012, he was also director of the Research and Expertise Centre for Survey Methodology (RECSM) at the Pompeu Fabra University. He was also one of the founders and first chairman of the European Survey Research Association (ESRA).

Academic career

Studies

Political decision making

Over a long period, Saris was involved with Irmtraud Gallhofer in an applied research project studying political decision making on the basis of governmental meeting minutes and notes of advisers. After developing reliable instruments for analysis, the team applied their approach on varied decisions of the Dutch government as well as major decisions in world history, such as decisions concerning the start of the First and Second World War, the Cuban Missile Crisis and the use of the atomic bombs in 1945. The first phase of the study was directed on the argumentation of individual decision makers. In a later phase, the research was extended to the study of collective decision making by the same governmental groups. The result was that the decision makers were making relatively simple arguments with respect to serious and far-reaching outcomes of war and peace.

Statistical aspects of structural equation models (SEM)

Worried about the testing procedures of structural equation modelling, Saris worked together with Albert Satorra to improve these procedures. They together developed different procedures to evaluate structural equation models. The final product was a procedure for detecting misspecification in these models taking into account the power of the tests. A program (JRule) was developed for these tests by William van der Veld.

Improvement of measurement in survey research

Application of SEM showed how large the errors were in survey data. Therefore, he directed his research on procedures to improve the measurement instrument such as the application of continuous response scales instead of category scales. In doing so, he detected that people could provide responses in very precise but different ways. This variation in response functions would be perceived as measurement error if it was not detected. In order to prevent this problem he suggested to use two fixed reference points on a scales points where there could be no doubt what they meant. Also studies were undertaken to improve measurement by use of computer assisted data collection, especially in order to remove the effect of the interviewer in the data collection. The latter research led to the development in 1986 of the Telepanel a procedure comparable with the Web survey procedure but at a time before the web existed. Given the well known fact that most people do not know much about many problems discussed and inspired by the decision theoretical approach discussed in a previous project a research line was developed to develop a decision aid for people to participate in decision making with respect to complex issues. In this context a decision aid, the Choice questionnaire, was developed by him in cooperation with Peter Neijens and Jan de Ridder. In this procedure the respondents had to evaluate necessary information about the issue before they were asked to make their choice. Because of the lack of validity of the proposals for measurement instruments specified even by experts in the different fields, he created together with Irmtraud Gallhofer the three steps procedure for designing survey questions. The suggestion is that use of this procedure guarantees the validity of the proposed questions.

Evaluation of survey measurement instruments with respect to reliability and validity

When he realized that one could indeed improve survey measures but one could never reduce the errors to zero, he decided to change his research in the direction of estimating the size of the systematic and random errors in the measures in order to be able to correct for measurement error. Inspired by the work of Frank Andrews (1984) a research program has been developed in cooperation with 11 different research groups in Europe from different countries. First the approach to evaluate measurement instruments, the so-called Multitrait-Multimethod (MTMM) experiment has been evaluated and a new model for MTMM data was developed in cooperation with Frank Andrews. This so-called True Score model allows for the separation of random and systematic errors. Also a new design, The Split-ballot MTMM design, for MTMM experiments was developed in order to reduce the number of repetitions of the same questions in cooperation with Albert Satorra and Germa Coenders. Next, data have been collected with this design within the telepanel and the ESS and the data were analysed by members of his research group in RECSM to determine the reliability (complement of random errors) and validity (complement of systematic errors) of measures.

Prediction of the quality of surveys

Probably the most important development in his research took place when he realized that one could never study the reliability and validity of all questions. This would be too expensive and time consuming. The alternative he suggested was to develop a coding system for characteristics of the questions and do a meta analysis trying to predict, on the basis of the coded characteristics of the questions, the reliability and validity of these questions. If these predictions were good enough the prediction procedure could also be used for other questions that were not studied so far but of which the same characteristics were coded. It turned out that the prediction of the quality of question, determined in the MTMM experiments, on the basis of the characteristics of the questions, was quite good. This led him to the idea to design a computer assisted expert system which uses all available information of data quality to predict the quality of new questions. This program, called Survey Quality Predictor or SQP was first developed by him in MS-DOS and later transformed in a Windows version. At present there is a new version (SQP 2.0) made by Daniel Oberski based on 3700 questions evaluated in MTMM experiments (SQP). Users can add their own questions in the data base and receive evaluations of the quality of the questions with respect to reliability and validity. Users get also information about possible improvements of the questions. The importance of this program is that the evaluation is not limited to the questions that have been evaluated but can be used for any questions for which the characteristics are coded in the system. All these questions and quality predictions are available to all users free of charge. In this way the knowledge base of questions and quality predictions is growing day by day. Momentarily the program contains already 67.000 questions introduced by more than 2000 unique users of the program.

Awards

As a member of the CCT of the European Social Survey he became laureate of the Descartes Prize 2005, for the best collaborative research. In 2009, he received the Helen Dinerman award by the World Association for Public Opinion Research (WAPOR) in recognition for his lifelong contributions to the methodology of public opinion research. In 2011, he received a doctor honoris cause of the University of Debrecen (Hungary). In 2013, he received the important service to survey research prize of the ESRA. In 2014, he was awarded together with Daniel Oberski by the American Association for Public Opinion Research with the Warren J. Mitofsky Innovators Award for the Survey Quality Predictor (SQP 2.0) and his contribution to the improving questionnaire design.

Notable contributions

• Test for misspecifications in SEM (JRule)
• Variation in response functions (VRF)
• Fixed reference points
• Telepanel
• Choice questionnaire
• Three steps procedure
• True score MTMM model
• Splitt Ballot MTMM design and model (SB-MTMM)
• Survey quality predictor (SQP)
• European Survey Research Association (ESRA)
• Simple procedure to correct for measurement errors

Most relevant publications

Saris has authored and co-authored numerous publications. Some of his main relevant publications are listed below, by main topics.

Books

• W.E. Saris, I.N. Gallhofer (Eds) (1987) Sociometric Research II: data analysis. London. MacMillan Press.
• W.E. Saris, I.N. Gallhofer (1987) Sociometric Research I: data collection. London. MacMillan Press.
• Gallhofer I.N. and W.E. Saris (1996) Foreign Policy Decision Making: A Qualitative and Quantitative Analysis of political Argumentation. Westport, Connecticut: Praeger Publishers.
• I.N. Gallhofer & W.E. Saris. (1997) Collective choice processes. A qualitative and quantitative analysis of foreign policy decision-making. Westport, Connecticut: Praeger Publishers.
• Saris W.E. & P. Sniderman (2004) Studies in Public Opinion: Attitudes, nonattitudes, measurement error and change. Princeton, Princeton University Press.
• Saris W.E. and I.N. Gallhofer (2007) Design, evaluation and analysis of questionnaires for survey Research. Hoboken, USA. Wiley.
• Saris W.E. and I.N. Gallhofer (Second Edition, 2014) Design, evaluation and analysis of questionnaires for survey Research. Hoboken, USA. Wiley.

Political decision making

• Gallhofer I.N. and W.E.Saris (1996) Foreign Policy Decision Making: A Qualitative and Quantitative Analysis of political Argumentation. Westport, Connecticut: Praeger Publishers.
• I.N. Gallhofer & W.E. Saris. (1997) Collective choice processes. A qualitative and quantitative analysis of foreign policy decision-making. Westport, Connecticut: Praeger Publishers.

Structural Equation models

• W.E. Saris, L.H. Stronkhorst (1984) Causal Modelling in non-experimental research. Amsterdam: Sociometric Research Foundation.
• Satorra, A.; Saris, W.E. (1986). "Power of the likelihood ratio test in covariance structure analysis". Psychometrika. 50: 83–90. doi:10.1007/bf02294150. S2CID 122574513.
• Saris, W.E.; Satorra, A.; Sörbom, D. (1987). "Detection and correction of structural equation models". Sociological Methodology. 1987: 105–131. doi:10.2307/271030. JSTOR 271030.
• Saris, W.E.; Satorra, A.; de Pijper, W.M. (1991). "A comparison of several approximations to the power function of the likelihood ratio test in covariance structure analysis". Statistica Neerlandica. 45 (2): 173–185. doi:10.1111/j.1467-9574.1991.tb01302.x.
• Saris, W.E.; Satorra, A.; van der Veld, W.M. (2009). "Testing Structural Equation Models or Detection of Misspecifications?". Structural Equation Modeling. 16 (4): 561–582. doi:10.1080/10705510903203433. hdl:10230/46205. S2CID 56293445.

Improvement of measurement

• W.E. Saris, C. Bruinsma, W. Schoots, C. Vermeulen (1977) The use of magnitude estimation in large scale survey research. Mens en Maatschappij, 369–395.
• Saris, W.E. (1988). "A measurement model for psychophysical scaling. In". Quality & Quantity. 22 (4): 417–433. doi:10.1007/bf00171631. S2CID 121292822.
• Saris W.E. Words are sometimes not enough to express the existing information. In Fennema M., C.v.d.Eijk and H.Schijf In Search of Structure. Amsterdam, Het Spinhuis. 63–73, 1998.
• W.E. Saris (1987) Variation in Response Functions: a source of measurement error in survey research. Amsterdam. Sociometric Research Foundation.
• Saris, W.E; Revilla, M.; Krosnick, J.; Schaefer, E.M. (2009). "Comparing questions with agree/disagree response options to questions with item specific response options". Survey Research Methods. 16: 561–582.
• Neijens, P.; de Ridder, J.A.; Saris, W.E. (1992). "An instrument for collecting informed opinions". Quality and Quantity. 26 (3): 245–258. doi:10.1007/bf00172428. S2CID 122986326.
• Saris, W.E.; de Pijper, W.M. (1986). "Computer assisted interviewing using home computers". European Research. 14: 144–150.
• W.M. de Pijper, W.E. Saris (1986) The formulation of interviews using the program Interv. Amsterdam. Sociometric Research Foundation, pp. 168
• W.E. Saris (1991) Computer-assisted interviewing. In: Sage Series Quantitative applications in the social sciences, nr. 07-080
• W.E. Saris CASIP: A complete automated system for information processing in family budget research, Eurostat report, 230, 1993
• Saris W.E. Ten years of interviewing without interviewers: The telepanel. In Couper M.P., R.P.Baker, J.Bethlehem, C.Clark, J.Martin, W.L.Nicholls II, J.M. O’reilly (EDS) Computer assisted survey information collection. New York Wiley, 409–431, 1998.

Multi trait Multimethod research

• W.E. Saris, A. van Meurs (eds.): Evaluation of measurement instruments by meta-analysis of multitrait multimethod studies. North-Holland.
• W.E. Saris, F.M. Andrews(1991) Evaluation of measurement instruments using a structural modeling approach. In: P. Biemer, R. Groves, L. Lyberg, N. Mathiowetz, S. Sudman (eds.): Measurement errors in surveys. John Wiley & Sons.
• W.E. Saris and A. Münnich (Eds) (1994) Multitrait Multimethod approach to evaluate measurement instruments. Budapest, Eötvös University Press, 280.
• Scherpenzeel, A; Saris, W.E. (1997). "The validity and reliability of survey questions: a meta analysis of MTMM studies. In". Sociological Methods and Research. 25: 341–383. doi:10.1177/0049124197025003004. S2CID 122375500.
• Saris, W.E.; van Wijk, T.; Scherpenzeel, A. (1998). "Validity and reliability of subjective social indicators: the effect of different measures of association". Social Indicators Research. 45: 173–199. doi:10.1023/A:1006993730546. S2CID 142109857.
• Corten, I.W.; Saris, W.E.; van der Veld, W.M.; Aalberts, C.E.; Kornelis, C. (2002). "Fit of different models for Multitrait-Multmethod Experiments". Structural Equation Modeling. 9 (2): 213–232. doi:10.1207/s15328007sem0902_4. S2CID 13749533.
• Saris, W.E.; Aalberts, C. (2003). "Different explanations for correlated disturbance terms in MTMM studies". Structural Equation Modeling. 10 (2): 193–214. doi:10.1207/s15328007sem1002_2. S2CID 122262960.
• Saris, W.E. (2004). "A new approach to evaluating the quality of measurement instruments: the split-ballot MTMM design". Sociological Methodology. 2004: 311–347. CiteSeerX 10.1.1.484.3241. doi:10.1111/j.0081-1750.2004.00155.x. S2CID 37588358.
• Revilla, M.; Saris, W.E. (2013). "The Split-ballot Multitrait-Multimethod Approach: Implementation and Problems". Structural Equation Modeling. 20 (1): 27–46. doi:10.1080/10705511.2013.742379. hdl:10230/28329. S2CID 121064182.

The development of a procedure to predict the quality of survey questions

• Gallhofer, Irmtraud (2004). "Operationalization of social science concepts by intuition". Quality and Quantity. 38 (3): 235–258. doi:10.1023/B:QUQU.0000031328.25370.e9. S2CID 143523418.
• Saris W.E., W.vander Veld and I.Gallhofer (2004) Development and improvement of questionnaires using predictions of reliability and validity. In Presser et al. (Eds) Methods for testing and evaluating survey questionnaires. Hoboken, Wiley, 275–299.
• Saris W. E., and I.N. Gallhofer 2007. Estimation of the effects of measurement characteristics on the quality of survey questions. Survey Research Methods, 1, 31─46.
• Saris W.E. and I.N. Gallhofer (2007) Design, evaluation and analysis of questionnaires for survey Research. Hoboken, USA. Wiley.
• Saris W., D. Oberski, M. Revilla, D. Zavala, L. Lilleoja, I.Gallhofer and T. Gruner (2011) The development of the program SQP 2.0 for the prediction of the quality of survey questions, RECSM Working paper 24
• Saris W.E. and I.N. Gallhofer (Second Edition, 2014) Design, evaluation and analysis of questionnaires for survey Research. Hoboken, USA. Wiley.
• Saris, W.E; Revilla, M. (2016). "Correction for measurement errors in survey research: necessary and possible" (PDF). Social Indicators Research. 127 (3): 1005–1020. doi:10.1007/s11205-015-1002-x. hdl:10230/28341. S2CID 146550566.

External links

• Willem E. Saris Google Scholar profile
• Willem E. Saris Research Gate profile
• Sociometric Research Foundation