Talk:Cognitivism (psychology)

Latest comment: 11 years ago by Gregbard in topic Merger proposal


Untitled

edit

This is a very tidy little page. The section on criticisms needs elaboration, in the light of a lot of recent work both by cognitive scientists and by philosophers (John Haugeland, e.g., or Vincent Descombes, or Charles Taylor). To be fair to cognitivism, of course, the section on the project itself should be elaborated a good deal.

Right, I'm sorry but i had to change bits. Cognitivism is NOT differentiated from Behaviorism by its assumptions of positivism (behaviorism was just as positivistic in this sense). Cognitivism is the theory that thought is the use of internal rules or algorithms on internal symbols and that's what i've written. BScotland.

Hi there,
Actually, the article never claimed this. It simply said that positivism was one of the central tenants of cognitivism and that cognitivism replaced behaviourism as the most popular paradigm for understanding mental function.
I've reverted the article back to the last edit by APH and will re-incorporate some of the recent changes powing to the following issues:
"mental function can be best described via the analogy of the digital computer"
Cognitivism has never made the claim that the digital computation is the best description of mental function, simply that computation (information processing) is the best description, however it might be instatiated. Many cognitivist theories posit purely analog information processing functions (e.g. most of psychophysics)
"Theoretically cognitivism assumes that mental functioning is best seen as being the operation of 'rules' (algorithms) on internal 'symbols' (i.e. the mind/brain is a digital computer (see representationalism)"
Again, many cognitivist theories are decidely anti-symbolic, particularly connectionism. Paul Smolenky (ref below) explicitly calls connectionism 'sub-symbolic'. Connectionism is however, rule bound, although the rules simply describe how the nodes should pass information between them (hence the phrase 'parallel distributed processing'), rather than to complete any particular high-level task. The rules to complete the high-level task are an emergent property of the rules for sub-symbolic processing (e.g. delta rule, back propagation etc)
Smolensky, P. (1989). Connectionist modeling: Neural computation/mental connections. In L. Nadel (Ed.), P. Culicover, L. A. Cooper, R. M. Harnish (Assoc. Eds.), Neural connections, mental computation. Cambridge, MA: MIT Press/Bradford. 49-67. [Reprinted in J. Haugeland, (Ed.). (1997). Mind Design II: Philosophy, Psychology, Artificial Intelligence, MIT Press/Bradford Books.]
- Vaughan 13:53, 17 Sep 2004 (UTC)

Well, your call, man. However, I was under the impression that connectionism was NOT a kind of cognitivism, although it IS an information processing theory (ie. it posits the idea that cognition is computation). Chomsky's own work is about as un-empirical as you can get, but is undeniably cognitivist, hence the reason i felt that the first section on methodology was misleading.

'Connectionism is however, rule bound, although the rules simply describe how the nodes should pass information between them (hence the phrase 'parallel distributed processing'), rather than to complete any particular high-level task. The rules to complete the high-level task are an emergent property of the rules for sub-symbolic processing (e.g. delta rule, back propagation etc)'

This is a highly controversial statement. Many connectionists are absolutely explicit and clear that their connectionist models are NOT rule bound. Psychophysics predates cognitivist theories, and when people talk about cognitivism, it is not generally psychophysics to which they refer (although, to repeat, psychophysics probably IS an information processing view of cognition).

One more thing. The use of the word 'demonstrated' sounds as if Chomsky actually carried out experiments which proved the existence of 'mental states'. He did not. Instead, he posited various thought experiments which proved this to his own satisfaction. Many others in other traditions (for example, the tradition of Wittgenstein) would very much deny that Chomsky proved his point, so if it was ok with everyone i would prefer if it that word was changed to 'claim'. BScotland

Hi there, some replies to your points...
>"However, I was under the impression that connectionism was NOT a kind of cognitivism,
>although it IS an information processing theory (ie. it posits the idea that cognition is
>computation)."
I would disagree and there's plenty of examples where connectionist authors explicity align themselves with cognitive explanations. e.g "[connectionist models] hold out the hope of offering computationally sufficient and psychologically accurate mechanistic accounts of the phenomena of human cognition" from p11 of Rumelhart and McClelland's Parallel Distributed Processing ISBN 026268053X. Incidentally, the book is subtitled Explorations in the Microstructure of Cognition. See also ISBN 0198524269.
>"Chomsky's own work is about as un-empirical as you can get, but is undeniably cognitivist,
>hence the reason i felt that the first section on methodology was misleading."
As Chomsky's work is based on observation, it is indeed empirical. However, I presume you mean experimental, which much of it is not. The article doesn't claim that all cognitivist theories must be derived from experimental work (this would discount much of the philosophy of mind work in this area for example) but that mental processes are for the most part amenable to experimental investigation (note the 'in principle' qualification in 2nd sentence of 'Theoretical approach' section).
>>'Connectionism is however, rule bound, although the rules simply describe how the nodes should
>>pass information between them (hence the phrase 'parallel distributed processing'), rather than to
>>complete any particular high-level task. The rules to complete the high-level task are an emergent
>>property of the rules for sub-symbolic processing (e.g. delta rule, back propagation etc)'
>This is a highly controversial statement. Many connectionists are absolutely explicit and clear
>that their connectionist models are NOT rule bound."
Connectionist models would not work without being rule bound. They are explicity rule bound on the node level - e.g. "Learning is implemented in connectionist models by rules which determine how the weights of the connections between units are changed." (from p15, McLeod, Plunkett and Rolls Introduction to Connectionist Modelling of Cognitive Processes ISBN 0198524269) and on the behavioural level - e.g. "Rule-following behaviour may be realised in connectionist networks without any need to have the rules represented explicitly in the network" (p14 of Ellis and Humphreys Connectionist Psychology ISBN 0863777872). The important point is that the rules developed at the behavioural level are not set a priori, they are developed by the network as a response to its training or execution. If you can cite any connectionists that specifically deny that connectionist models are rule-bound I would be very surprised.
>"One more thing. The use of the word 'demonstrated' sounds as if Chomsky actually carried out
>experiments which proved the existence of 'mental states'. He did not. Instead, he posited various
>thought experiments which proved this to his own satisfaction. Many others in other traditions
>(for example, the tradition of Wittgenstein) would very much deny that Chomsky proved his point
>so if it was ok with everyone i would prefer if it that word was changed to 'claim'."
Fair point and I agree it 'claimed' is more appropriate than 'demonstrated' in this context.

- Vaughan 17:23, 18 Sep 2004 (UTC)

With respect you are eliding between 'cognitive science' (which is nowadays merely a synonmym for 'psychology that is not openly behaviorist') to 'cognitivism', which is, as i've said, the belief that cognition is the algorithmic manipulation of internal symbols (cognitivist themselves like to do this, to make it seem as if cognitivism is more popular than it actually is). To assume that all cognition is algorithmic manipulation (etc. etc.) is to merely assume that cognitivism is correct. So your quote: '[connectionist models] hold out the hope of offering computationally sufficient and psychologically accurate mechanistic accounts of the phenomena of human cognition"' merely confirms that connectionists believe in cognition and that this is computational in nature (ie the information processing hypothesis), which i never denied (i.e. i never denied they said it). Again, quoting the title: Explorations in the Microstructure of Cognition merely proves, again, that connectionists believe in cognition (who doesn't?) not what the mechanism of that cognition is.

'If you can cite any connectionists that specifically deny that connectionist models are rule-bound I would be very surprised.'

Yeah I can and I'm amazed (genuinely flabbergasted) that you haven't come across them. However you'll have to wait until i can get into the office on monday for quotes. However as well as this you have to remember that cognitivists tend to play with words in this respect (and i mean, a lot). As John Searle never fails to point out, cognitivists invariable leap from the statement that cognition is describable as algorithmic, to the belief that cognition is IN ACTUAL FACT the product of internal rules/algorithms.

Also some connectionists have attempted to build bridges between connectionism and cognitivism and posit a 'rule following' superstructure on a connectionist 'base' (Stephen Pinker tries this trick). So the quotes you have don't surprise me: i knew that SOME connectionists stated such things (my own personal feeling is that they are deluding themselves but that's just me). BScotland.

Incidentally, just put 'rule' and 'connectionism' into google: and came up with : 'Carl Bereiter describes connectionism as the notion that cognition IS NOT GOVERNED BY RULES, but rather, that approximations to rule-based behavior can be achieved through processes that do not involve rules as agents in the process.'

http://carbon.cudenver.edu/~lsherry/cognition/connectionism.html

Not of course a proper academic reference, but nevertheless indicative of how the word 'connectionism' is generally understood. Generally connectionism is used to describe (in 'ordinary language' that is, as the way it is usually used) models of cognition are not rule bound. I am genuinely astounded that you haven't come across these definitions (although, as you point out, and i never denied, some people DO think rule based cognitive models can be produced on connectionist networks' BScotland

'Both connectionist and symbolic systems can be viewed as computational systems. But they advance quite different views of what computation involves. In the symbolic approach computation involves the transformation of symbols according to rules. ...The connectionist view of computation is quite different....it focuses on causal processes by which units excite and inhibit one another and does not provide either for stored symbols or rules that govern their manipulations.' (Bechtel and Abrahamsen, 'Connectionism and the Mind', Blackwell, 2002, p2).

'The rise of cognitivism in psychology....has been characterised as a Kuhnian revolution....as (its) paradigm developed, the idea that cognition involved the manipulation of symbols became increasingly central...These symbols....were entities which could be stored in and retrieved from memory and transformed according to rules. In the 1980s however, an alternative framework for understanding cognition emerged....connectionist...models. ' Bechtel and Abrahamsen, 'Connectionism and the Mind', Blackwell, 2002, p1).


<:Hi there,

Of course, Bechtel is by no means an unbiased source (see 'The Case for Connectionism', Philosophical Studies, Vol 71). Incidentally, in that very same article he argues that connectionism is a cognitivist theory (see section 'Expanding the Conception of Cognition') and has also argued that connectionism is not incompatible with the representation of symbol-like propositional states such as beliefs (see Bechtel, 1988, Connectionism and the Philosophy of Mind: An overview, reprinted in ISBN 0631167633) Just because he argues it's an alternative theory for understanding cognition, doesn't mean it's not cognitivist.>

I knew that: to quote Bechtel again: 'Some connectionists agree with Fodor [...]'s contention that humans carry out explicit symbol processing [...] and hence have directed their efforts to implementing systems of rules and representations in networks' (Bechtel and Abrahamson op cit p163). As I said I think these people are deluding themselves, but I'm aware that the distinction between 'symbolic' and 'connectionist' is a lot more analogue than some digital theorist might think (and there's a lesson there).

<Rule bound nature of connectionist models.
This is certainly a complex issue as it depends on exactly at what level we're talking about. Bechtel is arguing for the lack of a priori rules governing the manipulation of pre-speficied internal symbols. There are many examples of connectionist models that learn rule based behaviour (for example Joanisse and Seidenberg's recent model that they taught to map sequences of words to meanings) however they do not rely on strict, internal symbolic representations to complete the rule-governed behaviour. The representations are distributed throughout the network.> Again, it depends what you mean by a 'rule'. If you gloss is as 'algorithm' as I usually do, then this makes the sense clearer. There's a book (which i have now lost the reference for but which i could find again easily enough) which argues that connectionist 'rules' aren't rules as symbolists would interpret them (i.e. aren't digital algorithms) but are closer to 'heuristics' or guidelines: as you say, which emerge from the context and situation, rather than being pre-programmed. It was this distinction i was discussing. (on the other hand 'Rumelhart and McClelland's classic anti-rule, connectionist theory...' (http://www.americanscientist.org/template/BookReviewTypeDetail/assetid/26506;jsessionid=baa51x4nz7ba1Z)).

I should note that McClelland himself states that: 'One view of language, originating with Chomsky , championed by Fodor and Pylyshyn and widely pursued by Pinker, holds that abstract symbolic rules play a central role in the human language processing. This claim is part of a broader view that human cognitive mechanisms are symbolic, modular, innate, and domain-specific. An alternative view, from Rumelhart and McClelland, challenges the need for the use of rules....' (he defines as rule as) 'We address a specific notion of rules held by Pinker and his collaborators, in which rules are discrete, categorical, and symbolic objects used in a specialized, innate language module.' (i.e. what i meant by an 'algorithm' ...I assume that by 'discrete and categorical' he means what i meant by 'digital') ((http://www.cnbc.cmu.edu/~jlm/papers/RulesOrConnections.pdf)

I really do think that connectionism as that word is used in ordinary language (well by psychologists) usuall means 'not using rules' even apart from my own personal prejudices.

Also, I've removed the addition "specifically as a kind of Turing machine", as I think this is a controversial point in itself and not widely accepted among cognitive scientists. Even Pylyshyn (see ISBN 026266058X) does no go this far and many see computation purely as a metaphor (e.g. Parkin, ISBN 0863776736). I've not been able to find this claim in either the Oxford Dictionary of Psychology, or any number of books on cognitive or neural information processing. Although I would be glad to hear about other sources. P.S. Enjoyable debate on an interesting topic, many thanks.
- Vaughan 11:07, 21 Sep 2004 (UTC)

http://www.ecs.soton.ac.uk/~harnad/Papers/Py104/searle.comp.html

'The question is, "Is the brain a digital computer?" And for purposes of this discussion I am taking that question as equivalent to: "Are brain processes computational?"...Just to keep the terminology straight, I call the view that all there is to having a mind is having a program, Strong AI, the view that brain processes (and mental processes) can be simulated computationally , Weak AI. and the view that the brain is a digital computer, Cognitivism....This paper is about Cognitivism...I want to begin the discussion by trying to state as strongly as I can why cognitivism has seemed intuitively appealing. There is a story about the relation of human intelligence to computation that goes back at least to Turing's classic paper (1950), and I believe it is the foundation of the Cognitivist view. I will call it the Primal Story: We begin with two results in mathematical logic, the Church-Turing thesis (or equivalently, the Churchs's thesis) and Turing's theorem. For our purposes, the Church-Turing thesis states that for any algorithm there is some Turing machine that can implement that algorithm. Turing's thesis says that there is a Universal Turing Machine which can simulate any Turing Machine. Now if we put these two together we have the result that a Universal Turing Machine can implement any algorithm whatever. But now, what made this result so exciting? What made it send shivers up and down the spines of a whole generation of young workers in artificial intelligence is the following thought: Suppose the brain is a Universal Turing Machine. ....Since there is no universal agreement on the fundamental questions, I believe it is best to go back to the sources, back to the original definitions given by Alan Turing. According to Turing, a Turing machine can carry out certain elementary operations: It can rewrite a 0 on its tape as a 1, it can rewrite a 1 on its tape as a 0, it can shift the tape 1 square to the left, or it can shift the tape 1 square to the right. It is controlled by a program of instruction and each instruction specifies a condition and an action to be carried out if the condition is satisfied.' (note: by 'instruction' i understand that Turing (and Searle) mean 'algorithm' or what i have described as a 'rule').

Searle's article has been criticised (sometimes vehemently) but to the best of my knowledge, no one has ever denied his statement that cognitivism is the belief that the brain is a digital computer (as that phrase was defined in Turing's original papers). On the contrary: most cognitivists seem to act as if the mind/brain is so obviously a digital computer that it is not necessary to offer any evidence that this is the case.

'and many see computation purely as a metaphor'. Well they SAY that, but that's obviously (well obvious to me anyway) simply a rhetorical get out. When challenged on the fact that they have produced no evidence (whatsoever) that the brain/mind is a digital computer they tend to respond 'well of course this is only a METAPHOR...we don't really MEAN it.' Then they go off and happily start to talk as if the brain was a digital computer again.

I can only add that if it is a metaphor it's not a very good one. It doesn't illuminate anything or make anything clearer. On the contrary: as the brain/mind is NOT a digital computer it makes things more confusing.

As you may have noticed, I don't think much of cognitivism, so I am biased, but I do think that many psychologists attempt to muddy the waters and blur the distinction between connectionism and cognitivism because this makes it less obvious that they have been working on a fundamentally flawed and misleading paradigm for the last forty years. BScotland.

Incidentally, i think the article is still misleading: connectionism may or may not be the same as cognitivism, but i think it is undeniably an information processing model: i.e. connectionists undeniably state that cognition is computation. It is the kind of computation they are discussing that matters.

BScotland.

Hi everyone. Separated out Searle's views from Penrose's as they are quite different as can now be seen in the Criticism section. Still need some tidy up on these and better references. More importantly cognitivism and cognitive psychology are being confused here as somewhat noted above. Cogntivism was (is?) the dominant paradigm (hate that word) in cognitive psychology but there are (now) a number of alternatives. I think the intro and body need to be totally redone. When I have time I can make an attempt here? martinu 15:24, 4 Mar 2005 (UTC) martinu

Given that I'm not a cognitivist people might be surprised that I'm making this criticism, but, looking through the article now, I wonder if it isn't too biased AGAINST cognitivism. The section in which it is actually stated what it 'is' seems very short, and the 'criticism' section very long. Perhaps someone might want to write an piece on (for example) the history of the concept? (starting with Chomsky and Herbert Simon etc.). BScotland 14:32, 11 January 2006 (UTC)

Gorman...

I had to stop reading about 10 paragraphs ago so please excuse if this doesn't address your particular argument. The major problem with cognitivism is ontological; i.e., chicken and egg. Where do your rules come from? Where do your "knowledge stores" come from? From whence does reasoning proceed?

These are issues raised in the Wiki article from the start. Any "cognitive architecture" that you folks put forth seems to be computational, and computations require a priori input (here is were Kant and Chomsky come into play). As a programmer (interested in thought) I don't see a practical backbone for cognitivism as an explanatory mechanism for human thought because you require "rules" "architectures" and the like.

I rather prefer that rules are the result of human ingenuity rather than vice versa.

Gorman...


Are you all sure about this here?

edit

I suppose the original authors of this article are long gone, but anyway, here's what I've found. A quick internet search of "cognitivism" doesn't turn up any sources that remotely resemble the material in this article. Several points:

  1. The majority of source come from educational psychology, i.e. learning theory. This is where the term has currency.
  2. Cognitivism is almost always defined as a reaction against behaviorism (which is mentioned in this article, but buried beneath some other material).
  3. Most sources mention Jean Piaget as an important inspiration of the movement. Noam Chomsky and the founders of cognitive psychology are also sometimes mentioned.
  4. Few sources mentioned information processing at all. No sources mention positivism.
  5. This articles defines cognitivism in a way that sounds suspiciously similar to computationalism (and while I agree that there are lot's of people who were both cognitivist and computationalist at the same time, these are still slightly different positions -- I think you can be cognitivist without committing yourself to an information processing model. Am I wrong?)
  6. The primary criticism of cognitivism (in learning theory) is constructivism.
  7. I can't see how phenomenology (in general) can critique cogntivism, since it precedes it historically. Hubert Dreyfus is a stretch.
  8. Roger Penrose is way off topic, as a critic. Even John Searle and Hubert Dreyfus are off topic. These people are refuting computationalism, not cognitivism, for the most part.

This is not really a field I know a lot about, so I would really rather that an expert take a crack at this article. If no one appears to fix it, I will knock it down to stub size based on internet sources in a month or two. ---- CharlesGillingham (talk) 03:20, 23 June 2008 (UTC)Reply

I decided to remove the first paragraph of the criticisms sub-section as per some of the above users statements. That paragraph has been recreated below so that it can be easily accessed should a set of sources be found for it.
"Phenomenologists and hermeneutic philosophers have criticised the positivist approach of cognitivism for reducing individual meaning to what they perceive as measurements stripped of all significance. They argue that by representing experiences and mental functions as measurements, cognitivism is ignoring the context (cf contextualism) and, therefore, the meaning of these measurements. They believe that it is this personal meaning of experience gained from the phenomenon as it is experienced by a person (what Heidegger called being in the world) which is the fundamental aspect of our psychology that needs to be understood: therefore they argue that a context-free psychology is a contradiction in terms. They also argue in favour of holism: that positivist methods cannot be meaningfully used on something which is inherently irreducible to component parts. Hubert Dreyfus has been the most notable critic of cognitivism from this point of view. Humanistic psychology draws heavily on this philosophy, and practitioners have been among the most critical of cognitivism."
Travza (talk) 00:09, 7 April 2011 (UTC)Reply


Am I the only one to find the section on theoretical approach immensely biased? http://en.wikipedia.org/wiki/Positivism#Positivism_in_science_today explicitly describes how Positivism is still an accepted framework. Saying that "Cognitivism becomes ever more specious once the collapse of positivism as a theory of meaning is acknowledged" assumes a fact that is not in any way self evident. The sentence before it is also a blatant and pointless attack. Maybe, if sourced, these points could be put in the criticism section. Placing them in theoretical approach seems extremely non neutral. Penultimate sentence read: "All of these assumptions come from a school of metaphysics known as naturalism and have been severely criticized as they have not been able to provide any adequate support for their claims and assumptions." Removing the two unsourced attack sentences.

Keller Scholl (talk) 18:12, 14 January 2012 (UTC)Reply

"Positivism" is an irrelevant putdown

edit

The wiki article on positivism states, and I agree, that positivism is the philosophical (i.e. not scientific) belief that scientific knowledge is the only valid knowledge. More specifically, "positivist" is a term used in epistemology. Hence it not logically possible to describe a scientific field as either positivist or not positivist. Obviously, within a scientific field you have to assume that the only valid knowledge WITHIN THAT FIELD is scientific knowledge, but calling this assumption "positivist" distorts the language. No claim is being made about nonexistence of nonscientific knowledge OUTSIDE the field. I feel quite certain that, whatever "cognitivism" might reasonably be defined as, its practitioners view it as part of science and not part of philosophy. Hence describing it as "positivist" is at best anachronistic.

Now admittedly there are a very few cases where "positivism" has an accepted and useful meaning within a scientific or quasi-scientific field, but this is not one of them.

One case where the term "positivist" is meaningful is "positivist sociology." That is an older term that actually depended on Comte's earlier usage of the term, which is not the same as modern usage (see e.g. [1]). In effect, "positivist sociology" merely means "the historically first stab at a fully scientific sociology." Similar terms were used in related fields such as criminology. However using this term for cognitivism cannot possible be correct, because cognitivism split off from the already fully scientific approach of behaviorism.

A second case is in the theory of legal interpretation, where legal positivists claim that validity of a law is unrelated to its merits--rather far afield from this discussion.

It seems to me that when philosophers like Searle apply the term "positivist" in cases like this, they may be claiming that cognitive studies has not yet made a full split from philosophy. This is a confusing question, because philosophers do in fact continue to make contributions to cognitive science as well as to several other scientific fields. However when they do so, to the extent that they make successful contributions they are doing science rather then traditional philosophy. In any case they are certainly not doing epistemology.

Adding to the confusion, cognitive studies does in fact deal with the practical question of how people arrive at what they think they know (or act as if they know), which almost sounds like epistemology but isn't. Epistemology deals with the logical validity of knowledge, not necessarily directly with its acquisition process. And in any case, cognitive scientists as such are not making claims about the validity of, for example, divine revelation. Some of them do go too far and make such claims, but they are acting out of their element. Burressd (talk) 22:04, 7 January 2013 (UTC)Reply

References

  1. ^ Thomas Meaney, "The Religion of Science and Its High Priest," The New York review of Books, OCTOBER 25, 2012 at http://www.nybooks.com/articles/archives/2012/oct/25/religion-science-and-its-high-priest

Merger proposal

edit

The very recently created article Cognitivism (philosophy of education) overlaps significantly with this article, and has little new to offer. I propose to merge it into this one. hgilbert (talk) 03:40, 16 July 2013 (UTC)Reply

  • Oppose - I disagree with the merger and your opinion. The educational perspective is significantly different from the psychology perspective or even the philosophy perspective and the less overlap the better. Stmullin (talk) 15:12, 16 July 2013 (UTC)Reply
  • Oppose - In general, I would love for there to be single comprehensive articles. However, the political reality is that people from different academic areas don't respect each other on Wikipedia the same way they might in a university setting. This leads to edit warring, and wrongful deletion of good material. So, the best way to accomodate this situation is to have separate articles, and in the main article, have sections with a summary for each of the others. Greg Bard (talk) 17:17, 19 July 2013 (UTC)Reply

Ertmer & Newby (1993) should be cited

edit

I use Ertmer and Newby (1993) whenever I need a refresh on behaviorism, cognitivism and constructivism. I was just reading that chapter, then looking for additional cognitivism roots and citations, and opened this article. The format is REALLY similar, and the titles "How does learning occur" and "What is the role of memory" are exact copies of the subtitles used in each section (behaviorism, cognitivism, constructivism) of Ertmer and Newby's chapter. Unless there is a more seminal source, the format is clearly copied from that work. In addition, one can see how the content was summarized from that work; simpler words, but same progression of thoughts. They should receive credit: [1]

References

  1. ^ Ertmer, P. & Newby, T. (1993). Behaviorism, cognitivism, constructivism: Comparing critical features from an instructional design perspective. Performance Improvement Quarterly, 6(4), 50-72.