User:Agnese marino basc/sandbox/Approaches to Knowledge (LG seminar)/Group 8/Evidence

Evidence in health

Evidence based medicine (EBM) is about “integrating individual clinical expertise and the best external evidence”. EBM epistemologically grades evidence, and prioritises “hard” data, such as evidence from randomised controlled trials, systematic reviews and meta-analyses, over “softer” observational studies/testimonials. On the one hand, this helps clinicians make the best-informed, unbiased decisions. However, it has been flagged up that randomised-control trials can lead to a top-down ‘cookbook’ approach, as the word in these highly-graded studies is taken as law, or the “gold standard”. We only learn how the treatment works for an ‘average, randomized patient, and this evidence is then presented with authority and sanctity. This is then problematic to the field of evidence-based medicine and can result in ill-advised treatments.

Evidence in humanities

• In sciences, the confirmation theory is often used the systematic study of theory evaluation, shows remarkable lacunas when it comes to the methodology of the humanities.
•  Developments in humanities and conformation theory invite us to reconsider this situation
  • First, due to the fast uptake of empirical and computational methods in several humanities disciplines, the humanities are presently very much in flux, and much more amenable to methodological elucidation
  • Second, confirmation theory has over the past decade significantly broadened its scope, and made contact with developments in mathematics and computer science.

Confirmation theory

• Confirmation theory: concerns the support that empirical and other evidence provides for scientific hypotheses.

• humanities theories are often represented more adequately by structured sets of probability functions over empirical data.
• Relative to empirical data and various theoretical criteria, we do not merely choose the best fitting probability function as our best theory. Rather we choose between several sets of such functions, or in statistical parlance, we choose between models. The major advantage of representing the nature and evaluation of theories from the humanities in this way is that it provides more room for accommodating confirmation-theoretic phenomena like incompatible concepts and fuzzy evidence.
• In the humanities, evidence and theory are often not quantitative, let alone statistical in nature.Some humanities disciplines are primarily informed by methods of interpretation or hermeneutics, which seem to resist a formal and confirmation- theoretic representation. Moreover, the rigidity and uniformity of statistical methods seems to be in stark contrast to the plurality of methods that we find in humanities disciplines. And finally, humanities research is often concerned with the unique and specific, and not, as are statistical methods, with the repeatable and generic.
• However, the goal of involving confirmation theory is certainly not to cover all theory evaluation in the humanities. The proposal is to employ confirmation theory for an assessment of particular approaches within the humanities, namely computational and empirical ones. Our challenge is to integrate those new methods with an open eye for the specifics of existing humanities scholarship. And for this task, formal philosophical methods are certainly not far-fetched.

New methods and missing methodology

• Last two decades have seen the fast growth of a number of new branches in humanities scholarship, centered on the application of empirical (based on, concerned with, or verifiable by observation or experience rather than theory or pure logic) and computational methods. We find these new methods in a wide array of humanities’ disciplines
  • They range from linguistics and cognitive musicology, in which researchers simulate language production and musical hearing on a computer, to archeology, museology, philosophy, and the study of religion, in which empirical studies from psychology, ethnography, and sociology are brought to bear on more traditional ways of theorizing.
  • Much of the research in the humanities is carried out by means of scholarly methods: close reading of texts, analysis of concepts, and interpretation of practices.

• Compared to sciences: In analytic philosophy of science, insofar as it is concerned with research methods and the justification they lend to scientific theories, these scholarly methods have hardly been targeted.
•  The empirical, computational, and other methods used in the humanities are often not home-grown. Rather, they have been taken over from neighboring fields of research, and hence accommodated to the specifics of their domain of application on a case-by-case basis.
  • study of religion, for example, is routinely informed by psychological and ethnographic research. But the statistical and experimental tools employed for such interdisciplinary work derive by and large from the social sciences.
• There is a scientific approach to social sciences. In the social sciences there is little room for the exploratory nature of theorizing – hypotheses are fixed in advance and tested against the collected data – while in the humanities theory formation is deliberately done in tandem with data collection. In the same vein, available methods from the social sciences do not accommodate the possible incompatibility of theoretical concepts with the empiricist conception of measurement and fact. In short, the presuppositions of empirical research methods borrowed from the social sciences sit badly with particular aspects of humanities scholarship.

Fuzzy evidence and incompatible conceptual schemes

• phenomenon of incompatible schemes can be encountered in any humanities discipline that harbors multiple vocabularies.
  • Eg: the difficult relation between archeological findings as physical objects and as artifacts that served a purpose in prehistoric society and culture.  Establishing this rela- tion requires us to confront the empirical archeological facts, written in a mod- ern and empiricist vocabulary, with ideas from anthropology and cultural theory, written in a language determined by the culture under investigation.
  • Similar confrontations of theoretical frameworks can also be found in the study of religion. Findings from the psychology and the sociology of religion are typically cast in the empirical vocabulary that matches the ontology of quantitative social science, while the content of those findings pertains to experiences and events that are normally recorded in a wholly different vocabulary, e.g., of magic or religion.
• With respect to fuzzy evidence, the key observation is that in the humanities there is often only a weak confirmatory relation between empirical findings and theory.
  • empirical facts are still amenable to a variety of interpretations, each associated with their own confirmatory value for the empir- ical facts at issue
  •  theories may themselves be subject to fuzziness in content, owing to the fact that the theories have not been spelled out in full detail or operate with incompatible conceptual schemes. To date, there is no comprehensive understanding of empirical confirmation in settings where evidence is fuzzy in these ways.
  • The confirmatory practice of the natural and social sciences is comparatively straight- forward, often consisting of data collection and hypothesis testing against a fixed background of theory and terminology. The humanities, by contrast, employ a wider variety of confirmatory relations, which are often set against an imprecise and contested theoretical background. Conceptual change is more frequent in the humanities than in other disciplines.

Taken from: The Humanities’ New Methods- A Reconnaissance Mission by Jan-Willem Romeijn

'Evidence in Physics' (Louise)

It seems natural to think that the main sources of evidence in Physics are data and experimental results. However, these are not the only evidence used in this discipline. In the next paragraphs, we will discuss the different types of evidence used in Physics.

Primary sources of evidence used in Physics and Astrophysics are often data from experiments. Most physicists will use results that they have experimentally found to support their claim. However, for the data to be considered as correct it is often important to be sure that the experimental results would be similar if the experiment was reproduced in similar conditions. To give a few examples, the evidence used to support the claim that light is a wave mostly come from the results from the double-slit experiment.

Another source of evidence used in physics -which is quite similar to the previous one- is scholarly articles reporting findings from experiments. Articles from famous scientific journals (such as Nature Physics), papers from renown physicists (such as Stephan Hawking's work on Blackholes) or thesis from PHD students can often be considered as evidence.

It is important to notice that most forms of evidence in Physics are only theoretical. Indeed physicists are not able to verify everything threw experiments. This is particularly true in certain areas of physics such as astrophysics or quantum physics were scientific deal with very abstract ideas. This leads to new sources of evidence that could seem quite unnatural in other disciplines.

For example, another form of evidence often used in Physics is the evidence that "nothing has shown that it is wrong". Often, physicists come to assume that something must exist in order to explain something else. This leads to new theories that are sometimes very complicated and even impossible to verify experimentally. But the fact that they seem to comply with other theories and results from previous experiments is by itself a form of evidence.

Finally, it is also important not to confuse proof and evidence in Physics. Indeed, 'proof' does not leave room for error were 'evidence' allows progress and evolution. As we can never be sure in Physics it is better to speak about evidence.

Summary of the first chapter of the book "Credible and Actionable Evidence - The Foundation for Rigorous and Influential Evaluations" by Christina A. Christie, Stewart I. Donaldson, Dr. Melvin M Mark

Chapter 1- Examining the Backbone of Contemporary Evaluation Practice Credible and Actionable Evidence

Evaluation Theory

One of these theory forms is evaluation theory, which is largely prescriptive theory that “offers a set of rules, prescriptions, prohibitions, and guiding frameworks that specify what a good or proper evaluation is and how evaluation should be done” (Alkin, 2012)

-       How to understand the nature of what we evaluate ?

-       How to assign value to programs and their performance ?

-       How to construct knowledge ?

-       How to use the knowledge generated by evaluation ?

In summary, being well-versed in contemporary theories of evaluation practice can enhance our ability to make sound choices about gathering evidence to answer key evaluation questions.

The decisions about how best to go about collecting credible and actionable evidence to answer the key evaluation questions are typically thought to be contingent on the nature of the questions to be answered and the context of the setting.

Stakeholders are provided with a wide range of choices for gathering credible and actionable evidence, which reinforces the idea that neither quantitative nor qualitative nor mixed-method designs are necessarily superior or applicable in every applied research and evaluation context

Design and Methods

Assumptions about social inquiry and scientific paradigms seem to color views about which designs and methods provide the most credible and actionable evidence.(Developped in Chapter 2 of the book)

Research on Evaluation

Research on evaluation holds great promise for advancing our understanding of how best to practice evaluation in contemporary times in general and, more specifically, how best to gather credible and actionable evidence

Debates About Credible and Actionable Evidence

How do evaluators gather credible and actionable evidence to answer the wide range of evaluation questions they face across diverse and highly variable contexts ?

What Counts as Credible Evidence in Applied Research and Evaluation Practice (Donaldson, Christie, & Mark, 2009).

The Rise and Fall of the Experimenting Society

Experimenting Society (Campbell, 1991).His vision for this utopia involved rational decision making by politicians based on hardheaded tests of bold social programs designed to improve society.

RISE : Detailed understanding of threats to validity, multiple type of validity, bias control, and the implementation of rigorous experimental and quasi-experimental designs in real-world or field settings were advanced following this theory.

FALL :

• Evidence did not yield credible evidence in a timely and useful manner,
• Evidence judged to be poor by experimental scientific standards
• Inspiring the field to develop new tools, methods, and evaluation approaches

An Evidence-Based Global Society

The promise of an evidence-based society and the accelerating demand for credible and actionable evidence has led to the recent proliferation of evidence-based discussions and applications.

For example :

• Health care and medicine – Sackett, 2000 ; Sackett, Rosenberg, Gray, & Haynes, 1996
• Mental health – Norcross Beutler, & Levant, 2005
• Management – Pfeffer & Sutton, 2006
• Executive coaching – Stober & Grant, 2006
• Career development – Preskill & Donaldson, 2008
• Public policy – Pawson, 2006)
• Education – Gersten & Hitchcock, 2009)

Movement toward evidence-based decision making now appears highly valued across the globe, multidisciplinary in scope, and supported by an ever-increasing number of practical applications.

"In God We Trust—All Others Must Have Credible Evidence"

Less agreement, even heated disagreements, about what counts as credible and actionable evidence.

Recent Debates About Evidence

1970-80sThe great quantitative–qualitative debate captured and occupied the field (see Reichhardt & Rallis, 1994)

Also known as the Paradigm wars, which seemed to quiet down a bit by the turn of the century (Mark, 2003).

2003 Donaldson and Scriven invited a diverse group of applied researchers and evaluators to provide their visions for a desired future.

While the symposium concluded with some productive discussions about embracing diversity and integration as ways to move forward, it was clear there were lingering differences and concerns about what constitutes quality applied research, evaluation, and credible evidence.

2005 Donaldson and Christie concluded, somewhat surprisingly, that Lipsey and Scriven agreed that RCTs are the best method currently available for assessing program impactand that determining program impact is a main requirement of contemporary program evaluation.

What Counts as Credible Evidence ?

While there seemed to be much support for the notion of using evidence to continually improve efficiency and effectiveness, there appeared to be growing disagreement and confusion about what constitutes sound evidence for decision making.

Claremont Graduate University “What Counts as Credible Evidence?” = full vetting of best ways to produce credible evidence from both an experimental and nonexperimental perspective was facilitated in the hope that the results would move us closer to a shared blueprint for an evidence-based global society.

Evidence in the Social Sciences

Evidence must be used by social scientists in order to understand and explain the dynamics of our economic, cultural and political lives.When claims are made four types of evidence are used to support it:

1.Anecdotal evidence:

a)Provides weak support for an argument evidence

b)Acts as a powerful counterexample for dismissing an argument

An anecdote is a specific example usually grounded in personal,secondary or incomplete evidence.Anecdotal evidence is usually found in journalism,book reviews or short essays.

2.Testimonial evidence

a)Provides moderately strong or supportive evidence

b)Can render rich empirical evidence about specific phenomena

In research concerning social sciences credible citations and sources are used.The testimony of credible experts can strengthen an argument.Testimonial evidence is usually found in magazines and journalism.

3.Statistical evidence

a)Provides moderately strong or supportive evidence

b)Summarizes,indexes or models general phenomena

In social science research this type of evidence can bring generalized and transportable knowledge about broad phenomena and trends.Statistics from different sources usually vary or even counter each other so giving reports from multiple sources is usually done.

4.Analogical evidence

a)Provides strong or supportive evidence

b)Explains either by comparison to a known phenomenon or common metaphor

Analogical evidence aims to explain a concept by comparing it to something already well known and understood.It generally offers new perspectives and they often appear in literature reviews that set up a social problem.

Irina-Summary of Phil Howard's "Types of Evidence in Social Research"

Nina:

Evidence in History:

After the second world war, historical archives were closed to the public in France. Archives are the main source of evidence in the field of history because they tend to be reliable and precise. Therefore, the lack of these important documents left France with a sense of “missing truth”. The country was divided, and each group had their own conflicting version of the truth with their own evidence. Some claimed that the French had fought during the occupation and that most of the population were part of the revolution. For example, Robert Aron, a French author argued that the Vichy government collaborated to lessen the violence of the German rule and in turn help the revolution. He presents it as a calculated risk that in turn helped most of the population. However, Robert Paxton, an American historian deconstructed this myth in his book “La France de Vichy”. Indeed, this historian had access to the archives in the USA and his version of events seemed more reliable than the political leaders. His book and theory left the French feeling uneasy, showing us the power of evidence. Indeed, this piece of evidence led to many new ones coming to light. In 1969 the documentary “Le chagrin et la pitié” shows the life in Clermont-Ferrand during the occupation. This movie describes the French as Vichy followers and mainly preoccupied with surviving and not fighting against the occupation. This movie, and valuable piece of evidence produced by Marcel Ophuls was prohibited until 1981. Another aspect of evidence during this period was the testimony of the Jewish people. However, these are extremely subjective and come directly from extremely painful memories. These testimonies were deeply emotional and holocaust survivors were extremely hesitant to share them. Indeed, they felt as though the world wasn’t accepting, especially with the “resistant” image politicians were putting forward. In addition to this, they carried a sense of guilt and were seen as victims who accepted their terrible fate. However, with the rise of holocaust deniers in the medias, survivors felt as though it was their duty to share their stories. These important pieces of evidence we’re not considered legitimate at first. For example, the Italian author, Primo Levi had a very hard time finding a publisher for his book: “If this is a man”. In 1961, the trial of Adolf Eichmann in Israel relies once again on evidence, found in multiple archives and testimonies. This highly media followed trial is followed by a string of movies documenting and explaining the precise process in the camps. For example, the movie “Shoah” directed by Claude Lanzmann comes as a choc in 1985.

Nowadays, France recognizes their collaboration and actively tries to participate in commemorations for the victims of this collaboration. Indeed, the state organizes ceremonies and sanctions holocaust deniers. In 1995 the president Jacques Chirac officialy recognizes the collaboration for the first time. In consequence, we can see how the work of historians helped appease the memory of this somber period in France and helped restore a more universal truth.

Evidence in History and Art

If we study the discipline of history, we often fail to find evidence in a verbal or written form. In this case we can consult art, objects or artifacts. This will provide us with unique cases of evidence that differ from the approaches to the concept we consider common. Art in its every form can hold various pieces of information and further evidence which is displayed through subjective as well as objective perception, visualization and interpretations. If an object’s meaning and the evidence it holds is approached artistically, certain aspects of it become understandable and significant that would not have been otherwise. We are able to sense the purpose it serves, especially if we can allocate it to a specific time period and therefore put it in context. Consequently, this allows us to study and understand historical periods in depth and in their diversity as we can compare different pieces of art which hail from the same time. Thus it is possible to distinguish different portrayals of similar or identical objects through various periods and contexts and find evidence of social changes, movements or mentalities. Art creates visual evidence of an artist’s subjective perception and intentions as well as our own. This connect historical views of a topic to contemporary ones. What do we want to see and what are we looking for? How does this relate to the artist’s intentions and the art’s original purpose? The way in which an individual approaches and interprets art reveals and characterizes their interests and their queries as well as their approach to specific topics. Through art we are able to establish why certain subjects emerge more frequently than others and why they change their appearances and relate this information to a historical context. Consequently, art enables us to understand history more deeply by giving us a subjective view which corresponds to the time era we are looking at.

Taken from: The Journal of Interdisciplinary History Vol. 17, No. 1, The Evidence of Art: Images and Meaning in History (Summer, 1986) – Published by: The MIT Press

-Camilla

Evidence in Medicine

Evidence-based medicine approaches said concept as a guidance to our-perform past treatments in terms of care of individual patients. It is defined as the integration of clinical expertise, patient's values and evidence in the decision making related to patients health care. News in the medical world grow at a fast pace and sometimes clinicians feel the pressure of accepted facts rapidly becoming old and new knowledge being published every day by medical journals. EBM uses evidence acquired from rigorous clinical research, meaning its sources are mainly research studies in the form of controlled and randomised experiments. Additionally, medical practice uses other forms of evidence such as patient's histories in relation to new diagnosis tests. Within this discipline, evidence is crucial in terms of tracking patient's progress and evaluating tried treatments and interventions.

Evidence based medicine can be simply defined as a tripartite method that englobes individual clinical expertise, best external evidence and patient values & expectations.

This method has inspired other disciplines such as education which uses the same tripartite schema to monitor teaching effectiveness and learning patterns.

Evidence and Mass surveillance

In the era of mass surveillance we live in, a considerable amount of facts and information of our acts and thoughts are monitored and saved as evidence. The record of our DNA, fingerprints, biometric passports, our daily life on CCTV and moreover our data threats dangerously our privacy, democracy and freedom.

First implemented as a mean to reduce crime and terrorism, mass surveillance has expanded to every citizen. One might now question its uses in an ethical way.

Governments retain data from their citizen and use them sometimes for coercive measures preventing crime from happening. This effectiveness and ethic of this method is undoubtedly discussable, especially regarding the very concepts of freedom and democracy.

However governments aren't the only one using our data.

Indeed, the body of our evidence is used to interpret, watch and predict our behavior, and thus, companies can make profit out of it. Supermarkets now redesign their shops based on how we walk through, to make us buy more and online companies collect our data to create commercials more likely to appeal us. Creative companies, such as Netflix, shape their productions according to the taste of their consumers, their data (when they watch a movie, when they stop, what kind of films they like, etc.) collected and analyzed. Arts are then also affected by mass surveillance, and become more and more a commercial product. It could be also used as a powerful tool by a government to spread ideas through what seems to be an enjoyable movie or song.

Mass surveillance has also a very dangerous impact on insurance companies. They, and will more and more given the progress of technology and the lack of regulation, collect our evidence to be sure we respect their contracts and suggest us tailored offers. Put into practice, this means that people could have less chances to get good insurances because of the evidence of their every day life exposed to the companies.

The concept of privacy slowly disappears from our societies and whilst whistleblowers such as Edward Snowden who revealed the global surveillance made by the NSA in 2013 continue to raise the awareness, we still agree, consciously or not, with the share of our data, even though overuse of our evidence are occurring more frequently than ever, increasing the power of governments and companies and subsequently reducing the one of citizens.

George Orwell's dystopia 1984 is becoming a reality, and it now depends on us to decide wether we want 'Big Brother' to watch us or not.

Reliability of Evidence in Criminal Investigation

The reliability of much of the forensic evidence that has been heavily relied on since the beginning of court trials and criminal investigations has been brought under scrutiny for not being as conclusive as once was thought.

Studies of wrongful convictions due to DNA exonerations have found that use of many forensic evidences to be almost as prone to leading to wrongful convictions as eyewitness errors.

Here I will look at a few of the most commonly used forensic evidence techniques and their scientific reliability as evidence in convicting a criminal.

Some forensic science techniques have already been taken out of use in criminal investigation due to their inaccuracies, such as voiceprints, bullet lead analysis and many indicators of arson.

Forensic Bite Mark Identification

· Forensic odontology is the comparison of suspected bite marks, usually in the flesh of the victims, with the dental makeup of suspects.

· The claim being that a forensic dentist can accurately associate a bite mark to the individual set of teeth that produced the bite mark found on the crime scene.

· Originally, forensic odontology was used to identify victims of natural disasters. This involved identifying from a finite number of candidates. When this technique was moved into identifying bite marks, the number of potential suspects is much higher. The medium recording the dental indents is also a lot less reliable than having the actual physical teeth, as flesh records only a small portion of the persons teeth and is not an ideal medium for recording the teeth.

· No scientific studies had been done to prove these methods as reliable in convicting people, their popularity in cases has grown from one large case (Marx) in which the teeth provided the final useful bit of evidence needed for conviction.

· The method was denounced when many doubts were researched. One being the changeability of flesh as a record of bite marks due to the elasticity of skin, unevenness of the bite surface, and swelling and healing.

· Another problem was the bias of suspects provided by the police that the bite marks were being compared to, often limited selection of people, forcing the forensic dentist to make a connection that is perhaps not there.

· The uniqueness of human dentition has also not been scientifically proven.

Hair as Evidence

· Due to limitations in identifying the source if a hair, scientists will never say that two hairs match. They are restrained to stating that a hair is 'similar to' or 'consistent with' a known hair.

· Scientists are able to conclusively say whether the material is hair or fibre, human or other animal, the racial origin, method of removal, whether the hair is chemically treated and its growth phase when removed.

· Sufficient DNA testing required to identify the owner of a hair is only possible if sufficient root tissue remains on the hair. If not, mitochondrial DNA analysis can be undergone but even this has been shown to only eliminate 10-13% of hairs that are microscopically similar.

Fingerprints as Evidence

· The interpretation of fingerprints found at a crime scene relies on the expertise of latent print examiners. The accuracy of their decisions, however has been under scrutiny due to concerns about the reliability of their decisions.

· Latent prints are the fingerprints left at the crime scene. Exemplar prints are collected from a subject in a lab under controlled conditions and are of a higher quality. These two prints are compared by latent print examiners using their expertise instead of a quantitative method.

· Latents are often unclear and distorted as well as being small and smudged, also being on complex surfaces such as overlapping with other prints or texture.

· The software Automated Fingerprint Identification System (AFIS) provides a list of candidate exemplars that are then examined. However the exemplars selected in this was are more likely to be similar to the latent than other comparison methods, increasing the chances of examiner error and making fingerprint analysis much less reliable.