Skip to Main Content
It looks like you're using Internet Explorer 11 or older. This website works best with modern browsers such as the latest versions of Chrome, Firefox, Safari, and Edge. If you continue with this browser, you may see unexpected results.

Evidence-Based Decision Making: 6As Framework

This guide is intended to provide background, information, and resources on the process and framework of Evidence-Based Decision Making (EBDM)

Streaming Video

Go directly to one of these relevant NSCC Streaming Video Collections. Or watch some selected videos (right).

Off Campus Access

Working from home or from the beach? No problem! To access some of the databases off campus, you may be asked to login. Just sign in using your W # and PASSWORD.

Ask - ask an answerable question


It is important when making a decision that we translate our issue or problem into an answerable (focused) question.  One tool to use when doing this is PICOC Problem, Intervention (or success factor), Comparison, Outcome/Objectives, Context

PICOC Example

Two departments in your organization are merging.  However, it is well known that the culture within the two departments are very different and you are worried that these cultural differences might impede a successful merger and impact outcomes moving forward.  While you understand that department cultural differences matter you want some evidence-based advice on how to move forward.

QUESTION: Does a difference in departmental culture affect successful integration of two unique departments?


Recommended Reading, Watching, & Tools

Nutt, P. C. (1999). Surprising but true: Half the decisions in organizations fail. The Academy of Management Executive, 13(4), 75-90. (PDF)

CEBMa's Ask Manager App (app for iPhone of Android)

Acquire information

 

Once you have determined your question you then need to find information - this can include:

Practitioners - The professional experience and judgment of practitioners in the field.  This might include managers, consultants, business leaders and other practitioners.  Unlike intuition, opinion or belief, professional experience is accumulated over time through reflection on outcomes of similar actions taken in similar situations.  Professional experience refelects specialized knowledge acquired by repeated experience and practice. (Barends & Rousseau, 2018, p. 6)

 

The scientific literature - Findings from research studies published in academic journals. Many of us learn about new research on professional courses, at conferneces, and through professional organizations; however, new research is always being produced and in order to include up-to-date evidence from the research in your decisions it is essential to know how to search for studies and how to be able to judge how trustworthy and relevant they are (Barends & Rousseau, 2018, p. 8).  For assistance in searching and evaluating up-to-date research contact your NSCC Campus Library.

 

The organization - Data, facts and figures gathered from the organization (i.e., incoming student survey).  Organizational evidence can come in many forms: financial data, business measures, employees, organizational data, etc.  It is evidence from the organization that is essential to identifying problems that require attention - it is also essential to determine likely causes, possible solutions, and what is needed to implement solutions. (Barends & Rousseau, 2018, p. 9).
 

Stakeholders - The values and concerns of people who may be affected by the decision (i.e., NS Apprenticeship, students, etc.)  (Barends & Rousseau, 2018, p.6)  Stakeholders are any individuals or groups who may be impacted by an organization's decisions and their consequences.  Internal stakeholders include employees, managers,and board members.  External stakeholders can include suppliers, students, the government and the public at large. Gathering evidence from stakeholders isn't just important for ethical reasons, understanding stakeholder values and concerns can provide a frame of reference from which to analyse evidence from other sources (Barends & Rousseau, 2018, p. 10).

QUESTION: Does a difference in departmental culture affect successful integration of two unique departments?


Recommended Reading, Watching, & Tools

Appraise

 

It is important to critically appraise evidence in order to judge the trustworthiness and relevance of the evidence. Evidence is never perfect and can be misleading in different ways - evidence can be over stated so that what appears to be a strong claim turns out to be based on a single, not particularly reliable piece of evidence. (Barends, Rousseau, & Briner, 2014, p. 12)

It is common in organizations that long-standing ways of doing things may never have been evaluated to see whether it works or not (i.e., "it's how we do it here").  All evidence needs to be critically appraised and systematically assessed for trustworthiness and relevance.


Recommended Reading, Watching, & Tools

Evidence Hierarchy


Mody, N. (2017). Questioning the evidence-based pyramid. Retrieved from https://acrlog.org/2017/12/20/questioning-the-evidence-based-pyramid/

 

CAT Critical Appraisal App

Critical Appraisal App

Here you can find more about the app we have developed to help managers and consultants critically appraise the trustworthiness of scientific studies published in academic journals.

Aggregate

 

After completing your appraisal of the information found you must then aggregate it - weigh and pull together the evidence. In almost any situation it is possible to gather many different types of evidence from different sources.  You need to determine what evidence to pay more attention to and why? (Barends et al, 2014).  You will have information from the research literature, organizational data, stakeholder needs, as well as practitioners/experts within the organization.  At this stage you need to weigh the needs and abilities of the organization with your best evidence to determine the best possible outcome. 

In some cases you there may be no evidence found in the research literature or the organization so you will have no option but to make your decision based on professional expertise of colleagues or pilot different approaches to see what works best. (Barends et al, 2014). A current example of this is happening in educational institutions across Canada - due to COVID 19 restrictions we are teaching virtually (often with tools and software we didn't know existed prior to March 2020).  In this case we don't have the option of referring to research literature on how to best proceed - we are using our in-house expertise (online learning designers, curriculum designers, educational technology specialists) and organizational abilities to move forward.

Within the concept of evidence-based (medicine, management, nursing, etc.) we sometimes are limited in our ability to use the "best evidence" within our organization.  The best medication to treat disease X may cost $10,000 a pill - putting it out of reach of most patients; likewise, providing every NSCC student across the province with high speed internet and a computer may have met the needs of our online transition, but within our structure, budget and capabilities it wasn't possible.

In order to make the best decision within our organization/department we may not be able to use the "best evidence" - by aggregating our evidence within the scope of our organization we can make the best decision based on the best evidence available to us.

 

 

The next step in the evidence process is to apply the evidence that we found into our decision making process - after asking, acquiring, appraising and aggregating we need to ask if the evidence applies to our organization.  Is the evidence generalizable to our context - this is especially important if it comes from people outside the organization or from external sources such as research literature. (Barends & Rousseau, 2018)


Recommended Reading, Watching, & Tools

 

Assess

 

 

The final step in the EBDM process is to assess -evaluate the outcome of the decision taken.  This is a step that is often missed within organizations - without assessment we are unable to fully determine if we were successful.

 


Recommended Reading, Watching, & Tools

Evaluation Toolkits

Centre for Innovation in Campus Mental Health

Ontario Centre of Excellence for Child and Youth Mental Health: Program Evaluation Toolkit

MEERA (My Environmental Education Evaluation Resource Assistant)

Better Evaluation: Sharing Information to Improve Education

Article

Postsecondary Education Metrics for the 21st Century

http://www.heqco.ca/en-ca/Research/ResPub/Pages/Postsecondary-Education-Metrics-for-the-21st-Century-.aspx

Best Practices in Student Persistence and Completion: A Program Evaluation of Three Student Services Units In a Rural Community College  

https://irl.umsl.edu/cgi/viewcontent.cgi?article=1073&context=dissertation

Videos

From the Ontario Centre of Excellence for Child and Youth Mental Health - Evaluation = Evidence

https://www.cymh.ca/Modules/ResourceHub/?id=686bc8ab-3301-4ad9-ac37-e3280bed6c0e  (4 ½ minutes)

Types of program evaluation and evaluation design

https://www.youtube.com/watch?v=XNR3KZzDoGQ  (6 minutes)

Types of Evaluation

https://cyfar.org/different-types-evaluation   (7 minutes)

 

Websites

Canadian Evaluation Society - https://evaluationcanada.ca/about-us

Campus Intelligence: Reimaging Data in Higher ED -   From Good Intentions to Evidence-Based Success: A Better Way for Student Affairs

 

Book

Wise, V., Davenport, Z. R., Zhang, N. (2019). Student affairs assessment, evaluation, and research : A guidebook for graduate students and new professionals (American series in student affairs practice and professional identity ; 1). Springfield, Illinois: Charles C Thomas, Publisher.