In order to assess (measure) something, we need to know what we want to measure and how to measure it. How do we measure learning? How do we measure competency? I would like to define learning as “changes in behaviour that result from experience.”
To assess learning, we need to know how to measure the changes in behaviour. In our vocational education and training environment, the description of the standards “behaviour” expected from a competent individual, is described by Units of Competency, the building blocks of Nationally Recognised Training.
Units of competency describe the different outcomes that individuals need to be able to achieve through a structured list of “Elements”, and the different conditions that apply to each outcome (performance criteria). The outcomes and the performance criteria are not enough to determine the “standard behaviour expected”, to create an efficient assessment system. In our VET system, trainers and assessors are required to unpack (interpret) the unit of competency in consultation with the industry, because the unit of competency doesn’t include information about the specific context and conditions where individuals will apply the skills and knowledge.
Research and compliance statistics, shows that a majority of training providers don’t tend to find out if the training had an impact on what people are doing on the job or in the real world. Rather, they tend to only assess what they can recall from the training. That means that we are not regularly doing what most professionals do: assess their work, because the objective of our work is to prepare learners for employment or further studies, and we must assess whether that is the case.
Different Ways to Assess Learning
Testing with recall questions about the content really isn’t much of an assessment at all. In a real-life assessment, we want to know if a person can use the skills and knowledge to perform a job task up to the industry standard requirements. In other words, we want to know if learners:
- have developed the required skills,
- have developed the required knowledge, and
- are able to apply the required skills and knowledge at the right time and within the right context, using the right tools, following the right procedures, protocols, etc.
In other words, assessing behavioural changes!
Direct and Indirect evidence
We make judgements (a conclusion made on the basis of evidence and reasoning) about competency by collecting direct or indirect assessment evidence.
Direct evidence includes samples of what people do, which tends to be the strongest evidence, and therefore the basis of a competency-based assessment judgement. The assessor can see the candidate can perform the task in a real-life or simulated scenario.
Indirect evidence includes monitoring and perceptions. These can come from supervisors, peers, customers, and others. Indirect evidence is less strong than direct evidence because it’s more subjective, and the “third person” collecting indirect evidence may not necessarily use the same assessment criteria.
An ideal assessment might combine direct and indirect measures, what we call a “triangulation” of assessment evidence.
Measuring competency is difficult because we have to first agree on what exactly we are measuring. And we could argue that the process of unpacking units of competency has been inconsistently used by trainers and assessors, producing despair definitions of learning for the same benchmark!
Triangulating data (by looking at direct and indirect evidence) is often worthwhile. It increases the chance that you are getting a decent picture of the candidate’s performance in a real context.
Assessment practices is the biggest challenge that training organisations are facing, and a critical factor that underpins the credibility of our vocational education and training system. Recently the Department of Education and Training presented a discussion papers about the “Quality of assessment in vocational education and training”. Insources is organising the first VET Assessment Practices Summit in Sydney (28-29 April) Reserve your seat now! This event will provide thought provoking presentations, panel discussions, workshops, group work activities and networking opportunities for participants.
