Revelian is continuously looking for ways to improve our products and services to ensure our clients receive best of breed psychometric assessments and technology platforms.

In 2012 Revelian’s Research and Development team, consisting of Organisational Psychologists and Software Engineers, embarked on a project that would see a significant change in how the Revelian Cognitive Ability Test (RCAT) and our separate abilities tests for verbalnumerical an abstract reasoning are delivered. Specifically, this involved moving from fixed-form test delivery to Linear on the Fly (LOFT) to improve the candidate experience, psychometric properties, and test security.

What is LOFT?

LOFT is a well known and respected method of assessment item delivery often used in high stakes settings including education and recruitment, where test security, validity and fairness are of critical importance. Under the LOFT model, each test taker receives a unique set of items that are equivalent in terms of content variation and difficulty.

Considered to be a significant improvement to fixed-form delivery where candidates are streamed one of several predetermined parallel forms, LOFT ensures that no two candidates receive the same set of items. This heightens the security of the test and ensures that candidates do not gain from sharing item content and responses.

How it works

When a candidate logs in to begin the RCAT, 51 items are selected from a large item bank and streamed to the candidate. Both item type and item difficulty are controlled from candidate to candidate.

Item Type

The items are content balanced such that each individual receives 17 of each item type (verbal, numerical, abstract) with the order of item type being randomised. That is, the first item streamed to a candidate will be a random selection of either a verbal, numerical or abstract item. The second item will be randomly selected from the two remaining item types, with the third item therefore representing the item type not yet selected. Having now established the ordering of these 3 general item types, this same systematic ordering is retained throughout the assessment.

Item Difficulty

Each item is randomly selected from carefully constructed item groupings to ensure difficulty levels are controlled and equivalent across candidates. Similar to the fixed-form approach, item groupings are generally ordered from easiest to most difficult so that the RCAT becomes more difficult as candidates progress through the assessment.

The sophisticated controls described above ensure that while each candidate experience is unique in terms of the specific items streamed, equivalence in content and difficulty are maintained so that appropriate performance comparison across candidates is possible.
This LOFT model of item streaming is presented graphically in the figure below.

Loft Diagram Revelian

Each candidate is randomly presented with a pre-set number of questions from each difficulty level “bucket” from the item bank. Imagine this diagram multiplied by hundreds of items. Each candidate gets a random, fair and completely unique exam.

Benefits

Adopting the LOFT model of item streaming offers many benefits to Revelian clients and candidates. Four of the most significant are explained below.

  1. Heightened test security: The LOFT model of item streaming greatly enhances the security of the RCAT. Clients can be confident that our technology and method of delivering assessments remains cutting edge and that the results achieved by candidates remain valid and reliable, without the risk of assessment compromise.
  2. Unique candidate experience: The LOFT model of item streaming ensures that every candidate is streamed a unique set of items in a unique order. This allows the candidate experience to remain novel while also clearly demonstrating that no benefit can be gained by attempting to share items and answers. As a result, candidates can be confident that their score is being compared to legitimate scores obtained by other candidates. Clients have assurance that candidate scores are true representations of their cognitive ability and are not influenced by previous practice or access to assessment items.
  3. Innovative approach: The LOFT model of item streaming ensures that Revelian remains innovative in its development and maintenance of the RCAT, and the psychometric properties of this assessment continue to exceed industry standards.
  4. Comparable to previous model: The LOFT model of item streaming remains completely comparable to the previous fixed-form approach. Extensive effort has been undertaken to ensure the transition to this new approach does not impact RCAT scoring, reporting and interpretation. As such no changes are necessary to your understanding and application of the RCAT, nor is any additional training required. The candidate experience also remains unchanged, with the same user-friendly functionality, clear instructions, example questions and comprehensive support options.