OCMT
OCMT has been designed over 10 years of research and deliberation by all 24 Colleges in Ontario as a Math Assessment to increase student success and retention. The diagnostic assessment is used to identify strengths and areas of improvement in math skills. Upon completing the diagnostic assessment, students upgrade their skills using highly interactive mastery-based modules that provide them with feedback throughout the learning experience. The summative assessment is used to place students in their appropriate math course at the institution.
OCMT was developed by Vretta through collaborative efforts between content and research experts from all 24 colleges and schools in Ontario in partnership with the Ministry of Education and the Ministry of Advanced Education and Skills Development. To view the information brochure, click here.
Through the research conducted by the CSAP, a custom framework was developed as the foundation for both the assessment items and the remedial modules. The framework comprises of two equally important dimensions which form the basis for test item validity analysis:
Content Topics and Sub-topics
To ensure that the Assessment for Learning program is custom tailored for the specific requirement of the math course, mathematical topics and sub-topics are identified to form the content dimension of the assessment framework. The framework can be expanded or adapted to suit the needs of a particular college or program to include modules which focus specifically on concepts for business, technology, or programs such as health, and the skilled trades, including topics such as geometry and trigonometry.
Performance Expectation
The items and remedial modules were developed corresponding to the broader goals of mathematics teaching and learning: (A) Knowing (B) Applying (C) Reasoning.
The framework was reviewed by an advisory committee made up of College Mathematics faculty and secondary school Mathematics consultants.
The framework formed the basis for classifying assessment items which had been submitted by college faculty, eliminating duplicates, enhancing quality, and the filling of blank cells in the matrix through creating new items as required.
Item Types
Both constructed response (written) and selected response (multiple choice) items are included. The former provides a more reliable estimate of student ability than the latter because guessing is eliminated and the use of a computerized format enables a range of acceptable responses to be established for each item.
Language
The item database was fully translated/adapted into French from its original English format.
Variants
Variants of many items were developed by Vretta’s proprietary algorithm and then selected on the basis of realism and ease of computation and (after field testing) their psychometric properties.
The items were then pre-tested in secondary schools, in which every item was responded to by between 20 and 100 students and the results given detailed psychometric analysis using both classical and IRT (item response theory) methods. At the same time, each item was reviewed both by the teachers whose classes were used for pre-testing, by the CSAP advisory committee, by officials at EQAO and at the Ministry of Education.
Following these analyses, each item was either eliminated, modified or retained unchanged. This item database was then used as the basis for field trials of the overall assessments.
It is critically important to note that the OCMT is more than just the test item and remedial module database itself. Of equal importance is the test design whereby the items are combined into individual tests. This design ensures that all tests are:
- unique for every test taker;
- equally representative of both content and performance expectation dimensions of the assessment framework;
- long enough (i.e. a sufficient number of test items) to enable reliable estimates of test takers’ abilities but short enough to complete in a reasonable length of time.
Read more on the test design here (page 19 and 20).
The interactive, voice-enabled remedial modules have been customized and aligned to the learning objectives of the framework. They provide students with the following learning experiences:
Mastery-based Learning
Topics are broken down to micro-steps for students to visualize and conceptualize and engage with mathematics.
Various types of interactive practice scenarios with feedback mechanisms help students master concepts.
Individualized Learning
Students can progress through the modules and master concepts at their own pace.
Differentiated Learning
Provides teachers with the ability to use the modules in class, or assign them as homework, to enrich the learning experience.
