新永資訊有限公司

Facets 性能評價分析軟體

Facets is designed to handle really tough applications of unidimensional Rasch measurement. It constructs measures from complex data involving heterogeneous combinations of examinees, items, tasks, judges along with further measurement and structural facets. It is designed to handle flexibly combinations of items of different formats in one analysis. Item types include dichotomies, rating scales with up to 255 categories, Poisson counts and Bernoulli trials. Multiple different measurement models can be included in the same analysis, including paired-comparisons, rank- order, rating scales, partial credit and dichotomizations involving from 1 to 255 facets. Measures can also be fixed (anchored) individually or by group mean, facilitating equating and linking across test sessions. Quality-control fit evaluation of all measures is provided. Unexpected data points are identified. Bias, differential item functioning and interactions can be measured. Weighting schemes can be implemented. Up to 1 million examinees etc. can be included in one analysis. Typical applications have over 90% missing data. Since organizing the data for input to Facets and then interpreting its output can be challenging, it is recommended that simpler approaches be tried first. Facets has been used successfully to construct measures for medical staff performance, patient performance, public speaking, sports performance by individuals and teams, and consumer preferences. Training seminars are held regularly.

Description of FACETS program

This program extends the objective measurement principles of Georg Rasch beyond conventional tests and attitude surveys into more complex areas such as judged performances, and sub-tests of items replicated across tasks. Facets has already been applied in many areas including Drama Competition, Laboratory Technician Certification, Handicapped Patient Assessment, Stain Resistance of Paints, Public Speaking Competence, Graduate-School Candidate Selection, Teacher Assessment and Essay Grading.

Judged tests can be analyzed: Facets can analyze the ratings awarded by judges to examinees who are being tested on their performance on a number of different skill items for each of several tasks. The choice of judging design is yours: from complete designs in which every judge rates every examinee, to minimal designs in which any judge only rates a few of the examinees on a few of the items. The only constraint is that there be enough overlap or rotation in the judging design to enable each parameter to be uniquely positioned within one overall frame of reference.

Heterogeneous tests can be analyzed together: Not only can Facets perform analyses as diverse as dichotomous responses and judge- awarded ratings, but Facets can process them together simultaneously in one analysis to estimate one ability measure for each examinee.

Mathematical flexibility: The analyst specifies to Facets the number of facets in the data, and, for each facet, its name and the names of all the parameters included in the facet. The analyst also specifies the form of the measurement model which represents the way in which the facets interacted to produce the observations, and which observations apply to a given model, so that multiple measurement models can be used simultaneously in one analysis. For a complex judged test, the measurement model could be equivalent to:

where Bn = ability of examinee n, Am = difficulty of task m, Di = difficulty of skill item i, Cj = severity of judge J, Fk = difficulty of category k relative to category k-1, Pnmijk = probability of rating of k under these circumstances, Pnmijk-1 = probability of rating of k-1

• Input data: Facets can process up to 255 facets, containing millions of elements in total, and millions of individual observations. The structure of the data file, though simple, usually requires the analyst to reformat the raw data file before analysis. A utility program Facform is provided, which will usually accomplish this for small data files.

• Elements can be grouped within facets so that summary statistics can be produced by group.

• Measures can be pre-set by element or by group to facilitate equating. Rating scale calibrations can also be pre-set. Results of earlier analyses can also be used as starting values for later analyses.

• Secondary analysis can be performed to identify interaction (bias) between pairs of elements. Location, size and significance of interactions are reported, e.g., interaction (bias) between individual judges and individual items.

• Rating scales can be defined as applying to all elements, or scales can be specified to be particular to each item, or each judge or any element of any facet, up to 255 distinct scales, with up to 255 categories per scale..

• Individual misfitting observations are reported. No need to edit the data file in order to eliminate misfitting observations from further analysis.

• Weighting of observations allows examination board to determine the contribution of each sub-test to the estimate of person measure.

• Data formats include: dichotomous, rating scales, partial credit, Bernoulli trial (e.g. number of hits on target) and Poisson count (e.g. number of spelling mistakes in a long essay).

• Windows runs under Windows-XP/98/2000/NT etc., but not 95 nor 3.1. Windows version analyzes millions of people, item, judges. DOS version analyses size up to 32,000 people, items and judges combined.

• Facets is not 508 compliant.