Frequently Asked Questions
Is there a written or oral component to the exams?
There are no other components of the exam except for ESL which has the listening test.
The large number of students selected for the initial item calibration process were carefully chosen to represent the spectrum of ability levels from the novice to the typical, competent third semester college student. The results of the exam were found to correlate closely with the ability level determined by other means, such as standardized written tests and oral proficiency interviews.
Periodically the exam results are reevaluated by giving the test to BYU students taking classes at the various levels, and results of those trials show the test results remained indicative of placement level.
Can I set my own calibration levels?
Schools are encouraged and expected to calibrate the cutoff scores in the placement table to correlate with each one’s combination of curriculum, staff, and typical incoming student ability. After any calibration process, please send us a total of the tests used so that we can deduct them from your invoice.
The calibration procedure starts by giving the exam to some students who have just completed the language courses into which you want to place incoming students. It is best if ten or more students for each course are tested, although fewer may be used if the results are pooled over multiple semesters. It is also better, when possible, to select students who have taken the courses from different instructors.
The scores for these tests are then grouped and ordered by course level. The median score for each group is found, and the cutoff scores are set half-way between them. The high end cutoff score may be approximated from the results of the highest course tested, or, with a large enough sample, may be set between one and two standard deviations (depending on circumstances) above the average score for that course.
After you have determined the cutoff scores go to http://webcape.byuhtrsc.org/admin/nwc-admin-login.php and log in. Once there, click on Licenses then the language you are adjusting.
Occasionally you may wish to reevaluate and possibly change the cutoff scores by repeating the calibration process. This is done to account for shifts in typical incoming student ability, and for changes in teaching staff or curriculum.
Exam Accessibility for those with Disabilities
WebCAPE is currently not compatible with most screen readers at this time. It is expected for for the institution to make accommodations for the student via other testing methods, just as is done for written tests.
Student exam retakes / result access
Students can retake the exam with exactly the same name, ID and email as a previous attempt by pressing the ‘Continue’ button at the bottom of the registration page. The result of both exams will be recorded and shown on the result page.
If they press the ‘Resume’ button instead, then they will see the results of the most recent completed attempt. If they have not completed the exam and press the ‘Resume’ button, then they will be given the option of continuing the exam in progress or starting a new attempt, either with different information or restarting with the same information.
We recommend establishing and publishing a policy for retakes that makes sense for your situation. Using the highest, lowest, average, first or last score are all viable approaches.
The default note on the student result report page suggests that a score near a cutoff could mean that an other attempt be made. This is why multiple attempts are allowed.
With the external interface it is possible to authenticate students using the ID number, which uniquely identifies a student. Then one can deal with multiple attempts however one wants to do so.
Can I review the exam questions? Can I add my own?
BYU spent over 5 years and a great deal of research to develop and accurately calibrate the weight of each question. These questions and their weight are the value behind the exams. Therefore we cannot allow anyone to see the set of questions. Should the information inadvertently be leaked, the validity of the test would be compromised, which would be to the detriment of everyone who relies on the test for accurate placement.
There are many hundreds of possible questions for each language, and each question was exhaustively examined by a panel of language teaching and testing experts. Then each question’s performance was analyzed by statistical methods in several trials. Still, from time to time a question is removed when it is found that it is not performing up to expectations. Be sure to let us know if you have concerns about a specific question so we can look at it more closely.
Because of this, it is also not possible to modify and/or extend the exam questions. To do so without the proper rigorous testing would decrease the accuracy and validity of the exams.
How long does the test take?
Due to the adaptive nature of the test the time will vary. On average most students take 20 – 25 minutes to complete the test.
If most of the first several questions asked are answered right or wrong, the test length is much less, usually around 5 minutes. This is because these questions are used to determine a starting ability level. The test is not designed to to accurately place too low or high of ability, only the first THREE semesters OR HIGHER. There simply aren’t enough items calibrated beyond the first three semester placements to provide consistent evaluation among the higher levels.
It is possible for students of uneven ability to take longer than average. The problem is that uneven ability causes the system to thrash. In other words, the system tries to zero in on a score, but then, just as it is settling in on an ability level estimate, the student gives an answer that sends the system seeking a new target.
At this time the system ends the exam under two conditions: when confidence in the estimated ability level is high enough, or else when all questions at the estimated ability level are exhausted. Adding a time or question limit would reduce the accuracy level of the exams.
ESL reading and listening usually take longer due to the large amount of material that the student must read or listen to. If their ability is uneven, expect much longer times.
One last note. Since professors are experts with very high ability, they will always be able to finish a test fast (even when answering incorrectly). To get a better feel for the adaptive nature, try to answer the questions at the ability level of a 1st or 2nd year student.
How can I limit access to specific computers?
Access to the WebCAPE exams for your school can be restricted to computers with specific public IP addresses or ranges of addresses you supply to us. Other computers either may be denied access or require a password. There is an extra charge for this type of setup.
Also, keep in mind students do not know the link, nor are they familiar with the fact that the exams are available on the web. You are also able to change the exam password as frequently as you’d like.
What if I want to use the exam, keep our own data?
There is an interface that allows you to program your own student authentication, data collection, and result management system on your own servers. Only the exam itself comes from our server. Link: http://webcape.byuhtrsc.org/webcape-eei.html
Also, UNC has done some amazing work integrating the exams with their system. They wanted us to share a link to their blog that gives an overview of what they did. http://mindofaninnovator.com/online-language-placement-test/
How do I access the exam results?
Exam results can be browsed on-line or retrieved automatically in bulk.
The result management web page allows recent exams to be selected by entering a date. The results can be sorted by date, name, score, exam language, or a user specified status. Details of the exam, such as survey answers, recomended placement, and summary information about the questions seen can be accessed by selecting one of the students listed. The basic link for results is: http://webcape.byuhtrsc.org/result You will need your accountID and password.
Also, once a list of students has been selected, the corresponding result details can be exported in plain text or in a form suitable for spreadsheets and databases.
It is also fairly simple to have a computer program you create (or have created) retrieve exam results automatically and process them according to your needs. A link explaining that retrieval is: http://webcape.byuhtrsc.org/nwcresulthelp.html#autoexport
Is there a minimum score or maximum?
Since there are nearly a thousand questions for each language, and the questions for a given exam are chosen randomly, it’s difficult to nail down the maximum score. What we can do is give a list of the maximum score received for each exam language so far:
(As of 2006)
Spanish Exams Given: 302,000 Top Score 956
French Exams Given: 65,499 Top Score 766
German Exams Given: 25,871 Top Score 948
It’s too soon to tell for the ESL and Russian exams.
Note that these scores include exams taken by faculty, and so are probably very close to the ‘actual’ maximum.
**Note — On rare occasion we have seen a score get a negative result close to zero. This is basically a zero. With the algorithm and adaptive nature of the exam there is a possibility of this happening.