08 September 2003

The Policeman's Beard is Half-Constructed

Prof. Felten’s Freedom to Tinker href=“http://www.freedom-to-tinker.com/archives/000440.html”>on
computerized essay grading software:

Another possibility … is that there is something simple
— I’ll call it the X-factor — about an essay’s language or structure
that happens to correlate very well with good writing. If this is
true, then a computer program that looks only for the X-factor will
give “accurate” grades that correlate well with the grades assigned by
a human reader who actually understands the essays. [...]
game will be up as soon as students and their counselors figure out
what the X-factor is and how to maximize it. Then the SAT-prep
companies will teach students how to crank out X-factor-maximizing
essays, in some horrendous stilted writing style that only a
computerized grader could love.

I’m wondering how the href=“http://barillari.org/blog/academics/princeton/administration/odus/reports.html”>Committee
on Discipline would respond if a student used a computer program
to generate essays designed to earn high marks from essay-grading
software. (I wouldn’t expect Princeton to use software, but if the
University ran out of TAs —- perhaps due to href=“http://barillari.org/blog/issues/immigration/wu_jie.html”>immigration
rulings —- software might become more attractive.)

I could write a program which takes a topic as input. It could visit
JStor, find related papers in a
keyword search, chop out quotes (properly cited, of course),
intersperse them with nonsense sentences drawn from SAT word lists,
transitional operators like “because,” and topic-specific vocabulary
(drawn from the set of words in the papers that occurred a few
standard deviations more often than background vocabulary). I could
then submit the essay a few minutes later, without actually writing a
single word of it. Would this run afoul of the University’s
guidelines on original work?

Unfamiliar with the title? See href=“http://www.robotwisdom.com/ai/racterfaq.html”>this page.

No comments:

Post a Comment

About Me

blog at barillari dot org Older posts at http://barillari.org/blog