Spoken language processing: People versus machines

A fondamental challenge we must meet for computers to eventually process spoken language as effectively as humans is to capture the immensely rich fund of information we have in our heads that is NOT in the speech signal. This information is what gives us the ability to supply acoustic cues when these are degraded or missing, or to zero in on one speaker amid a chorus of other voices. While the powerful statistical methods currently used in speech recognition and synthesis have brought some success and useful applications, future progress will depend crucially on a deeper knowledge and greater use of this information. Some of this information is applicable to all languages, and some of it is specific to individual language types. In my discussion, special attention will be given to the processing of spoken Chinese.