Project 11: Lexical decoding of speech using sub-phonemic features
This project addresses the lexical coding and decoding of FPD by means of probabilistic subphonemic feature representations that are automatically derived from the speech signal. Subphonemic feature vectors for each 10 millisecond interval in the unfolding speech signal provide an excellent window on the fine phonetic detail across many acoustic and articulatory dimensions. The main aim is to improve computational modelling of HSP by using key techniques from ASR. The starting point will be existing models of ASR and HSP, such as the conventional HMM-based ASR models (Nijmegen), exemplar-based models (Leuven & Sheffield), and the ASR-based model of HSP called SpeM (Nijmegen).
For three related issues (see below) we will first carry out a corpus-based survey, followed by psycholinguistic experiments on production and comprehension. This work is interdisciplinary and also addresses fragmentation within linguistic phonetics, and within computational modelling. It will be carried out in collaboration with Cambridge, Leuven and Sheffield.
Methods We will use subphonemic feature vectors to address the extent to which FPD mediates lexical competition in speech comprehension and language production. Recent studies show that listeners are highly sensitive to FPD present in laboratory speech, and phonetic analyses of speech corpora have shown that the FPD of acoustic realisations in speech corpora bear the traces of lexical competition in the speakers’ mental lexicons. Almost all studies focused on durational differences. The sequences of probabilistic feature vectors representing speech tokens provide a unique opportunity to investigate a much broader palette of potential differences in FPD, ranging from place and manner to nasality and intensity of frication. This will be applied e.g. to (1) the FPD found for the Dutch singular-plural system (morphology) and (2) temporal alignment of features of prosodic information, in particular the intonation contour, with subphonemic detail at the segmental level, with special emphasis on lexical disambiguation during comprehension, which is a completely uncharted domain of inquiry.
Year 1 will involve corpus-based research on acoustic encoding, focused on FPD in lexical competition in Dutch and English, in close collaboration with Cambridge and the Nijmegen ESR working on Project 2. Beginning also in Year 1, we will carry out comprehension and production experiments to assess the role of FPD (including prosodic alignment) in mediating lexical competition, and in the singular/plural system, in Dutch and English. In Year 2, we will focus on computational modelling for ASR and HSP, especially comparing lazy and greedy learning algorithms to evaluate performance that uses more (‘episodic’) or less (‘abstract’) signal information.
Young researchers One ER (Gubian) is based at Nijmegen. He will probably visit (i) Cambridge, to study FPD in morphology; (ii) Cambridge or Sheffield, to study prelexical representation; (iii) Leuven, to learn about alternative modelling techniques.
Links Project 11 links to Theme I (especially Projects 2 & 3), and to Projects 8, 9 and 10. This work links computational modelling with phonetics and morphology; and addresses fragmentation within linguistic phonetics, and within computational modelling.
Working on this project: » Dr Louis ten Bosch » Dr Helmer Strik » Prof Roger Moore » Dr Odette Scharenborg » Dr Michele Gubian
