International Journal of Pattern Recognition and Artificial Intelligence

In this paper, a minimum error rate pattern recognition approach to speech recognition is studied with particular emphasis on the speech recognizer designs based on hidden Markov models (HMMs) and Viterbi decoding. This approach differs from the traditional maximum likelihood based approach in that the objective of the recognition error rate minimization is established through a specially designed loss function, and is not based on the assumptions made about the speech generation process. Various theoretical and practical issues concerning this minimum error rate pattern recognition approach in speech recognition are investigated. The formulation and the algorithmic structures of several minimum error rate training algorithms for an HMM-based speech recognizer are discussed. The tree-trellis based N-best decoding method and a robust speech recognition scheme based on the combined string models are described. This approach can be applied to large vocabulary, continuous speech recognition tasks and to speech recognizers using word or subword based speech recognition units. Various experimental results have shown that significant error rate reduction can be achieved through the proposed approach.

This paper gives an overview of a research system for phoneme based, large vocabulary continuous speech recognition. The system to be described has been applied to the SPICOS task, the DARPA RM task and a 12000 word dictation task. Experimental results for these three tasks will be presented. Like many other systems, the recognition architecture is based on an integrated statistical approach. In this paper, we describe the characteristic features of the system as opposed to other systems: (1) The Viterbi criterion is consistently applied both in training and testing. (2) Continuous mixture densities are used without any tying or smoothing; this approach can be viewed as a sort of ‘statistical template matching’. (3) Time-synchronous beam search is used consistently throughout all tasks; extensions using a tree organization of the vocabulary and phoneme lookahead are presented so that a 12000 word task can be handled.

This paper describes a Continuous Speech Understanding System that allows information services to be accessed through the telephone line. It accepts queries within a restricted semantic domain, expressed in free but syntactically correct natural language, with a lexicon of the order of 800 words. In the implementation here described, a user can access an electronic mailbox or a train information service through a PABX telephone line.

The architecture of the system is based on two main modules that represent and use different knowledge sources. A speaker independent recognition module generates, for each utterance, a lattice of word hypotheses which is the interface to an understanding module that performs the syntactic and semantic analysis. The recognition module is based on Hidden Markov Models of subword units, and performs the acoustic decoding process according to a beam search strategy. The understanding module finds the most likely sequence of words and represents its meaning in a format which facilitates the access to a database. It makes use of a modified caseframe analysis guided by the word hypotheses scores.

Experiments were performed with 600 sentences from 10 speakers on the E-Mail application task. Using 15 Gaussian mixtures per state, a word accuracy of 75.7 was obtained with a test vocabulary of 787 words and no linguistic constraints. Linguistic processing of the corresponding lattices achieved a sentence understanding rate of 82%.

Speech-to-text conversion of French necessitates that both the acoustic level recognition and language modeling be tailored to the French language. Work in this area was initiated at LIMSI over 10 years ago. In this paper a summary of the ongoing research in this direction is presented. Included are studies on distributional properties of French text materials; problems specific to speech-to-text conversion particular of French; studies in phoneme-to-grapheme conversion for continuous, error-free phonemic strings; past work on isolated-word speech-to-text conversion; and more recent work on continuous-speech, speech-to-text conversion. Also demonstrated is the use of phone recognition for both language and speaker identification.

The continuous speech-to-text conversion for French is based on a speaker-independent, vocabulary-independent recognizer. In this paper phone recognition and word recognition results are reported evaluating this recognizer on read speech taken from the BREF corpus. The recognizer was trained on over 4 hours of speech from 57 speakers, and tested on sentences from an independent set of 19 speakers. A phone accuracy of 78.7% was obtained using a set of 35 phones. The word accuracy was 88% for a 1139 word lexicon and 86% for a 2716 word lexicon, with a word pair grammar with respective perplexities of 100 and 160. Using a bigram grammar, word accuracies of 85.5% and 81.7% were obtained with 5 K and 20 K word vocabularies, with respective perplexities of 122 and 205.

A major step in the process of speech understanding is the acoustic-phonetic decoding which can be defined as the automatic mapping of the continuous speech wave into a set of predetermined linguistic units such as phones, diphones, syllables, etc. This paper relates to the approach of this problem which consists in exploiting an explicit description of all kinds of available knowledge about the speech communication phenomena, in the general framework of an artificial intelligence knowledge-based system.

We will first recall the main difficulties of acoustic-phonetic decoding with a practical example. We will then present the APHODEX system that we have been designing for the past eight years, in terms of software architecture and of knowledge representation and reasoning. The practical evaluation of this system will then be carried out at the different levels of feature extraction, segmentation and labelling. Finally, we will discuss the limitations of our approach and present the ongoing effort to overcome these limitations, especially through the use of abductive reasoning.

The design of current acoustic-phonetic decoders for a specific language involves the selection of an adequate set of sublexical units, and a choice of the mathematical framework for modelling the corresponding units.

In this work, the baseline chosen for continuous Spanish speech consists of 23 sublexical units that roughly correspond to the 24 Spanish phonemes. The process of selection of such a baseline was based on language phonetic criteria and some experiments with an available speech corpora.

On the other hand, two types of models were chosen for this work, conventional Hidden Markov Models and Inferred Stochastic Regular Grammars. With these two choices we could compare classical Hidden Markov modelling where the structure of a unit-model is deductively supplied, with Grammatical Inference modelling where the baseforms of model-units are automatically generated from training samples.

The best speaker-independent phone recognition rate was 64% for the first type of modelling, and 66% for the second type.

This paper describes a phonetic typewriter and a dictation machine that utilize the underlying statistical structure of phoneme or character sequences. The approach of using syllable or character trigrams is applied to language source modeling. The language source models are obtained by calculating trigram probabilities from a large text database. These models are combined with the HMM-LR continuous speech recognition system.^3,6 The phonetic typewriter is tested using 274 phrases uttered by one male speaker. The syllable source model achieves a 94.9% phoneme recognition rate with the test-set phoneme perplexity of 3.9. Without the syllable source model, the phoneme recognition rate is only 73.2%. A trigram model based on characters is also evaluated. This character source model can reduce the syllable perplexity significantly to 7.7, compared with 10.5 of the syllable source model. The character source model achieves a 78.5% character transcription rate for the 274 phrase utterances. The experimental results show that a syllable source model and a character source model are very effective for realizing a Japanese dictation machine.

This paper describes an overview of a continuous speech recognition system composed of an acoustic processor and a linguistic processor. The system deals with 843 conceptual words and 431 functional words. We have constructed an acoustic processor using a modified learning vector quantization method (MLVQ2) for phoneme recognition. The phoneme recognition score was 85.5% for 226 sentences uttered by two male speakers. The linguistic processor is composed of a processor for spotting bunsetsu units (i.e. units similar to a “phrase” in English) and a syntactic processor. The structure of the bunsetsu unit is effectively described by a finite-state automaton, the test-set word-perplexity of which is 230. In the processor for spotting bunsetsu units, using a syntax-driven continuous-DP matching algorithm, the bunsetsu units are spotted from a recognized phoneme sequence and then a bunsetsu unit lattice is generated. In the syntactic processor, the bunsetsu unit lattice is parsed based on the dependency grammar, which is expressed as the correspondence between a FEATURE marker in a modifier-bunsetsu and a SLOT-FILLER marker in a head-bunsetsu. The recognition scores of the bunsetsu units and conceptual words were 75.2% and 88.9% respectively for 226 sentences uttered by the two male speakers.

In this paper, we introduce a Korean large vocabulary speech recognition system. This system recognizes sentence utterances with a vocabulary size of 1160 words, and is designed for an automatic telephone number query service. The system consists of four subsystems. The first is an acoustic processor recognizing words in an input sentence by a Hidden Markov Model (HMM) based speech recognition algorithm. The second subsystem is a linguistic processor which estimates input sentences from the results of the acoustic processor and determines the following words using syntactic information. The third is a time reduction processor reducing recognition time by limiting the number of candidate words to be computed by the acoustic processor. The time reduction processor uses linguistic information and acoustic information contained in the input sentence. The last subsystem is a speaker adaptation processor which quickly adapts parameters of the speech recognition system to new speakers. This subsystem uses VQ adaptation and HMM parameter adaptation based on spectral mapping.

We also present our recent work on improving the performance of the large vocabulary speech recognition system. These works focused on the enhancement of the acoustic processor and the time reduction processor for speaker-independent speech recognition. A new approach for speaker adaptation is also described.

In this paper, several tone recognition schemes for continuous Mandarin speech are discussed. First, an SCHMM is used to model the acoustic features of a syllable for tone discrimination. Parameters extracted from the F0 and energy contours of the syllable by discrete Legendre orthonormal transform are used as the recognition features. Then, a scheme using two-layer network is proposed to cope with the difficulty resulting from the declination effect on the F0 contour of the declarative sentential utterance. The declination effect is modeled by a sentence-level HMM on the upper layer and the acoustic features of each tone are modeled by a state-dependent SCHMM on the lower layer. Lastly, the coarticulation effect coming from neighboring syllables is considered in the scheme using context-dependent model. Performance of these recognition schemes was examined by simulations. A recognition rate of 86.34% was achieved.

This paper presents a speech recognition method based on the distribution of two-dimensional cepstral (TDC) coefficients. For each recognition unit, a TDC matrix is calculated. A set of selected TDC coefficients forms a pattern to represent this speech segment. By assuming the Gaussian distribution of the TDC coefficients, a statistical model for a class of speech patterns is generated. The recognition process is to evaluate the probability of a TDC pattern belonging to a specific pattern class and to find the model which gives the highest probability. This method is applied to the recognition of Mandarin syllables. The experimental result shows that the proposed method is very promising for syllabic languages such as Mandarin.

Learning is a critical research field for autonomous computer vision systems. It can bring solutions to the knowledge acquisition bottleneck of image understanding systems. Recent developments of machine learning for computer vision are reported in this paper. We describe several different approaches for learning at different levels of the image understanding process, including learning 2-D shape models, learning strategic knowledge for optimizing model matching, learning for adaptive target recognition systems, knowledge acquisition of constraint rules for labelling and automatic parameter optimization for vision systems. Each approach will be commented on and its strong and weak points will be underlined. In conclusion we will suggest what could be the “ideal” learning system for vision.

Within the field of structural pattern analysis, algorithms for inference of discrete mathematical models from samples are an important area of research. This paper gives an extensive survey of state-of-the-art methods for data-driven inductive inference of finite-state automata. In addition to providing notationally consistent descriptions of the methods’ fundamental mode of operation, aspects such as sequential learning, advantages and disadvantages, and the extension to stochastic automata are also addressed.

In the pattern classification problem, it is known that the Bayes decision rule, which separates k classes, gives a minimum probability of misclassification. In this study, the prior probability of each class is unknown and the conditional density functions are known. A set of unidentified input patterns is used to establish an empirical Bayes rule, which separates k classes and which leads to a stochastic approximation procedure for estimation of the priors. This classifier can adapt itself to a better decision rule by making use of unidentified input patterns while the system is in use. The resulting probability of misclassification can be made arbitrarily close to that of the Bayes rule. The results of a Monte Carlo simulation study with normal, uniform and exponential distributions are presented to demonstrate the favorable prior estimation for the empirical Bayes rule.

This paper considers the problem of detecting lines in images using a pyramid architecture. The approach is based on the Hough Transform calculation. A pyramid architecture of size n is a fine-grain architecture with a mesh base of size processors each holding a single pixel of the image. The pyramid operates in an SIMD mode. Two algorithms for computing the Hough Transform are explained. The first algorithm initially uses different angles, θ_j’s, and its complexity is O(k+log n) with O(m) storage requirement. The second algorithm computes the Hough Transform in a pipeline fashion for each angle θ_j at a time. This method produces results in O(k * log n) time with O(1) storage, where k is the number of θ_j angles, m is the number of ρ_i normal distances from the origin, and n is the number of pixels. A simulation program is also described.

In this paper, a new feasible approach is proposed to determine the 3-D location and orientation as well as the radius of a cylinder pasted with a known-height label. The proposed approach uses a rectangle-shaped standard mark composed of two point-pairs for performing monocular image analysis.

A monocular image of a labelled cylinder is first taken. Image processing and numerical analysis techniques are then used to set two point-pairs located on the boundary of the visible part of the cylindrical label. According to 3-D imaging geometry, the location and orientation as well as the radius of the cylinder relative to the camera are uniquely determined by using 3-D vector analysis and simple algebraic computation. Owing to the full linearity of the,derivation, this approach can achieve high speed requirement. In addition, no prerequisite restriction is imposed on the image-grabbing process and there is no need to perform the calibration of the focal length since it can be derived. Experimental results, with mean errors less than 3%, prove the feasibility of the proposed approach.

In the paper, a learning algorithm for Hopfield associative memories (HAMs) is presented. According to the cost function that measures the goodness of the HAM, we determine the connection matrix using a global minimization, solved by a gradient descent rule. This optimal learning method can guarantee the storage of all training patterns with basins of attraction that are as large as possible. We also study the storage capacity of the HAM, the asymptotic stability of each training pattern and its basin of attraction. A large number of computer simulations have been conducted to show its performance.

Learning when limited to modification of some parameters has a limited scope; capability to modify the system structure is also needed to get a wider range of the learnable. In the case of artificial neural networks, learning by iterative adjustment of synaptic weights can only succeed if the network designer predefines an appropriate network structure, i.e. the number of hidden layers, units, and the size and shape of their receptive and projective fields. This paper advocates the view that the network structure should not, as is usually done, be determined by trial-and-error but should be computed by the learning algorithm. Incremental learning algorithms can modify the network structure by addition and/or removal of units and/or links. A survey of current connectionist literature is given on this line of thought. “Grow and Learn” (GAL) is a new algorithm that learns an association at one shot due to its being incremental and using a local representation. During the so-called “sleep” phase, units that were previously stored but which are no longer necessary due to recent modifications are removed to minimize network complexity. The incrementally constructed network can later be finetuned off-line to improve performance. Another method proposed that greatly increases recognition accuracy is to train a number of networks and vote over their responses. The algorithm and its variants were tested on recognition of handwritten numerals and seem promising especially in terms of learning speed. This makes the algorithm attractive for on-line learning tasks, e.g. in robotics. The biological plausibility of incremental learning is also discussed briefly.