FLAT INITIALIZATION OF CI MODEL PARAMETERS

TRAINING CONTINUOUS MODELS

CI models consist of 4 parameter files :

• mixture_weights: the weights given to every Gaussian in the Gaussian mixture corresponding to a state
• transition_matrices: the matrix of state transition probabilities
• means: means of all Gaussians
• variances: variances of all Gaussians

To begin training the CI models, each of these files must have some initial entries, ie, they must be "initialized". The mixture_weights and transition_matrices are initialized using the executable mk_flat. It needs the following arguments:

FLAG	DESCRIPTION
-moddeffn	CI model definition file
-topo	HMM topology file
-mixwfn	file in which you want to write the initialized mixture weights
-tmatfn	file in which you want to write the initialized transition matrices
-nstream	number of independent feature streams, for continuous models this number should be set to "1", without the double quotes
-ndensity	number of Gaussians modeling each state. For CI models, this number should be set to "1"

To initialize the means and variances, global values of these parameters are first estimated and then copied into appropriate positions in the parameter files. The global mean is computed using all the vectors you have in your feature files. This is usually a very large number, so the job is divided into many parts. At this stage you tell the Sphinx how many parts you want it to divide this operation into (depending on the computing facilities you have) and the Sphinx "accumulates" or gathers up the vectors for each part separately and writes it into an intermediate buffer on your machine. The executable init_gau is used for this purpose. It needs the following arguments:

FLAG	DESCRIPTION
-accumdir	directory in which you want to write the intermediate buffers
-ctlfn	control file
-part	part number
-npart	total number of parts
-cepdir	path to feature files - this will be appended before all paths given in the control file
-cepext	filename extension of feature files, eg. "mfc" for files called a/b/c.mfc. Double quotes are not needed
-feat	type of feature
-ceplen	dimensionality of base feature vectors
-agc	automatic gain control factor(max/none)
-cmn	cepstral mean normalization(yes/no)
-varnorm	variance normalization(yes/no)

Once the buffers are written, the contents of the buffers are "normalized" or used to compute a global mean value for the feature vectors. This is done using the executable norm with the following flag settings:

FLAG	DESCRIPTION
-accumdir	buffer directory
-meanfn	file in which you want to write the global mean
-feat	type of feature
-ceplen	dimensionality of base feature vector

The next step is to "accumulate" the vectors for computing a global variance value. The executable init_gau, when called a second time around, takes the value of the global mean and collects a set of (vector-globalmean)² values for the entire data set. This time around, this executable needs the following arguments:

FLAG	DESCRIPTION
-accumdir	directory in which you want to write the intermediate buffers
-meanfn	globalmean file
-ctlfn	control file
-part	part number
-npart	total number of parts
-cepdir	path to feature files - this will be appended before all paths given in the control file
-cepext	filename extension of feature files, eg. "mfc" for files called a/b/c.mfc. Double quotes are not needed
-feat	type of feature
-ceplen	dimensionality of base feature vectors
-agc	automatic gain control factor(max/none)
-cmn	cepstral mean normalization(yes/no)
-varnorm	variance normalization(yes/no)

Again, once the buffers are written, the contents of the buffers are "normalized" or used to compute a global variance value for the feature vectors. This is again done using the executable norm with the following flag settings:

FLAG	DESCRIPTION
-accumdir	buffer directory
-varfn	file in which you want to write the global variance
-feat	type of feature
-ceplen	dimensionality of base feature vector

Once the global mean and global variance are computed, they have to be copied into the means and variances of every state of each of the HMMs. The global mean is written into appropriate state positions in a means file while the global variance is written into appropriate state positions in a variances file. If you are using the scripts provided with the SPHINX package, you will find these files with "flatinitial" as part of its name in the model_parameters directory.
The flat means and variances file can be created using the executable cp_parm. In order to be able to use this executable you will have to create a copyoperations map file which is a two-column file, with the left column id-ing the state *to* which the global value has to be copied, and the right column id-ing the state *from* which it has to be copied. If there are "nphones" CI phones and each state has "nEstate_per_hmm" EMITTING states, there will be ntotal_Estates = nphones * nEstate_per_hmm lines in the copyoperations map file; the state id-s (on the left column) run from 0 thru (ntotal_Estates - 1). Here is an example for a 3-state hmm (nEstate_per_hmm = 3) for two phones (nphones = 2) (ntotal_Estates = 6; so, state ids would vary from 0-5):

0   0
1   0
2   0
3   0
4   0
5   0

cp_parm requires the following arguments.

FLAG	DESCRIPTION
-cpopsfn	copyoperations map file
-igaufn	input global mean (or variance) file
-ncbout	number of phones times the number of states per HMM (ie, total number of states)
-ogaufn	output initialized means (or variances) file

cp_parm has to be run twice, once for copying the means, and once for copying the variances. This completes the initialization process for CI training.