Step 5: Implementing the voice analyser

In this step, you will implement the VoiceAnalyzer class and the related parts in the GuitarTuner class. This makes the application able to analyse the voice frequency.

VoiceAnalyzer is given a target frequency. It then reads audio input data, analyses the voice, and emits the difference of the main frequency of the voice as compared to the target frequency periodically. To restrict the complexity of the calculations needed, only a fraction of the audio data is taken to be analysed.

Implementing the GuitarTuner class methods related to the audio input

1. Implement the GuitarTuner initAudioInput method so that m_format_input, m_audioInput, and m_analyzer are set up based on comments.
   #include <QAudioInput.h> ... /*! Inits audio input. */ void GuitarTuner::initAudioInput() { // Set up the input format. m_format_input.setFrequency(DataFrequencyHzInput); m_format_input.setCodec("audio/pcm"); m_format_input.setSampleSize(16); m_format_input.setChannels(1); m_format_input.setByteOrder(QAudioFormat::LittleEndian); m_format_input.setSampleType(QAudioFormat::SignedInt); // Obtain a default input device, and if the format is not // supported, find the nearest format available. QAudioDeviceInfo inputDeviceInfo( QAudioDeviceInfo::defaultInputDevice()); if (!inputDeviceInfo.isFormatSupported(m_format_input)) { m_format_input = inputDeviceInfo.nearestFormat(m_format_input); } // Create new QAudioInput and VoiceAnalyzer instances, and store // them in m_audioInput and m_analyzer, respectively. // Remember to set the cut-off percentage for voice analyzer. m_audioInput = new QAudioInput(inputDeviceInfo, m_format_input, this); m_analyzer = new VoiceAnalyzer(m_format_input, this); m_analyzer->setCutOffPercentage( m_guitarTunerUI->getMicrophoneSensitivity()); }

   Select All
2. Add implementation into GuitarTuner modeChanged as follows. In the if (isInput) branch, which is executed when the input mode is changed to be active, stop output and start input based on code comments. The else branch, which was otherwise implemented in Step 3.3, also needs to be changed as follows. Add code to stop audio input and voice analyser, before the line // Set up the audio output.
   /*! Receives a mode changed signal (\a isInput) */ void GuitarTuner::modeChanged(bool isInput) { // If the mode must be changed to input mode: if (isInput) { // Stop audio output and audio generator. m_audioOutput->stop(); m_voicegenerator->stop(); // Start the audio analyser and then the audio input. m_analyzer->start(m_guitarTunerUI->getFrequency()); m_audioInput->start(m_analyzer); } // Else: else { // Stop audio input and audio analyser. m_audioInput->stop(); m_analyzer->stop(); // Set up the audio output. // If the current frequency of voice generator // is not the same as the target frequency selected in the UI, // update voice generator's frequency. if (m_voicegenerator->frequency() != m_guitarTunerUI->getFrequency()) { m_voicegenerator->setFrequency(m_guitarTunerUI ->getFrequency()); } // Start the voice generator and then the audio output. m_voicegenerator->start(); m_audioOutput->start(m_voicegenerator); // Call setMaxVolumeLevel() to modify the maximum volume level. setMaxVolumeLevel(m_guitarTunerUI->getVolume()); // If the voice is muted, the voice is suspended // in the outputStateChanged slot. } }

   Select All

Implementing the VoiceAnalyzer class

The voice analyser collects voice samples from the audio stream at regural intervals, analyses them with a Fast Fourier Transform (FFT), and then signals the results.

1. Add implementation into the VoiceAnalyzer constructor as follows. Init the m_format, m_frequency, m_position, and m_fftHelper properties. m_totalSampleCount stores the amount of voice samples to be taken; equation 6 in the Math Behind the Curtains of GuitarTuner is a formula for this (explanations are also included in the document). Set m_totalSampleCount to be equal to the following:
   ,
   where is the same as the M_SAMPLE_COUNT_MULTIPLIER macro. Prepare m_samples to hold m_totalSampleCount values. Set m_maximumVoiceDifference so that the following equality holds:
   m_maximumVoiceDifference = log_2(TargetFrequencyParameter) * 12.
   The reasons are given in Section 3.1. of Math Behind the Curtains of GuitarTuner. Init the cut-off percentage with the setCutOffPercentage method.
   VoiceAnalyzer::VoiceAnalyzer(const QAudioFormat &format, QObject *parent): QIODevice(parent), m_format(format), m_frequency(0), m_position(0), m_fftHelper(new FastFourierTransformer()) { Q_ASSERT(qFuzzyCompare(M_SAMPLE_COUNT_MULTIPLIER, float(2)/(M_TWELTH_ROOT_OF_2 -1.0))); m_totalSampleCount = qRound(qreal(PrecisionPerNote) *TargetFrequencyParameter *M_SAMPLE_COUNT_MULTIPLIER); m_samples.reserve(m_totalSampleCount); int i = 2; int j = 1; for (; i < TargetFrequencyParameter; i *= 2) { j++; } m_maximumVoiceDifference = j*12; setCutOffPercentage(CutOffScaler); }

   Select All
2. Implement the VoiceAnalyzer start and stop methods. The start method must initialise m_stepSize, which is the interval of the samples taken to be analysed. Equation 4 in the Math Behind the Curtains of GuitarTuner is the formula for this; explanations are also provided. Set m_stepSize to sampleRate divided by 2 * frequency * TargetFrequencyParameter, where frequency is the provided parameter. Store the frequency with m_frequency; finally, open the parent QIODevice in write-only mode. The stop method must close the parent QIODevice and clear m_samples which are already collected.
   /*! Opens the parent QIODevice. Sets up the analysation parameters and target frequency to be the given \a frequency. */ void VoiceAnalyzer::start(qreal frequency) { m_stepSize = (qreal) 1.0 * m_format.sampleRate() / (TargetFrequencyParameter*2*frequency); m_frequency = frequency; open(QIODevice::WriteOnly); } /*! Closes the parent QIODevice, thus the voice is not analysed anymore. Resets the m_samples QList. */ void VoiceAnalyzer::stop() { m_samples.clear(); m_samples.reserve(m_totalSampleCount); close(); }

   Select All
3. Implement the VoiceAnalyzer writeData method. In writeData, obtain each m_stepSize voice sample with the getValueInt16 method, and append the value into the m_samples list. If the m_totalSampleCount samples have been appended into m_samples, call the analyzeVoice method to analyse the voice and emit appropriate signals, after which m_samples must be reset.
   /*! Called when \a data is obtained. Stores each m_stepSize sample into a QList to be analysed. */ qint64 VoiceAnalyzer::writeData(const char *data, qint64 maxlen) { const int channelBytes = m_format.sampleSize() / 8; int sampleSize = m_format.channels() * channelBytes; int m_stepSizeInBytes = m_stepSize*sampleSize; // assert that each sample fits fully into the data Q_ASSERT((m_position % sampleSize)==0); const uchar *ptr = reinterpret_cast<const uchar *>(data); while (m_position < maxlen) { if (m_samples.size() < m_totalSampleCount) { m_samples.append(getValueInt16(ptr+m_position)); } else { analyzeVoice(); m_samples.clear(); m_samples.reserve(m_totalSampleCount); // fast forward to the last possible position m_position += ((maxlen - m_position) / m_stepSizeInBytes) * m_stepSizeInBytes; break; } m_position += m_stepSizeInBytes; } m_position -= maxlen; return maxlen; }

   Select All
4. Implement the VoiceAnalyzer getValueInt16 method. Use reinterpret_cast to cast ptr into const quint8* or const quint16* pointer. Then read and scale the value appropriately, depending on the audio format sampleType, sampleSize, and byteOrder. Use the qFromLittleEndian and qFromBigEndian methods to make the code be align-safe. Finally, return the obtained value.
   /*! Interprets \a ptr as a pointer to int value and returns it. */ qint16 VoiceAnalyzer::getValueInt16(const uchar *ptr) { qint16 realValue = 0; if (m_format.sampleSize() == 8) { const qint16 value = *reinterpret_cast<const quint8*>(ptr); if (m_format.sampleType() == QAudioFormat::UnSignedInt) { realValue = value - M_MAX_AMPLITUDE_8BIT_SIGNED - 1; } else if (m_format.sampleType() == QAudioFormat::SignedInt) { realValue = value; } } else if (m_format.sampleSize() == 16) { qint16 value = 0; if (m_format.byteOrder() == QAudioFormat::LittleEndian) value = qFromLittleEndian<quint16>(ptr); else value = qFromBigEndian<quint16>(ptr); if (m_format.sampleType() == QAudioFormat::UnSignedInt) { realValue = value - M_MAX_AMPLITUDE_16BIT_SIGNED; } else if (m_format.sampleType() == QAudioFormat::SignedInt) { realValue = value; } } return realValue; }

   Select All
5. Implement the VoiceAnalyzer setCutOffPercentage method. This method must call m_fftHelper setCutOffForDensity with a parameter, which should be some fraction of the maximum density that can be noticed in a signal. The following formula is a good choice for a parameter:
   CutOffScaler * cutoff * m_totalSampleCount * X,
   where X is either M_MAX_AMPLITUDE_8BIT_SIGNED or M_MAX_AMPLITUDE_16BIT_SIGNED.
   /*! Takes a \a cutoff number between 0 and 1, scales it with CutOffScaler, multiplies it with maximum density, and then gives it to the fft helper. */ void VoiceAnalyzer::setCutOffPercentage(qreal cutoff) { cutoff = CutOffScaler*cutoff; if (m_format.sampleSize() == 8) { float t = cutoff*m_totalSampleCount*M_MAX_AMPLITUDE_8BIT_SIGNED; m_fftHelper->setCutOffForDensity(t); } else if (m_format.sampleSize() == 16) { float t = cutoff*m_totalSampleCount*M_MAX_AMPLITUDE_16BIT_SIGNED; m_fftHelper->setCutOffForDensity(t); } }

   Select All
6. Implement the VoiceAnalyzer frequency, getMaximumVoiceDifference, and getMaximumPrecisionPerNote methods. These are ordinary getters, which return the corresponding values m_frequency, m_maximumVoiceDifference, and PrecisionPerNote, respectively.
   /*! Returns the current target frequency. */ qreal VoiceAnalyzer::frequency() { return m_frequency; } /*! Returns the maximum absolute value sent by the voiceDifference() signal. */ int VoiceAnalyzer::getMaximumVoiceDifference() { return m_maximumVoiceDifference; } /*! Returns the maximum precision per note near the target frequency. */ int VoiceAnalyzer::getMaximumPrecisionPerNote() { return PrecisionPerNote; }

   Select All

The analyzeVoice method is already implemented because it is not very closely related to the topic of this hands-on lab, that is, working with Qt Mobility Multimedia classes. In the analyzeVoice method, the maximum density index (the index corresponding to the main frequency) is calculated from m_samples with m_fftHelper. Depending on the result, the lowVoice, voiceDifference, and correctFrequency signals are emitted. The value emitted in the voiceDifference signal represents the difference of the obtained main frequency of the voice and the target frequency.

Now the building should be successful. However, you cannot change the input mode to be active. This will be corrected in the next step.

If the application is not working properly, download and use the files in this zip file.

You can test the changes in this step by changing the GuitarTuner constructor code temporarily so that modeChanged is called with a true parameter, corresponding to the input mode.