Audio is an integral part of many IoT applications, including consumer products such as speakers and headphones, wearables and medical devices (e.g., hearing aids), automation and industrial control applications, entertainment systems, and automotive infotainment units.
IoT audio can be broadly categorized into three types: streaming (i.e., music, voice, and data), voice recognition/commands, and wireless via Bluetooth and Wi-Fi connectivity (e.g., streaming multi-channel audio over Wi-Fi to a home surround sound system). However, designing high-quality, uninterrupted acoustic audio subsystems can be challenging when engineers must adhere to the strict constraints required for IoT-based devices.
More complex designs are required to include advanced features such as voice recognition, such as allowing drivers to control the infotainment system in their car in the same hands-free manner as a cell phone. Since the MCU is at the heart of all these audio systems, it is important to select an MCU that integrates the necessary audio technologies required to design a reliable noise-free audio system. This paper explores the audio technologies that can be used to design such systems.
Components of an audio subsystem
IoT audio involves three main activities: steam high-quality voice/data, wireless transmission, and voice reorganization control. Figure 1 shows the most important building blocks in an embedded system.
This block diagram shows the more important building blocks of the audio processing subsystem
Note that many of these features can be integrated into modern MCUs, such as the Cypress CYW43907 with integrated Wi-Fi 802.11n used in this example.Some of the important audio technologies that may be included in an IoT-based system include:
Music applications
Audio-enabled MCUs allow engineers to decode MP3/4 streams used by most popular media players and content providers. Many designs also need to support WMA and Apple's AAC decoding, which requires additional processing power. In consumer audio applications, low-cost audio MCUs can often be utilized by managing digital music streams from audio accessories such as digital speaker sets.
In these applications, a frame of PCM audio data (encapsulated in a USB audio class format) arrives every 1 ms through one of the processor's SPI/I²C serial channels. Depending on the source, the audio stream may arrive in one of several formats (i.e., left-aligned, right-aligned, I2S, etc.). However, some low-cost codecs can only accept certain formats. In these cases, the MCU plays an important role in ensuring that the data is correctly aligned before it is fed to the codec.
Since not all audio sources use the same sampling rate, the codec must also adapt its sampling frequency to the source or rely on the MCU to convert the sampled data stream to a common data rate (see Figure 2). In these cases, the MCU must manage the stream to avoid under- or overload conditions that can lead to mutes, pops, and audio discontinuities that can cause data loss and disrupt the user listening experience. Note that the audio MCU can also be used to implement other functions of the audio subsystem, such as controlling lighting during audio playback.
The audio MCU may need to perform format conversion, sample rate adjustment, and stream management, as well as support audio user interfaces.
To implement audio in a wide range of applications, audio MCUs need to support a variety of audio technologies. Figure 3 shows examples of these audio technologies.
sound technology
Audio Codecs
Audio codecs are the main front-end component of an audio system. Many MCUs built for IoT applications support codec functionality in hardware. This allows the system to reduce the size of digital audio samples to speed up wireless transmission (saving power) and save storage space (reducing the strain on internal memory capacity). The codec may support various audio standard formats such as AAC, AC-3, and ALAC. to do so, it requires a decoding access unit (AU), which is implemented before any audio post-processing (e.g., DSOLA, SOLA). When used with standard audio formats such as AAC, AC-3 and ALAC, audio is categorized in such a way that subsequent audio samples are within the prescribed format specified in the audio packet data stream. Packet spacing is also managed to allow for minimal cross jitter and uninterrupted operation in the presence of congestion. the AU payload size allows for the execution of any concealment that needs to be performed.
Baseband Processing
A baseband signal is the fundamental group of frequencies in an analog or digital waveform that can be processed by electronic circuits. A baseband signal can consist of a single frequency or a group of frequencies or, in the digital domain, a stream of data sent over a non-multiplexed channel. Baseband is defined as the baseband (signal/second) mixed with the carrier signal to produce a modulated signal. Note that in MCUs supporting IoT audio, the audio codec integrates baseband processing and RF on a single chip. the audio codec can be implemented in a variety of wireless transceivers to provide voice data and/or music functionality. The codec also features mono and stereo channels for audio output, as well as stereo inputs.
Packet Loss Concealment and Data Replication
Excessive latency, packet loss, and high latency jitter can impair communication quality. The likelihood of sudden packet loss increases with network load and results in interruptions that can be heard by the user. Robust audio transmission over Wi-Fi can be enhanced with advanced features such as Cypress's packet loss concealment technology. The system architecture source/receiver is as follows: one source captures the audio, multiplexes the PCM data through the RTP stream structure, and synchronizes the clock with all receivers connected to the PLC source.
Note that the performance of the communication link depends on the quality of the link budget performance. This link budget is determined by three factors: transmit power, transmit antenna gain, and receive antenna gain. For example, reliable communication over an 802.11 network is possible if the power of the link path minus the loss of available space is greater than the minimum received signal level of the receiving radio (see Figure 4).
The performance of a communication link depends on the quality of the link budget performance
Speech intelligibility enhancement (SIE)
Background noise in the audio system can reduce the intelligibility of speech. If the noise exceeds a certain level, the speech may be difficult for the user to understand. The availability of real-time continuous speech recognition on embedded devices requires a system that enhances the intelligibility of noise-impaired speech. Selecting an MCU that supports porting and optimization of a commonly used Large Vocabulary Continuous Speech Recognition (LVCSR) system can simplify development.
Wake-up phrase detection
This advanced feature enables users to turn on the system hands-free by activating the device with their voice.
Efficient Multicast to One or More Speakers
Multicasting is a network addressing method used to send messages to a group of targets simultaneously using the most efficient strategy. Messages are delivered only once through each link in the network, and copies are created only when the next link splits to multiple destinations, usually at network switches and routers. However, like the User Datagram Protocol (UDP), multicast does not guarantee delivery of the message stream, which can lead to message discarding or unorganized message delivery. Reliable Multicast (RMC) provides acknowledgments for multicast packets (packets only) so that certain specific multicast packets can be delivered reliably. The transmitter selects the receiver with the weakest RSSI to acknowledge the frame. In an IoT environment, implementing RMC means that the Wi-Fi transmitter chooses one of the many Wi-Fi receivers to acknowledge frame reception. The transmitter selects the receiver with the weakest RSSI to acknowledge the frame. The implementation uses an operational framework containing proprietary RMC information elements to notify and enable the acknowledger. The implementation also contains RMC-specific Wi-Fi driver commands to set the multicast MAC address and to enable and disable the RMC.
For audio and video with fixed and symmetric transmission delays, time synchronization requirements are met; for example, RMC can rely on highly accurate timing and synchronization for smooth cell-to-cell transmission of voice, video, and mobile data. Achieving highly accurate and precise timing is not easy from a technical point of view, so it is important to find implementations that can be verified to meet application requirements.
Framing formats, forward error correction and packet replication
For audio streaming, it is critical that the clock is synchronized with all Wi-Fi receivers. One approach is to use a common clock for both source and receiver devices, often called a wall clock or system clock (STC). First, each receiver (receiver) synchronizes its STC (wall clock) with the source/transmitter's STC (master wall clock). Each receiver can now recover the transmitter's clock because the timestamp inserted by the source (available in the extended header of each RTP packet) reflects the sampled moment of the media relative to the common clock.
The STC is based on the Grandmaster clock values outlined in the 802.1AS specification. Since all receiver devices are aware of the correlation between the STC and the source device's media clock (as it relates to the RTP or media timestamp), each receiver can reconstruct a copy of the source device's RTP media clock and queue its output accordingly for proper rendering. Transparent clocking is where the hardware/ucode can timestamp packets received and transmitted as close as possible to the MAC/PHY interface. While this clock value is not used for playback, it can be used to measure jitter throughout the system and perform a full performance analysis.
Example of a smart home audio system
To understand IoT audio in context, consider the smart home example and the role audio can play in improving the overall functionality of a smart home system. A home becomes a smart home when the devices and appliances in it can communicate with each other and the people who live there. By increasing our interconnectivity, smart homes are improving our quality of life and increasing our security.
One of the main use cases for audio in the smart home is storing and sharing audio over Wi-Fi or Bluetooth. The choice of Wi-Fi over BLE varies by application and depends on range and audio quality requirements. For example, a home controller can play a specific sound in each room of the house if someone rings the doorbell at the door, rather than just plugging in the bell in one part of the house. Similarly, the controller can limit the sound to specific rooms, such as not in a nursery for babies. Embedded controllers help to process this audio and make the system smarter by managing various output control functions.
Playback Audio Systems
Replay audio systems have become an important application in the audio market. Wireless audio replay systems are at the heart of the smart home, bringing together many different smart devices in the home and making intelligent decisions on behalf of the user. For example, an audio system can control the lighting patterns in a house based on the music that is currently playing. It can also use text-to-speech conversion to read user notifications or emails aloud. Users also have the option of creating zones in a multi-room audio system by using networkable audio devices, such as wireless speakers in different rooms of the house. This approach creates an entire ecosystem to ensure that the home is always operating at peak efficiency while minimizing interactions with the people who live there. To create such an ecosystem, IoT designers need to select an embedded microcontroller with the right performance and audio-based features that have been optimized for IoT applications.
Digital signal processing effects
Audio signal processing in the digital domain is an important part of any audio system before transmitting audio data over a wireless link. This processing typically involves measuring, filtering and/or compressing the audio analog signal. Embedded MCUs with integrated DSP functionality enable effects such as the addition of a digital mixer and support for remote control functions. With a 5-band equalizer for each channel, audio playback can be cleverly integrated with most sequencer applications to create a powerful studio system.