The idea of a thinking (and learning) machine has been upon us since before the dawn of computing. The 1920 Karel Čapek play R.U.R–roughly translated as Rossum’s Universal Robots, and the origin of the term “robot” as we know it today–deals with synthetic creatures that think for themselves. Even in the late 1700s, Wolfgang von Kempelen created an automaton known as “The Turk,” which appeared as though it could play chess on its own.
Machine Intelligence: Imagined, Theorized, Realized
On a more real-world computational level, Alan Turing proposed the concept of the “unorganized machine” in 1948, which would use a series of interconnected nodes for decision making. This is similar to the “neural network” concept that we know today, thrust into the larger consciousness in 1991 when the Terminator declared that its “CPU is a neural-net processor; a learning computer.”
Despite somewhat “mixed” reviews of how computer-based consciousness may ultimately turn out (Čapek’s robots also eventually turn on their masters), humans have barged ahead with development. We’ve created devices like Alexa and Google Home that, while not “intelligent” per se, can answer questions and perform control tasks in an intelligent manner. The general paradigm of such devices is that a simple process listens for a wake word/phrase to start, and responds to further queries using offsite cloud computing power.
But what about other simple voice recognition tasks, processing motion patterns, or even working with video? Would it be possible to use a small embedded chip such as those used on Arduino boards for this task locally (at the “edge”) rather than relying on the cloud? The short answer is “Yes,” though the real answer is, “It’s complicated.” The simple example presented below runs on an Arduino Nano 33 BLE Sense (N33BS), with no additional hardware.
Hands-On With Arduino Machine Learning: Simple Speech Recognition
If the “hello world” of microcontroller boards is blinking an LED, then the equivalent for machine learning could be to control it with voice commands. The N33BS has not just an impressive processing unit–nRF52840 microcontroller running at 64MHz, with 1MB or Flash, and 256KB of SRAM–but also boasts an array of onboard environmental sensors.
The Arduino Nano 33 BLE features a variety of sensors—perfect for machine learning
Among these onboard sensors is a microphone input, which we’ll use here to recognize the words “yes” and “no,” flashing an onboard LED green or red in response. It also lights up blue when it hears a noise that can’t be resolved as either option.
To get this going, first install the Arduino IDE and configure the IDE to work with the Arduino Nano 33 BLE via the boards manager:
Tools > Board > Boards Manager, search for nano 33 ble, and install the Arduino mbed-enabled boards package that comes up.
With that bit taken care of, go to
Sketch > Include Library > Manage Libraries and search for/install the Arduino_TensorFlowLite library (not the ESP32 version).
While you’re at it, you might want to also install the Arduino_LSM9DS1 to interpret the NEEBS’ accelerometer data, though this isn’t required for this experiment.
Green, yes; red, no; blue, unknown
Open up the micro_speech example in the TensorFlowLite library, hit the arrow key to compile, and upload. Once finished, you can then speak yes/no commands at your board. I found it was much better at recognizing “no” than “yes.” Your results may be different, as it’s not trained to me individually.
Train Your Arduino using Edge Impulse
While this might show that an Arduino is capable of simple speech recognition, using two pre-programmed words of questionable accuracy limits your options. Ideally you’d want to define your own parameters, which you can do via Edge Impulse.
This platform allows data capture via your phone, and includes tools for cleaning up your dataset when constructing a model. Capturing data with the Arduino microphone itself is also possible, and would be closer to what it actually hears. However, it’s also less convenient. This is important, as you’ll need in the order of 10 minutes of samples, ideally with a variety of people saying your trigger words. Background noise samples are also required, so the processor knows what’s not important.
If you want to try this yourself, search for Edge Impulse and create an account there. They provide instructions for teaching your own model. The collection and processing method appears to be quite good, but be prepared for quite a bit of recording time, plus a bit of “data hygiene” to produce a robust model for the N33BS to use.
Beyond Speech: Arduino Image Sensor & More Projects
While speech is the obvious ML target here, you can also train a model to recognize inputs from its onboard inertial measurement unit (IMU), or other onboard sensors. You can even analyze data from an external camera sensor with the right training and setup. Arduino itself has a piece on getting started with machine learning on Arduino. Instead of Edge Impulse, it uses Google Colab for model training.
Conclusion
So how does this all turn out? For now, it means that you can trigger an LED with just your voice. And with the proper training, you can get an Arduino board to recognize and respond to custom commands or various sensor inputs. Where neural network-trained AI, and ML in general, goes from there, only time will tell. Perhaps we’ll have all the benefits of machines that can intelligently do our bidding, or even entertain us, without the nefarious “side effects” that some see as a possibility.
