In order to have the tree “understands” what the user is talking about and make it respond to the user, we used voice recognition to translate the speech of user into text.
The voice recognition is done by p5.speech.library which can translate audio input into text.
Store information in a database
To store the worries in the tree, I used M-Lab’s database server. Basically, you need to register an account at M-lab, create a database, then a collection. Each account will have an API key, you will be able to use that API key to access your database, upload data and view all the documents you have sent to that database.
Easy Tutorial on how to use M-lab to create database and upload data
M-lab’s tutorial (Also easy to follow)
These two websites provide some very good tutorials on how to set up a database at M-Lab and store data into it.
While M-lab’s own sample code is written in Jquery, I used p5.js to do the same thing. I majorly used HttpPost() function to push information (voice text array) into my database.
User Interface Programming
All animations are done by After Effects and are embedded and organized into the interface using p5 DOM library.
To add all the light effects to the tree, we used p5.serialport.js library to do serial communication. Basically everytime when we want the computer side to trigger the tree to change its behavior, we send one-byte binary (a number) from the computer to Arduino by doing p5-to-Arduino serial communication.
We used Jumbo RGB LEDs, Neopixel RGB LED strips as well as some white Jumbo LEDs.
We first tested all the behaviors of the lights by just doing Arduino, using some switches and a sound sensor.
After we made sure that everything worked well on Arduino, we used serial communication to allow the computer to control the behaviors of all lights.
We created the physical tree using laser cutting and soldering techniques.