Blind Date

iphone_x.png

 

iphone_x.png

Worry Capsule Tree

An interactive installation composed of a physical tree and a web application.

Worries surround us every day. However, as time passed, we always realize how trivial those problems are in comparison. Worry Capsule Tree is an interactive installation that listens and stores people’s current worries and sends those back to them in the future, thus allowing its users to retrieve their previous worries and revisit the past selves.

The project contains two parts, a tree that the user can talk to and a web application where the user can refer to as a guide.

 

PROJECT OVERVIEW

Worry capsule tree is a 6-week interactive installation project teamed with Hau Yuan

SHOWCASE

ITP Winter Show, New York 2017. 12. 17 - 2017. 12. 18

Maker Faire, New York 2018. 09. 22 - 2018. 09. 23

MY ROLE

Voice Recognition | Database Server | Physical Computing | Fabrication | Front-end Development | Experience Design

TECHNICAL FEATURES

Design hand sketch | sketch | After effects | illustrator

Coding Voice Recognition (p5.speech.js) | Database Server (M-Lab) | Interface (After Effects/p5.js/Sketch) | Serial Communication(Arduino)

Physical Computing Arduino MEGA | USB microphone | Jumbo RGB LEDs | Neo-pixel RGB Lights

Fabrication Soldering | Laser Cutting

 
 

USER FLOW

tree_IxDBehaviour.png
tree_workflow.png

When the user comes to the installation, he will see our tree with a computer and a microphone in front of it. The interface on the computer screen will instruct the user to say “hello” to the tree. The greeting from the user will trigger the tree to communicate with him and encourage him to tell it the current worry he holds. The tree will listen, remember and store the worry from the user. When the worry is being stored, the tree installation will be gradually lit up. In the end, the user will find his worry being stored in one of the capsules on the tree. The user can choose to tell his contact information to the tree to get his worries retrieved in the future.

 

DESIGN

PHYSICAL TREE

 

WEBSITE INTERFACE

DESIGN PROGRESS

IDEATION

  1. Brainstorming

  2. Make design plans

USER TESTING

  1. Test with users to decide user behavior

  2. Develop the interactions

FAST PROTOTYPE

  1. Create different possible designs

  2. User testing with the prototypes and develop tree design

DEVELOPMENT

  1. Physical computing

  2. Programming

  3. Fabrication

IDEATION

The original idea of Worry Capsule Tree comes from one of the creators Yuan’s desire to release daily stress by keeping his anxiety into a safe place. After Yuan talked to me, this idea of telling and storing worries reminded me of the time capsule, whose function is to help people store memories that are accessible a long time later. 

Both of us liked the concept of storing current anxiety and retrieve it in the future. On one hand, it helps release the stress we have now, and on the other hand, it provides us with a chance to re-visit our past selves and see how much we have overcome and grown.

Since we decided to build something that deals with emotions and memories, the questions of:

“how would the users best communicate with this capsule?” and “how do they emotionally engage with the device” becomes very important.

These questions triggered us to first think about the appearance of our device. Our first thought is the image of a tree. Trees have the ability to absorb substances from the environment and convert them to nutritious fruits. We think it’s a metaphor for transforming people’s anxiety to their assets in the future. Therefore, we decided to build a worry capsule tree which can respond to the users and store their worries. This has brought us to the very first stage of our design.

 

USER TESTING

In order to know which way makes people feel more comfortable to interact with the tree and release their anxiety, we did user testing with 10 people.

Based on our research, more than half of the testers prefer speaking, in particular murmuring to the tree because they believe that creates an intimate relationship between them and the tree.

We finally decided to create an interactive capsule tree which can listen to its users and store their worries.

 

FAST PROTOTYPE


Once we decided the main interaction behavior, we built two fast tree prototypes. We ask different users’ opinions on our designs. We decided to follow my (Xiran’s) proposal, a more sculptural design because we believed it looked more interesting and can create more intimacy between our tree and the users. We adapted the design later and finished with the final outcome as shown in the video.

Yuan’s prototype

Yuan’s prototype

Xiran’s prototype

Xiran’s prototype

 

DEVELOPMENT

CODING

1

Voice recognition

In order to have the tree “understands” what the user is talking about and make it respond to the user, we used voice recognition to translate the speech of user into text.

The voice recognition is done by p5.speech.library which can translate audio input into text.

 

2

Store information in a database

To store the worries in the tree, I used M-Lab’s database server. Basically, you need to register an account at M-lab, create a database, then a collection. Each account will have an API key, you will be able to use that API key to access your database, upload data and view all the documents you have sent to that database.

Easy Tutorial on how to use M-lab to create database and upload data

M-lab’s tutorial (Also easy to follow)

These two websites provide some very good tutorials on how to set up a database at M-Lab and store data into it.

While M-lab’s own sample code is written in Jquery, I used p5.js to do the same thing. I majorly used HttpPost() function to push information (voice text array) into my database.

 

3

User Interface Programming

All animations are done by After Effects and are embedded and organized into the interface using p5 DOM library.

 

4

Serial Communication

To add all the light effects to the tree, we used p5.serialport.js library to do serial communication. Basically everytime when we want the computer side to trigger the tree to change its behavior, we send one-byte binary (a number) from the computer to Arduino by doing p5-to-Arduino serial communication.

 

PHYSICAL COMPUTING

We used Jumbo RGB LEDsNeopixel RGB LED strips as well as some white Jumbo LEDs.

We first tested all the behaviors of the lights by just doing Arduino, using some switches and a sound sensor.

After we made sure that everything worked well on Arduino, we used serial communication to allow the computer to control the behaviors of all lights.

Animated GIF-downsized (1).gif
 

FABRICATION

We created the physical tree using laser cutting and soldering techniques.

 
IMG_2003.JPG
 
 

Tubeman is happy!

an interactive web experience

Tubeman is happy! is an interactive web application. The users can use it to create, download and share gifs of themselves moving like a tubeman. By encouraging the users to imitate the tubeman and enabling them to create and share gifs of themselves moving, Tubeman is happy! is trying to create simple and sharable happiness.

Key Words

Interaction Design, Web Development

Time Frame

4 weeks

Progress

Ideation, User Testing, Prototyping, Iteration, Showcase

Toolkit

Adobe XD, Illustrator, WebRTC, WebSocket, Node.js, Javascript/HTML/CSS

My Role

Interaction Designer and Web Developer

Showcase

ITP Winter Show, 2018

 

How it Works?

1

When entering the web application, the users will see an image of the tubeman. They will also see themselves in a camera feed.

 

2

Following the instruction, they will then imitate the movements of tubeman (as shown in the picture), pose and take 18 pictures.

 

3

Users can add a word to describe their moods at the moment.

 

4

At the end of this experience, a gif will be generated.

 

5

The users can access to the gif by entering emails of themselves and then be able to share that “stupidity” and happiness with others.

gif1.gif
 
 
 

Project Setup For the Show

projectsetup_e794bbe69dbf-1.png
 

User FLow

userjourney-02.png

The user will first see and go to the gif booth. They will then look at the instruction computer and follow a series instructions which basically guide them to look at a frame by frame breakdown of a tubeman’s movements, take pictures one frame by one frame imitating the tubeman and at the end, get a gif made of the pictures taken previously. They will be able to download and share the gifs.

All the previous taken gifs, if allowed by the users, will all be displayed on a display page that everyone can see.

 

At the show

I developed this project on my MacBook and showed it during the ITP winter show in a larger scale as a “gif booth” installation. Hundreds of people have visited my project, moved, laughed and had fun.

Here is a glimpse of the HAPPINESS!


 

Why do I want to make it?

Tubeman always moves in a weird yet entertaining way. However, it shares some strong and magical happiness through its movements.

Happiness can be subtle and simple. I have noticed there are many simple things in life that easily makes people happy. Like a smile from a stranger on the street, a weird shape cloud in the sky, as well as the funny movement of a tubeman.

Technologies have made it simple for us to share moments in our lives with others much easier. Living in a social media era, it is natural for us to capture moments in our lives and share them. By sharing a photo of us in a certain time and space, we also share our emotions and connections to that physical space to people in another place.

Therefore I want to create an experience in which people can share a form of simple happiness easily to their families and friends through technologies. This idea leads me to create “Tubeman is happy!”.

By encouraging the users to imitate the tubeman and enabling them to create and share gifs of themselves moving, “Tubeman is happy!”is trying to make people happy while allowing them to share that happiness to others.

 


 
tubeman.gif

Fly Your Love

Send your love to a friend through an AR paper plane.

 

Mockup.png
 
 
 
 
cloud.png

OVERVIEW

 
 

Project Brief

Fly your love is a mobile AR app that aims to create a new type of chatting experience among people of different physical places. Users are encouraged to utilize their body movements to send messages based on where they currently are located.

My Role

User Experience Designer - Persona, Storyboard, User Interview, User Testing, Prototyping

Swift Programmer

Tools

Adobe XD, Ai, Ps

Sketch

Xcode

 
 
 
 
cloud.png

DEMO

 
 
 
cloud.png

HOW IT WORKS

 
loginin.gif
 
send a plane.gif
 
ar.gif
 
receiveAR.gif
 
ezgif.com-video-to-gif.gif

Log in to the app

 
evaflyplane.gif

Click to send a plane

 
evaAR.gif

Record the AR plane in space

 
receiveAReva.gif

Receive a plane from a friend

 
 
cloud.png

IDEATION

 
Artboard 1.png

COLLABORATOR

helps students find collaborators cross departments much more easier

mockup.png

Introduction

Collaborator is a mobile application prototype. It aims at helping students in the same educational institute to find collaborators cross departments. A user can log into this app either post a project idea and look for collaborators with certain skills, or find a project that requires skills he has to become a collaborator of others.

Time Frame

3 weeks

Role

UX/UI Designer

Methods

Ideation / User Research / Hand Sketching / User Journey / User Flow / Wireframe / Prototyping / User Testing / AB Testing

Tool Kits

Adobe PhotoShop / Adobe Illustrator / Adobe XD / Balsamiq


 

Background

How can I find my collaborator?

As a student who is interested in emerging technologies and new media arts, I am always trying to express my understandings about this world by combining different technologies, art and media formats. However, as I am eager to explore new approaches to express my ideas, I also constantly encounter the problem of not having enough experience in a particular area I am interested in. So I always have this idea to collaborate with other people. I want to contribute what I am good at and at the same time learn from my teammates who are experts in things I am less experienced with.

However, I sometimes find finding a collaborator not an easy mission. For instance, if I want to build an interactive installation. and I am most experienced in physical computing and less experienced in visual design or fabrication. I really want to collaborate with some people who have strong visual design skills and fabrication skills. However, when I ask around, people who are available are not quite strong in terms of visual design or fabrication, yet the strong candidates have different schedules with me and could not spare time to collaborate. I also send emails in my department email list and ask people to forward things around, but sometimes people just do not really read emails. Anyway, various situations make finding a collaborator really difficult.

So a strong willing just emerges from the bottom of my heart “I want a platform where I am able to find partners!” I really wish there is one in my school which could allow me to post a collaborator request to all students on campus and also potentially allow me to become collaborators of others.

 

User Research & Insights

I am most interested in these questions


When you need to find collaborators, what do you often do?

What tools or approaches do you use?

Among those tools or approaches, which one is your favorite and why?

Are you satisfied with the current tools you have to find collaborators?

Are there things you want to improve?


interviewHighlights-03.png

Since my targeted audience are college students who have the needs to find cross-department collaborators, I interviewed 5 students who will be the potential users. I asked about their personal experience finding partners.

These students come from different backgrounds including computer science, fine arts, architecture and design. When they are creating projects, they often run into the need to collaborate with other people.

From the interviews, I found that

The most used approaches people use to find collaborators outside their own majors is either by asking a friend to recommend candidates or sending emails to other majors’ email list.

 

current user journey

user journey-02.png
currentuserjourney-01.png
 

Pain Points

efficiency-04.png

The first pain point is that the current methods (asking friends/sending emails) are not efficient enough. Asking a friend might end up taking a lot of time - what if my friends do not know anyone I know? And passing emails around might lead to a situation that people do not reply efficiently.

macthing system-02.png

The second pain point comes when there are several candidates to choose. Since people are collaborating with people outside their our area of studies, if they have multiple potential collaborators to choose from, sometimes it might be tricky to decide the most ideal candidate. How do I make sure this person has the right skills I need? How do I know someone will be a better collaborator than someone else? Though talking to each individual helps to make a decision, Still, is there any better method besides that?

 

Competitive analysis

Tisch Talent Guild

Through my research, I found a student-oriented organization (web platform) at Tisch School of the Arts at New York University which aims at fostering student collaborations.

Screen Shot 2019-04-05 at 12.48.57 PM.png

“…whose goal is to foster creativity and collaboration between the various departments in New York University’s Tisch School of the Arts.”

Q2from the Tisch Talent Guild Official website

Q2from the Tisch Talent Guild Official website

TTG’s Targeted Audience

Although the platform claims to foster collaboration among various departments, they are more targeting towards majors related to performing, acting and video production. Other majors like Interactive Telecommunications Program or Game Design, which focus more on design and technologies are less included.


TTG’s Collaborator-Seeking System

ttguser_flow-06.png

TTG’s User Profile - People show their Resume/Portfolio

after clicking in a profile under the “talent” section

after clicking in a profile under the “talent” section

after clicking in a profile under the “collaborator” section

after clicking in a profile under the “collaborator” section

Pros

It solves the "collaborating-matching problem” by providing portfolio and resume of the talent.

TTG has provided a great platform for students in different majors to gather together and collaborate. It serves the purpose of both searching for projects and collaborators. It also have a great talent pool in which people showcase their talents by providing resume or portfolio.

Cons

You have to send an email once you find an ideal collaborator, still not efficient enough.

The “project” and “stories”, “collaborator” and “talent” sections are a bit overlapping.

When clicking into a project page, the information hierarchy is a bit messy.

Once you find a project you are interested in or an ideal collaborator, you will still have to go through the “send an email and wait” process. You have to move from one platform to another in order to get into connection with people.

In terms of information spreading, website platform does less a good job than mobile apps.

 

Solution

Based on the user research and existing platform research, I decided to create a mobile app which foster collaborators among students in different majors in Tisch School of Arts at NYU. The user can use it to find projects they are interested in or collaborators they need. The choice of mobile app is made because of the belief that communication via mobile apps works more efficient than on a website.

 

User Flow

Board 1.png
 

Wireframe

Based on this user flow, I used Balsamiq to create some wireframes for user testing.

I designed three scenarios for my users to test the app.

  1. You are new to this app. You are creating a dance and technology project. You would like to use this app to find a dancer.

  2. You are an actor. You currently are not engaged in any show. You go on this app to see if you will find some projects you are interested to participate in.

  3. You have a cinema project proposal, you want to use this app to post your idea and see if it will attract some people to collaborate with you.

Login and Find a Dancer

IMG_7481.JPG
 
find a dancer.gif
 
 

Find a Project

IMG_7332.JPG
 
find a project.gif
 
 

Submit a Project

IMG_7335.JPG
 
Submit a Project.gif
 

Insights from user testing

Pay more attention

Scene_3.png
 
Scene_3.png
 

IMPOSE

A Motion Capture Live Performance

Why do social norms and stereotypes still exist ? How exactly does society impose these norms ?

IMPOSE is an interactive, immersive motion-capture driven live-performance that explores different social norms, their beginnings and how they continue to be enforced by society on an individual.

 
 
 
 
5L5A2135.2019-03-03 21_57_59.gif

One Versus All

a collective memory of people born during the Chinese “one-child policy” era



research project | interactive installation

 
“独生” on the installation means “the only child”

“独生” on the installation means “the only child”

Role

Independent Work Designer and Developer

Showcase

IMA Spring Show, NYU Shanghai, China | 2017. 05 NYU Shanghai, Interactive Media Arts 17', Capstone Project

#Processing #WebCam #OpenCV #Fabrication #Projection Mapping #MadMapper

 

project breif

different and unique individual stories, same collective memory and culture

One Versus All is an interactive installation composed of 36 equal square size wooden boxes, a web camera, a computer and a projector. All 36 boxes together form a large box, and the surface of each smaller boxes is a screen. The installation will passively react to the cumulative number of the viewers who view this installation. As the number of people who visit this installation adds up, the web camera, which is detecting the faces of viewers will send a signal to the computer, and the written code will trigger the projector to project more videos onto the little screens.

This installation draws inspiration from both the impressive power of repetition art and the stories behind the Chinese one-child policy.

One-child policy is like a task that is repeatedly assigned to each Chinese family, and each family will produce similar yet very different outcome based on this policy. While all one-child persons have their very special personal experience, all of their stories inevitably and ultimately form a larger, collective and unified story.

The video content comes from 36 interviews of 36 different one-child individuals. All in all, “One Versus All” is trying to reveal that unified and collective story of such a policy to different people who have not necessarily experienced it.

 

Inspiration & Concept

Repitition

At the time I was informed about the IMA capstone, I was taking an interactive installation class and was obsessed with repetition art, which is used in some of the installations that were introduced to us during classes. I found that repetition, when used in art, is able to enlarge and proliferate the impression people can get from one item to a much larger level and therefore makes the installation greatly impressive.

One-child Policy

In the mean time, I was also reading a book named One Child written by Mei Fong, which is about her observations and thoughts of the Chinese one-child policy. Her words have provided me with a new perspective to view the one-child policy, which I have personally experienced for my entire life. Thinking about this policy, I suddenly realized that I have very little knowledge about it and have never carefully thought about it and its impact.

In order to make this capstone something not only would showcase the knowledge I have learned in IMA, but also a project that would be meaningful to me and more people, I decided to combine my favorite art form, which is repetition art and my most interested topic, which is the one-child policy together and create an interactive installation.

Based on my understandings, one-child policy is like a task that has been repeatedly assign to different Chinese families. When finishing this task, each family will produce similar yet very different outcomes. And the outcomes were all the one-child persons. Each of them have their own very special personal stories yet from a larger picture, all their stories inevitably and ultimately form a bigger collective and unified story that reflects one-child policy. Therefore I was hoping my project can start from the perspective of personal stories to show people some glimpse of the policy which they might or might not experienced by themselves.

 

Installation Construction

Choice of Box

I decided to build my installation using 36 equal-size boxes. Each box represents one  person I interviewed and the story of him/her.  The reasons I choose to use boxes are because first of all, I feel like each person has a box in their hearts, where many stories are stored. And secondly, the flat surfaces of boxes make it easier to project things on.

Choice of number “36”

I choose the number "36″ because one-child policy lasts for exactly 36 years (1980-2016).

Choice of Laser Cut & Color:

In order to remind people that my installation is about one-child policy, I laser cut the cover of the one-child certificate, which each one-child person would get when born and spray painted them into red, which is the color of the certificate.

Interview and insights

36 Interviews:

In order to get to know stories of people from a greater diversity, I interviewed people not only from our school, but also friends of my friends, people I met during my trip, people I met during some random events, and even strangers. I ended up asking for about 50 persons, and some rejected my interview.  I got around 40 people to participate at the end. I chose 36 of the interviews which I thought are more interesting to become the materials of my videos.

I interviewed people from three different ways: face-to-face interview, Skype/phone-call interview as well as online survey. I tried to get to talk to each of my interviewees personally to let them know more about my project and the concept behind it. If I was not able to meet them on face, I will try to contact them through either email or WeChat.

I am interested in:

1.  What is the first phrase that pops up in your mind when you think about “one-child” policy and why?

2. Are you aware of being a “one-child”? If so, when did you start to realize that?

3. What kind of comments have you ever received from people growing up as the only kid in your family ? What are those comments? And who gave those comments?

4. What’s your most impressive moment during your childhood?

5. What is your personal understanding or opinion on one-child policy?

With people giving different answers, I sometimes also asked some follow-up questions.

It is very interesting that the interviews got many similar answers as well as different answers. For instance, for the first question, many people answered “loneliness”. And many also answered “two-children policy”, which is a new policy following up the one-child policy that each Chinese family are allowed and probably encouraged to have two but no more that two children.

I also got to know more about people’s stories and their opinions on one-child policy, which we are all so familiar yet unfamiliar with. As we talked and discussed,  we touched upon topics that are not only about our growth, but also gender inequality/equality, the aging of the population as well as the problem of supporting the old people.

I also asked each interviewees to share either a photo/video clip from their childhood or a picture of something that they think is pretty valuable to them when they grow up. About half of them gave me pictures yet due to the concern of privacy, the other half rejected.


After I conducted all the interviews and collected all information, I started to think about how should I present these interesting stories to my audience through a visual perspective. While I was contemplating about this, I got a great advice from Susie. She suggested that I could present the most interesting parts of my interview in the format of bullet comments and transfer these comments into videos. I took her suggestion and then took the key words that I thought are most interesting from each of my interviews and used Premiere to transfer all of them into small videos.

*Bullet comment is a popular way of writing comment to videos. The viewers can write down their opinions on the particular part they are watching in a video and when the next person watch the same video at the same part, he/she will be able to view the previous comments left by previous viewers. And these comments would appear on the screen like bullets passing by.

Mechanism and Technologies

My idea of this project is that smaller stories form a larger collective story and each of the stories is a part of all the stories as a whole. Based on that, I also want to include my audiences, who are individuals(one) to the whole project(all). While they view this project, they are both independent people by themselves and also part of the whole project and a member of all the viewers who have participated in my installation.

I think one way that best make my viewers feel that they are participating is to let my installation react to the number of people who visit the installation—the more people have come, the more videos will be triggered. I want to make the viewers feel like they are contributing to adding the videos and helping to build the whole project.

Therefore I decided to use web camera to detect human faces (count the number of people) and use processing to trigger mad mapper to map all the videos onto my installation.

I started with building a grid in processing and then tried to connect processing to mad mapper via syphon.

I later tested my code with test videos.

When I was doing the test, because I only had a few videos and each of them are of small size, I did not really encounter problem running my code. But later when I tried with all my 36 finished videos, I found processing having problem running them.

I realized that processing is not really made for playing many films in a grid at the same time. But in order to achieve my goal of interaction, I ended up making 5 different larger videos. The size of the larger video equals the total size of the 36 films together. And each will having different numbers of smaller videos on it, with the first one with the least videos and the last one with all films playing. In this way, I was able to achieve my goal without changing too much of my original plan.

 

Reflection and Development

Success

I am happy I got to know much more about both repetition art and one-child policy.

I am happy that I got to practice and utilize what I have learned in IMA including: Processing(JAVA), Projection Mapping(MadMapper), Video/Audio Editing(Premiere), Laser Cut, Spray Painting, Interview people, etc..

I am happy that I finished an independent interactive installation project.

Development

Software: I did not really realize that processing is not really made for running videos, if possible, I want to explore other softwares in the future to deal with my videos.

Mechanism: The web camera does not seem to be the best tool to count the number of people. I want to explore other ways to count the number of people and edit my code accordingly.

Video Content: So for the videos are working but not working in the perfect way, I want to edit the videos to make them more readable. In addition, I want to figure out a way to make people be more aware that these videos are about “one-child” policy.

More User Testing.

 

 

 
Snapseed 3.jpg
 

Image Processing Archive

 

Oil Painting Effect

I used processing to recreate an oil painting effect based on photos I took.

BEFORE

before.JPG
1.jpg
4.jpg
5.jpg

AFTER

after2.png
after5.png

Glitch Animation

Glitch visual effect created by processing based on one of my illustrations.


swirly effect

AFTER

BEFORE

TheStarryNight.jpg

UNTITLED

 
theStarryNight.gif
 

Creative Coding Visual Archive

Rain Drops A Cloud

Rain Drops a Cloud is a creative coding project that aims at representing the romantic loop relationship between rain and cloud in an exaggerated way.


Latte Lover

Latte Love is a project that uses p5.js to simulates the interesting progress of making a cup of latte, especially the natural process of different liquids merging together. In the codes, I use mainly Particle system, Collision function and flow-field to create the visuals.


Dandelion

Sketch created by p5.js library. I connected the visual to an Arduino, a microphone and several potentiometers. By serial communication, I was able to create a "blowing the dandelion away" effect by blowing at the microphone. I was also able to live change the sketch by using the potentiometers.


Threads

Random threads patterns created by p5.js library.


Small animations

 
cover.gif