Tools for Interactive / Locative Projects


Recently Noticed…

At the beginning of the semester, each student shared something they recently noticed in digital or interactive culture…

My sister recently got a job at Mother NY, an independent advertising and marketing company that works to give creative solutions to their clients. To get a better idea for the company, I decided to look on their website. They have a great portfolio of creative, interactive work done for a variety of clients. The two that stood out the most for me was their work for James Patterson and Virgin Mobile.

For famous thriller author James Patterson, they decided to do something new to add more intensity to reading one of his novels. They made a self destructing book. With only 24 hours to read, it added more intensity to the usual reading experience in addition to making it more interactive and immersive. 1,000 copies were released, all connected to each other. Each reader could sabotage the other, by stealing and gaining time. James Patterson also promoted the book over social media, that also, appropriately enough, were self destructing. This is an example of great marketing to make reading much more of an interactive experience.
Another example is their work for cell phone retailer, Virgin Mobile. Advertisements can be dull and fail to connect with their audience. To change this, Mother attempted to create a more immersive and interactive advertising experience. To do this they utilized webcam’s to bring consumers more into the content. The advertisements played like a typical ad. However, every time the audience blinked, the scene of the ad would change. Conversation and dialogue would continue but it would be of new characters in new places and environments. The campaign drew a lot of attention and worked well, creating many unique impressions among consumers.

CHRIS:    One particularly surprising and interesting piece of interactive technology (among others) that I saw recently is found at the North Carolina Museum of Natural Sciences which I visited last month. The interactive exhibit was housed inside a special unit of the museum called the Naturalist Center. This center was special because it featured many real specimen bodies, furs, skeletal systems, molds, organs, and more which could be handled and examined by its visitors to learn more about them.

The digital interactive portion of the Naturalist Center were 2 raised white tables on the far side of the room. Behind these surfaces were more shelves and tables which housed many different smaller organisms or parts encased in small black boxes. To learn more about the organism, the visitor must simply set the box on the front of the white table. Wooden step stools were placed in front of the white table for children. A wireless tag inside the box communicates with the white table telling it what the organism is, and relevant information is projected down onto the table from above. Using some sort of camera-read motion or hand tracking, different tabs on the right of the information can be ‘pressed’ (just by touching the table, no touch screen or actual button) which will then bring up more information such as what sounds it makes, where it can be found, what they eat, and more.

I was instantly surprised by this technology and wanted to know how it worked. It was simple enough for a little kid to use, but contained so much complex information in a simple format. It seemed to work like magic, suavely presenting the information on an easy to read display right in front of you. I’m sure this kind of interactive technology could be implemented in many different applications and fields for a wide range of uses to present information, stored in a faux-physical space.

MCKENZIE:  I explored the website Tangible Interaction and they have some very intriguing projects on there. The Digital Graffiti Wall in the Chanel store located in New York City really interested me because I absolutely love street art. Graffiti gives anyone the opportunity to express themselves and the fact that they were using a digital spray can takes it to a whole new level in the digital art world. I love that the digital art community keeps growing with new ideas and I’m very excited to see what artist, including myself, come up with in the future.

MELISSA:   The newest interactive work of art that I have seen and was in awe by was a wall piece at Disneyworld. When waiting in line for one of their rides, you tend to get bored. Disney has added interactive displays to occupy your time as you wait and make it a little more fun. One of these interactive displays was in the Peter Pan line. It starts out as bells that hang at varying levels and as you walk by or reach up, the bells move and make noise. They also have it switch to butterflies that flutter to you and land on you when you are in front of the screen and Tinker Bell flutters around as well. Unfortunately, I was unable to take any pictures or video of the display as we had too many people around.

It was the first time I had seen this type of display and I thought it was so unique and interesting, it made me curious to know how it worked. I had the privilege of getting to learn a little bit in Arizona of how that kind of technology works and it was cool to see. The digital world has grown so much over the years and it is fascinating to see what new things people can come up with. I enjoy being a part of this culture


CHARLES:    Bullet Time

In 2013, a small demo that was created in 7 days took the internet by storm. It was a simple concept that many found to infections and ingenious. Its premise was:

Time moves only when you move.

As long as you stood still, time moved at next to nothing; but move around, turn your head, or jump and time moved along with you. This changed the game from a standard action game to a puzzle game. Players were able to fulfill their action star dreams as you could literally watch bullets fly right by you in slow motion. The game exuded style as you slipped through streams of bullets. As you pulled off an insane chain of actions you would find yourself reciting the name of the game in a rhythmic fashion.


Fast forward to 2016, SUPERHOT had its full release, expanding on the original game’s concept. SUPERHOT has gained a cult following and you can already see its influence show up in other games. In fact, I myself was riveted by the idea so much that I have a mode in my game, anti|piracy, called SUPERDOT where time moves only when you move. It is one of the most innovative game design ideas I saw in 2016 and you will find yourself chanting:


You can watch the SUPERHOT trailer here:

Interactive Project Pitch

After spring break, students in Art2018 will pitch an idea to the class for a project they would like to work on for the remainder of the semester. Each student will have about 10 minutes to present the pitch, with 5 minutes for questions / discussion. After all the presentations, students can break into teams or choose to work solo on their Interactive project. Projects may take the form of interactive installations, online games, or art experiences that use social media or mobile devices. Each pitch should include about 20 slides in a powerpoint / google slides presentation. Every slide should include VISUAL examples, such as screen grabs of inspiration or links to  web-based interactive works.


Presentation OUTLINE (~20 Slides)

Table of Contents


1 – Title of Project / Teaser Pitch (one line)

2 – Overview of Concept / Interactive Art / Game play type /  Audience experience / Story scenerio

(short paragraph)


3 – Inspiration from Art & Design / image examples

4 – Inspiration from Game world / image examples

5 – Inspiration from Emerging Media or Current events, Science, Psychology, Social Issues / examples


6 – Visual Concept / Style / Mood / Color examples

7 – Settings / Backgrounds / Locations

8 – Interactive Design Strategy / Abstract or Narrative ?

9 – Primary Characters / Creatures / Villains / Forces ?

10 – Props / Special Objects / Magical elements ?


11- Target Audience / Secondary Audience

12 – Communication with Players (tools for connecting)

13 – Technology audience will need to Interact or Play

(what devices, software, online access will your audience need?)

14 – Physical elements needed to Play or Experience

(objects, tools, places, transportation needed to play?)

15 – Goals for Game Play / Primary goal / Secondary gains


16 – How does this project proposal make use of available skills and talents?

17- Artistic Skills Needed to Create (be as specific as possible)

18 – Digital Skills Needed to Create (be as specific as possible)

19 – Technology needed to Create (software, hardware, online access)

20 – Materials needed to Create ( art supplies, objects, tools)


21 – Outline briefly the timeline of the installation or game / how long is the audience experience? / when will it launch? ideal start and stop dates?

Climate Remix / Projection Project


Our idea is to create interactive projections on the theme of CLIMATE REMIX. The first step will be to create original visual content to first test indoors. Building on the ideas we used with Fishnetstockings, the project will include layers of media… Video backgrounds, Kinect to create interactivity, and possibly text or data pulled from live feeds about climate change. Sound can also be an element ( Dean is interested in contributing to an audio bank related to climate and weather). Students in my other mixed-media classes are creating an image bank of Climate themed collages and animations that can be integrated and remixed. (See collaborative folders on google drive with Climate Remix IMAGEBANKS and MASHUPS)
Students in our interactive media class are contributing via their own skills:
Charles is researching how to program the piece using UNITY, with advice from Pete Willemsen and Logan Sales.
McKenzie is creating textures and collage backgrounds to be mashed into videos
Melissa is creating floating elements as PNGs with transparent backgrounds (such as snowflakes, flames, and other signs of climate change.)
Paul is creating video backgrounds, possibly integrating video blocks, environments, or animations made with the motion capture characters. Themes involve weather and seasons.
Chris can choose to assist with visual assets ( images, videos) or work on interactive code with Charles, by using the wii or Kinect.
Dean can create a collections of audio files for use with the videos on the climate themes.
Joellyn can also assist with additional video shoots if desired in MMADlab chromakey studio, and demo techniques of digital compositing layers for a rich visual background.
Kathy McTavish may visit later, to lend advice about creating code and using multiple projectors in installations. We will discuss issues with creating environmental installations. Later we will attempt to project onto SNOW, pointing projectors out windows from upper classroom onto snow below. The interactivity may not work outside, but this is an experimental project, dealing with Climate change, weather, temperature, wind, water, fire… IT would be especially cool if it could somehow tap data on climate or current weather conditions / that may somehow impact the visuals.

Jan-Feb Class Plan for Art2018

Jan. 25–Meet in classroom, discuss recently noticed finds. Charles demo of game. Learn to use Fuse to make 2 characters for game or animation (FBX files). Possibly design a character that could be used for Dance Conference as well. Think about movements. Add more images to Motion Capture research reports in google slides or powerpoint. Export research slideshow as powerpoint or movie file and upload to class google folder so that Joellyn can post your research to the class blog.

Feb.1–Meet in MMAD Lab. Chris & Melissa suit up. Rebecca Katz-Harwood will lead a short workshop in movement. Capture Melissa & Chris. Slow simple motions are easier to capture. Be done with capture by 4 pm. (Noah leaves) Go to classroom… Bring your avatars / Fuse characters, brainstorm ways to use mo cap with your characters for narrative or experimental uses.  Storyboard either animations or games or both. Make backgrounds in Photoshop for game or animation.

Feb.8–Meet in MMAD Lab. Lisa & Noah will have added motion capture skeletons to avatars. Show animated characters in Unity. Figure out how to add animated characters to Charles’ game or backgrounds for animation. Discuss / Brainstorm Snow projection project.
Tentative Schedule in Feb and Mar:
I would like to do the SNOW PROJECTION project before SPRING BREAK. I would love you to be scouting locations for projection onto and from around campus. Shoot photos of cool ideas. Think about what themes or interactivity might be integrated into the snow projections.
Feb. 15— Meet in MMADLAB. Demo completed short animation or game.  MOVE to CLASSROOM / Begin planning snow projections.
Feb. 22–Meet in Classroom / work on SNOW PROJECTIONS / interactivity and videos and test with projectors. Try to Project onto snow.
AFTER SPRING BREAK: Meet in Classroom. Discuss Future projects / Pitch Ideas to class. DEMO / experiment with Dean’s Max MSP audio project.

Motion Capture Research

The semester has begun for Art2018 students in this experimental class, introducing creative projects that use emerging interactive media. Lisa Fitzpatrick has been team teaching and covering for Joellyn while she is busy with a family emergency. (Thanks to Lisa!!!)

Last week Lisa gave this MOTION CAPTURE RESEARCH Assignment, see below for Joellyn’s additions:

Write a 2 page research paper about motion capture. Use this assignment to learn about it for yourselves. You can write about different uses for motion capture, different ideas you have for its use, where the markers should go on the motion capture suits, what kind of motion is useful in games or animation, how motion capture is used in science or medicine or sports or whatever. Whatever is interesting to you. We have the Vicon system and I am thinking to use Blade with you. We also have the Optitrack portable system. Please email this to me (Lisa Fitzpatrick) and Joellyn Rock.
Joellyn’s Additions to the Motion Capture Research Assignment:
Next step:
Please find visual examples (images, screen grabs, links to video and other media) to support your research. Each student’s research will be posted here on the blog with links and images to illustrate what you are finding and examples of exciting uses of motion capture in digital art projects. I am very interested in the ways that motion capture might be integrated into class creative work. I know Lisa has plans for using Fuse and Blade softwares with the system in the MMADlab.