Vincent – Augmented Reality Project

Video presentation of the augmented reality project for the Creative Technology Master – Middlesex University/London.
Softwares: Blender 3D and Unity.
Credits:
Carol Sant’anna
Matteo Becalli

    Small Object of Delight Project (SOD)

• Technical inspiration
  For what concerns technical inspiration, the first idea has to be connected with the animation movie Wall-E. In this movie, a robot that can’t properly speak and have no real facial expression – except for the eyes – expresses his emotions through actions and other associations with the world around him. Whenever he is afraid he transforms himself into a box or when he is sad he just angles his camera-eyes to fit the natural sad expression. This movie shows how not human-like characters or humanized characters can interact and express emotions in a way that the user can easily understand and identify itself with it.Wall-e The second inspiration is a project made in the School of Design at the Victoria University of Wellington, by two students. The project is called Pinokio, and it’s a lamp modified with servo motors, programmed arduino and processing that follows the user and interacts with it in a way that is completely “natural” for a lamp. Pinokio doesn’t want to be put to sleep or switched off, he plays with the user and tries to catch its attention.

• The third inspiration is a character known by every office users, his name is “Clippy” and it’s an annoying clip character that is presented as the Office suite assistant in the Microsoft software. Although it was criticized by users and Microsoft employees, Clippy was initially cancelled from new Office suite versions, but eventually came back as the main character in the gamified product Ribbon Hero 2, an interactive tutorial made by Microsoft to help users understand new Office features and challenge them in competitive games inside the suite.

• Concept – Project idea
  The idea of creating a cartoon character that is a sunflower was inspired by the expressionist artist Vincent Van Gogh. Van Gogh’s famous paintings, the “Sunflowers” represent conceptually what the expressionism trend was: human feelings transformed into art/painting. The group understands that this is exactly what interaction means: express yourself with actions and having some feedback from the other. Furthermore, the sunflower always follows the sun – so, in the project, the sun is the user. Vincent, the sunflower, than follows the user actions, and respond to it in a very charming and entertaining way.The concept and design of the project is to compile an AR (Augmented Reality) software that can be than upgraded to fit into portable devices, smartphones and tablets. The idea is to combine artificial expressions with a human-like character to create a pet or a companion for the user that represents a delightful virtual object. The character is humanized and tries to catch the users attention without being annoying – the virtual pet has no “serious” functions, it is just to entertain the audience. The scenario imagined is a typical office worker that can entertain itself by playing and watching the character in his virtual world.On the other hand the group wanted to challenge itself by exploring software and technologies group members have never tried before like 3D modeling and Unity programming. This represents one of the most important points of the project, although its realization and concept creation has been done in relation to what was asked in the brief, the group exploration in new technologies and new software skill acquirement represents one of the most important achievement of this project.

• Technical Part – Software and logic
  The project is an AR program compiled to recreate a humanized character that can interact with the user via the camera position. This process is made in Unity 3D thanks to the AR toolkit extension. The character has been created in Blender and imported into the software after the modeling, texturing, rigging and animation processes. The program is based on Vuforia examples and is compiled to identify an image target, usually called tracker, trough a camera to reproduce on top of it the 3D character in a virtual environment based upon image information like angles, distance and position. The interaction with the user happens thanks to virtual buttons, a virtual entity created with Unity upon the image tracker that are used to trigger the character movements. The scripting part has been compiled in C# via event handlers that trigger events inside the update function, a function called every frame. These event handlers manage animations calls,translations and rotations events of the 3D object.  

• Personal experience
  The biggest challenge in the Project was to learn 3D modeling from scratch – as I never had that experience before. So the first thing to do was research and choose which software should be used for that. After talking to some classmates and teachers Blender was chosen as it is open source, free, the most used platform for gaming (and our character has some gaming features) and has a good integration with Unity – the chosen software to do the AR. The character was first drawn on paper, with some different options of face, body and hands/feet. The most simple one was chosen, as to model a flower would be already a hard work to do. After having some Blender sessions and watching some tutorials,  the character creation began. Some bugs happened during the process and John Cox helped a lot with those and with some main doubts about the Blender interface and how to build the character on it. “Vincent” the sunflower was then built, textured, rigged and animated. Some changes from the initial drawing had to be done to better adapt to the 3D features.The construction process took quite a long time, during 3 to 4 weeks and for some moments the group though of changing the character to a more simple one. Than a little ball was created and animated, in case the sunflower did not work out as expected. Fortunately, after numerous tweeking over the character and its animation, the result was quite good. Although, the repertory needs to be expanded, the design, animation and integration with Unity’s platform worked well.

• Conclusion
  The final result of the project, both visual and technically, was considered good. Extra movements and actions from the character would complete the application, and some tweeks are needed on the Unity integration. Some uncertainty and doubts may have emerged during the process of the project, but the group learnt a lot from its development and it was worth doing it.