The brave new world of Virtual Reality, Augmented Reality and Mixed Reality has a lexicon all its own. This prompted Jason Peterson, chairman of Go Digital Media Group and CEO of ContentBridge Systems, and Ramón Bretón, CTO, 3rd i QC, to literally come to terms with this world. Peterson, an Entertainment Merchants Association (EMA) board member, and Bretón, an EMA member, in turn sought input from Philip Lelyveld of the USC Entertainment Technology Center, to fashion a dictionary to better enable content producers and agency creatives to communicate when working in this arena.
The EMA has long maintained a digital steering committee that sets standards for the entertainment industry. In the past, EMA has set standards for mezzanine files, metadata files and core definitions for the digital supply chain that were quickly adopted by the entertainment community. Continuing in that tradition is this work done by Peterson, Bretón and Lelyveld. Below is a sampling of definitions in that VR, AR and MR dictionary.
360° Video / Spherical Video / Immersive Video – A video recording where a view in every direction is recorded at the same time, and during playback the user has control of the viewing direction.
3DOF (3 Degrees of Freedom) – A piece that reacts to user head movement along three axes: pitch, yaw, and roll. See “3DOF+” and “6DOF.”
3DOF+ (3 Degrees of Freedom Plus) – A piece that reacts to user head movement along three axes: pitch, yaw, and roll, and also includes limited interactivity based on head movement, such as parallax movement, lighting effect changes, etc.
6DOF (6 Degrees of Freedom) – A piece that reacts to user head movement along three axes: pitch, yaw, and roll, and also reacts to body movement along three axes: forward/backward, left/right, and up/down.
Ambisonic Audio – A method of capturing and playing back a 360° sound sphere.
Augmented Reality (AR) – Computer rendered image or data that is overlaid over the real world where your brain is actually located. It is the addition of sensory input to your brain while your brain is getting its normal sensory input from its surroundings.
Avatar – A representation of the user in a virtual space.
Binaural Audio – Reproductions of sound the way human ears hear it. In fact, the word “binaural” literally just means “using both ears.” When you listen to a binaural recording through headphones, you perceive distinct and genuine 360° sound. Binaural recordings frequently use a “binaural dummy head”, a model of a human head complete with anatomically correct ears and ear canals, with a microphone located at the base of each ear canal.
Cybersickness – See “Simulator Sickness”.
Directional Sound – A technology that concentrates acoustic energy into a narrow beam so that it can be projected to a discrete area, much as a spotlight focuses light. Focused in this manner, sound waves behave in a manner somewhat resembling the coherence of light waves in a laser.
Empathy – The intellectual and/or emotional connection with the subject(s) of a piece, which tends to be stronger in VR compared to traditional visual media, due to the immersive nature of VR technology.
Equirectangular Projection or Mapping – Translating a spherical source into a rectangular presentation. One artifact of this mapping format is horizontal stretching towards the top and bottom of the image, as the poles are stretched to the entire width of the image, as in a two-dimensional map of the earth. See “Flat File.”
Eye Tracking – A technology that monitors eye movements as a means of triggering changes in the content being consumed. For example, software interactions based on where the user is looking, or increasing the bit rate to the portion of a streaming 360° video that is currently being viewed by the user.
Field of View – The extent of the observable world that is seen at any given moment. With VR, MR, and AR applications, a wide field of view simulating the human visual experience provides a more immersive viewing experience. Early generations of VR hardware had limited fields of view.
Flat File – A 360°-video file which is suitable for viewing on a video monitor as it displays the entire range of captured images, as opposed to the limited field of view presented at any one time in a VR headset. Typically uses equirectangular mapping.
Flicker – A visible artifact at refresh intervals on display devices, commonly caused by insufficiently high refresh rates. For VR applications, a minimum refresh rate of 90 frames-per-second is recommended.
Foveated Rendering – A developing technology which uses eye tracking to maintain maximum resolution for portions of the VR image currently being viewed, while lowering the resolution of portions of the VR image not being viewed, thereby lowering the overall bitrate of the program.
Gaze Input – A method of triggering events in a VR experience based on the user maintaining the same head position for a certain amount of time, thereby indicating that the user is looking at a certain location. Some experiences may display a reticle (usually a small dot) in the center of the field of view to aid in aligning the HMD over the desired object.
Haptic Feedback –The use of the sense of touch in a user interface design to provide information to an end user. The resistive force that some “force feedback” joysticks and video game steering wheels provide is a form of haptic feedback. (Often referred to as simply “haptics”.)
Head Mounted Display (HMD) – A pair of goggles or a full helmet with a tiny monitor in front of each eye. Because there are two monitors, images can (some can be monoscopic) appear as three-dimensional. In addition, most HMDs include a head tracker so that the system can respond to head movements.
Head Tracking – A technology that enables the VR software to determine where the user’s (apostrophe) head is in a predefined space. Head tracking in VR is normally used together with hand tracking and, in some instances, even finger tracking.
Horizontality –Maintaining a level horizon in the captured or rendered image in order to prevent or minimize the effects of simulator sickness.
Immersion – Deep mental involvement. Immersion into virtual reality is a perception of being physically present in a non-physical world. The perception is created by surrounding the user of the VR system in images, sound or other stimuli that provide an engrossing total environment.
Inside-Out Tracking – Positional tracking that uses cameras and/or sensors located within or on the VR headset.
Interactive Parallax – In a stereoscopic VR piece, changes in the user’s field of view via head movements that cause foreground objects to move or change position at rate independent from objects in the background, mimicking human binocular vision. See “Parallax”
Interpupillary Distance – The distance between the center of a person’s pupils. High-end VR headsets allow this distance to be set for each user, increasing the comfort and effectiveness of the HMD.
Judder – An instance of rapid and forceful shaking and vibration.
Latency – See “Motion-to-Photon Latency.”
Mixed Reality (MR) – A variant on Virtual Reality in which part computer rendered 3D elements and part photographed real elements are combined into an immersive experience that simulates a user’s physical presence in the environment.
Monoscopic – A VR piece which presents the same view to the left and right eyes. See “Stereoscopic.”
Motion-to-Photon Latency – Discrepancy between user interaction and the resulting response in the VR image. Increased motion-to-photon latency can contribute to a break in the illusion of immersion, as well as simulator sickness.
Outside-In Tracking – Positional tracking that uses cameras and/or sensors located outside of the VR headset, either mounted on walls or free-standing. See “Positional Tracking.”
Parallax – The change in position of an object when the user changes their viewpoint. In stereoscopic VR pieces, the amount of positional change varies for objects located at different apparent distances from the user. See “Interactive Parallax”.
Pitch – In regards to VR, titling the head up and down, rotating along the horizontal axis through the ears.
Positional Tracking – Using cameras and/or sensors to determine the location of the user in their physical space, to translate that information to the user’s position in a virtual space. See “Inside-Out Tracking” and “Outside-In Tracking.”
Postural Instability – A symptom of simulator sickness that causes an inability to keep the body in a stable or balanced position, sometimes even after the user has removed the VR headset.
Presence – The degree to which your brain believes you are present in a particular place. Virtual reality tries to trick your mind into believing you are actually somewhere else.
Refresh Rate – The frequency with which the image on a computer monitor or similar electronic display screen is refreshed, usually expressed in hertz. Virtual reality requires at least 90Hz or the motion may make people sick.
Roll – In regards to VR, tilting the head side to side, rotating along the horizontal axis from the nose to the back of the head.
Room-Scale – A type of VR technology where the user is intended to move around their physical space, instead of remaining in a fixed position.
Sensory Conflict – The condition of a person’s senses indicating something different than what their inner ear experiences. For example, when a VR experience places the user in a fast-moving rollercoaster, but they are motionless in their physical space. Sensory conflict is believed to be a contributing factor to simulator sickness.
Simulator Sickness – A subset of motion sickness that is typically experienced by pilots who undergo training for extended periods of time in flight simulators. Symptoms of simulator sickness include discomfort, apathy, drowsiness, disorientation, fatigue, vomiting, and many more.
Stereoscopic – A VR piece which presents different views to the left and right eyes, mimicking human binocular vision. See “Monoscopic.”
Stitching – The process of combining multiple images or video streams that are from overlapping fields of view into one large field of view in a higher resolution image or video. This is often utilized to create 360° video.
Vection – The illusion that the user is moving caused by objects moving in some portion of their field of view. Vection is believed to be a contributing factor to simulator sickness.
Virtual Reality (VR) – In pure VR, your brain is getting all its sensory input from a time and place other than where your brain is located, and you are able to interact with that other time and place as if your body were actually there. Commercial and technological realities often mean this is a computer rendered 3D environment that is intended to be immersive, often interactive, and simulate a user’s physical presence in the environment. However, guiding a robot with a camera where your VR headset is displaying the reality around the robot is Virtual Reality. You are virtually, but not really, there. Implementation is usually through a virtual reality headset.
Virtual Reality Sickness – See “Simulator Sickness.”
XR – A “catch-all” term to refer to any of the “realities”, VR, AR and/or MR.
Yaw – In regards to VR, moving the head side to side, rotating along the vertical axis down from the center of the top of the head through the throat.
DISCLAIMER: These definitions are an attempt to collate what the authors feel are the most accurate, clear and concise definitions available from public and private sources. The contributors to this article did not invent all of these definitions, although they were polished where it was felt it was necessary. Special thanks to Philip Lelyveld at the University of Southern California’s Entertainment Technology Center for lending his expertise in VR and AR.
