Internet of Sensing
- [Fleet Week, San Francisco, 2014 - Jeff M. Wang]
- Overview
The Internet of Senses (IoS) - often also referred to as the Internet of Sensing - is a futuristic concept for a hyper-immersive, multisensory digital experience. It extends beyond traditional screens and speakers to blend digital content with the physical world, creating sensations of touch, taste, and smell.
This is achieved through a network of advanced sensors, actuators, and emerging technologies like brain-computer interfaces (BCI).
The Internet of Senses (IoS) is a futuristic concept that will enable a hyper-immersive, multisensory digital experience by connecting human senses to the digital world.
Moving beyond current screen-based interactions, IoS aims to blend digital content with the physical world to create realistic sensations of touch, taste, and smell.
This technology is expected to be enabled by a convergence of advancements in artificial intelligence (AI), extended reality (XR), haptics, and high-speed networks like 6G.
- Key Concepts and Technologies
The IoS relies on several key technologies to deliver multisensory experiences:
- Brain-Computer Interfaces (BCI): IoS envisions using the brain as an interface, potentially allowing for the exchange of cognitive senses with minimal physical effort. Non-intrusive BCIs could allow users to navigate AR interfaces simply by thinking.
- Extended Reality (XR): This umbrella term includes Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR). Lightweight AR glasses and contact lenses are predicted to project digital objects into the physical world with high realism.
- Advanced Haptics: While haptic feedback exists today, the IoS will feature advanced wearables that replicate the sense of weight and motion, allowing users to physically interact with virtual objects.
- Olfactory and Gustatory Systems: Research into digitally recreating taste and smell is underway. Prototypes exist, such as a device that artificially recreates any flavor on a user's tongue.
- Artificial Intelligence (AI): AI and machine learning will process vast amounts of data in real-time to generate personalized sensory experiences based on a user's physiological responses and preferences. AI will also enhance safety by using sensory technology to detect deepfake calls or identify odors.
- Next-generation Networks (5G/6G): Real-time transmission of sensory data will require ultra-low latency and high-bandwidth capabilities. Networks like 5G and 6G are critical infrastructure for the IoS.
- Potential Applications and Uses
IoS has the potential to transform numerous industries and aspects of daily life.
- E-commerce: Online shoppers could feel the texture of clothing or smell a perfume before purchasing it.
- Entertainment: Gaming and film will become fully immersive, allowing users to feel the warmth of a desert scene or the chill of an arctic one.
- Education and Training: Students could learn history by "walking" through a virtual Roman marketplace and smelling the air. Surgeons could practice with realistic haptic feedback.
- Healthcare and Well-being: Sensory-driven therapies could treat PTSD or chronic pain. AI-powered diagnostics could detect diseases like Alzheimer's through smell or speech patterns.
- Retail: Online shopping could be enhanced with multisensory feedback, such as being able to "feel" the fabric of a garment or "smell" a perfume before buying it.
- Accessibility: For people with disabilities, BCI could restore autonomy by allowing them to control devices with their thoughts, and sensory-enhanced interfaces could open up new ways to experience the world.
- Social Connection: Immersive telepresence will allow for realistic remote meetings and even physical interactions like hugs over long distances.
- Limitations and Ethical Challenges
The potential benefits of IoS are accompanied by significant limitations and ethical concerns that must be addressed.
- Privacy and Surveillance: IoS technology could collect vast amounts of intimate personal and physiological data. Concerns exist over who owns this data and how it will be protected from misuse by corporations or governments.
- Reality Confusion: As digital and physical experiences blur, individuals may struggle to distinguish reality from fabrication. This could enable manipulated perception, false memories, and digital hallucinations.
- Addiction and Psychological Impact: Hyper-realistic virtual experiences could lead to addiction, psychological distress, and detachment from the physical world. The potential for a "synthetic happiness" problem exists if people begin to prefer digital reality.
- Security: Multi-sensory experiences rely on robust security, as vulnerabilities could expose deeply personal data or allow for the manipulation of perception.
- Interoperability and Standardization: The lack of uniform standards could hamper the seamless exchange of sensory data between devices from different manufacturers.
- Economic Inequality: If advanced sensory technology is expensive and exclusive, it could exacerbate the digital divide, creating a new form of inequality.
- Internet of Senses vs. Internet of Sensing
The terms Internet of Senses and Internet of Sensing are sometimes used interchangeably, but a key distinction can be made.
- Internet of Sensing: Primarily focuses on using a network of sensors to collect data from the physical world, which is the foundational technology of the Internet of Things (IoT). For example, smart home sensors gather temperature or motion data.
- Internet of Senses: Represents the next evolution, using that network to deliver information to the human user in a multi-sensory and hyper-immersive way. It goes beyond sensing the environment to digitally replicating and augmenting human sensory experiences.
- How IoS Blends Physical and Digital Experiences
The IoS will fundamentally change human-computer interaction by eliminating many current barriers.
- Sensory input from the mind: With technologies like BCI, mental commands could directly manipulate the digital world. This bypasses the need for physical actions like typing or swiping, creating a more seamless link between thought and action.
- Sensory output as a "sense": The system will be able to provide feedback in the form of a direct sensation, rather than a visual or auditory notification. For example, the system could produce a specific scent to notify you of an event.
- Blurring the line between real and virtual: IoS will allow remote interactions to feel like physical presence. It could enable you to smell the ocean breeze while on a virtual beach, or feel a handshake during a remote business meeting.
- Challenges and Considerations of IoS
While offering transformative potential, the IoS also presents significant ethical and practical challenges.
- Privacy and security: The transmission of sensory and brain data involves highly personal information, raising major concerns about data security and privacy.
- Ethical implications: The ability to digitally control or influence someone's senses raises complex ethical questions about free will, identity, and the nature of reality.
- Cost and accessibility: Early IoS technologies, particularly advanced BCIs, are likely to be very expensive, potentially creating a new digital divide.
- The Internet of Things (IoT) vs. The Internet of Senses (IoS)
The Internet of Things (IoT) is a broad concept of interconnected devices collecting and exchanging data to create smarter environments, whereas the Internet of Senses (IoS) is a future evolution of IoT focused on creating immersive, multi-sensory digital experiences by enabling human senses like sight, sound, touch, taste, and smell to interact with the internet.
1. Internet of Things (IoT):
- Definition: A vast network of physical objects—"things"—embedded with sensors, software, and other technologies that enable them to collect and exchange data over the internet.
- Focus: Collecting data from various devices to improve efficiency, automation, and the quality of life in consumer, industrial, and commercial settings.
- Examples: Smart home devices, connected cars, industrial sensors for manufacturing, and agricultural monitoring systems.
- Key Aspect: The primary function is data collection, transmission, and basic analysis to enable smart decision-making and automated processes.
2. Internet of Senses (IoS):
- Definition: An extension of the IoT focused on replicating or enhancing human sensory experiences over a network, enabling digital interactions to feel "real".
- Focus: Blending digital experiences with the local environment and allowing remote interaction with people and devices as if they were present.
- Examples: Immersive virtual reality, remote control of machinery using tactile feedback, online shopping with sensory experiences, and merging the five senses with reality.
- Key Aspect: It goes beyond data collection to create rich, multi-sensory digital and virtual experiences, leveraging advancements in AI, virtual reality, and haptic technology.
3. The Relationship:
- Evolutionary Step: The Internet of Senses can be seen as the next phase or a specialized application within the broader framework of the Internet of Things.
- Sensors are Foundational: IoT devices often utilize sensors to collect data, and IoS builds upon this by focusing on how these sensory data streams can create immersive human experiences.
- Focus Shift: While IoT is about "connecting things to the internet," IoS is about "connecting senses to the internet" to facilitate deeper, more intuitive interactions.