Pixel Streaming vs. On-Device (Local) Rendering — A comprehensive business guide.
(Disclaimer: I am a Founder at METAVRSE, a 3D platform for on-device rendering, so I am inherently biased. I have tried my best to remain neutral and provide the best advice for both technologies)
For brands and organizations looking to leverage 3D game engines to build more immersive and engaging solutions for retail, marketing, training, architecture, digital twins, collaboration, design and manufacturing, the decisions to use PC, Mobile, Console or VR can be a bit complicated. One way to ensure that your experience hits all of these is to publish to the web, but bypassing the app stores comes with its own questions; Do you choose on-device (local) rendering or pixel streaming?
Pixel streaming and on-device (local) rendering are two distinct methods used in computer graphics to deliver interactive experiences to users over the open web without app downloads. These technologies prevail when organizations are looking to create robust 3D experiences and deliver these to the maximum number of devices, without having to build and maintain multiple apps across iOS, Android, MacOS, Windows, VisionOS, Console, etc. Pixel streaming and on-device rendering offer the flexibility and open architecture of the web, with the power and fidelity of native app solutions.
While both approaches aim to provide high-quality visuals and responsive interactions, they operate in fundamentally different ways and offer unique advantages and disadvantages.
Pixel Streaming involves rendering graphics remotely on powerful servers and then streaming the resulting video frames to the user’s device in real-time. This approach offloads the computational burden from the user’s device to the server, allowing users to experience high-fidelity graphics and complex simulations even on low-powered devices such as smartphones or tablets. Pixel streaming is commonly used in cloud gaming services, virtual desktop infrastructure (VDI), and remote visualization applications.
Meta’s Oculus has made it clear that pixel streaming is not allowed on their Quest VR products.
A company that is using this technology with great success is Holo-Light, an enterprise-grade platform for XR applications in engineering & training where a select few people require access to massive files (ie. CAD models for design/build). Typically, organizations will build on Unreal Engine and use cloud render farms to deliver this content to users. Sometimes there is a strange glow to the output and quality of the stream depends on the users network connection.
On-device (local) rendering, in contrast, refers to the process of generating graphics directly on the user’s device using its local hardware resources, such as the CPU and GPU. This approach offers low latency and high responsiveness since the rendering is performed locally, without the need for data transfer over the network. On-device rendering is typically utilized in video games, augmented reality (AR), and virtual reality (VR) applications where real-time interactions and immersion are critical. A new, web-based platform for building and deploying experiences on-device is METAVRSE Engine, used for applications such as retail, marketing, training and collaboration that require a lot of people to access the experience from any device via the browser. Older devices may not be powerful enough to enable larger, higher-fidelity experience on-device.
USE CASES, PROS & CONS
Pixel streaming relies on powerful servers with high-end GPUs to handle the rendering workload, while the user’s device can be relatively lightweight and inexpensive since it only needs to decode and display the streamed video. Pixel streaming relies heavily on a stable and high-speed internet connection to deliver smooth and artifact-free streaming. Any latency or network congestion can result in input lag and degraded visual quality.
- Best use cases: High-quality 3D product demos, interactive training modules in industry, gaming. Security sensitive projects like military, oil and gas or others that require no data to be stored on device. Aerospace and defense customers will appreciate the fact that data is never transferred to the device. This is a security risk. By streaming pixels the classified data remains on a secure cloud or local server.
- Advantages: Pixel streaming makes it possible to display very complex and graphically sophisticated 3D models (>100M polygons) that would not be possible on normal mobile devices. It shifts the computing load to powerful servers in the cloud.
- Disadvantages: Pixel streaming is network dependent, so if you have a stable internet, it is great, if not, it suffers from jitter and glitching artifacts in the scene. Pixel Streaming bandwidth costs are very low (about the same as 4K video), but the GPUs in the cloud are extraordinarily expensive (compared to on-device rendering ~10,000x more), so using this for mass consumer projects is untenable.
As mentioned in Matthew Ball’s book ‘The Metaverse’, he refers to Tim Sweeney (CEO of Epic Games) view on the subject that ‘device-based computing will advance at a faster pace than reliable broadband connectivity’. This makes sense since devices follow Moore’s Law and increase in power fairly evenly distributed, whereas networks are fragmented and increase in power asymmetrically and unpredictably around the world. If you can control the network, and you serve a few people that require ultra high fidelity experiences (BIM, CAD, etc.) then pixel streaming is the only option to deliver these projects to myriad devices via the web. This comes with a cost, but typically the costs are irrelevant because the value being created is exponentially more (ie. selling faster, reducing design time or improving critical training). There is already a robust global infrastructure capable of both on-device rendering and pixel streaming.
On-device rendering requires a capable GPU and CPU on the user’s device to generate and render graphics in real-time, which may limit the compatibility with older or less powerful devices. On-device rendering operates independently of the network and can provide consistent performance regardless of internet connectivity, making it suitable for wider reach, less glitchy experience, good for real-time play or work sessions in multiplayer (ie. social games, MMOs, XR, collaboration, training, sales, design, spatial websites, etc.)
- Best use cases: Web-based applications, websites, product visualizations, e-commerce (vStores), casual games, interactive educational content, 3D maps and collaboration where a lot of people will access the experience either alone or multiplayer. Communications systems can also be made available through this method.
- Advantages: Since On-device rendered content displays directly from the browser, it does not require powerful hardware on the user side and is easily accessible. It offers broad compatibility with different device types and browsers. You build one version and deploy everywhere instantly. The web is open and not subject to platform fees. Instant publish allows for immediate changes without app downloads or updates. Web developers and Javascript engineers can work in their native coding languages.
- Disadvantages: The higher the fidelity, the newer the device required to power it. Pushing large projects requires applying limits to polygons, textures and does not allow for ray tracing (yet!). Web 3D creation is relatively new, so the tools and examples are limited.
On-device Rendering is better suited to reach a large number of users because it runs directly in the web browser regardless of the user’s hardware. It is ideal for applications that require broad accessibility, such as casual games, 3D websites, online shopping (vStores), collaboration and communication and educational platforms. Smaller 3D projects have the potential to be rendered on over 5 billion devices globally that support WebGL. With the introduction of new mobile and XR chipsets (Apple Vision Pro’s M2 and Oculus Quest 3’s XR2 Gen 2) the experiences that are able to run on modern devices is getting pretty impressive. Native browser support for WebGPU and WebXR will allow creators to build and deploy rich, immersive experiences to desktop, mobile and headworn devices with the freedom of the open web.
COSTS
One of the major deciding factors in which of these technologies to use comes down to economics. Pixel streaming, while an incredible technology, is roughly 10,000 times more expensive than on-device rendering. This is because you are using super-powerful rendering and each user requires their own dedicated GPU in the cloud.
Pixel streaming: $0.10/user/minute* or ~$6.00/user/hour (*Eagle 3D Streaming).
On-Device Rendering: $0.000001/user/minute* or ~$5.00/user/year (*METAVRSE Estimated)
More specialized services are as low as $3.00/hour (ZeroLight)
DECISIONS, DECISIONS.
The decision between pixel streaming and on-device rendering is a critical one for innovation and technology leaders, as it shapes the trajectory of business ventures in various industries. Both approaches offer distinct advantages and challenges, necessitating careful consideration to maximize business success.
Pixel streaming presents an attractive proposition for businesses seeking to deliver high-quality experiences to a narrow audience, regardless of device capabilities. Its scalability and potential for market expansion are particularly advantageous in industries like training, industrial, manufacturing and military. However, dependencies on network reliability and high operational costs require thorough analysis before implementation.
On-device rendering offers greater control over the user experience and enhances privacy and security. Yet, challenges such as hardware compatibility and content distribution strategies require close attention. The fragmentation of devices in the market, varying in processing power, screen size, and operating systems, necessitates careful optimization efforts to guarantee smooth and consistent performance across all platforms. The low-cost deployment of on-device rendering lends itself to casual games, mass consumer marketing, ecommerce (vStores), collaboration and learning at global scale.
Ultimately, the decision should align with business objectives, target audience, and resource constraints. Whether prioritizing accessibility and reach with pixel streaming or performance and control with on-device rendering, the choice is yours to make. Hopefully this guide has been helpful in making it easier for you to decide.
###
If you are looking to build a custom 3D project in marketing, retail, collaboration or training, feel free to contact me directly and we can help guide you in the right direction. For speaking engagements: alan@metavrse.com
Our products include: METAVRSE Engine, Meetopia, TheMall
###
MORE RESEARCH
