rhit2015

Thanks for hosting an AMA!

1. I'm wondering about the latency of remote rendering, especially since the user or environment might be moving while data is sent to the server, processed, and sent back. Do you find that it's useful for the server to make predictions about the user's movement? Or for the device to make final corrections on the servers output before displaying it?

I've done some cloud gaming and it's usually pretty smooth, but I could see where dealing with the physical 3D world could require even lower latency or higher detail.

Edit: I see that the documentation mentions head pose prediction but not making corrections after remote rendering.

2. Do you see MR devices being used mostly alone or as groups? Do you see a role in local mesh networking between devices to improve the accuracy of sensors or the number of polygons each device can display?

3. Do you expect compression techniques to improve for live streaming of 2D or MR data? Looking up some quick numbers, your video below mentioned about 16Mbps for a particular model (30fps, two eyes), Netflix 1080p (probably 24 fps) is 5Mbps, Stadia 1080p (probably 60 fps) is about 10Mbps. Netflix has an advantage in that they can spend more time preprocessing and the future is already known. Increased framerate requires more data, but not linearly because the changes per frame become smaller and more predictable for each marginal frame. On a MR device, you could potentially accept an intermediate dataset that allows greater compression because you have enough processing ability to finish final steps that expand the data. So I'm not familiar with advanced compression techniques and don't have an intuition here, but it seems like there's room for noticeable improvement. Do you agree? And do you see this improving soon? Or will it only be a focus as consumer use becomes more common?

Thanks for the response!

Reading this AMA has been a cool way for me to be more aware of what's going on and I'll be thinking a little more seriously about working in the field in the future.

That 40-100Mbps recommendation looks really high to me for a single user. I'm guessing there will be ways to improve that eventually, but a tool that has room for improvement is still better than no tool.