It seems like all the major players are taking game streaming seriously. I'm still not very sure about the viability.
There's the obvious DRM win. Accessibility will be higher with way lower or non-existent hardware costs. These probably look very enticing on a cooperate slide deck. Not to mention the subscription model.
On a more technical side game streaming is only really desirable for games that don't run well on commodity hardware (see AAA 3D action titles). Latency is very important for these titles and the general rule of thumb is that 50ms of input latency is readily noticeable to humans. Average home connections have ~20-30ms round trip latencies. These speeds are not a reality everywhere or consistent. Combined with hardware input, world simulation, and rendering latencies it can be difficult to get latencies low enough to be comfortable.
Modern games just have the client run its own world simulation and rectify with the server later to hide the network latency. This strategy won't be possible with streaming. Is there some alternate optimization that can be made for streaming? The architecture will definitely be simpler with the main-frame paradigm. Maybe if there's only one client it could be feasible to send some batteries included chunk of frames that can be easily constructed based on the next set of inputs. Is there any hint that progress can be made in this direction?
If the latency problem can't be solved, I'm definitely bearish or game streaming. Latency insensitive games for the most part are not difficult to run on commodity hardware.
I have no idea how reliable it is, but quoting wikipedia [1]:
"Testing has found that overall "input lag" (from controller input to display response) times of approximately 200 ms are distracting to the user. It also appears that (excluding the monitor/television display lag) 133 ms is an average response time and the most sensitive games (fighting games, first person shooters and rhythm games) achieve response times of 67 ms (excluding display lag)."
These numbers are from the PS3/XBox 360 area, which certainly sold massively well. Since display lag is not included in these 67ms, this is what these streaming services would have to aim for. It doesn't sound completely insane given the 30ms round trip latencies of home connections.
On a more subjective note, I think it's going to be really hard hunting down high end gamer PCs. On the other hand lots of games don't run particularly smoothly on consoles (especially "normal" PS4 and XBox One, rather than PS4 Pro or XBox One S) that these services are directly aiming at. I can perfectly imagine a stadia version of The Witcher 3 running circles around the PS4 version that lags so much... But I don't think it would hold against the PC version.
But I don't game much at all anymore, so my knowledge is mostly outdated. The last games I played were 80 days and A Dark Room on my iPhone (both pretty good games). So I guess I'm now more in the target demographics of Apple Arcade.
I followed up on the wiki article. The citation was a just eurogamer article that had little to with human perception of latency. It did have this nugget: “Criterion said it was aiming for 50ms latency [for burnout paradise]” which agrees with what I’ve heard making handheld electronics. The goal was always <50ms latency.
Personally I noticed a significant improvement switching to a 165hz monitor in basically every first person game. The only difference here is going from 16ms to 6ms frame granularity (10ms off the worst case). I’m not sure how much is visual smoothness or latency, but I’d gladly take refresh rate/fps over UHD.
I’m definitely in the enthusiast group though. I didn’t even realize that console players dealt with such high latency. It will be very interesting to see how the market plays out.
Hm... the topic is quite complex. Human perception is actually much slower than many people want to believe. Parts of the human vision system react no faster than around 1/25th of a second (other parts are faster - its complicated!). Hearing is similarly divided. AFAIK, auditory and visuak stimuli are processed together and merged over a rather large time window (to the point where one system can induce illusions in the other). But we are able to perceive time delays within a complex stimulus with very high precision.
I am not totally convinced that a high refresh rate monitor allows human vision to respond faster. I would rather suspect that most game loops are VSync-locked and a higher refresh rate leads to better input sampling and processing.
Maybe I should set up a simple test where I show a non-interactive sequence at various framerates and ask users to identify the highest framerate. And then repeat with interactive camera controls.
>I am not totally convinced that a high refresh rate monitor allows human vision to respond faster.
Have you ever watched a high-level FPS player? They can aim and shoot in the span of about ~1/5 of a second, occasionally even faster. That's 200ms. At 25hz, that's 40ms. A big part of it is going to be increased input processing, but a lot of it is muscle memory at that skill level. At 165hz, that's 6ms. With two equally skilled players, one will see the other up to a whole 34 ms sooner, in a process that takes about 200ms total. That's going to show up as a highly statistically significant advantage. Of course, the increased sense of connectedness to the game at higher rates is another big advantage, but these effects combine. Pit one skilled shooter against another at equal refresh rates, but with one dealing with 34ms of additional input latency, and they're not going to play as well as they should.
Competitive players explicitly disable vsync, because the higher the game framerate, the faster it processes input. Also, the sooner you see updated information on the screen mid-refresh, but that doesn't help as much as a wholly higher refresh rate.
All of this only matters if cloud gamers were playing against locally-rendered players in the same match though.
There is a reaction delay but it is added on top of all the other delays. If the monitor is delayed by 1/25th of a second and your perception is delayed by 1/25th of a second then the worst case latency is 2/25th of a second. That is a 100% increase in reaction delay. If you think a 100% increase isn't significant what do you think about the 250% (or more) increase that streaming will cause?
I am not arguing against the existence of such an effect. I am loathe to state that the cause is that your eyes see more images per second on screen. I am not convinced that a higher framerate improves perception of the image content when over ~50Hz. I'm rather suspecting that the traditional coupling of game simulation and input processing rates to screen refresh rate is the cause: games seem more responsive because input sampling is more precise when the game loop runs faster.
Console gamers have long been conditioned to accept high input lag with laggy televisions, controllers and low-dexterity games and game mechanics.
Quake III makes all modern games feel slow. But that hasn't had the slightest effect on the typical gamers mindset. If only certain type of games work with cloud gaming then AAA publishers will only make those type of games. Exactly like the case with EA and microtransactions.
Is this a typo? Anyway, audio feedback is a critical part of games. Pressing the "fire" button and waiting 100ms for the sound of the gun would be insanely noticeable
The inverse is also true. Sound triggers and local animations are often used to mask latency in games like DOTA which need a server round trip before your commands have any affect on the game world. It works remarkably well to make games feel responsive even when they’re not.
However, with these streaming systems I’m not sure developers will be able to use tricks like that. It seems like these tech stacks are running all code remotely. I feel pretty skeptical about whether they can make the game feel good enough this way.
Actual video gamers right now go out of their way to get displays with very low input lags. 1080p144hz TN low input lag displays are used quite a bit in esports to shave a few msec. You literally win more often if you react faster. People aim to get TVs with sub 16ms input lag.
However that's not most of the market. The appeal of cloud gaming is for single player non-competitive games. These are the most pirated games and the games where input lag matters the least.
You can probably do even better than 20-30ms round trip if you're one of the large Cloud companies with data centers everywhere. Amazon so far doesn't seem interested, but Google and Microsoft definitely have the infrastructure to pull it off.
Sure you won't get high end PC gamers, that's not really the target demographic here. It's the millions of console and mobile gamers.
I can easily notice the difference between 20 and 40 ms in FPS games. I have gotten out of FPS's due to keeping up with the hardware requirements, but a streaming service like this could make it possible. That being said, if they cant stay below 50ms I probably wouldn't waste my time.
The Stadia trial convinced me that streaming is a near-term viable option. Latencies were not an issue. Bandwidth was, but when I got to play on a 50Mbps+ connection any issues I had encountered disappeared.
Moreover I was able to play a AAA title on my 2011 Macbook Air and that was fantastic.
What kind of games were played? Whether or not latency is an issue is hugely dependent on the title. A game like Sekiro: Shadows Die Twice where the bulk of the game play revolves around precision timing is going to be killed by any significant amount of latency. Unless it does something smart like trying to compensate for the latency on the client end (e.g. make a block succeed even if it was late, if it suspects that the player blocked at the right time from their perspective when viewing the delayed stream) I can't see how a game like that would be playable on Stadia.
> Unless it does something smart like trying to compensate for the latency on the client end
This is actually already done in many AAA titles. The idea is to basically always be showing a prediction of what will have happened in ~70 ms, and update your state model accordingly. Given a datacenter to run the games on and all of the announcements about being able to take a snapshot of the state, this doesn't sound super far-fetched.
But stadia can't use these techniques. The game engine and rendering isn't done on the client side. Normally a network game the client displays the user's actions immediately, and the lag is only seen from other player's perspectives.
On a streamed game, all interactions need a round trip before the user sees any response.
Doing anything where mispredictions result in a few frames of doing the wrong thing escape to the user is going to make games feel really inconsistent. If the game is running at 60fps and you have a system that can predict with 99% accuracy (which seems wildly optimistic) you’re still going to have mispredictions at a pretty high rate.
Not to mention that the game will have to be written especially to allow for the simulation to be rolled back and forth. For example a misprediction triggers a sword swing it turns out you didn’t make that traps the player in a state for a second. You end up giving the player a horrible experience where the game seemingly has a mind of its own. Or you can roll back and interpolate from the bad state back to a good one which would be ideal. Then the middle ground is to rollback and have a cut in the action back from misprediction. Then you realise this is happening every second or two.
Game state prediction doesn't cut it, or rather there's no point to it. The rendering is done server-side - remember they're streaming HD AAA games to televisions and tablets. There's always going to be a round trip between pushing a button and something happening onscreen. Google did say something about device inputs being sent straight from the controller to the server. Maybe this means they will cut down on latency between the input and the server, but it's still a trip back down from the server to the display on top of that.
I'd be quite satisfied with a reasonably priced tier that guaranteed 40ms round trip or better 99.8% of the time. The HFT levels could shave that time off and start stacking 9's for exponentially more money.
Bloodborne on PS Now? That sounds like hell. Plenty of later bosses have very narrow dodge windows. I already have a physical PS3 and PS4 so there's no reason for me to get PS Now. A couple co-workers of mine used it and none of them liked it. It works okay for RPGs, especially turn based RPGs where there is zero emphasis on timing or reactions.
If you're close enough to the server it's possible for network latency to not add much compared to the combined latency from the tv + console + wireless controller. (most modern tvs add at least 20ms, even on "game mode", some add more than 100ms).
It's a different story on a pc with a decent gaming monitor though.
For a networked multiplayer game, the total amount of network lag is effectively the same, except that the network lag is moved from between client and server to client and "display". Now that the server and client are now just one instance, there is zero lag there.
Considering that pretty much all multiplayer games in existence perform some sort of local prediction / state interpolation to hide the lag on the local machine, cloud only multiplayer will be considerably worse. After all you can no longer hide the lag locally, since you're not computing anything on the local machine, so the minimum precieved lag will go from 0 (for movement of your own player character) to the RT between the datacenter and your PC :/
Not entirely true. Many networked games use client side prediction, and collision detection (at least for collisions against terrain and walls, weapon hits are often computed on the server to curb cheating).
If you have a 100ms latency to a stadia server, then there's a 200ms latency between pushing forward on the stick and your character moving forward. This is not the case on a networked game, the client would start moving your player immediately (even if there is a 100ms delay to send those updates to the server).
This is only relevant if you're deliberately splitting the game across multiple servers (MORE work) so that people living in different cities can play against each other and have low latency to their own server. Other than that there's no reason to run one multiplayer game on more than one server.
Rather than purely streaming frames, you could push a frontend client in e.g. webgl + wasm on demand, cache artifacts locally, and send view-independent texture/mesh streams down. Doing so is technically more complex than having a rigid client/server split, but if they start with full streaming they can progressively make the client more intelligent, without being burdened with any specific hardware/client infra choices.
It's also easy to port edge-rendered games to many platforms. It makes sense for MS to ally with Sony and shift their effort to cloud before both the Xbox and PlayStation are little more than rich web clients.
This seems a lot more feasible. In fact, that's what the Nvidia Shield essentially is (although the edge computation is done with the customer's desktop computer).
I see this as being a more plausible implementation for say multiplayer twitcg shooters. Market needs to scale up a lot for this to be viable though...
Latency isn't an issue with games that have high time to kill or are purely exploration based, which seems to be something Stadia developers have noticed in the examples.
There will have to be local rendering for nearby objects and particles, but this is feasible.
DRM isn't totally likely, REnouveau does exist, and basic reverse engineering tools could be created to document the most common game engine functions.
Twitch shooters will never be replaced by this though.
Streaming based DRM doesn't need cloud rendering to be effective. The client can download 3D models and use prediction and local collision detection while the server is still responsible for the secret sauce like combat rolls, item drops, missions. Reverse engineering is still possible but even the private MMORPG server scene is actually pretty "dead" compared to the actual games they are cloning. Adding content is expensive and replicating content is sometimes impossible so the developers basically end up building what amounts to a custom game which uses the same rendering engine as an existing game. Becoming an indie dev and working for free on your own game ideas is a far better strategy although the chances of success are low.
> On a more technical side game streaming is only really desirable for games that don't run well on commodity hardware (see AAA 3D action titles).
I think it's actually more or less the opposite?
Twitch platformers like Celeste and Cuphead can't afford input lag, as they require very quick and precise movements from players. It so happens that these types of games almost always have low system requirements. Rhythm games and fighting games tend to be similarly lightweight. Street Fighter 5 won't run on a potato, but the hardware required is far from state-of-the-art.
By contrast, Assassin's Creed Odyssey and Red Dead Redemption 2 already contain large amounts of input lag when played locally according to Digital Foundry, and consumers don't seem to care much. This isn't to say that adding more latency is a good idea—and I have no idea if the existing latency could be whittled down to compensate for streaming—but it clearly doesn't bother people much.
Modern games just have the client run its own world simulation and rectify with the server later to hide the network latency. This strategy won't be possible with streaming. Is there some alternate optimization that can be made for streaming?
I'm working on a system that unifies single player and multi-player development. Think of it like zeit.co's "NodeJs Now" but for games, with a local-only configuration option. The way my system relates with streaming, is that the client is scaled way down in complexity. Basically, the sync protocol is a stream of positional updates from the server. The client becomes little more than a screen entity display. Then, the developer has the option to add a bit of "own world simulation" to further hide latency. (Or, in the case of the game I've implemented, just "own entity simulation" is enough.)
This kind of nuanced discussion of latency is something I only see among really serious gamers.
If you look at things like the Steam hardware survey, it becomes quickly apparent that the hardcore gamer (e.g. someone who has a high refresh rate display or similar and cares about this) appears likely a very small part of the overall gaming landscape, which is a very broad church now.
I think game streaming will ultimately win out hugely - “ordinary” (for lack of better term) gamers simply don’t care about the technical disadvantages. Huge numbers of console gamers already often add 100ms or more lag via their TV’s slow image processing and don’t even notice or care. I don’t think the average gamer is as latency sensitive as many readers here might be.
Most people are going to see that they can have a high end console/PC-like experience for a fraction of the cost of owning a high end console/PC, for the vast majority that is surely an appealing prospect. Finally, John Carmack himself believes the latency issue can be mitigated well enough for most things - that’s good enough for me ;)
My only real concern in all this is the future of video game mods. This is a pretty big thing now and hard to imagine this surviving well in a streaming based future. This would be a loss, creatively and functionally, for some people.
USA has no (local) competition for internet, it’s almost all regional monopolies or duopolies where one (cable TV company) is expensive and ok and the other (copper wire phone company) is slow and unreliable.
Some places (mine) are, by local ordinance, unable to receive high speed internet because the company that owns the rights/stupid ass poles that they run everything on along the road offers high speed internet access ALREADY. . . .
SO COOL!!
How do they do it? By defining high speed as dial-up speeds in the ordinance they helped the local small, technologically inept government craft!
6-9ms is not fibre here - I was referring to copper ADSL that is available pretty much everywhere in the UK (apart from some really isolated tiny villages/hamlets)
We care about the 50ms latency because the current games are designed for the platforms with <50ms latency. If the platform is changed, so the game too.
> Latency insensitive games for the most part are not difficult to run on commodity hardware.
Majority of people don't have a PC with a dedicated GPU. Or PC at all. Thanks to the Smartphone.
That said, I don't like the current trend and I hope they failed on streaming video game.
>Majority of people don't have a PC with a dedicated GPU. Or PC at all. Thanks to the Smartphone.
Problem is, smartphones still are shit for input (lol touchscreen) in any precise manners so what point is there in trying to run intensive games on it via streaming.
Outside of most FPS and some really tight-action platformers streaming pioneered by OnLive is more than enough especially for the casual gamer.
When OnLive was operational I had no trouble playing Batman (something something Arkham?) on their American servers from a MacBook Air in Sweden. I had a similar experience later with LiquidSky which was running regular games downloaded from Steam in their cloud and streaming video to my laptop.
If they do cooperate on creating a similar service, this will definitely be a net win for the gamers.
However, if that's what happens, and if the service is successful, we might end up seeing less games tailored to run on the console's hardware with low latencies, and we'll end up having cloud-optimised games (much like we have mobile-optimised games now). But this is pure speculation at this point.
I remember using Gaikai a bit. It was quite usable with non-reflex based games like From Dust. But that's quite common these days. I sometimes RDP over wifi to my gaming PC when to lazy to transfer saves for games like Factorio or KSP.
Still I have a hard time imagining first person games, since I despise those with significant input lag. Even just on a local PC some games have a horrible delay unless you run very high frame-rates (quite a bunch of those Unreal Engine) and I always have a hard time getting into those.
Now with an additional 20-30ms network latency... for this to be close to acceptable, they would have to get the on server delay close to nothing at the very least. I doubt that will happen while games are still release on multiple platforms.
The casual gamer is perfectly happy with cookie clicker or Fortnite on mobile. 2D games should be able to run on almost any platform and 3D games can be toned down until they run well too. The only type of gamer that isn't served by the things above is the "core" gamer that wants high fidelity 3D graphics and real time experiences. Really the only need that streaming serves is to make DRM possible which means the companies want to have complete control over the user experience and they certainly are going to abuse that control as much as they can.
Most 3D games don't even have to be toned down. Because if you stream video, you don't care if you send a video of a 2D game or a 3D game. And that's the main appeal of such services: you don't need to spend large sums of money to be able to play modern games in high or ultra high settings. All you need is a beefy internet connection with low enough latency.
Also keep in mind how important multi player is for most big titles now. If you add up player -server-player latency it's actually the same. Given more consistent performance on the rendering/frontend side it's quite possible the overall experience will turn out better.
Client-side prediction does a very good job of hiding the network latency. Granted a lot of games run their servers at lowish ticks/sec (people do complain though) which is 30ms worst case at 32tps.
You can do similar tricks in a cloud setup also though; notably you can do hit detection based on what the user was seeing when they entered the input rather than what the server was doing later. This is pretty similar to client side prediction.
When playing online against other opponents, having the gaming machines be very close to each other might make up part of the latency introduced by the distance between the player and the gaming machine. Maybe.
Your first point is key though. Piracy is a big deal. People can say until they're blue in the face that it "helps", or that "they wouldn't have bought the game anyway", and for some unknown percentage its true, and for some other unknown percentage its a straight lie.
If they go from "optionally play a game via streaming", to "this game MUST be played via streaming" eventually, they get the second group (the ones who WOULD pay if they couldn't get it for free) instantly.
They also lose everyone not interested in streaming games (like myself). I'm sure they've run the numbers, but I know plenty of people who actively purchase games and are not at all interested in streaming them.
The other consideration is bandwidth. Streaming a game is far more intensive than just playing it and many US households are under tight data caps (not to mention throttling under peak hours). I just am not convinced by the advantages, for much of the gaming demographic.
I’ve played Bloodborne on PS Now and it works great. This game is really dependent on fast reactions and smooth camera panning. I think if you try it out you might learn something.
I recall we all said the same crap about streaming HD movies with a monthly fee too. I think the only true problem with streaming gaming is that the quality of games will diminish even more.
Won't have much to do with net neutrality really since they will want the traffic onto non public fiber as fast as possible for more direct routes which will have a bigger effect
Yeah the bandwidth needs are going to be high enough that laying dedicated fiber is not only an advantage in terms of latency but also absolutely necessary because the regular lines will be at maximum capacity if everyone streams video games at 20mbit/s but ISPs will still refuse to upgrade their infrastructure.
Honestly I just don't get why every single time there is discussion about game streaming, the topic of lag as a potential show stopper comes up, and every time it's a hypothetical discussion, which doesn’t make sense at all - the solutions are out there! Don’t assume - just verify it.
And when you do, you will find out that there isn’t any actual issue. I’ve played games like The Witcher 3 via streaming over three years ago.
Input/output lag simply isn’t an issue, and I’m your average German VDSL user, nothing special.
From my point of view, there is no sense in any further discussion - the empirical data is already in.
Yeah, speed of light is actually a lot faster than people think. Honestly things such as controller and display lag are probably bigger than the round trip latency. And experimentally, anyone who has tried Stadia at GDC or I/O has said that latency wasn't an issue.
The bigger limiting factor I think will be bandwidth, especially in America with the awful internet provider situation.
"Yeah, speed of light is actually a lot faster than people think"
This would only be relevant if you had a direct fibre link in a straight line from your house to the service. It's packet switching\routing etc. that introduce latency. And even if you DID have direct fibre, the worst case scenario for distance - server on the other side of the world, but with no other latency accounted for - is about 200ms, even half of which would be noticeable in many games.
I'm not saying it's impossible to do game streaming, but you can't just divide the distance by the speed of light and say it's low. That's not really the issue.
Most of those hops would be on ISP equipment with 10G or 100G interfaces, modern routers forward packets in tens to hundreds of microseconds and switches can do it in a few microseconds, so as long as there is a bit of distance it will account for most of the delay, and SOL is 2/3 of its speed I'm vacuum. Switches can even sends out packets on a port before it has been fully received on the incoming port
True, though routers with 100G interfaces do not have a lot of buffer space. But most ISP's make sure that isn't happening as it its pretty bad for their customers experience. I know some ISP's do not put enough effort into that at all but every single one I worked for did (Europe). In this case Google can put effort into direct peering relationships to make sure that their customers have a good experience.
Right, which is exactly big Cloud companies such as Azure and GCP have a huge advantage over most other smaller companies, as they have their own dedicated backbone, allowing them to minimize the number of hops.
> Honestly things such as controller and display lag are probably bigger than the round trip latency.
The problem with latency is that it adds up, lag from the server, lag from the TV, lag from my brain, lag to my fingers, lag from the controller to the machine, lag back to the server, lag in processing and then start again.
Client-Server lag might be acceptable in isolation but not as part of a system.
Yep, I played a couple of the Arkham games on streaming services, the first one way back on OnLive before Sony bought them. It was fine.
I now play games with some friends and while I'm using a local PC, they use Shadow PCs (ie a VM with a GPU in a datacentre vaguely near them) and it's fine.
Maybe it'd be worse on an LTE/5G connection, but my phone is getting 25ms to the nearest Speedtest.net instance right now, which doesn't seem substantially terrible.
I would be much more concerned about the pricing of all of these things, than the technical issues. How many gaming subscription services am I going to need to add to my video subscription services?
From a hard theory network perspective, having higher bandwidth, lower latency, lower error rate on transmission path combined with closer geographical proximity to servers seem to help mitigate increases in latency.
However, if you argue the problem is solved by your empirical evidence gained by playing Witcher 3 of all games, I disagree with your point. Play a real time strategy game, a MOBA, or a first person shooter. 70ms~ (a report for Stadia reported 70ms-130ms, may be stale data) of lag is noticeable to people. On the other hand, I don't mind streaming a turn based game.
There's the obvious DRM win. Accessibility will be higher with way lower or non-existent hardware costs. These probably look very enticing on a cooperate slide deck. Not to mention the subscription model.
On a more technical side game streaming is only really desirable for games that don't run well on commodity hardware (see AAA 3D action titles). Latency is very important for these titles and the general rule of thumb is that 50ms of input latency is readily noticeable to humans. Average home connections have ~20-30ms round trip latencies. These speeds are not a reality everywhere or consistent. Combined with hardware input, world simulation, and rendering latencies it can be difficult to get latencies low enough to be comfortable.
Modern games just have the client run its own world simulation and rectify with the server later to hide the network latency. This strategy won't be possible with streaming. Is there some alternate optimization that can be made for streaming? The architecture will definitely be simpler with the main-frame paradigm. Maybe if there's only one client it could be feasible to send some batteries included chunk of frames that can be easily constructed based on the next set of inputs. Is there any hint that progress can be made in this direction?
If the latency problem can't be solved, I'm definitely bearish or game streaming. Latency insensitive games for the most part are not difficult to run on commodity hardware.