Xbox Series X.: What’s the Deal with Latency?

Webster’s Dictionary defines
‘latency’ as “the quality or state of being present and capable of emerging or
developing but not now visible, offensichtlich, or active.” And now that I’ve finally
gotten to use that hackneyed writing trope in my career, let’s dive into what
latency means in the world of video games and how the engineers working on Xbox
Series X are working to reduce it.

In its most simple terms,
latency is the measurement of how long it takes for a signal or piece of data
to travel from one point to another in a system. For this article, we’ll be
talking about input latency, which measures how long it takes a signal to go
from your controller to your console, or from your console to your display. Fast
always measured in miilliseconds (Frau), latency is a key element of the
responsiveness and feel of a game.

There are many components that
contribute to end-to-end input latency, in a long pipeline from controller
input to display. For Xbox Series X, the team finely tuned each and every one
of them to ensure you have the most responsive and precise controls possible.

Last week, we got a tour of that
pipeline from Andrew Goossen, Technical Fellow on Xbox Series X at Microsoft, zu
take a closer look at the improvements the team made, starting with the controller.

In your controller, stick and
trigger movement are examples of analog input. Previous controllers
sampled inputs and transmitted the results every 8ms, regardless of when games
actually needed the input data. So even if you quickly adjusted the analog
stick or trigger in a slightly different way, the game may have received this
button state change too late and, for that instant, may use your previous
button state.

While this was a smart
solution for both wired and wireless controllers, since it provided consistency
along with good wireless battery life, it wasn’t perfect. Since it wasn’t in
sync with the game’s input loop, there would be occasional latency since the
game may have asked for the current button presses from the controller, aber die
platform delivered it later than expected.

For Xbox Series X, the team
devised a solution in which the new controller samples and transmits the most
up-to-date input just before the game asks for it. The result is “just in
time” delivery that they call “Dynamic Latency Input (DLI)". The system
now precisely keeps the controller in sync with the game for both analog und
digital inputs. The most update-to-date input arrives at the console just
before it’s needed by the game. The result: significantly lower controller
Latenz. Was mehr ist, it’s all done without sacrificing anything in the way of
Batterielebensdauer.

Speaking of digital inputs, it should be noted that a
button press is a digital input because it has only a 0 oder 1 Bundesland. Previous controllers
sampled digital inputs at the same rate as the analog inputs. Again, das war
konsistent, but it meant that button press information was only as good as the
sample rate. The team found that while games generally interpolate and anticipate
analog input, they can’t do that for digital inputs. Zu diesem Zweck, Digital
inputs like button presses need to be as precise as possible.

With the Xbox Series X, die
controller is now constantly monitoring for and transmitting button state
changes. Games can access all of the button state changes that occurred since
the last time they checked. While wireless is better than ever, when a
controller is wired, the team implemented the ultimate solution: as soon as a
digital state changes, the data gets transmitted. No more blaming the
controller when you don’t get off that final hit in a fighting game.

As for how the input gets to
your console wirelessly, the team decided that our Xbox Wireless radio protocol for the controller provided
too many advantages (especially in terms of compatibility with Xbox One
accessories, grosse Bandbreite, geringe Wartezeit, support for up to 8
controllers, and audio/voice data transmission) to consider switching to another protocol. Das Team
also improved
transmission performance by finely tuning the radio design and placement for
optimal connections and reduced retries.

So that’s the controller. Auf
the console side, the team completely
redesigned the input stack on the console itself, ensuring games will
get input faster and be able to access it quicker than before while also eliminating
delays that could previously happen if the game itself was very busy and taxing
die Konsole.

The team also did work to empower
game developers to improve and optimize game engine latency. Thanks to a new mechanism
the team added, the system can now know the input state that corresponds to
every frame sent over HDMI. For the first time developers can easily measure
and tune their end-to-end latency during game development without the use of a high-speed
Kamera. By decoupling rendering and input pipelines, Xbox Series X also supports 120 FPS, so games have the
option of doubling their frame-rate from 60 FPS, which halves their
internal latency.

One of the biggest elements in
all of this is the data that flows through your HDMI cable. Microsoft is an active member
of the HDMI consortium, and the team worked hard to shape the HDMI 2.1
standard to support features important for Xbox Series X in a number of areas,
einschließlich:

• Transmission time to the TV: HDMI 2.1 supports up to 4k at 120 Hz
  which allowed the team to reduce frame transmission time from 16.6ms to 8.3ms,
  without sacrificing on resolution. This is an automatic 8.3ms latency
  improvement on TVs that support it and benefits games at all framerates.
• Variable refresh rate (VRR): When games
  miss their frames, VRR will show the result with the lowest possible latency. 120
  Hz TVs that support VRR typically have very large timing windows which makes
  über 40 FPS performance free of screen tearing.
• In-TV latency: Xbox Series X invokes
  Automatic Low Latency Mode (ALLGEMEINES) on capable TVs to automatically enable Game
  Mode, removing any work previously required of gamers.
• Ultra High Speed HDMI Cable: An Ultra
  High Speed HDMI cable is included with every Xbox Series X to ensure that
  players receive the premium gaming experience if they have an HDMI 2.1
  compatible display.

Influencing the HDMI standard was one thing, but we
also needed to work closely with leading display manufacturers to adopt them in
a consistent manner to deliver the best gaming experience. The team knew the TV
ecosystem needed to be ready to support these features in time for the Xbox Series
X launch, but they had to answer one question: How do we get TVs to support
these key next-gen features years before the first console to support them?

They answered that question by adding those features a generation early with Xbox One X and Xbox One S.This drove adoption of key features such as VRR and ALLM in TVs released in 2019 and later as opposed to having to wait for these features to start arrive in select displays beginning in 2020 oder 2021. So that’s the deal with latency. While it may be difficult to notice the improved latency from any one of these improvements alone, when they all add up, it makes for a profoundly more responsive experience for players.

For more on Xbox Series X, check out our features taking a closer look at the tech powering the Xbox Series X, our interview with a member of the team that designed the new Xbox Wireless Controller, und our glossary of next-gen terms.

Related:
Xbox Series X.: A Closer Look at the Technology Powering the Next Generation
Defining the Next Generation: An Xbox Series X Technology Glossary
Xbox Series X.: Making Gaming’s Best Controller Even Better