A QoE Perspective on HTTP/2 Server Push

Motivated by our results obtained in our adoption and performance analysis (paper available online), i.e., the use of server push can have positive and negative effects on the page load time, we investigated if humans are able to perceive the measured improvements or detriments.

To this end, we selected 28 pushing websites out of a set of 526 pushing websites, that look correct, are not offensive and have a reasonable page load time (<10s). We automate the Chome Browser to visit these sites under different protocol configurations (HTTP/1.1, HTTP/2, HTTP/2 without Push) and utilize the Browsertime framework to capture the loading process as video.

Based on these videos, we create a pair-wise comparison as a crowdsourcing study (still available online), where the particpants decide which of the aforementioned versions loaded faster. For more details, we refer to our paper at the Internet-QoE'17 Workshop.

Overview of Participants

To obtain ratings from a controlled environment complementing the (potentially noisy) crowdsourcing study, we first conducted a laboratory study at our institute located in Aachen (Germany). As both yield similar voting distributions, we thus merge the results obtained from both studies for the remaineder of this study. For more details about the overall study procedure and implementation we refer to our paper.

In both studys, before the users are asked to rate the individual page loads, we ask them to provide their demographic data. The overview is shown in the following table.

Overview of study participants.
Set Users Gender
(♀, ♂, ○)		 Age
<25, 25-31, >31		 Expertise
(-, ∅, +)		 Online [h]
<4, 4-8, >8
Lab 28 6, 12, 1 6, 20, 6 0, 9, 19 7, 11, 10
Crowd 323 72, 246, 5 143, 119, 61 7, 95, 221 86, 130, 107

Userstudy Results

In the following figure, we show the combined results (lab + crowd) of our userstudy. Please note that the y-axis shows support for the individual protol variants and is centered around No Difference (0).

Figure 1: Combined results (Lab + Crowd) for all 28 websites and 3 conditions. The y-axis shows support for both protocol variants and is centered at No Difference (0).

Comparing user votes between each individual condition allows us to evaluate whether push is the determining factor in the user-perceived performance between HTTP/1.1 and HTTP/2. Thus, we investigate whether there are differences between votes for HTTP/1.1 and HTTP/2 (cf. Figure 1, top vs. middle), when push is enabled or disabled. First, we observe that HTTP/2 push was perceived faster for pages p1 – p16 by ≥ 59.09% of votes compared to HTTP/1.1 (C2, green bars).
However, considering the case HTTP/1.1 vs. HTTP/2 without push (C1), we see that those websites already had a vote towards HTTP/2 (> 50%) and likely already benefit from other HTTP/2 properties than push. Only p13 and p15 exhibit changes in their votes in favor of HTTP/2, when push is enabled. In this case push contributes to the faster perception of HTTP/2. In the following, we discuss possible effects causing this resulting perception for two case studies.

Case Studies

a) HTTP/2 Server Push perceived faster than HTTP/1.1

p13 is a sports website with only 13 resources. Here we find HTTP/1.1 to perform well since we observe only little HOL Blocking. When using HTTP/2, the server does not ad-here to the priorities specified by the browser. As a result, some requested resources are not delivered in the order requested by the browser, impacting the rendering performance and diminishing the overall HTTP/2 performance (HTTP/1.1 is faster).
However, with HTTP/2 push the server pushes two render critical stylesheets, that contain links to font files. Without push the browser would have to request the stylesheets explicitly and subsequently after retrieving the stylesheets request the missing font files. Thereby, pushing the stylesheets reduces the overall time (H2 push is faster than H1).

b) HTTP/1.1 perceived faster than HTTP/2 Server Push

p22 is an example of harmful push usage. Six render-critical resources are pushed, i.e., the resources are necessary to render above-the-fold content. However, those pushed resources do not only arrive in an suboptimal order for the browser to process, they also delay the initial HTML response, hindering the browser to discover additional resources beyond the pushed resources (e.g. 3rd party content) required to render.
When comparing HTTP/1.1 and HTTP/2 with push, the first render event is delayed by 277ms in the case of push, i.e., the page is rendered partially sooner but incomplete. Interestingly, although the page starts to render later in the HTTP/2 case the overall rendering process is faster using HTTP/2 push compared to HTTP/1.1, still 70.59 % subjects voted in favor of the earlier HTTP/1.1 scenario.

Discussion

Based on our results, we argue that push is no general silver bullet to optimize the Web. When used properly, it can lead to websites that are perceived to load faster than their HTTP/1.1 or plain HTTP/2 counterparts. However, we also observe cases in which push leads to slower loading times and to websites that are not perceived to load faster. Here, the usage of push is detrimental for the website operator and we find reasons for this to be rooted in numerous page-specific aspects.