Enable HTTP/2 on HAProxy 1.8 and temporarily fix the bad request problem with the Nginx backend using FastCGI/uWSGI.

9th January 2018 – Update

As pointed out by Krzysztof Bąk/webmind.pl, the issue mentioned here is already fixed in HAProxy 1.8.3. See the announcement haproxy-1.8.3 for more information, so upgrade your package.

[...]
Users of 1.8 must upgrade. Please don't report bugs affecting older 1.8
versions now, so that efforts are focused on fixing new bugs.
[...]

$ apt-cache policy haproxy

haproxy:
  Installed: 1.8.3-1ppa1~xenial
  Candidate: 1.8.3-1ppa1~xenial
  Version table:
 *** 1.8.3-1ppa1~xenial 500
        500 http://ppa.launchpad.net/vbernat/haproxy-1.8/ubuntu xenial/main amd64 Packages
        100 /var/lib/dpkg/status
     1.6.3-1ubuntu0.1 500
        500 http://mirror.de.leaseweb.net/ubuntu xenial-updates/main amd64 Packages
        500 http://security.ubuntu.com/ubuntu xenial-security/main amd64 Packages
     1.6.3-1 500
        500 http://mirror.de.leaseweb.net/ubuntu xenial/main amd64 Packages

HAProxy configuration

Look at the initial HAProxy configuration.

[...]
frontend web
  bind :::80 v4v6
  bind :::443 ssl crt /etc/haproxy/ssl/
  mode http

  redirect scheme https code 301 unless { ssl_fc }
[...]

Advertise support for both HTTP/2 and HTTP/1.1 protocols.

[...]
frontend web
  bind :::80 v4v6
  bind :::443 ssl crt /etc/haproxy/ssl/ alpn h2,http/1.1
  mode http

  redirect scheme https code 301 unless { ssl_fc }
[...]

Reload HAProxy configuration.

$ sudo systemctl reload haproxy

The bad request issue

Firefox 57.0.3 and Google Chrome 63.0.3239.108 web-browsers return 400 HTTP error code on the POST requests.

Excerpt from the HAProxy log.

Dec 29 23:48:34 sleeplessbeastie haproxy[20076]: ::ffff:37.131.142.96:40002 [29/Dec/2017:23:48:34.953] web~ web-statistics-production/statistics 0/0/0/-1/1 400 188 - - CH-- 9/5/7/7/0 0/0 "POST /index.php?action=getDashboardLayout&date=today&format=JSON2&idDashboard=1&idSite=3&module=Dashboard&period=day HTTP/1.1"

Excerpt from the Nginx log.

::ffff:37.131.142.96 - - [29/Dec/2017:23:48:34 +0000] "POST /index.php?action=getDashboardLayout&date=today&format=JSON2&idDashboard=1&idSite=3&module=Dashboard&period=day HTTP/1.1" 499 0 "https://statistics.sleeplessbeastie.eu/index.php?module=CoreHome&action=index&idSite=3&period=day&date=today" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:57.0) Gecko/20100101 Firefox/57.0"

400 and 499 HTTP error codes clearly state that there is some kind of client issue.

The solution for bad request issue

The issue and solution are described on the HAProxy mailing list – Re: HTTP/2 Termination vs. Firefox Quantum e-mail and Firefox Bug 1427256 – connection header http2.

You need to ignore client aborts in the Nginx backend by enabling uwsgi_ignore_client_abort or fastcgi_ignore_client_abort directive.

Sample uWSGI configuration.

location ~ .php$ {
  if (!-f $request_filename) { return 404; }
  include uwsgi_params;
  uwsgi_modifier1 14;
  uwsgi_pass unix:/run/uwsgi/app/php/socket;
  uwsgi_ignore_client_abort on;
}

Sample FastCGI configuration.

location ~ ^/index\.php(.*)\$ {
  fastcgi_pass unix:/var/run/php/php7.0-fpm.sock;
  fastcgi_index index.php;
  include fastcgi_params;
  fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
  fastcgi_ignore_client_abort on;
}

This solution fixed my problems with the Icinga web-interface, Nextcloud, and Piwik/Matomo.