Manual:Cloudflare

CloudFlare (cloudflare.com) is a commercial content delivery network with integrated distributed denial of service (DDoS) defence. As it acts as a reverse proxy and domain name server to your website, it can provide a useful IPv6 transition mechanism if your hosting provider doesn't provide native IPv6.

As with other reverse proxies, such as Squid and Varnish, there are configuration issues with any server that places itself in-line between the user and Mediawiki. CloudFlare needs to be told what to cache (and when to discard outdated content) and needs to be told not to replace MediaWiki's custom 404 page (which invites the user to create a missing article) with its own "SmartErrors" page. Likewise, MediaWiki (or Apache) needs to be aware of any intervening trusted proxies to ensure that the user's IP address (and not that of CloudFlare's servers) is listed in Special:RecentChanges or logged by Extension:CheckUser.

Advantages and limitations
CloudFlare does not currently charge per megabit/second or per gigabyte or terabyte of data transferred; (as of 2014) the basic service, with some limitations, is free. For sites receiving high-volume requests for the same unchanged content (such as images, which typically account for more than 80% of bandwidth costs) placing a busy domain behind a service like CloudFlare can substantially reduce its cost of operation. As fewer requests are being made to the origin servers, in some cases the site may run faster.

Individual page view counters are broken under CloudFlare, as most requests never reach the origin servers and the individual wiki site owner does not have access to CloudFlare's logs. CloudFlare does provide analytics to tell how many visit your site, but not on a per-page basis.

The free version of CloudFlare does not support SSL. A paid version ($20/month per domain, no limit on number of subdomains) provides some SSL support, but content would be decrypted and re-encrypted at CloudFlare's servers - a potential 'man in the middle' vulnerability.

CloudFlare takes control over DNS for your entire domain; this may be a problem if you use domains in which individual subdomains are shared between users with a service like freedns.afraid.org or if you use other services which expect to be your DNS provider.

CloudFlare places your site behind multiple anycast servers in multiple countries. This can be an advantage if being geographically closer to your users makes your site appear faster, but can be a legal disadvantage for sites which are targets of libel tourism or are dealing in politically-sensitive matters. A Wikileaks-like site might not want a US server in the data path if discussing sensitive information about US intelligence agencies; a site discussing activities lawful in its own country but illegal in one or more of the CloudFlare server locations might also wish to avoid using the service and keep their content distribution network at home.

Like any third-party free or paid service, there is also the risk that what's free today becomes an expensive paid service tomorrow, becomes slow or unstable or simply disappears. Be prepared to switch your domain registration's DNS entries back to your original service (or another provider) if the 'freemium' CloudFlare service goes away in the future.

Integration with MediaWiki
MediaWiki's caching strategy is designed to work with open-source reverse proxy servers such as Squid and Varnish. On a single-server configuration, this looked like:

When an anonymous user requested a page, Squid (or Varnish) checked whether it had a copy already stored. If one was available, it was served directly without even querying the origin server. If not, the request would pass through to Apache and MediaWiki.

This saved significant amounts of processor time (as MediaWiki was no longer repeatedly regenerating the same HTML pages for frequently-read pages) but had various impacts on MediaWiki server configuration:
 * The address of the Squid proxy had to be kept out of Special:RecentChanges, instead using the user's IP address from the X-Forwarded-For header.
 * The proxy server had to be notified when a page was changed, so they could discard the outdated version. This was done (for a one-server configuration) from LocalSettings.php with entries like:
 * The proxy had to avoid caching dynamic content which frequently or pseudo-randomly changes, such as special pages or output from specific extensions such as any version of Extension:DynamicPageList or Extension:RandomSelection.
 * The user's IP address had to be removed from the upper-right corner for anonymous IP users, $wgShowIPinHeader=false;
 * Page view counters should be disabled in LocalSettings.php, $wgDisableCounters=true, as caching means not all page views reach the MediaWiki/Apache server
 * Pages displaying a "You have new messages" banner or similar flags could not be cached
 * Pages for logged-in users (as they contain login names or identifying info) could not be cached
 * Protection against image hotlinking had to move out of Apache and onto whichever server was facing the user
 * IPv6 support also moved onto whichever server was facing the user, although MediaWiki did still need to be reasonably current (version 1.19 or later) as older versions were buggy in logging IPv6 user addresses to Special:RecentChanges

MediaWiki sends HTTP headers (such as "cache-control: private" if something should not be cached, or an expiry time if something can be cached for a limited time) which are recognised by Squid. Squid sends "X-forwarded-for:" to indicate the IP of a user to a MediaWiki installation behind a Squid proxy. MediaWiki also sends HTTP PURGE requests to notify Squid that an item has changed and needs to be discarded.

As MediaWiki was designed to work with Squid (and Varnish can be configured to act in the same manner), the MediaWiki installation and the cache are tightly integrated.

CloudFlare in some cases provides a function which could serve a similar role, but implements it in a non-standard or incompatible manner. A MediaWiki installation could operate behind CloudFlare, but there are some configuration issues to be addressed.

Anonymous IP user identification
If a user connects directly to a MediaWiki installation (on Apache), the user's IP address is reported by PHP in $_SERVER['REMOTE_ADDR'] and no further configuration is required to get the information into Special:RecentChanges

If a MediaWiki installation is behind Squid or Varnish, MediaWiki must be configured to use the "X-forwarded-for:" header.

If a MediaWiki installation is behind CloudFlare, that server will present the user's IP in a non-standard format. CloudFlare at one point recommended changing core MediaWiki code to use $_SERVER['HTTP_CF_CONNECTING_IP'] but their proposed patch is, by their own admission, only suitable for "MediaWiki versions 1.18.0 and older".

This leaves mod_cloudflare as the simplest currently-available option. This module, downloadable from CloudFlare's site, is installed directly into Apache (not MediaWiki) and restores the original user's IPv4 or IPv6 address if CloudFlare is connecting to Apache directly. (If the MediaWiki/Apache server is sitting behind Squid, and that in turn is behind CloudFlare, the IP addresses in recentchanges will break).

Cache control
To prevent CloudFlare from caching certain parts of your site, it is possible to use Page Rules.

By default, MediaWiki sets certain HTTP headers (such as "cache-control: private" for logged-in users) to control whether its dynamically-generated content is cached and when it should expire. For purely static content, such as image files, the headers are generated by the web server.

CloudFlare may respect these, depending on how it has been configured. From My Websites &rarr; settings &rarr; Page rules it is possible to create three rules per domain (more for paid users). Each matches a pattern (such as *example.org/wiki/* for all wiki pages on example.org and its subdomains, in standard view) and allows configuration of:
 * Custom caching - allows the cache control headers to be overridden, "Cache everything" is the most aggressive and may be OK for static image files but certainly not wanted for anything else. Special:Recentchanges or other dynamic content should use the least aggressive settings.
 * Edge cache expire TTL - overrides the time after which CloudFlare requests a new copy of a page or file from the origin server
 * Browser cache expire TTL - indicates the time after which the user's browser should request a new copy of a page from CloudFlare

It's best to cache static image files aggressively, ordinary wiki pages less aggressively and special: pages not at all.

HTTP purge
When new content is uploaded to a MediaWiki, the wiki software will request that each cache server (as listed in $wgSquidServers) discard the outdated version of the page or image. This does not apply to third-party web proxies such as those listed in Extension:TrustedXFF.

The message, as sent by MediaWiki, looks like: PURGE http://wiki.example.org/images/0/01/Some_image_recently_replaced.jpg

Squid will honour this request if it comes from a trusted IP address. CloudFlare likely will not. There is a means to manually request one outdated file be discarded (from 'My Websites' settings for one domain &rarr; CloudFlare Settings &rarr; Cache Purge, click "purge single file" and enter URL of file to purge) but this is cumbersome. CloudFlare's API provides a function which can also perform this task, but there is currently nothing available to invoke this API automatically from MediaWiki.

Error 404 handling
By default, CloudFlare enables a "feature" which it brands as SmartErrors. This replaces the originating site's 'error 404' pages with a CloudFlare search page which lists other, related pages which do exist on the same site and allows the user to search the web. This "smart error" page, which displays in US English instead of your site's local language, may contain advertising or direct users to obscure, little-known external web search engines.

MediaWiki, when given a request for a page which doesn't exist http://en.wikipedia.org/wiki/Like_this_one will display a custom error page which has the 404 code but invites the user to create the page (or indicates the page was deleted, protected, or "no such special page"). MediaWiki sites need this page left as-is so that clicking on a red link invites a user to create a new article. SmartErrors on every 404 page breaks this.

SmartErrors may be disabled for an entire domain on initial import of the domain to CloudFlare or deactivated later on a per-domain basis from the list of domains by clicking 'Apps', scrolling down to "SmartErrors" and turning the app 'off'. They may also be turned off for specific paths using "page rules" in the settings for each domain.

Image hotlinking
Many web-based forums are infamous for allowing users to link directly to images hosted on other sites. Their site appears to be displaying the image, but is using bandwidth that the 'hot-linked' target site's operators must pay for. A common defence is for Apache webmasters to deploy mod_rewrite to look at the "referer:" (sic) line of each request and reject any that hotlink images (.jpg, .png, .gif et al.) for use on pages on some other site.

With any form of caching server, the origin servers are no longer user-facing. Any anti-hotlinking code needs to be removed from the origin webserver (such as Apache); anti-hotlink protection is available on CloudFlare if needed.

IPv6
All current MediaWiki versions are able to log an IPv6 user address to Special:RecentChanges if one is provided. No changes to MediaWiki's LocalSettings.php is required.

IPv6 support is enabled on whichever server is facing the user; Apache for a stand-alone install, Squid/Varnish for sites using these as a reverse proxy, CloudFlare's server for sites where CloudFlare is user-facing. There is no requirement that the communication from the user-facing server back to Apache support both IPv4 and IPv6 and no benefit to enabling both for any back-end link.

My websites → settings → CloudFlare settings → Automatic IPv6 may be set to "full" for each domain to enable IPv6 for your site.