![Addy Osmani](https://web.dev/images/authors/addyosmani.jpg) Addy Osmani [X](https://twitter.com/addyosmani) [GitHub](https://github.com/addyosmani) [Bluesky](https://bsky.app/profile/addyosmani.bsky.social)

<br />

Learn how you can use [Puppeteer](https://developer.chrome.com/docs/puppeteer) APIs to add server-side
rendering (SSR) capabilities to an Express web server. The best part is that
your app requires very small changes to code. Headless does all the heavy
lifting.

In a couple of lines of code you can SSR any page and get its final markup.

```javascript
import puppeteer from 'puppeteer';

async function ssr(url) {
  const browser = await puppeteer.launch({headless: true});
  const page = await browser.newPage();
  await page.goto(url, {waitUntil: 'networkidle0'});
  const html = await page.content(); // serialized HTML of page DOM.
  await browser.close();
  return html;
}
```

> [!NOTE]
> **Note:** This document uses ES modules (`import`), which requires Node 8.5.0+ running with the `--experimental-modules` flag. You can use `require()` statements if you prefer. [Read more](https://nodejs.org/api/esm.html) about Node's ES Modules support.

## Why use Headless Chrome?

You may be interested in Headless Chrome if:

- You've built a web app that isn't being indexed by search engines.
- You're hoping for a quick win to optimize [JavaScript performance](https://web.dev/articles/optimize-long-tasks) and improve [first meaningful paint](https://developer.chrome.com/docs/lighthouse/performance/first-meaningful-paint.md.txt).

Some frameworks like Preact
[ship with tools](https://github.com/developit/preact-render-to-string)
that address server-side rendering. If your framework has a prerendering
solution, stick with it instead of bringing Puppeteer and Headless Chrome into
your workflow.

### Crawling the modern web

Search engine crawlers, social sharing platforms, [even browsers](https://lynx.invisible-island.net/lynx.html)
have historically relied exclusively on static HTML markup to index the web
and surface content. The modern web has evolved into something much
different. JavaScript-based applications are here to stay, which means that in
many cases, our content can be invisible to crawling tools.

The Googlebot, our Search crawler,
[processes JavaScript](https://developers.google.com/search/docs/crawling-indexing/javascript/javascript-seo-basics)
while making sure it doesn't degrade the experience of users visiting the site.
There are some [differences and limitations](https://developers.google.com/search/docs/crawling-indexing/javascript/fix-search-javascript)
that you need to account for when designing your pages and applications to
accommodate how crawlers access and render your content.

## Prerender pages

All crawlers understand HTML. To ensure crawlers can index JavaScript, we need a
tool that:

- Knows how to run *all* types of modern JavaScript and generate static HTML.
- Stays up to date as the web adds features.
- Runs with little to no code updates to your application.

Sounds good right? **That tool is the browser**! Headless Chrome doesn't care
what library, framework, or tool chain you use.

For example, if your application is built with Node.js, Puppeteer is an easy way
to work with headless Chrome.

Start with a dynamic page that generates its HTML with JavaScript:

**public/index.html**

```html
<html>
<body>
  <div id="container">
    <!-- Populated by the JS below. -->
  </div>
</body>
<script>
function renderPosts(posts, container) {
  const html = posts.reduce((html, post) => {
    return `${html}
      <li class="post">
        <h2>${post.title}</h2>
        <div class="summary">${post.summary}</div>
        <p>${post.content}</p>
      </li>`;
  }, '');

  // CAREFUL: this assumes HTML is sanitized.
  container.innerHTML = `<ul id="posts">${html}</ul>`;
}

(async() => {
  const container = document.querySelector('#container');
  const posts = await fetch('/posts').then(resp => resp.json());
  renderPosts(posts, container);
})();
</script>
</html>
```

### SSR function

Next, take the `ssr()` function from earlier and beef it up a bit:

**ssr.mjs**

```javascript
import puppeteer from 'puppeteer';

// In-memory cache of rendered pages. Note: this will be cleared whenever the
// server process stops. If you need true persistence, use something like
// Google Cloud Storage (https://firebase.google.com/docs/storage/web/start).
const RENDER_CACHE = new Map();

async function ssr(url) {
  if (RENDER_CACHE.has(url)) {
    return {html: RENDER_CACHE.get(url), ttRenderMs: 0};
  }

  const start = Date.now();

  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  try {
    // networkidle0 waits for the network to be idle (no requests for 500ms).
    // The page's JS has likely produced markup by this point, but wait longer
    // if your site lazy loads, etc.
    await page.goto(url, {waitUntil: 'networkidle0'});
    await page.waitForSelector('#posts'); // ensure #posts exists in the DOM.
  } catch (err) {
    console.error(err);
    throw new Error('page.goto/waitForSelector timed out.');
  }

  const html = await page.content(); // serialized HTML of page DOM.
  await browser.close();

  const ttRenderMs = Date.now() - start;
  console.info(`Headless rendered page in: ${ttRenderMs}ms`);

  RENDER_CACHE.set(url, html); // cache rendered page.

  return {html, ttRenderMs};
}

export {ssr as default};
```

The major changes:

- Added caching. Caching the rendered HTML is the biggest win to speed up the response times. When the page gets re-requested, you avoid running headless Chrome altogether. I discuss other [optimizations](https://developer.chrome.com/blog/headless-chrome-ssr-js-sites.md.txt#optimizations) later on.
- Add basic error handling if loading the page times out.
- Add a call to `page.waitForSelector('#posts')`. This ensures that the posts exist in the DOM before we dump the serialized page.
- Add science. Log how long headless takes to render the page and return the rendering time along with the HTML.
- Stick the code in a module named `ssr.mjs`.

### Example web server

Finally, here's the small express server that brings it all together. The main
handler prerenders the URL `http://localhost/index.html` (the homepage)
and serves the result as its response. Users immediately see posts when
they hit the page because the static markup is now part of the response.

**server.mjs**

```javascript
import express from 'express';
import ssr from './ssr.mjs';

const app = express();

app.get('/', async (req, res, next) => {
  const {html, ttRenderMs} = await ssr(`${req.protocol}://${req.get('host')}/index.html`);
  // Add Server-Timing! See https://w3c.github.io/server-timing/.
  res.set('Server-Timing', `Prerender;dur=${ttRenderMs};desc="Headless render time (ms)"`);
  return res.status(200).send(html); // Serve prerendered page as response.
});

app.listen(8080, () => console.log('Server started. Press Ctrl+C to quit'));
```

To run this example, install the dependencies (`npm i --save puppeteer express`)
and run the server using Node 8.5.0+ and the `--experimental-modules` flag:

Here's what an example of the response sent back by this server:

```html
<html>
<body>
  <div id="container">
    <ul id="posts">
      <li class="post">
        <h2>Title 1</h2>
        <div class="summary">Summary 1</div>
        <p>post content 1</p>
      </li>
      <li class="post">
        <h2>Title 2</h2>
        <div class="summary">Summary 2</div>
        <p>post content 2</p>
      </li>
      ...
    </ul>
  </div>
</body>
<script>
...
</script>
</html>
```

#### A perfect use case for the new Server-Timing API

The [Server-Timing](https://w3c.github.io/server-timing/) API communicates
server performance metrics (such as request and response times or database
lookups) back to the browser. Client code can use this information to track
overall performance of a web app.

A perfect use case for Server-Timing is to report how long it takes for headless
Chrome to prerender a page. To do that, just add the `Server-Timing` header to
the server response:

```javascript
res.set('Server-Timing', `Prerender;dur=1000;desc="Headless render time (ms)"`);
```

On the client, the [Performance API](https://developer.mozilla.org/docs/Web/API/Performance_API)
and [PerformanceObserver](https://developer.mozilla.org/docs/Web/API/PerformanceObserver)
can be used to access these metrics:

```javascript
const entry = performance.getEntriesByType('navigation').find(
    e => e.name === location.href);
console.log(entry.serverTiming[0].toJSON());

{
  "name": "Prerender",
  "duration": 3808,
  "description": "Headless render time (ms)"
}
```

### Performance results

The following results incorporate most of the performance
[optimizations](https://developer.chrome.com/blog/headless-chrome-ssr-js-sites.md.txt#optimizations) discussed later.

On an example app, headless
Chrome takes about one second to render the page on the server. Once the page is
cached, DevTools **3G Slow emulation** puts
[FCP](https://web.dev/articles/user-centric-performance-metrics) at
**8.37s faster** than the client-side version.

|   | First Paint (FP) | First Contentful Paint (FCP) |
|---|---|---|
| Client-side app | 4s | 11s |
| SSR version | 2.3s | \~2.3s |

These results are promising. Users see meaningful content much quicker because
the server-side rendered page **no longer relies on JavaScript to load + shows
posts**.

## Prevent re-hydration

Remember when I said "we didn't make any code changes to the
client-side app"? That was a lie.

Our Express app takes a request, uses Puppeteer to load the page into
headless, and serves the result as a response. But this setup has a
problem.

The **same JavaScript that executes in headless Chrome** on the server **runs again**
when the user's browser loads the page on the frontend. We have two places
generating markup. [#doublerender](https://www.youtube.com/watch?v=MX0D4oZwCsA)!

To fix this, tell the page its HTML is already in place.
One solution is to have the page JavaScript check if `<ul id="posts">` is
already in the DOM at load time. If it is, you know the page was SSR'd and can
avoid re-adding posts again. 👍

**public/index.html**

```javascript
<html>
<body>
  <div id="container">
    <!-- Populated by JS (below) or by prerendering (server). Either way,
         #container gets populated with the posts markup:
      <ul id="posts">...</ul>
    -->
  </div>
</body>
<script>
...
(async() => {
  const container = document.querySelector('#container');

  // Posts markup is already in DOM if we're seeing a SSR'd.
  // Don't re-hydrate the posts here on the client.
  const PRE_RENDERED = container.querySelector('#posts');
  if (!PRE_RENDERED) {
    const posts = await fetch('/posts').then(resp => resp.json());
    renderPosts(posts, container);
  }
})();
</script>
</html>
```

## Optimizations

Apart from caching the rendered results, there are plenty of interesting
optimizations we can make to `ssr()`. Some are quick wins while others may be
more speculative. The performance benefits you see may ultimately depend on the
types of pages you prerender and the complexity of the app.

### Abort non-essential requests

Right now, the entire page (and all of the resources it requests) is loaded
unconditionally into headless Chrome. However, we're only interested in two
things:

1. The rendered markup.
2. The JS requests that produced that markup.

**Network requests that don't construct DOM are wasteful**. Resources like
images, fonts, stylesheets, and media don't participate in building the HTML
of a page. They style and supplement the structure of a page but they don't
explicitly create it. We should tell the browser to ignore these
resources. This reduces the workload for headless Chrome, saves bandwidth, and
potentially speeds up prerendering time for larger pages.

The [DevTools Protocol](https://chromedevtools.github.io/devtools-protocol/)
supports a powerful feature called
[Network interception](https://chromedevtools.github.io/devtools-protocol/tot/Network#event-requestIntercepted)
which can be used to modify requests before they're issued by the browser.
Puppeteer supports network interception by turning on
[`page.setRequestInterception(true)`](https://github.com/puppeteer/puppeteer/blob/main/docs/api/puppeteer.page.setrequestinterception.md)
and listening for the
[page's `request` event](https://github.com/puppeteer/puppeteer/blob/main/docs/api/puppeteer.httprequest.md).
That allows us to abort requests for certain resources and let others continue
through.

**ssr.mjs**

```javascript
async function ssr(url) {
  ...
  const page = await browser.newPage();

  // 1. Intercept network requests.
  await page.setRequestInterception(true);

  page.on('request', req => {
    // 2. Ignore requests for resources that don't produce DOM
    // (images, stylesheets, media).
    const allowlist = ['document', 'script', 'xhr', 'fetch'];
    if (!allowlist.includes(req.resourceType())) {
      return req.abort();
    }

    // 3. Pass through all other requests.
    req.continue();
  });

  await page.goto(url, {waitUntil: 'networkidle0'});
  const html = await page.content(); // serialized HTML of page DOM.
  await browser.close();

  return {html};
}
```

> [!NOTE]
> **Note:** I use an allowlist to allow all other types of requests to continue. This may preemptively avoid any gotchas that arise from aborting more resources than necessary.

### Inline critical resources

It's common to use separate build tools (such as `gulp`) to process an app and
inline critical CSS and JS into the page at build-time. This can speed
up first meaningful paint because the browser makes fewer requests during
initial page load.

Instead of a separate build tool, **use the browser as your build tool**!
We can use Puppeteer to manipulate the page's DOM, inlining styles, JavaScript,
or whatever else you want to stick in the page before prerendering it.

This example shows how to intercept responses for local stylesheets
and inline those resources into the page as `<style>` tags:

**ssr.mjs**

```javascript
import urlModule from 'url';
const URL = urlModule.URL;

async function ssr(url) {
  ...
  const stylesheetContents = {};

  // 1. Stash the responses of local stylesheets.
  page.on('response', async resp => {
    const responseUrl = resp.url();
    const sameOrigin = new URL(responseUrl).origin === new URL(url).origin;
    const isStylesheet = resp.request().resourceType() === 'stylesheet';
    if (sameOrigin && isStylesheet) {
      stylesheetContents[responseUrl] = await resp.text();
    }
  });

  // 2. Load page as normal, waiting for network requests to be idle.
  await page.goto(url, {waitUntil: 'networkidle0'});

  // 3. Inline the CSS.
  // Replace stylesheets in the page with their equivalent <style>.
  await page.$$eval('link[rel="stylesheet"]', (links, content) => {
    links.forEach(link => {
      const cssText = content[link.href];
      if (cssText) {
        const style = document.createElement('style');
        style.textContent = cssText;
        link.replaceWith(style);
      }
    });
  }, stylesheetContents);

  // 4. Get updated serialized HTML of page.
  const html = await page.content();
  await browser.close();

  return {html};
}
```

This code:

1. Use a [`page.on('response')`](https://github.com/puppeteer/puppeteer/blob/main/docs/api/puppeteer.httpresponse.md) handler to listen for network responses.
2. Stashes the responses of local stylesheets.
3. Finds all `<link rel="stylesheet">` in the DOM and replaces them with an equivalent `<style>`. See [`page.$$eval`](https://github.com/puppeteer/puppeteer/blob/main/docs/api/puppeteer.page.__eval.md) API docs. The `style.textContent` is set to the stylesheet response.

### Auto-minify resources

Another trick you can do with network interception is to **modify the responses
returned by a request**.

As an example, say you want to minify the CSS in your app but also want to
keep the convenience having it unminified when developing. Assuming you've
setup another tool to pre-minify `styles.css`, one can use `Request.respond()`
to rewrite the response of `styles.css` to be the content of `styles.min.css`.

**ssr.mjs**

```javascript
import fs from 'fs';

async function ssr(url) {
  ...

  // 1. Intercept network requests.
  await page.setRequestInterception(true);

  page.on('request', req => {
    // 2. If request is for styles.css, respond with the minified version.
    if (req.url().endsWith('styles.css')) {
      return req.respond({
        status: 200,
        contentType: 'text/css',
        body: fs.readFileSync('./public/styles.min.css', 'utf-8')
      });
    }
    ...

    req.continue();
  });
  ...

  const html = await page.content();
  await browser.close();

  return {html};
}
```

### Reuse a single Chrome instance across renders

Launching a new browser for every prerender creates a lot of overhead.
Instead, you may want to launch a single instance and reuse it for rendering
multiple pages.

Puppeteer can reconnect to an existing instance of Chrome by calling
[`puppeteer.connect()`](https://pptr.dev/api/puppeteer.puppeteer.connect)
and passing it the instance's remote debugging URL. To keep a long-running
browser instance, we can move the code that launches Chrome from the `ssr()`
function and into the Express server:

**server.mjs**

```javascript
import express from 'express';
import puppeteer from 'puppeteer';
import ssr from './ssr.mjs';

let browserWSEndpoint = null;
const app = express();

app.get('/', async (req, res, next) => {
  if (!browserWSEndpoint) {
    const browser = await puppeteer.launch();
    browserWSEndpoint = await browser.wsEndpoint();
  }

  const url = `${req.protocol}://${req.get('host')}/index.html`;
  const {html} = await ssr(url, browserWSEndpoint);

  return res.status(200).send(html);
});
```

**ssr.mjs**

```javascript
import puppeteer from 'puppeteer';

/**
 * @param {string} url URL to prerender.
 * @param {string} browserWSEndpoint Optional remote debugging URL. If
 *     provided, Puppeteer's reconnects to the browser instance. Otherwise,
 *     a new browser instance is launched.
 */
async function ssr(url, browserWSEndpoint) {
  ...
  console.info('Connecting to existing Chrome instance.');
  const browser = await puppeteer.connect({browserWSEndpoint});

  const page = await browser.newPage();
  ...
  await page.close(); // Close the page we opened here (not the browser).

  return {html};
}
```

#### Example: cron job to periodically prerender

TO render a number of pages all at once, you can use a shared browser instance.

```javascript
import puppeteer from 'puppeteer';
import * as prerender from './ssr.mjs';
import urlModule from 'url';
const URL = urlModule.URL;

app.get('/cron/update_cache', async (req, res) => {
  if (!req.get('X-Appengine-Cron')) {
    return res.status(403).send('Sorry, cron handler can only be run as admin.');
  }

  const browser = await puppeteer.launch();
  const homepage = new URL(`${req.protocol}://${req.get('host')}`);

  // Re-render main page and a few pages back.
  prerender.clearCache();
  await prerender.ssr(homepage.href, await browser.wsEndpoint());
  await prerender.ssr(`${homepage}?year=2018`);
  await prerender.ssr(`${homepage}?year=2017`);
  await prerender.ssr(`${homepage}?year=2016`);
  await browser.close();

  res.status(200).send('Render cache updated!');
});
```

Also, add a `clearCache()` export to **ssr.js**:

```javascript
...
function clearCache() {
  RENDER_CACHE.clear();
}

export {ssr, clearCache};
```

## Other considerations

### Create a signal for the page: "You're being rendered in headless"

When your page is being rendered by headless Chrome on the server, it may be
helpful for the page's client-side logic to know that. In my app, I used this
hook to "turn off" portions of my page that don't play a part in rendering the
posts markup. For example, I disabled code that lazy-loads
[firebase-auth.js](https://firebase.google.com/docs/web/setup). There's no user
to sign in!

Adding a `?headless` parameter to the render URL is a simple way
to give the page a hook:

**ssr.mjs**

```javascript
import urlModule from 'url';
const URL = urlModule.URL;

async function ssr(url) {
  ...
  // Add ?headless to the URL so the page has a signal
  // it's being loaded by headless Chrome.
  const renderUrl = new URL(url);
  renderUrl.searchParams.set('headless', '');
  await page.goto(renderUrl, {waitUntil: 'networkidle0'});
  ...

  return {html};
}
```

And in the page, we can look for that parameter:

**public/index.html**

```html
<html>
<body>
  <div id="container">
    <!-- Populated by the JS below. -->
  </div>
</body>
<script>
...

(async() => {
  const params = new URL(location.href).searchParams;

  const RENDERING_IN_HEADLESS = params.has('headless');
  if (RENDERING_IN_HEADLESS) {
    // Being rendered by headless Chrome on the server.
    // e.g. shut off features, don't lazy load non-essential resources, etc.
  }

  const container = document.querySelector('#container');
  const posts = await fetch('/posts').then(resp => resp.json());
  renderPosts(posts, container);
})();
</script>
</html>
```

> [!NOTE]
> **Note:** Another handy method to look at is [`Page.evaluateOnNewDocument()`](https://github.com/GoogleChrome/puppeteer/blob/main/docs/api.md#pageevaluateonnewdocumentpagefunction-args) . It lets you inject code into the page and have Puppeteer run that code before the rest of the page's JavaScript executes.

### Avoid inflating Analytics pageviews

Be careful if you're using Analytics on your site. Prerendering pages
may result in inflated pageviews. Specifically, **you'll see 2 times the
number of hits**, one hit when headless Chrome renders the page and another
when the user's browser renders it.

So what's the fix? Use network interception to abort any request(s) that tries
to load the Analytics library.

```javascript
page.on('request', req => {
  // Don't load Google Analytics lib requests so pageviews aren't 2x.
  const blockist = ['www.google-analytics.com', '/gtag/js', 'ga.js', 'analytics.js'];
  if (blocklist.find(regex => req.url().match(regex))) {
    return req.abort();
  }
  ...
  req.continue();
});
```

Page hits never get recorded if the code never loads. Boom 💥.

Alternatively, continue to load your Analytics libraries to gain insight
into how many prerenders your server is performing.

## Conclusion

Puppeteer makes it easy to server-side render pages by running headless Chrome,
as a companion, on your web server. My favorite "feature" of this approach is
that you **improve loading performance** and the **indexability** of
your app **without significant code** changes!

If you're curious to see a working app that uses the techniques described
here, check out [the devwebfeed app](https://github.com/ebidel/devwebfeed/blob/master/server.mjs).

## Appendix

### Discussion of prior art

Server-side rendering client-side apps is hard. How hard? Just look
at how many [npm packages](https://www.npmjs.com/search?q=server+side+rendering)
people have written which are dedicated to the topic. There are countless
[patterns](https://en.wikipedia.org/wiki/Isomorphic_JavaScript),
[tools](https://github.com/GoogleChrome/rendertron), and
[services](https://prerender.io/) available to help with SSRing JS apps.

#### Isomorphic / Universal JavaScript

The concept of Universal JavaScript means: the same code that runs on
server also runs on the client (the browser). You share code between server and
client and everyone feels a moment of zen.

Headless Chrome enables "isomorphic JS" between server and client. It's a great
option if your library doesn't work on the server (Node).

#### Prerender tools

The Node community has built tons of tools for dealing with SSR JS apps. No
surprises there! Personally, I've found that
with some of these tools, so
definitely do your homework before committing to one. For example,
some SSR tools are older and don't use headless Chrome (or any headless browser
for that matter). Instead, they use PhantomJS (aka old Safari), which means your
pages aren't going to render properly if they're using newer features.

One of the notable exceptions is [Prerender](https://github.com/prerender/prerender/).
Prerender is interesting in that it uses headless Chrome and comes with drop-in
[middleware for Express](https://github.com/prerender/prerender-node):

```javascript
const prerender = require('prerender');
const server = prerender();
server.use(prerender.removeScriptTags());
server.use(prerender.blockResources());
server.start();
```

It's worth noting that Prerender leaves out the details of downloading and
installing Chrome on different platforms. Oftentimes,
[that's fairly tricky](https://pptr.dev/troubleshooting#running-puppeteer-in-docker)
to get right, which is one of the reasons why
[Puppeteer does for you](https://pptr.dev/faq#q-which-chrome-and-firefox-version-does-puppeteer-use).

*[YMMV]: Your mileage may vary