Server-Side Visualization With Nightmare

This is an extract from chapter 11 of Ashley Davis’s book Data Wrangling with JavaScript now available on the Manning Early Access Program. I absolutely love this idea as there is so much data visualization stuff on the web that relies on fully functioning client side JavaScript and potentially more API calls. It’s not nearly as robust, accessible, or syndicatable as it could be. If you bring that data visualization back to the server, you can bring progressive enhancement to the party. All example code and data can be found on GitHub.

When doing exploratory coding or data analysis in Node.js it is very useful to be able to render a visualization from our data. If we were working in browser-based JavaScript we could choose any one of the many charting, graphics, and visualization libraries. Unfortunately, under Node.js, we don’t have any viable options, so how otherwise can we achieve this?

We could try something like faking the DOM under Node.js, but I found a better way. We can make our browser-based visualization libraries work for us under Node.js using a headless browser. This is a browser that has no user interface. You can think of it as a browser that is invisible.

I use Nightmare under Node.js to capture visualizations to PNG and PDF files and it works really well!

The headless browser

When we think of a web-browser we usually think of the graphical software that we interact with on a day to day basis when browsing the web. Normally we interact with such a browser directly, viewing it with our eyes and controlling it with our mouse and keyboard as shown in Figure 1.

Figure 1: The normal state of affairs: our visualization renders in a browser and the user interacts directly with the browser

A headless browser on the other hand is a web-browser that has no graphical user interface and no direct means for us to control it. You might ask what is the use of a browser that we can’t directly see or interact with.

Well, as developers we would typically use a headless browser for automating and testing web sites. Let’s say that you have created a web page and you want to run a suite of automated tests against it to prove that it works as expected. The test suite is automated, which means it is controlled from code and this means that we need to drive the browser from code.

We use a headless browser for automated testing because we don’t need to directly see or interact with the web page that is being tested. Viewing such an automated test in progress is unnecessary, all we need to know is if the test passed or failed — and if it failed we would like to know why. Indeed, having a GUI for the browser under test would actually be a hindrance for a continuous-integration or continuous-deployment server, where many such tests can run in parallel.

So headless browsers are often used for automated testing of our web pages, but they are also incredibly useful for capturing browser-based visualizations and outputting them to PNG images or PDF files. To make this work we need a web server and a visualization, we must then write code to instance a headless browser and point it at our web server. Our code then instructs the headless browser to take a screenshot of the web page and save it to our file system as a PNG or PDF file.

Figure 2: We can use a headless browser under Node.js to capture our visualization to a static image file

Nightmare is my headless browser of choice. It is a Node.js library (installed via npm) that is built on Electron. Electron is a framework normally used for building cross-platform desktop apps that are based on web-technologies.

Why Nightmare?

It’s called Nightmare, but it’s definitely not a Nightmare to use. In fact, it’s the simplest and most convenient headless browser that I’ve used. It automatically includes Electron, so to get started we simply install Nightmare into our Node.js project as follows:

npm install --save nightmare

That’s all we need to install Nightmare and we can start using it immediately from JavaScript!

Nightmare comes with almost everything we need: A scripting library with an embedded headless browser. It also includes the communication mechanism to control the headless browser from Node.js. For the most part it’s seamless and well-integrated to Node.js.

Electron is built on Node.js and Chromium and maintained by GitHub and is the basis for a number of popular desktop applications.

Here are the reasons that I choose to use Nightmare over any other headless browser:

  • Electron is very stable.
  • Electron has good performance.
  • The API is simple and easy to learn.
  • There is no complicated configuration (just start using it).
  • It is very well integrated with Node.js.

Nightmare and Electron

When you install Nightmare via npm it automatically comes with an embedded version of Electron. So, we can say that Nightmare is not just a library for controlling a headless browser, it effectively is the headless browser. This is another reason I like Nightmare. With some of the other headless browsers, the control library is separate, or it’s worse than that and they don’t have a Node.js control library at all. In the worst case, you have to roll your own communication mechanism to control the headless browser.

Nightmare creates an instance of the Electron process using the Node.js child_process module. It then uses inter-process communication and a custom protocol to control the Electron instance. The relationship is shown in Figure 3.

Figure 3: Nightmare allows us to control Electron running as a headless browser

Our process: Capturing visualizations with Nightmare

So what is the process of capturing a visualization to an image file? This is what we are aiming at:

  1. Acquire data.
  2. Start a local web server to host our visualization
  3. Inject our data into the web server
  4. Instance a headless browser and point it at our local web server
  5. Wait for the visualization to be displayed
  6. Capture a screenshot of the visualization to an image file
  7. Shutdown the headless browser
  8. Shutdown the local web server

Prepare a visualization to render

The first thing we need is to have a visualization. Figure 4 shows the chart we’ll work with. This a chart of New York City yearly average temperature for the past 200 years.

Figure 4: Average yearly temperature in New York City for the past 200 years

To run this code you need Node.js installed. For this first example we’ll also use live-server (any web server will do) to test the visualization (because we haven’t created our Node.js web server yet), install live server as follows:

npm install -g live-server

Then you can clone the example code repo for this blog post:

git clone

Now go into the repo, install dependencies and run the example using live-server

cd nodejs-visualization-example/basic-visualization
bower install

When you run live-server your browser should automatically open and you should see the chart from Figure 4.

It’s a good idea to check that your visualization works directly in a browser before you try and capture it in a headless browser; there could easily be something wrong with it and problems are much easier to troubleshoot in a real browser than in the headless browser. live-server has live reload built-in, so now you have a nice little setup here when you can edit and improve the chart interactively before you try to capture it under Node.js.

This simple line chart was constructed with C3. Please take a look over the example code and maybe look at some of the examples in the C3 gallery to learn more about C3.

Starting the web server

To host our visualization, we need a web server. It’s not quite enough that we have a web server, we also need to be able to dynamically start and stop it. Listing 1 shows the code for our web server.

Listing 1 – Code for a simple web server that can be started and stopped

const express = require('express');
const path = require('path'); module.exports = { start: () => { // Export a start function so we can start the web server on demand. return new Promise((resolve, reject) => { const app = express(); const staticFilesPath = path.join(__dirname, "public"); // Make our 'public' sub-directory accessible via HTTP. const staticFilesMiddleWare = express.static(staticFilesPath); app.use('/', staticFilesMiddleWare); const server = app.listen(3000, err => { // Start the web server! if (err) { reject(err); // Error occurred while starting web server. } else { resolve(server); // Web server started ok. } }); }); }

The code module in listing 1 exports a start function that we can call to kickstart our web server. This technique, being able to start and stop our web server, is also very useful for doing automated integration testing on a web site. Imagine that you want to start your web server, run some tests against it and then stop it at the end.

So now we have our browser-based visualization and we have a web server that can be started and stopped on demand. These are the raw ingredients we need for capturing server-side visualizations. Let’s mix it up with Nightmare!

Rendering the web page to an image

Now let’s flesh out the code to capture a screenshot of the visualization with Nightmare. Listing 2 shows the code that instances Nightmare, points it at our web server and then takes the screenshot.

Listing 2 – Capturing our chart to an image file using Nightmare

const webServer = require('./web-server.js');
const Nightmare = require('nightmare'); webServer.start() // Start the web server.
.then(server => { const outputImagePath = "./output/nyc-temperatures.png"; const nightmare = new Nightmare(); // Create the Nightmare instance. return nightmare.goto("http://localhost:3000") // Point the browser at the web server we just started. .wait("svg") // Wait until the chart appears on screen. .screenshot(outputImagePath) // Capture a screenshot to an image file. .end() // End the Nightmare session. Any queued operations are completed and the headless browser is terminated. .then(() => server.close()); // Stop the web server when we are done.
.then(() => { console.log("All done :)");
.catch(err => { console.error("Something went wrong :("); console.error(err);

Note the use of the goto function, this is what actually directs the browser to load our visualization.

Web pages usually take some time to load. That’s probably not going to be very long, especially as we are running a local web server, but still we face the danger of taking a screenshot of the headless browser before or during its initial paint. That’s why we must call the wait function to wait until the chart’s <svg> element appears in the browser’s DOM before we call the screenshot function.

Eventually, the end function is called. Up until now we have effectively built a list of commands to send to the headless browser. The end function actually sends the commands to the browser, which takes the screenshot and outputs the file nyc-temperatures.png. After the image file has been captured we finish up by shutting down the web server.

You can find the completed code under the capture-visualization sub-directory in the repo. Go into the sub-directory and install dependencies:

cd nodejs-visualization-example/capture-visualization
cd public bower install
cd ..
npm install

Now you can try the code for yourself:

node index.js

This has been an extract from chapter 11 of Data Wrangling with JavaScript now available on the Manning Early Access Program. Please use this discount code fccdavis3 for a 37% discount. Please check The Data Wrangler for new updates on the book.

The post Server-Side Visualization With Nightmare appeared first on CSS-Tricks.

Routing and Route Protection in Server-Rendered Vue Apps Using Nuxt.js

This tutorial assumes basic knowledge of Vue. If you haven’t worked with it before, then you may want to check out this CSS-Tricks guide on getting started.

You might have had some experience trying to render an app built with Vue on a server. The concept and implementation details of Server-Side Rendering (from Github.

Why Should I Render to a Server?

If you already know why you should server-render and just want to learn about routing or route protection, then you can jump to Setting Up a Nuxt.js App from Scratch section.

Sarah Drasner wrote a great post on what Nuxt.js is and why you should use it. She also showed off some of the amazing things you can do with this tool like page routing and page transitions. Nuxt.js is a tool in the Vue ecosystem that you can use to build server-rendered apps from scratch without being bothered by the underlying complexities of rendering a JavaScript app to a server.

Nuxt.js is an option to what Vue already offers. It builds upon the Vue SSR and routing libraries to expose a seamless platform for your own apps. Nuxt.js boils down to one thing: to simplify your experience as a developer building SSR apps with Vue.

We already did a lot of talking (which they say is cheap); now let’s get our hands dirty.

Setting Up a Nuxt.js App from Scratch

You can quickly scaffold a new project using the Vue CLI tool by running the following command:

vue init nuxt-community/starter-template <project-name>

But that’s not the deal, and we want to get our hands dirty. This way, you would learn the underlying processes that powers the engine of a Nuxt project.

Start by creating an empty folder on your computer, open your terminal to point to this folder, and run the following command to start a new node project:

npm init -y # OR yarn init -y

This will generate a package.json file that looks like this:

{ "name": "nuxt-shop", "version": "1.0.0", "main": "index.js", "license": "MIT"

The name property is the same as the name of the folder you working in.

Install the Nuxt.js library via npm:

npm install --save nuxt # OR yarn add nuxt

Then configure a npm script to launch nuxt build process in the package.json file:

"scripts": { "dev": "nuxt"

You can then start-up by running the command you just created:

npm run dev # OR yarn dev

It’s OK to watch the build fail. This is because Nuxt.js looks into a pages folder for contents which it wills serve to the browser. At this point, this folder does not exist:

Exit the build process then create a pages folder in the root of your project and try running once more. This time your should get a successful build:

The app launches on Port 3000 but you get a 404 when you try to access it:

Nuxt.js maps page routes to file names in the pages folder. This implies that if you had a file named index.vue and another about.vue in the pages folder, the will resolve to / and /about, respectively. Right now, / is throwing a 404 because, index.vue does not exist in the pages folder.

Create the index.vue file with this dead simple snippet:

<template> <h1>Greetings from Vue + Nuxt</h1>

Now, restart the server and the 404 should be replaced with an index route showing the greetings message:

Project-Wide Layout and Assets

Before we get deep into routing, let’s take some time to discuss how to structure your project in such a way that you have a reusable layout as sharing global assets on all pages. Let’s start with the global assets. We need these two assets in our project:

  1. Favicon
  2. Base Styles

Nuxt.js provides two root folder options (depending on what you’re doing) for managing assets:

  1. assets: Files here are webpacked (bundled and transformed by webpack). Files like your CSS, global JS, LESS, SASS, images, should be here.
  2. static: Files here don’t go through webpack. They are served to the browser as is. Makes sense for robot.txt, favicons, Github CNAME file, etc.

In our case, our favicon belongs to static while the base style goes to the assets folder. Hence, create the two folders and add base.css in /assets/css/base.css. Also download this favicon file and put it in the static folder. We need normalize.css but we can install it via npm rather than putting it in assets:

yarn add normalize.css

Finally, tell Nuxt.js about all these assets in a config file. This config file should live in the root of your project as nuxt.config.js:

module.exports = { head: { titleTemplate: '%s - Nuxt Shop', meta: [ { charset: 'utf-8' }, { name: 'viewport', content: 'width=device-width, initial-scale=1' }, { hid: 'description', name: 'description', content: 'Nuxt online shop' } ], link: [ { rel: 'stylesheet', href: '' }, { rel: 'icon', type: 'image/x-icon', href: '' } ] }, css: ['normalize.css', '@/assets/css/base.css']

We just defined our title template, page meta information, fonts, favicon and all our styles. Nuxt.js will automatically include them all in the head of our pages.

Add this in the base.css file and let’s see if everything works as expected:

html, body, #__nuxt { height: 100%;
} html { font-size: 62.5%;
} body { font-size: 1.5em; line-height: 1.6; font-weight: 400; font-family: 'Raleway', 'HelveticaNeue', 'Helvetica Neue', Helvetica, Arial, sans-serif; color: #222;

You should see that the font of the greeting message has changed to reflect the CSS:

Now we can talk about layout. Nuxt.js already has a default layout you can customize. Create a layouts folder on the root and add a default.vue file in it with the following layout content:

<template> <div class="main"> <app-nav></app-nav> <!-- Mount the page content here --> <nuxt/> </div>
/* You can get the component styles from the Github repository for this demo */
</style> <script>
import nav from '@/components/nav';
export default { components: { 'app-nav': nav }

I am omitting all the styles in the style tag but you can get them from the code repository. I omitted them for brevity.

The layout file is also a component but wraps the nuxt component. Everything in the this file is shared among all other pages while each page content replaces the nuxt component. Speaking of shared contents, the app-nav component in the file should show a simple navigation.

Add the nav component by creating a components folder and adding a nav.vue file in it:

<template> <nav> <div class="logo"> <app-h1 is-brand="true">Nuxt Shop</app-h1> </div> <div class="menu"> <ul> <li> <nuxt-link to="/">Home</nuxt-link> </li> <li> <nuxt-link to="/about">About</nuxt-link> </li> </ul> </div> </nav>
/* You can get the component styles from the Github repository for this demo */
import h1 from './h1';
export default { components: { 'app-h1': h1 }

The component shows brand text and two links. Notice that for Nuxt to handle routing appropriately, we are not using the <a> tag but the <nuxt-link> component. The brand text is rendered using a reusable <h1> component that wraps and extends a <h1> tag. This component is in components/h1.vue:

<template> <h1 :class="{brand: isBrand}"> <slot></slot> </h1>
/* You can get the component styles from the Github repository for this demo
export default { props: ['isBrand']

This is the output of the index page with the layout and these components added:

When you inspect the output, you should see the contents are rendered to the server:

Implicit Routing and Automatic Code Splitting

As mentioned earlier, Nuxt.js uses its file system to generate routes. All the files in the pages directory are mapped to a URL on the server. So, if I had this kind of directory structure:

--| product/
-----| index.vue
-----| new.vue
--| index.vue
--| about.vue

…then I would automatically get a Vue router object with the following structure:

router: { routes: [ { name: 'index', path: '/', component: 'pages/index.vue' }, { name: 'about', path: '/about', component: 'pages/about.vue' }, { name: 'product', path: '/product', component: 'pages/product/index.vue' }, { name: 'product-new', path: '/product/new', component: 'pages/product/new.vue' } ]

This is what I prefer to refer to as implicit routing.

On the other hand, each of these pages are not bundled in one
bundle.js. This would be the expectation when using webpack. In plain Vue projects, this is what we get and we would manually split the code for each route into their own files. With Nuxt.js, you get this out of the box and it’s referred to as automatic code splitting.

You can see this whole thing in action when you add another file in the pages folder. Name this file, about.vue with the following content:

<template> <div> <app-h1>About our Shop</app-h1> <p class="about">Lorem ipsum dolor sit amet consectetur adipisicing ...</p> <p class="about">Lorem ipsum dolor sit amet consectetur adipisicing ...</p> <p class="about">Lorem ipsum dolor sit amet consectetur adipisicing ...</p> <p class="about">Lorem ipsum dolor sit amet consectetur adipisicing ...</p> ... </div>
import h1 from '@/components/h1';
export default { components: { 'app-h1': h1 }

Now click on the About link in the navigation bar and it should take you to /about with the page content looking like this:

A look at the Network tab in DevTools will show you that no pages/index.[hash].js file was loaded, rather, a pages/about.[hash].js:

You should take out one thing from this: Routes === Pages. Therefore, you’re free to use them interchangeably in the server-side rendering world.

Data Fetching

This is where the game changes a bit. In plain Vue apps, we would usually wait for the component to load, then make a HTTP request in the created lifecycle method. Unfortunately, when you are also rendering to the server, the server is ready way before the component is ready. Therefore, if you stick to the created method, you can’t render fetched data to the server because it’s already too late.

For this reason, Nuxt.js exposes another instance method like created called asyncData. This method has access to two contexts: the client and the server. Therefore, when you make request in this method and return a data payload, the payload is automatically attached to the Vue instance.

Let’s see an example. Create a services folder in the root and add a data.js file to it. We are going to simulate data fetching by requesting data from this file:

export default [ { id: 1, price: 4, title: 'Drinks', imgUrl: '' }, { id: 2, price: 3, title: 'Home', imgUrl: '' }, // Truncated for brevity. See repo for full code.

Next, update the index page to consume this file:

<template> <div> <app-banner></app-banner> <div class="cta"> <app-button>Start Shopping</app-button> </div> <app-product-list :products="products"></app-product-list> </div>
import h1 from '@/components/h1';
import banner from '@/components/banner';
import button from '@/components/button';
import productList from '@/components/product-list';
import data from '@/services/data';
export default { asyncData(ctx, callback) { setTimeout(() => { callback(null, { products: data }); }, 2000); }, components: { 'app-h1': h1, 'app-banner': banner, 'app-button': button, 'app-product-list': productList }

Ignore the imported components and focus on the asyncData method for now. I am simulating an async operation with setTimeout and fetching data after two seconds. The callback method is called with the data you want to expose to the component.

Now back to the imported components. You have already seen the <h1> component. I have created few more to serve as UI components for our app. All these components live in the components directory and you can get the code for them from the Github repo. Rest assured that they contain mostly HTML and CSS so you should be fine understanding what they do.

This is what the output should look like:

Guess what? The fetched data is still rendered to the server!

Parameterized (Dynamic) Routes

Sometimes the data you show in your page views are determined by the state of the routes. A common pattern in web apps is to have a dynamic parameter in a URL. This parameter is used to query data or a database for a given resource. The parameters can come in this form:

The value 2 in the URL can be 3 or 4 or any value. The most important thing is that your app would fetch that value and run a query against a dataset to retrieve relative information.

In Nuxt.js, you have the following structure in the pages folder:

--| product/
-----| _id.vue

This resolves to:

router: { routes: [ { name: 'product-id', path: '/product/:id?', component: 'pages/product/_id.vue' } ]

To see how that works out, create a product folder in the
pages directory and add a _id.vue file to it:

<template> <div class="product-page"> <app-h1>{{product.title}}</app-h1> <div class="product-sale"> <div class="image"> <img :src="product.imgUrl" :alt="product.title"> </div> <div class="description"> <app-h2>${{product.price}}</app-h2> <p>Lorem ipsum dolor sit amet consectetur adipisicing elit.</p> </div> </div> </div>
<style> </style>
import h1 from '@/components/h1';
import h2 from '@/components/h2';
import data from '@/services/data';
export default { asyncData({ params }, callback) { setTimeout(() => { callback(null,{product: data.find(v => === parseInt(}) }, 2000) }, components: { 'app-h1': h1, 'app-h2': h2 },

What’s important is the asyncData again. We are simulating an async request with setTimout. The request uses the id received via the context object’s params to query our dataset for the first matching id. The rest is just the component rendering the product.

Protecting Routes With Middleware

It won’t take too long before you start realizing that you need to secure some of your website’s contents from unauthorized users. Yes, the data source might be secured (which is important) but user experience demands that you prevent users from accessing unauthorized contents. You can do this by showing a friendly walk-away error or redirecting them to a login page.

In Nuxt.js, you can use a middleware to protect your pages (and in turn your contents). A middleware is a piece of logic that is executed before a route is accessed. This logic can prevent the route from being accessed entirely (probably with redirections).

Create a middleware folder in the root of the project and add an auth.js file:

export default function (ctx) { if(!isAuth()) { return ctx.redirect('/login') }
function isAuth() { // Check if user session exists somehow return false;

The middleware checks if a method, isAuth, returns false. If that is the case, it implies that the user is not authenticated and would redirect the user to a login page. The isAuth method just returns false by default for test purposes. Usually, you would check a session to see if the user is logged in.

Don’t rely on localStorage because the server does not know that it exists.

You can use this middleware to protect pages by adding it as value to the middleware instance property. You can add it to the _id.vue file we just created:

export default { asyncData({ params }, callback) { setTimeout(() => { callback(null,{product: data.find(v => === parseInt(}) }, 2000) }, components: { //... }, middleware: 'auth'

This automatically shuts this page out every single time we access it. This is because the isAuth method is always returning false.

Long Story, Short

I can safely assume that you have learned what Nuxt.js guide for more features and use cases. If you’re working on a React project and need this kind of tool, then I think you should try Next.js.

Routing and Route Protection in Server-Rendered Vue Apps Using Nuxt.js is a post from CSS-Tricks