Testing Batch Requests & Pagination in Node

Background noise: Node with HapiJS. The test spec files use the Request library & the Chai Assertion library.

The post focusses on writing simple & small tests for batch requests and pagination.

Consider a bakery menu app. It can list all the pastries any particular bakery offers. Pretty typical. It works just fine when the bakery has fewer items to list out, but becomes laggy when there is a huge list. So for performance reasons we set up a default size limit on our request sizes. This leads to pagination. Better than modifying the default which is a good safety standard, we can make the request in batches, and display pages.

Lot of ways to write for pagination in Node.

Typically your url will look something like this:
http://my-bakery-app}/bakeryName/pastries?limit=100&page=1

There is a difference between client-side query params and the backend API query params. I learnt about this distinction in a good article about Ember JS client side pagination setup with a JSON API Backend. This is a useful distinction to bear in mind as was evident when trying to test out the queryString params.

How to test for batch requests and pagination?

First things first. It's useful to run a single test in Node.

describe.only()
npm test -- --grep 'TEST DESCRIPTION/NAME'

These are the two options I use. If you are using npm for testing makes sense to run just one test. Even then it will be crucially slow. If you have more than 500 tests prepare to waste significant time.

Testing for the queryString in the URL

Chai Helpers

1. Chai-url

Chai-url: A chai assertion plugin for working with URLs

SETUP

const chaiUrl = require('chai-url');
chai.use(require('chai-url'));

TEST:
chai.expect(url).to.contain.path('/bakeryName/pastries?limit=100&page=1');

2. Chai-http

chai-http: HTTP Response assertions for the Chai Assertion Library.

SETUP

chaiHttp = require('chai-http');
chai.use(chaiHttp);

TEST
Assert that a Request object has a query string parameter with a given key, (optionally) equal to value
chai.expect(req).to.have.param('limit', 100)

While these are useful ways to test, we didn't want the test to be limited to the url or client-side params. We wanted to test the response.

Using the The Request library

We use the Request library in testing. It has more functionality than node’s native http client.It's good to know the different options you can use in it via the documentation. The first arg is always the url, then the callback function where we add our chai assertions. You can also use an options object as the first argument, just don't forget to include the uri/url key-value pair then.

Given a response has the same number of results you can be certain that changing the limit will affect the page numbers. Use the PostMan app to verify by checking the body response object.

The approach in this test is to check for different pages depending on the different limits we set. Higher limits should result in fewer pages, while lower limits means more pages to display.

  const {
    expect
  } = chai;

  // Testing for Pagination
  describe('GET /bakery/pastries in batches', function () {
    const BakeryUrl = `http://localhost:${settings.port}/bakery/pastries`;
    const qsObject = { limit: 10, page: 2 };
    return it('responds with pastries in batches- ', done =>
      request({
        url: BakeryUrl,
        qs: qsObject
      },
      // Callback function with our test assertions
      function (err, response, body) {
        expect(err).to.be.null;
        expect(response.statusCode, 'HTTP status code').to.eql(200);
        const pastriesList = JSON.parse(body);
        // 10 limit returns 16 pages
        expect(pastriesList.meta.total_pages).to.equal(16);
        return done();
      })
    );
  });

This is the test that will pass. You can then write an additional test that now validates pagination with a higher limit, which means the response pastriesList would receive fewer pages.

If you use fixtures, turn on recorder when tests are finally written, so that records/fixtures can be created without any cacheing related trouble.