Server-Side Pagination and Filtering with Angular and MongoDB

I am trying to get Server-Side Queries to work. I am using the excellent trNgGrid to accomplish this. If you don't know it yet, it is the plugin that I would recommend if you are working with tables and Angular.

You can get started with trNgGrid here: Moonstorm/trNgGrid

Unfortunately the Demo Page remains relatively silent about how to implement real-world usable server-side pagination. It mentions only the HTML side of things. But how to exactly request from your server, from your database or how to emply filtering and searching it does not say.

E.g. I have a database with 30.000 entries, this is how I request it and assign the API callback from the server to $scope.docs; which is the ng-model attached to my trNgGrid.

$scope.docs = DatabaseService.findAll().then(function(response) {
            return response.data;
});

Now how to combine above with the server-side trnggrid method onServerSideItemsRequested?

$scope.onServerSideItemsRequested = function (currentPage, pageItems, filterBy, filterByFields, orderBy, orderByReverse) { ... }

I want to request only 10 items and even more; the rest of the items residing on my server should be searchable, e.g. my fields are named:

"Name", "Address", "Gender". How to achieve that if I search for Name= Alfred with trnggrid's filter feature, that all 500 "Alfreds" are getting displayed?

I found a solution on how to achieve this. First, we implement server-side pagination, scope variables that limits result and delivers the correct "page" back to the API caller.

This is what my Express.js / Mongoose API does with the page and pagecount parameters which I am passing through to the server:

IP.find()
      .skip(req.params.page*req.params.pagecount)
      .limit(req.params.pagecount)
      .populate('stuff')
      .exec(function (err, Model) { 
            // callback
      }

That works as expected, giving back only the requested page and skipping previous ones.

On the client-side I figured out how to employ filtering with server side queries; this is how my Angular controller will work:

$scope.onServerSideItemsRequested = function (currentPage, pageItems, filterBy, filterByFields, orderBy, orderByReverse) {

    $scope.requestedItemsGridOptions = {
        pageItems: pageItems,
        currentPage: currentPage,
        filterBy: filterBy,
        filterByFields: angular.toJson(filterByFields),
        orderBy: orderBy,
        orderByReverse: orderByReverse,
        requestTrapped: true
    };

    console.log("FilterBy: ", filterBy);
    console.log("filterByFields: ", filterByFields);
}

This is how the API endpoint looks like in Mongoose - it will automatically build a query based on the filters a user entered in trNgGrid

var conditions = {};

for (var key in req.body.filterByFields) {
  if (req.body.filterByFields.hasOwnProperty(key)) {
    conditions[key] = req.body.filterByFields[key];
    console.log(conditions);
  } 
}

var query = IP.find(conditions);

return query
  .skip(req.body.requestedPageNumber*req.body.myPageItemsCount)
  .limit(req.body.myPageItemsCount)
  .populate('stuff')
  .exec(function (err, Model) {
     // callback
  }

Above code will automatically build a query according to the filters used in trnggrid. Data sanitation is something I am still working on; yet thanks to the power of JavaScript type casting should not be necessary at all.

Another important thing to do with server-side pagination is to let trNgGrid know of the total number of returned results, in order for it to be displayed on the pagination bar. To do this I employ Mongoose's count method:

return query
  .skip(req.body.requestedPageNumber*req.body.myPageItemsCount)
  .limit(req.body.myPageItemsCount)
  .exec(function (err, model) {
    if (!err) {
        Model.count(conditions,function(err,c){console.log(c);});
        res.status(200).send({
        status: 'OK', 
        error: 'Grid erfolgreich geladen', 
        type: 'success',
        model:model
      });

By using count on your Model and passing in the filters stored inside of conditions from Angular, you will the correct total amount of search results; which then you can pass back to your client-side.

Actually, to successfully pass back the count you have to callback inside the callback, if you know a more elegant solution I would be interested:

.exec(function (err, model) {
    if (!err) {
        Model.count(conditions,function(err,c){
          res.status(200).send({
            status: 'OK', 
            error: 'Grid erfolgreich geladen', 
            type: 'success',
            myItemsTotalCount: c,
            model:model,
        });
      });
    } 

As a further thought, it would be interesting to see server-side filtering be extended to enable negation with ! or greater-than and less-than modifiers. If you have an idea how to implement that in Mongoose and Express.js, let me know.