Modern Domino: REST Services

Modern DominoI’ve been doing a lot of development here lately implementing and tweaking a REST service for redpill Now to get some better performance from the service. During this evolution I’ve come to a few conclusions about how best to handle the delivery of data via a REST service and thought this might be useful to others.

Why are REST Services Modern?

REST services allow you to deliver data on demand. It can deliver pretty much any type data that would be useful to be consumed by any type of application. It doesn’t matter what platform that application is running on, we’re just talking about delivering pure data. This allows developers to build their own UI and logic to best display that data whether it be JSON, XML, CSV, etc.

By providing a REST service to your application data you make it possible for other developers to consume that data from any type of platform. This provides a level of robustness to domino applications that hasn’t been possible in the past. It’s also a great way to allow people to write their own visualizations (i.e. charts and graphs) for that data.

REST services can also allow consumers to POST data back. Again, this makes it possible for people to write an entire application around a REST service. Deliver the initial data via a GET request, user makes changes and sends those changes back via a POST request.

Common Problem

When looking at most modern REST services you’ll probably notice there is quite a bit of data provided in that service. This is the norm, however in the case of delivering data from a view or documents, the means by which you process that data before delivery can take quite a bit of time. If you’re providing real time UI functionality this is probably not the best way to go. You’ll want to split up your data to deliver only what the UI needs when it needs it.

In the scenario of building a data browser of somewhat hierarchical data I was delivering about 490K of data at a time, 3 levels worth (i.e. /root/dir/files). Click on a file and see all it’s properties, etc. But building this data was taking anywhere from 2 to 12 seconds for each request. In the context of a UI element waiting on this data, this is a HUGE performance issue and makes the user think something is wrong or didn’t happen. So they click again only to start the process over.  Not to mention, while performance on your machine my be good, when you point it at 10 or 20 times the amount of data that’s on your machine you’ll probably be surprised that you’re now looking at 1 to 2 minutes to deliver all that data.

 Only Deliver what’s needed

The solution to this issue is to only deliver the data the user requests and only the most minimal amount of data to fulfill that request. Don’t be tempted to send stuff that might turn out to be handy. Only include what’s absolutely needed. In my case, I changed my service to deliver data for each level and only what the UI needed at that moment. I didn’t include all the file properties and such, only the title and other necessary information needed for the display of the file. When the user clicks the file then I get the properties for just that file. This brought those response times down to the 200ms – 300ms range. Which actually made the data browser become useful and greatly improved the UX.

Domino, XPages and OpenNTF

Domino and XPages provide the ability to build your own REST service. Sure you can use the Domino Data Services, or even readViewEntries&outputFormat=JSON, etc, but sometimes you need very specific, custom data. In this case you’ll have to build your own REST service getter. You just use the framework provided by the REST Service component. With this component you can define multiple REST services and it provides a property called “doGet” where you point it to your custom JSON, XML or CSV builder. The results of the custom builder will be to return a string of JSON, XML, CSV, etc.

Using the OpenNTF Domino API you are provided a lot of cool methods that will assist you in building your custom REST service. There is improved collection iteration, you can now do something like:
for (Document doc : DocumentCollection){
//Do Something with each document
}

There is a Document.toJson() method. Also, Document implements Map, so you can treat a document as you would a java Map. The OpenNTF Domino API is a great resource for making your XPages Java development fall more in line with traditional Java development. While this isn’t REST service specific, it does help you when developing your custom REST service. Biggest thing about this API is that you no longer need to recycle, it takes care of that step for you.

Conclusion

Hopefully I’ve made a little bit of a case for trying your own REST service. The ability to deliver minimal data on demand to drive a UI is a very nice inclusion to Domino and XPages. While it’s been around for a while now it’s still a very useful tool and makes you aware of what you’re sending to a web browser and the ramifications of what you’re sending. I certainly learned a very important lesson to better mind what you’re sending to the browser and how you’re determining what to send.

Extra Credit….. Visualizations

The data for the below visualizations was delivered via a custom REST service

3 Comments

  1. Jeff Byrd says:

    That’s what I’m talking about! Excellent post.

  2. Thorben Hellweg says:

    I love the visualizations. Looks like D3.js…? Great post anyway!

  3. Thanks Jeff and Thorben! Yes, we’re using D3.js for those visualizations. Hopefully we’ll be able to include a Modern Domino: D3.js post before too long.

Leave a Reply

Your email address will not be published. Required fields are marked *

%d bloggers like this: