I was reading the book Subscribed: Why the Subscription Model Will Be Your Company’s Future - and What to Do About It and had a slack online meeting with a team building World app for our platform. With that app consumers or subscribers of the platform can view and manipulate visually the data space they have access to. The book is naturally discussing matters from business point of view, not technology. Yet the similarities struck me.

Stream of dipers

The book by the way is awesome, well written, entertaining yet informative. I’ve been sucked into subscription thinking already by some years ago, but I wanted to read the book and get new ideas and more thorough understanding of it. I believe I’m not the only one as a consumer to want subcscription for a lot of things. I would love to get dipers as a service and delivered to my house. We have 1 year-old identical twin girls at home and the amount of dipers needed is phenomenal. The same service could probably sell me additional products such as wet wipers, D-vitamin drops and all.

Ride a scooter

Another example of how brainwashed I’m to subscription is electric scooters. Tier and Voi occupied Tampere basically over one weekend. I tried Tier once and after that I wanted to have subscription. Not just any subscription but family pack. Gimme 5 users and unlimited use in Tampere for less than 50€ / month. I suggested that to Tier in Twitter and found out that subscription plans are coming (most likely). Compared to subscription plans with APIs, the difference is that there is no free tier (freemium). And now without the subscription plan you pay the initial fee everytime and the minute price. I want flat rate. One of the reasons to subscription success is that people don’t want to own things so much, but get access.

Car pool subscription

According to the book car manufacturers have understood the trend of not wanting to own. They have to understand and act or perish. Some of the manufacturers offer already subscription plan and allow you to change your car 8 times per year. This is not leasing since you are not given the option to buy the car at the end. You are not buying the car, you are byuing access to pool of cars.

Deliver my package

Lets say you order something (physical) from a webstore and it will be delived to you. Often delivery is mail service or courier. Instead of you checking from a web app the status/location of delivery each hour/day, you request updates to your phone as SMS (old fashion but works). In other words you subscribe to status of the package. You don’t need to check the status from the service just to see that nothing has changed, the frigging package is still in Poland.

Now apply that to any application with data needs similar the the above. Would you like to get updates when there really is a change or keep on polling the status and get zero change (delta) 99% of the cases?

The book also gives reasoning why the subscription model is better even for the company. You might take a hit in profit / ROI in the beginning, but eventually it makes more sense and anything else. Now that the business level is pretty much going to towards subcription, what about APIs? The same kind of trend is visible there, has been for a while already.

Problem with API developer experience

Traditionally companies start with REST APIS including the traditional methods GET, POST, PUT, DELETE, OPTIONS, PATCH. This results to normal client - server pattern which is tight coupling of the two. This also results to “old fashion” polling pattern. The application must be active and ask for updates periodically.

Consuming application has no idea if there’s an update on data or not, it just needs to check the status constantly. That is just stupid for all. Developer needs to build additional loops, status tracking and logic in the app and our APIs are bombarded with zero result for all. Of course API caching should handle most of the cases, but still. Useless polling is just stupid. That is also bad developer experience.

Solution

Publish/subscribe messaging, or pub/sub messaging, is a form of asynchronous service-to-service communication used for example in serverless and microservices architectures as well as chat apps. In a pub/sub model, any message published to a topic is immediately received by all of the subscribers to the topic. Pub/sub messaging can be used to enable event-driven architectures, or to decouple applications in order to increase performance, reliability and scalability.

Subscription to the resque

Now if the idea on subscription model familiar to us from Netflix, Dropbox, Amazon Prime, Fender Play and many others, it becomes obvious that subscription model is less of a hassle for applications as well. Application subscribes to the topic or content. When ever there is change in the data, new information is pushed to the application. Less hassle for the consumer (developer/app) - better experience.

One way to provide subscription to the data is webhooks. W3C has adopted the formerly known pubsub as websub in January 2018. WebSub is an open protocol for distributed publish–subscribe communication on the Internet. Initially designed to extend the Atom (and RSS) protocols for data feeds, the protocol can be applied to any data type (e.g. HTML, text, pictures, audio, video) as long as it is accessible via HTTP. Its main purpose is to provide real-time notifications of changes, which improves upon the typical situation where a client periodically polls the feed server at some arbitrary interval. In this way, WebSub provides pushed HTTP notifications without requiring clients to spend resources on polling for changes.

Under WebSub, there is an ecosystem of publishers, subscribers, and hubs. The subscriber needs to run a web accessible server so that hubs can directly notify it when any of its subscribed topics have updated, using a webhook mechanism.

Pubsub as a service

Pusher is one example of pubsub as a service. They provide “Stripe like” approach with help of libs. They have some official libs and then community driven libs. The total amount of libs is around 40. The concept is pretty simple. Subscribe to a channel and get the updates. Push a message to a channel and other subcribers see it in near realtime.

Publishing new content to the stream is pretty easy

// First, run 'npm install pusher'

var Pusher = require('pusher');

var pusher = new Pusher({
  appId: 'APP_ID',
  key: 'APP_KEY',
  secret: 'APP_SECRET',
  cluster: 'APP_CLUSTER'
});

pusher.trigger('my-channel', 'my-event', {"message": "hello world"});

And if you want to get the information, just subscribe (in this example case) to the channel and wait for events.

<script src="https://js.pusher.com/4.4/pusher.min.js"></script>
...
var pusher = new Pusher('APP_KEY', {
  cluster: 'APP_CLUSTER'
});
...
var channel = pusher.subscribe('my-channel');
... 
channel.bind('my-event', function(data) {
  alert('An event was triggered with message: ' + data.message);
});

Previously pub/sub has been the term I’ve used, but now I learned from the team that websub is the thing we are going to. The concept is pretty much the same and the result for the API consumer is 99% the same. It reduces the barrier, it makes things a lot easier.

Converting all your REST APIs to subscription probably does not make sense. Apply it where it makes sense. In addition, there’s plenty of other techniques to provide near realtime data stream to the apps.

MQTT is something you should also take a look at. MQTT is a machine-to-machine (M2M)/”Internet of Things” connectivity protocol. It was designed as an extremely lightweight publish/subscribe messaging transport. It is useful for connections with remote locations where a small code footprint is required and/or network bandwidth is at a premium.

In addition to that popular frameworks like Node.js have build-in support for data subscription approach since node.js is event driven. It’s in the framework’s architecture and blood. It reacts to changes and in that websub/hooks and such technologies fit in perfectly.


Some more to read from 100 Days DX