Why do I need to care?

Note! This post contains some academic sources which are not open access articles. Normally you can get access to those from Sci-Hub. I’m not saying you should use it, if it violates your principles. I don’t have problems with it though. Research results are intended to be used and often funded by taxes anyway.

Huge responsibility follows huge impact

If your API is popular, any changes or deprecation of it will have huge effects on the ecosystem. API deprecation may affect hundreds of clients as was discovered in research behind (“How do developers react to restful api evolution”).

In perfect world what is published stays the same for ever but with APIs that is not the case though (“How Does Web Service API Evolution Affect Clients?”). APIs have lifecycle just as any products do. The subject is more relevant now than ever since (research) has found that RESTful web APIs are more change-prone than JAVA APIs and WSDL services during API evolution.

Compared to statically linked APIs used previously (code level integration) the change is even bigger. Previously developer was able to use the library (for example libxml) as long as it provided value. Now APIs are dynamically linked (via web) which has turned the situation around. API provider dictates the pace of change, not the developer. As a API provider you should respect the power you have been given and make the forced transition from version to version as smooth as possible. (“Web API growing pains: Stories from client developers and their code”)

Some of the business reason to deprecate API are lack of 3rd party developer innovations (change from public to partner API perhaps), opposing financial incentive or competition, changes in technology, versioning and security concerns.

Breaking changes

Even before shutting it completely, there will be versions. Some of the changes will be backward compatible and does not require version change. Examples of breaking changes are:

Whether you deprecate the API or make a new version of it, the impact will be enormous. API versioning is a constant on-going debate. You can read about the strategies for example from (APIs You Won’t Hate). Thus you should not take versioning or API deprecation lightly. Dispite the big impact of deprecation, (researchers Wu et al) have found out that APIs driven frameworks are backward-incompatible and even the valuable deprecation messages are often missing (Robbes et al.; Brito et al.).

Perfection is an illusion

New APIs are prone to be changed often and that causes issues for the early adopters. You probably would like to design perfect API with one round, but in reality that is not possible. Changes are inevidable and a good strategy is to clearly mark in documentation which features are prototype/alpha/beta (i.e.features which are very likely to change).

In large scale study of API changes in Android ecosystem (McDonnell et al.) found out that client adoption is lagging behind. In other words evolution of APIs is too fast for consumers to adopt. That can be due to multiple reasons such as lack of communication between API owner and consumer (Wang et al), too tight migration and adaptation timetable or lack of roadmap.

Good practice is to write migration guides for API consumers in both cases. Keep in mind that great developer experience is not about taking the money and run, but about taking care of the customer even after we or the customer has moved elsewhere.

But even if you provide roadmap, mark experimental features in documentation, you have clear versioning and migration guides for different situations you are not done yet. The big question is how do I pass the information to API consumers? How can I be sure that all developers are aware of the changes and help we provide? Answer is you don’t. You should follow and analyze the discussions about your API in forums, try to push information to the consumers with email (you don’t know it they read or understand it), pass information in the API response headers (sunset header)(Sturgeon), push information to relevant social media channels, write blog posts about it and use blackout testing (planned shutdown of all API functions for a short window of time)(Blackout testing). The blackout tests are often praised by the developers: “These blackout tests will be super helpful in the transition. Thanks for setting those up!”[6]

Sunset comes eventually

All the above are examples of retirement plan, which you should have in case of killing an API permanently. Migration time to “sunset” should be at least 6 months. Google Maps and Youtube follow a one year announcement to deprecation policy. Of course there are exceptions in the general rule of retirement schedule. The reasons for faster pace changes are whenever it is required by law to make such changes, whenever there is a security risk or substantial economic or material technical burden. The last one is debatable, but often stated as a reason to deviate from general retirement plan.

Change and retirement rules

Espinha et al (2014) have listed six guidelines for API provider to follow regarding API changes and retirement:

  1. Do not change too often. For example Facebook has made breaking changes constantly and received a lot of feedback about it. Thus they have moved to quarterly based change management.
  2. Old versions of the API should not linger too long. Years long migration timetable leads to too relaxed adoptation and the migration is done at the last minute anyway. Google has learned the lesson with Google Maps API. In genetal you should not maintain old versions longer than 6 months to one year.
  3. Keep usage data of your system. Use API management or alike to see what features are used by which developer. Then you can target change information more accurately to API consumers. This is applied by Facebook.
  4. Use blackout tests. Make it a habit since API consumers love it. Blackout tests should be run before the sunset to give a “last warning” for API consumers and speed up the migration to new API version.
  5. Provide an example of interaction with the API (regarding changed parts)
  6. Clearly mark which of the features are mature and which as beta/alpha level and due to change.

I would add:

  1. Make additive changes when possible. If you want to add a new property to some resource, just add it. Old clients will just ignore it. If you want to make breaking changes to some resource, consider just creating a new resource.
  2. Take Preventative measures. Spend more time on design, prototyping, and beta-testing your API before you release it. If you just throw together the quickest thing that could possibly work, you’re going to be paying the price for that for a long time to come.

Keep your eyes open

Even if you do all of the above, you can’t be sure that all consumers get the information. Therefore you must have a damage control plan in place. Get ready for feedback and respond quickly. Be ready to redeem some of the customers. Keep an eye on Stack Overflow, because that’s where most of the developers go to seek for help ( Developer Survey Results 2018 ). For example “stripe-payments” tag has 5 877 questions, “twilio” 6 201 questions and “docusignapi” 3 623 questions.

Do what ever it takes to mimize the PR and customer satisfaction damage which is lurking behind the corner.

Some more to read from 100 Days DX