I’m kind of obsessed with API documentation and I’ve discussed about it multiple times in the 100 Days DX series. One of the reasons for obsession is the fact that sucky API documentation is the biggest reason to ditch any API. In addition to that current job in Platform of Trust has opened my eyes for the complexity of great API documentation when you have multiple APIs with versioning. That can not be achieved without a lot of automation and automation requires pattern recognition as well.
Data driven decision making is said to be direction towards what nearly all our decision making is going to. API Documentation should not be an exception. API Documentation development should be data driven. It should be based in facts and activities of the API consumers. It should match the needs of the API consumers. Of course you can make surveys, but the response rate is normally quite low and you get results quite rarely. You simply can’t run surveys all the time since it’s not free and API consumers do not want to spend their time in recurring weekly or even monthly surveys. They have better things to do.
You just can’t get all right in the beginning. API documentation should be improving all the time as long as you have resources. You need to learn from the API consumers and react to the data given insights.
Your APIs are monitored, right? You should get some hints which API endpoints are most used. You should put effort in improving that part in your API documentation. Also the opposite direction is pretty interesting. You should seek the failure patterns in the API logs. Reason for the erorrs might well be in your API documentation. API consumers just don’t understand something in there and keep on banging their heads on the 400 wall. Even if the consumers see what’s wrong with your documentation, they rarely contact you. They might post the answer for example in Stack Overflow instead. Anyway, you should analyse the successes and failures in your logs to find out spots to improve in documentation.
Searches in API docs and developer portal
Another obvious data to analyse is the search queries in API documentation and Developer Portal. What are people looking for? If you can see a pattern, you can “surface” the most wanted items a bit more to make discovery easier. Perhaps you’ll find a pattern that consumers aree looking for information that is not even available in your API docs or portal. Perhaps you should add it?
Intelligent API documentation would analyse the usage and build trends. Then in slow reacting pace might change the order of things. Of course that breaks the discovery of learned paths. Consumer might get familiar with your API documentation and then if it changes frequently, they feel lost again. Some level of stability in the documentation is needed. The obvious spots to make drastic changes in API documentation is for example version change. Then the developers most likely are mentally prepared for changes anyway.
Analyse application (open source)
The last idea is not possible always and might be a bit challenging. The matured and popular APIs might be able to analyse open source applications (programmably) and learn how APIs are used. The patterns might give you hints of the learning paths or maps of the API consumers. You might learn what are the key concepts they need to learn anyway and put effort on making those parts just brilliant in your API docs.
I saved the most common method for last. The consumers traverse the documentation back and forth. They jump to Developer portal and then return to API documentation. By analysing the paths you can see if your consumers are making detours to find what they are looking for. Of course you as a wise API product manager have created wanted customer path for your consumers to follow. You should be analysing the results of following the path constantly. Normally the analysis is conversion management but it can be more. Questions should be: do my consumers find the information needed fast? From which sections of the documentation are they leaving elsewhere? Where do they spend more time? and why?
Some more to read from 100 Days DX
While you are here...
Check out Platform of Trust in which I've worked to enable best possible developer experience!