Considering the importance of API docs, I’m a bit puzzled how stagnated the development seems. Researchers explore enriching content (snippets and discussion) from #StackOverflow, Stripe and a few probably use code generators and such, but what else? What we might see in the future is combination of sandbox and API docs. I know it sounds crazy, but some signs indicate that direction as we see more personalized solutions in API Docs. Perhaps the world is going towards more sandbox driven than API Docs driven support and onboarding. This is based on the fact that developers want to see and do instead of reading. Of course API Docs is there still but it’s not the place we throw new developers to. The above thinking lead me to principle:
Instead of offering documents to read, give a sandbox to explore and learn.
Static code examples as part of the documentation
API Documentation has been enriched with code examples for some time already. You don’t get much of “Wow!” yells even if you provide code examples with 5 or more programming languages. This example is from Twilio, which is also one of the traditional examples next to Stripe
Instead of providing and maintaining all code examples (can be automated though) yourself, some providers are exploring the opportunity to use crowdsourcing. Stack Overflow codesnippets are an invaluable code-centric knowledge base of small units of source code. Besides being useful for software developers, these annotated snippets can potentially serve as the basis for automated tools that provide working code solutions to specific natural language queries. Aim probably is not to replace all shared information with the crowdsourced, but enrich it.
Another one is live executable code examples as part of the API docs. That is like having runkit in your API docs. Developer is given code examples, but you are not able to modify those - someone else has decided the use cases.
From static to dynamic code examples
First step in bringing dynamics in the docs is to offer login for the developer. For the login users you can replace the tokens from placeholders to real one if you like. Thus the snippet should be reay to be used without further tuning. Of course when the code snippet is injected to a bigger code base, then the real token must be replaced to a variable anyway since it is most likely used elsewhere too and regenerated later. Besides you don’t want to put your credentials or tokens to code if you keep your code in public Github.
Some of the API providers such as Transferwise state the same - be careful with credentials - in API Docs. In those cases you most likely use a variable and add it to Travis or alike environment. But anyway, offering authentication-ready snippets makes the early exploration easier.
One rather low-hanging fruit for API docs providers is the add list of Stack Overflow questions related to endpoints in your API. Of course you should limit the listing to include only questions with accepted answers. You might want to apply other quality increasing filtering as well. Linking just about anything from SO is a risk.
Risks of copy-paste development
Leaking tokens is not the only risk. In analysis of 1.3 milllion Android app researchers Fischer et al “Stack Overflow Considered Harmful? The Impact of Copy&Paste on Android Application Security”:
- detected copied and pasted snippets in 200 672 (15.4%) apps,
- of these apps, 198 347 (15.2%) contain a question snippet and 40,786 (3.1%) apps contain an answer snippet,
- an overwhelming amount of apps contain an insecure codesnippet: 196 403 (15%) apps contain at least one,
- the top offending snippet has been found in 180,388 (13.81%) apps.
The results indicate that copy-paste is alive and used a lot. Thus the impact of provided code examples is huge. The largest number of sensitive apps that use insecure copied and pasted code snippets are 14 944 business apps, 4 707 shopping apps, and 4 243 finance apps.
What is a good code example?
At this point I started to wonder what makes a good code example. Luckily researchers Nasehi et al did a study 2010 “What makes a good Code Example - A study of Programming Q&A in StackOverflow”. In the research they analysed 163 unique Q&A threads, each had at least one answer containing code that gathered 4 or more points. Based on that Nasehi et al say that the attributes of a good code example are:
- Concise code. The code presented within answers is considered as concise if either it is shorter than similar code inside other answers to the same question, has less than 4 lines of code
- Using question context. When the question code has some flaws or is not working, the answer usually provides some corrective suggestions.
- Highlighting important parts. A considerable number of recognized answers (43 answers, each belongs to one of 36 different questions) start the discussion of the answer with highlighting the most important element of the solution.
- step-by-step solution. these answers present the solution in a detailed and ordered fashion. They might divide the code into some chunks, describing each small piece of code separately.
- providing links to extra resources. The expected length of an answer, and the time window for answering, make short answers more appealing for responders. To keep the answer short, the responder can provide links to external web sites that contain longer/more complex examples, contain examples with slightly different scenarios, and/or provide more detailed explanation.
Personal API docs
Now the code examples are in Stack Overflow (SO) or in gist. Or possibly linked to 3rd platform like in some cases of SO answers. Often that link is to codepen, jsfiddle or similar service. SO introduced runnable JS code 2014. Why did SO add runnable code feature?
- There’s no need to copy-paste the code into the post. It’s all embedded in the post automatically, so revision history and diffs just work.
- There’s no need to visit another site to get your answer. The best experience is one where your question and answer(s) are complete and on the same page.
- SO can guarantee performance and up-time. SO has high standards when it comes to performance and up-time, and want to make sure that the ability to run a snippet is always available.
As a API owner harvesting that information might become a bit tricky. As a API owner you want to know how the community is using your API and advising other developers to do API calls.
Bring codepen to Developer portal/sandbox
To take the idea a bit further… It could a mix of documentation in the background, you could bring codepen like features to it and let community offer examples as part of your documentation. Documentation is not in the center, but in peripheria. The same feature would let developer to build own “library” of API calls in the sandbox which has API docs (Remember “Using question context” as element of a good code example). Then the really meaningful code examples and rest of the API documentation would be in the same UI.
If the codepen would be part of your API Sandbox the code examples would be easier to track and analyze. As a API owner you would get intel about how community uses your API. Other values just like above with SO runnable :) The best experience is one where your code examples and API Documenation are complete and on the same page.
The hurdle to cross is the fact that users are already using codepen and alike platforms. Attracting them to use your service instead is something you need to solve. There must be some additional and significant value why they would make a change. Furthermore, in general “codepens” in the world developer is able to fiddle with all APIs in the world. But if you manage to attract enough developers to create content on your platform, that for sure will create value for the others as well. This would also acting against the principle of “go where the developers are already”. “If you build it, they will come” has been proven wrong multiple times.
Providing local sandbox
Viljami tipped about Seamless,”Ran into this yesterday. Feels pretty promising seamlessapis.com/#/”. I had to fiddle a bit with this one and then I realised that he was thinking probably more of the API development sandbox. I was approaching from API consumer side. But of course with this one you can build your own sandbox, which lead me to think about local sandboxes instead of provider’s sandbox.
Helsinki regional transport (HSL) provides full scale (with full data) local sandbox in docker container. That’s possible ofc due to open data content, but could be replicated with sample data in any context.
If you provide the machine readable specs for the APIs, developers can generate the APIs in local sandbox or use services like getsandbox.com
Crowdsourced tips and tricks in IDEs
At this point I took a small sidetrack to IDE side. A lot of developers use plethora of IDEs. I code mostly with python and have selected PyCharm as my IDE. Although these articles I write with Code. Those 2 are my IDEs I’m happy with. Stack Overflow and alike content have been tested and integrated to IDEs as well. One of the examples was found from “Mining StackOverflow to Turn the IDE into aSelf-Confident Programming Prompter”.
The idea is that developer can get supportive information without leaving the IDE and possibly for that reason drop out of flow state. In prompter developer’s activity (code) is analysed constantly and acts according to the principle of “recommender system shouldbehave like a prompter in a theatre: Ready to provide suggestions whenever the actor needs them, and ready to autonomously give suggestions if it feels something is going wrong.”
According to the research “participants agreed that Prompter is very useful when working on tasks in which the developer has poor experience, since the information bring in the IDE by Prompter helps the developer in enriching her knowledge about the task to be performed”
In another research from 2012 “Harnessing Stack Overflow for the IDE” Seahawk User Interface plugin for the open-source IDE Eclipse was introduced. It also uses SO as source for further tips and hints.
If you are really keen on the subject, check out Treude & Robillard’s article “Augmenting API Documentation with Insights from Stack Overflow” too.
The above would suggest that new developers onboarding unfamiliar API would also benefit from similar support.
Towards sandbox driven developer portal
Code examples are the core of the documentation and rest of the documentation is supporting in difficulties. We are not that for from sandbox environment. What if the developer is able to login to your developer portal which has basicly been built around sandbox environment. Developer forks given code examples to match his/her own needs while peeping details from API documentation. All the tokens and credentials are in place and developer can run any of the code created at any time and see the result. From there the developer can select code snippets tailored to specific needs and use cases. This would cannibalise Postman, Insomnia and alike services too. That would be another hurdle to cross since those apps can be used with any API.
In short the idea is that API providers build onboarding more on “Sandbox-first” principle than API docs and getting started pages. With enormously popular platforms this might work. Small API players probably should probably not even consider this option.
Building “one to rule them all solution” is tricky thing because developers want to use multiple tools all depending of the case. Furthermore versatility of pencode, jsfiddle, postman, insomnia and alike is hard to beat. Common working strategy is not to fight but collaborate.
What’s your opinion?
What do you think? Leave your comment or express your opinion in Twitter with hashtag #100DaysDX.
Some more to read from 100 Days DX
- #100 - The biggest open resource on Developer eXperience so far
- #99 - Hackers - the top 1% of engineers
- #98 - Invalid Open API spec files ruins the developer experience
- #97 - Lightweight API evaluation framework
- #96 - Embracing open source community driven tools development
- #95 - Exploring the multilevel nature of API Design Guides
While you are here...
Check out Platform of Trust in which I've worked to enable best possible developer experience!