You might have stumbled upon the famous API mandate given by Jeff Bezos. One of the items in the mandate was “All service interfaces, without exception, must be designed from the ground up to be externalizable. That is to say, the team must plan and design to be able to expose the interface to developers in the outside world. No exceptions.” That is still solid advice which will help you to minimise refactoring costs and prepares all APIs for future.
If you happen to be in position responsible for API Developer eXperience, it’s an intriguing yet challenging position! You are sitting in between the internal development team and customers (developers and their managers). Having this in mind, combining internal and external developer experience is not always a walk in the park. It’s more like Ying and Yang or two sides of a coin. It might occasionally require some compromises and accepting that not everything is optimal.
My job is the find the solutions how to enable maximum positive developer experience for the API consumers without killing the internal developer experience.
Here’s a few things I was faced with during the first months in managing the build of Platform of Trust DX. My responsibilities were not limited to APIs only, but included management of developer portal, API documentation and sandbox as well. But let’s discuss three situations related to managing internal and external DX.
I gave a talk about the topic in APIDays Finland 2019, but content is a bit different. So you get value from reading and watching both.
Top tools for API development
First the internal DX. According to The State of API 2019 report, API teams want tools that are easy to use, can be adopted quickly, and that fit into their existing toolset. This makes sense since who wants to work daily with poorly designed and inefficient tools while building and managing APIs. API teams are using an average of five different tools to develop high quality APIs. The top tools for API development include: API documentation tools (55%), functional testing tools (52%), CI/CD tools (51%), source control hosts (48%), and unit testing tools (47%).
While 67% of respondents say they do have a formal API documentation in place, only 37% view documentation as a top priority for their organization. The lack of focus on documentation as a top priority could prove costly for API providers. More than half of API consumers cite ‘accurate and detailed documentation’ as one of the most important factors they consider when evaluating an API. The only factors viewed as more important for consumers were performance and usability.
1. Microservice architecture vs exposed APIs
One of the first topics to discuss with acting architect was the amount and names of the exposed APIs. I was glad that development team had selected microservice style to build the platform. But soon I discovered that the exposed APIs reflect directly the internal structure of our platform. Bringing up this observation in the company Slack resulted to days long discussion with the architect. My point was that what makes sense internally - does not necessarily make sense for the external developers.
The capabilities of the platform are developed in small independed pieces (microservices). Each of them have API or more. Just exposing some of the APIs as is to 3rd party developers results to collection of hard to adopt and understand APIs. Understanding that while exposing your APIs, you need to step into the shoes of the consuming developer (external DX) - not the API developing developer (internal DX).
The clash between me and architect was due to thee fact that in his world internal developer experience is probably the only thing that matters to get things done smoothly. For me the external developer experience was and still is the most important thing. I want smooth and easy onboarding for our APIs to maximise customer base and satisfaction.
After several day long discussions in Slack we found an understanding that we are talking about the same subject (DX), but our approaches are just different and we have to find a compromise here and there. Some of the APIs are exposed as is, but the name has been tweaked. In some cases two or more internal APIs are smashed together in exposing layer while keeping them still separate in the Dark Side. Dark Side is the name our Chief Marketing Officer Pirkko Laitinen gave to all backend stuff.
2. RAML vs OpenAPI Spec
The second example is API documentation related. Another debate you might end up is which API description specification to use? Although OpenAPI Spec is the default answer for REST API description for most. The advantages of OpenAPI spec are:
- A large community and support-base
- High adoption rate, meaning lots of documentation
- Strong framework support
- Has the largest language support of any opensource framework
Our developers did not choose to use that but RAML instead. Why? OpenAPI Spec doesn’t allow for code reuse, includes, or extensions. With RAML developers were able to reuse definitions and pieces of the description again and again. The developers estimated the amount of redundancy process wise would be too much with OpenAPI Spec. As you know, good developers are lazy bastards and minimize the work load all the time. Having the internal process slick and enjoyable to work with was valued more than popularity of the specification format.
The advantages of RAML weighted more for us:
- Single specification to maintain
- Strong, visual-based IDE and online tooling with collaboration focus
- Allows for design patterns
- Easy to get started
Furthermore the poorer tool support for newer versions was not a problem for us. The selected approach did cause some extra work in getting the API documentation generation process done. I had defined that we go for the “Stripe style” 3-column model in which navigation is on the left and you can jump to wanted section in the documentation from there. In the middle you have the more detailed descriptions of options and endpoints. On the right you have code examples with various programming languages for the endpoints. Result looks like the below screenshot
If you want a thing done now, do it yourself
Our developers were occupied with core features - so I took it as a learning experience and started working on the solution on my own. Back to python and being the Jack Sparrow of Code…
Since we lean towards open source and hate building everything from scratch, I started to look for available open source implementations to use. Soon I found Slate which is ruby driven implementation of the wanted API Docs. Running the Slate in dynamic service contains some security issues, but that was not a problem for us, since we use the build feature in Slate to generate HTML page from the documentation and serve it with Github pages under own domain.
Slate uses markdown files to describe the content and I have raw RAML formatted API descriptions to start with. There was a little gap between what I have and what I want to see (the above picture). To get content from RAML files to Slate, I had to seek out tools to do some conversion. There was no tool to convert RAML files directly to Slate formatting. I had to convert RAML files to OpenAPI Spec first and from those to Slate content files. After that I had to solve how to inject code examples in the right spots in the documentation. All this has to happen automatically of course since managing the updates manually is a pain in the ass. Furthermore that would be against the selected CI/CD process we use in everything. As a result I had to write a small python program to do all needed operations: conversions, injecting code examples from files and generating the HTML content.
All that was added to Travis and now if new pull request is merged (after my review) to master in Github, the documentation is rebuilt and published automatically (if build was successful). The lesson learned here is that sometimes you need to do things somewhat not so directly, but you can find a way to satisfy pretty much everyone. I could have been an asshole and do the don Quijote fight against the windmills and require developers to use OpenAPI Spec to make my job easier. I decided not to. I wanted our developers to be productive and respect their well thought choices.
Too much manual work
During summer 2019 we got into situation that maintaining code examples manually would be too much work. We have 7 open APIs in the platform already and more to come. We had to start developing an automated solution. Some sort of code example generation tools exists but we were not able to find robust enough. So we decided to build one. It will be a perfect match to our needs and will be part of the automated API documentation generatioon toolset. The tool does not just generate the code but also runs every example to verify that they work as well. More about that later this fall.
3. Insomnia vs Postman
Postman is one of the most popular tool to dissect RESTful APIs made by others or test ones you have made yourself. Postman and alike tools can be used to get familar with APIs without writing any code. These tools are prettier GUI alternatives for testing APIs with cURL among other things. The point is that the same tool can be utilized by internal developers (building the APIs) and 3rd party developers consuming the APIs. One familar format to enable easy setup for 3rd party developers, is to export collections for the selected tool.
This is what Postman offers - collection with environment configurations. By importing the ready made collection of API calls the 3rd party can start exploring the API with own tools and own environment (desktop). As for GUI alternative is to offer browser based API console in which 3rd party developer can make requests to the APIs. Yet Postman and alike solutions are more versatile and often easier to use.
I asked our developers about why they chose Insomnia over Postman and reason is that again Insomnia fits better in the teams workflow and is considered more agile in development. The overall image of Postman inside the team was not that good:
- “They (Insomnia) tend to address the bugs and feature requests in GitHub pretty fast”.
- “whereas Postman refuses to provide a good fix for a critical issue causing issues to lots of people and has been reported years ago”
- “compared to my last experience with Postman it’s (Insomnia) easier for both new users and experienced users, as well as just having more useful features, and most critically - less bugs. I initially switched to it simply because the
hostsbug made Postman unusable”
The above reasons among others has turned our developers to use Insomnia instead of market leader Postman. Given the above situation my options as Developer eXperience lead are:
- Force the team to use Postman because that is most often expected
- Create / get a tool to convert Insomnia export to Postman collections
- Use Insomnia and offer needed configuration for that in public for 3rd party developers.
Option one is out of the question in short term. I will not force developers to use other tools than what they see fit. They have selected the tools for a reason and messing with that would cause more new problems than solve any existing ones. Second option is more likely to exist in some form (did not check yet) as a tool or service, but conversions contain always a risk to break something. Thus I did select to go against the mainstream even if I know that some of the API consumers want to see Postman collections. We will provide Insomnia settings and package to fiddle with our exposed APIs. Of course we provide detailed guide how to work with the Insomnia in minutes and start learning the opportunities and options of our APIs.
Some more to read from 100 Days DX
- #100 - The biggest open resource on Developer eXperience so far
- #99 - Hackers - the top 1% of engineers
- #98 - Invalid Open API spec files ruins the developer experience
- #97 - Lightweight API evaluation framework
- #96 - Embracing open source community driven tools development
- #95 - Exploring the multilevel nature of API Design Guides
While you are here...
Check out Platform of Trust in which I've worked to enable best possible developer experience!