CLI was the initial and only interface in computers in the beginning. There was no GUI. Can you image that. I started with Commodore64 and everything was done from command-line in that computer. Later on around 2001 I started learning Linux usage and then I became pretty familiar with the CLI and related tools. My personal relationship with CLI is love and hate. Sometimes it’s a bliss and occasionally a hell. The bliss is still winning though.
- What do developers use the CLI tools for?
- How those make life easier?
- How the CLI tools make us productive?
I asked in Twitter some of the developers near me their experiences and opinions. Here’s some of the things they raised. The discussion inspired one of the discussing participants to write a blog post about the subject.
Command-line tools or commmand-line interface are pretty unknown territory for every day computer users. But for some developers it’s the every day habitat. Every developer must know something of the commandline to survive the basics of development. Ordinary computer users are accustomed to use Graphical User Interface driven applications. At least some of the GUI apps have command-line interface too. For example you can launch Firefox browser:
$ /usr/bin/firefox --new-window https://100Daysdx.com/. Most of the ordinary users just double-click the Firefox icon in menu.
0. Less clunky menus
GUIs change and that pushes developers towards CLI as a developer friend says in Twitter “No need to constantly learn new guis with clunky menus and views”. Furthermore it seems that CLI tools change less frequently than GUI based tools.
1. Learning and showing
This is for what I use CLI most. As Developer eXperience Lead I do some demos and show some new customers how certain things are done in the Platform of Trust. In those cases I use CLI. So far I’ve managed without any specific “SDK-like” tool and used only ready-made scripts as examples. Some of the scripts use cURL, which is another command-line tool. Some are miniature Python apps which just make some API calls against Platform of Trust. The examples are collected in one “DX tests” folder. That’s it. That’s more MVP tool version. If you like cURL, you should also checkout HTTPie, which consists of a single http command designed for painless debugging and interaction with HTTP servers, RESTful APIs, and web services.
Another example is a getting started guide which contains code examples. I was developing a getting started guide for “How to create a simple digital twin with APIs?”. I collected the needed API calls in scripts and I can use them to show and tell some developers in event if needed. I actually did “live coding” once with the scripts only during spring 2019.
My method to onboard new API pretty much always contains some API testing with cURL fom command-line. That is more like experimenting, exploring and getting familiar with the API and how it is used. After I’m done with the fiddling, I take the lessons-learned to code (node/python) app. Lately I have started to use more Insomnia and Postman for the learning and less with the CLI.
2. Sharing related usage information in various environments
With GUI tools to share “how to” things with others, you need to take screenshots or record videos. That is quite painful and cumbersome compared to sharing a one-liner as text.
3. CLI tools are a must for CI
Command-line tools are vital in API development, while those enable automated continuous integration (CI) processes. Of course you’ll have the CI environment, but with CLI tools you flesh the CI skeleton. Without the tools, you would not be able to automate easily for example testing or validation.
CLI tools are valued highly in API development. Postman collections were discussed earlier in documentation related post. But here the topic is expanded to CLI tool “newman” (part of Postman offering) which can be used to run Postman collections as API tests as part of the CI process.
Even after the CI process, you most likely use CLI tools also in continuous delivery (CD). The usage might not be direct usage though, but hidden inside a service you use.
4. Do things at scale
A simple CLI command / a few can easily adjust configurations for a large group of systems at once. For example with simple command
sed 's/.$//' you can convert DOS/Windows newlines (CRLF) to Unix newlines (LF). Or substitute masses of words with another one
sed 's/foo/bar/'. You can even reverse a line
sed '/\n/!G;s/\(.\)\(.*\n\)/&\2\1/;//D;s/.//' (emulates “rev” Unix command). The last one is just horrible, but it does the job. These and other mass editing tools are handy if you need to change something in multiple places (files) and manually doing it would be just boring and really time consuming + error prone. Well, you can easy fuck things up with sed and awk as well…trust me, I know.
5. Something needs to be scripted and automated.
With help of CLI tools you launch full scale environments with multiple services with ease. The setup often contains plethora of CLI tools. That might sound complex and occasionally it is. One of the benefits though is that you can easily switch or tweak small step at a time without doing code level changes. Finetuning and altering the setup is mostly easy (if you know what you are doing).
6. Create awesome tools (and share the result)
Every developer creates custom tools while developing applications. Some of those tools are simple scripts, but some evolve to more “productised” and versatile solutions. The latter have huge value for other developers. Thus it’s no wonder that Github is full of different kind of tools for API development, testing, code generation, validation etc. Some of the tools combine punch of smaller tools into package which is used with the same root command. In other words tools development generates more tools.
In the above image referenced API toolkit is available in Github: Command line tools for openapi-enabled APIs
7. Use NPM or pip for Package Installs
Node Package Manager is easily the most popular tool for modern developers and it does not have a GUI. Python packages are also installed with CLI (pypi). Thus some level of CLI skills is necessary to perform as a developer in most cases.
8. Piping multiple commands
Piping multiple commands is a neat feature in CLI. You can put multiple tools in a chain of events in which one gets the input from previous one. Eventually you’ll have one-liner using plethora of CLI tools and neat result. As a simple example formatting JSON response from API can be achieved with
curl https://api-sandbox.oftrust.net/products/v1/ | python -m json.tool.
Or use jq prettyprint and formatting. Here’s an example to pick first item from the list
curl https://api-sandbox.oftrust.net/products/v1/ | jq '.ItemList'. jq is a lightweight and flexible command-line JSON processor. jq is like sed for JSON data - you can use it to slice and filter and map and transform structured data with the same ease that sed (discussed above), awk, grep and friends let you play with text.
9. Lint and mock API
With CLI tools you can esily lint and mock your API. Lint, or a linter, is a tool that analyzes source code to flag programming errors, bugs, stylistic errors, and suspicious constructs. In this case you don’t want to “lint” your source code, but machine-readable API spec files which acts as designs for your API. This refers to design-first approach. Of course linting your generated spec file is an option too.
I ran one of the linters available (stoplight.io/spectral/) against Platform of Trust Product API and found 0 errors, which is good considering that the OpenAPI spec file is converted from RAML.
spectral lint https://docs.oftrust.net/specs/oas/product-api.json
Mocking with REST APIs often refers to generating server code from machine-readable spec files before implementing the actual logic connections in the API. Mocking helps you to see how the API would behave and even let your customers get a sneak preview and provide feedback before putting lots of efforts in implementing it. Refactoring afterwards most likely will be more costly than finding the logical errors in design phase.
Multiple solutions exist for both but here’s a two examples from same provider (provided by Phil Sturgeon):
- Linting your API descriptions: stoplight.io/spectral/
- Mocking your API descriptions: stoplight.io/prism/
Do you still wonder why developers love CLI tools? The above few examples should give you an idea that CLI tools are the thing for developers. You should really consider providing such tools for your consumers (developers). The tools might not be directly related to your product but still attract developers near your product and consider using it too! CLI tools are B2D marketing elements!
How do I document my CLI tool?
Rule number 1 is that some first aid must be with the app. Detailed explanations and all can be in internet. The old school man pages is an option, but you can also just simply add
--options parameters to your app and it will print out a cheat sheet for the user. That can be considered as the medikit. Expanding documentation from there is almost always a good idea.
You could try the old school man pages. Second option is to go for bropages, which is maintained by the community. The slogan of this utility is just get to the point. The bropages are just like man pages, but it will display examples only. As its slogan says, It skips all text part and gives you only the concise examples for command line programs.
Third option Cheat is another useful alternative to man pages to learn Unix commands. It allows you to create and view interactive cheatsheets on the command-line.
Fourth option is plain website (perhaps gh-pages served under own domain) just like API docs. One of the good examples is Terraform CLI tool. Terraform “enables you to safely and predictably create, change, and improve infrastructure. It is an open source tool that codifies APIs into declarative configuration files that can be shared amongst team members, treated as code, edited, reviewed, and versioned.”
Distribution and usability
Ok, now you have CLI tool and it has some help pages. How to distribute it? Rule number 1 is that make it easy. Rule number 2 is that use the methods and locations which are already used by developers.
The first option is to put the code in Github and spread the word. Developers anyway seek tools from Github and google finds them as well occasionally. Second option requires more effort. Use package managers like NPM, pip, gem depending of your programming language. Installing new tool is just a simple one-liner.
npm install [package name]
pip install [package name]
gem install [package name]
That is easy and fast.
Lastly, make sure the tab autocomplete works. It really makes life easier and raises the developer experience with the tool.
Not black and white
As with a lot of other practices and opinions this is not black and white either. It’s not GUI or CLI. Quite a lot of the developers are mixed users depending of the tools. For example Git is something that I like to use both from CLI and desktop app. For the the branch management feels better to handle with GUI. Git(hub) is a service which I use from three interfaces: CLI, App and Webapp. Same goes for some developers for example with cURL vs Postman when fiddling with APIs.
I like the Postman too, but it starts to feel a bit too bloated and thus I use Insomnia instead (from GUI).
Some more to read from 100 Days DX
- #100 - The biggest open resource on Developer eXperience so far
- #99 - Hackers - the top 1% of engineers
- #98 - Invalid Open API spec files ruins the developer experience
- #97 - Lightweight API evaluation framework
- #96 - Embracing open source community driven tools development
- #95 - Exploring the multilevel nature of API Design Guides
While you are here...
Check out Platform of Trust in which I've worked to enable best possible developer experience!