Beyond the interface: hidden challenges of working with APIs as a product manager


I love working with APIs, and in the course of the years I have come to work with them in different ways, be it by consuming those provided by others or using them to service others, be it providing data or carrying out a particular functionality. For a product manager, the core work around APIs is, in my view, not distinct from standard PM work. Nevertheless, there are some unique challenges, but which when mitigated, are more than worth the trouble. Somewhat ironically, these start with adoption, but then tend to get trickier the more they are used.

(I also have the issue with the term ‘API Product Manager’, as it seems as meaningful as calling one’s self a ‘UI Product Manager’, but I admit this is a pedantic point, and the title is really a way of emphasising one’s typical focus.)

What are APIs, and why they are great

diagram

I will assume that the reader of this post will be aware of what an API is, but if one is looking for a definition, then the team at Postman has produced this great explainer. However for anyone who doesn’t know, very loosely speaking, APIs are what allow one software system to communicate to another, which in turn means that through them you can share information and/or trigger specific actions.

From this basic principle is also where their greatness lies. Through them you essentially expand the limits of your product, as it starts interacting with other services and programmes, which in turn magnifies where and how it produces value. In contrast to interacting with a UI, it is easier to automate or schedule an API call, thus reducing or completely removing the manual labour required to extract the information. This means that you are better entrenching your product, making churning a costlier proposition for the customer, but also it is a way of empowering your users to do things with your product that you might never have thought of. Finally, with the right API endpoints provided to them, your customers can sometimes build a surprising amount, which means you don’t need to build it for them.

Some of the challenges

Adoption of API endpoints takes time

I remember the excitement I felt when we released a particular API endpoint, which I was convinced would provide significant value to our customer base. This was a common feature request that came up regularly in customer interviews and support tickets, and it was aligned with one of our key value propositions. And yet it took weeks following the release for that endpoint to start receiving regular calls. Gradually it became more and more used, and the numbers were still growing after I had left the company. Though the value proposition was eventually validated, I had underestimated how slow the adoption rate would be. This shouldn’t have been the case when one considers several factors around APIs.

First of all, new endpoints are considerably less discoverable than new functionality you build within an app, and you cannot flag it in your UI while someone is using it. People also do not read your release notes in as much detail as you would like. Thus the promotion of your new releases needs as much, if not more, promotion as ‘ordinary’ features. This includes directly reaching out to customers who have either expressed an interest for such functionality in the past, or whom you otherwise would think is going to be a good fit. You don’t have to do this evangelising alone, of course, and your marketing, sales, and customer success teams are all key allies in this. On that note, make sure they know how this endpoint works, and whom it should benefit. (Ideally they would know this already, and possibly provided input during feature development).

Communication is only the first part. It will take time to build around the API, and the development work will also need to be scheduled together with other work. The questions then are on how to shorten the time to adoption. The actual design of the API endpoints themselves will be key here, ensuring that they are clear and that they address the core problem of the users. The nature of the problem will determine what this design will look like, but this means that, like for any product, you will need a good understanding of the use case of your users.

Additionally, your documentation will now be more important than ever. Clear documentation does not make integration easier just for people, but also LLMs will be better to guide developers on how to call and interact with your API, or even potentially for it to build it itself. If the documentation is in a RAG-friendly format (e.g. in a JSON file), this becomes even easier.

Updates are tricky

Changing an API can lead to a break in people’s workflow in a way that UI change will typically not. Hence the API’s endpoints and behaviour should be changed sparingly, and often the rollout will have to be communicated to your customers, and be more gradually in nature. Not all changes are breaking ones, but some do require more major changes. One typical way of managing this is to build a new version of your API, and gradually divert traffic to it, letting your user base know of any changes that they will need to implement on their end. You will need to maintain both sets of endpoints until you finally deprecate the first version.

Stripe has come up with an elegant solution for updating their API without fear of breaking integrations. Every user who starts calling the API remains ‘stuck’ on that version of the API, and as new versions get released, the user keeps using the original version. The user can decide to change the version in their own time. This, of course, does pose the challenge of having to maintain many versions of the same API, but in the field of payments where reliability is paramount, this seems like a wise trade-off.

Hard to know how your product is being used

One of the points of your API is that others can integrate with it and use it for their own needs, outside of your ecosystem. The downside of this is that you do not get to easily deduce how it is being used, beyond knowing who is calling your endpoints, which ones, and how often. If you want to know how these calls are being leveraged, then there are not many other options beyond talking to your users. This of course is something all PMs should be trying to do regardless, but I would argue that what sets APIs apart is their flexibility and the number of potential use cases they can enable. You are not going to be able to anticipate many of them, and they may be incorporated into workflows with tools you are unfamiliar with. Your metrics will tell you very little, so go and supplement them.

Victim of your own success?

You have to factor in that a particularly valuable API endpoint is going to receive lots of calls, which could easily go into the millions a day. Say you own a service that is part of an e-commerce payment flow and Black Friday comes along. Your API is going to have a spike in calls, which may overwhelm the system. Not only that, but the users of your API are not going to be humans but other machines, who can schedule calls on an ongoing basis, and can anyway make more requests than a human possibly could. An API that is unreliable is not going to be used, which means you have to ensure that its performance scales well, which in turn means you have to invest in the underlying infrastructure, which you can ramp up as well as down, so that you can control your infrastructure bills and only pay for more when you need it.

An example of APIs in the news

At the time of writing there has been controversy when JPMorgan Chase stated that it would start charging FinTechs for access to customer data via their API, with many arguing that the pricing would be a serious blow to Open Banking. Setting aside whether the price increase is justified, it is a demonstration of how crucial the right API from a particular source can be, and how changes to it can shake an industry.

Parting thoughts

The fact that APIs are mainly used by other systems is what allows them to have an outsized influence to other product world, allowing your system to interact with others, and become part of a whole that is greater than the sum of its parts. The different challenges covered here are consequences of being part of said whole, and the costs of coordination and interdependencies that follow. The impact of the work will take longer to be felt as you need to manage slower adoption of your system, not breaking everyone’s workflows when you introduce changes, having to be proactive in knowing how it is being used, and then being able to scale if it turns out you have indeed built something of value. This comes with the territory of building infrastructure that others come to rely on. It can be hard but when it works, you are enabling people to solve problems in ways you couldn’t have imagined, and having a cumulative impact that is much greater than you would have had on your own. And this, I think, makes the challenges more than worth it.



Posted

in

by

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *