API Security is More Than Testing

API Security doesn’t start with penetration testing. This isn’t to say testing isn’t important – it is! And at multiple phases. It isn’t always feasible to cover every single security scenario, but the most important ones should be considered. This is why it’s important for the business to understand what can go wrong, when, and how to get ahead of these issues as early as possible. The following advice is based on my years of testing and monitoring for issues as a security engineer, and implementing APIs as a developer. Use this as guidance, some might be applicable to your context and others might not be. There is a lot of information out there so I haven’t focused on technologies and solutions. This list is not exhaustive (give me a few more blogs for that…), but it should help should help you highlight the problem space and put you on a path to choosing the best mitigations.

0. Educate

The most important and earliest intervention for security is to educate. Ensure different expertise has relevant training and understands their role when it comes to security, and ultimately the quality of your application. Education is ongoing and can be applied to any of the below points.

1. Invest in Application Security early

When thinking about the risks we’re willing to accept, making a decision late in the development phase can be costly. Penetration testing will uncover the security that wasn’t thought about in the design – it’s like finding a brittle foundation after you’ve already built the house. The best case scenario is you patch the issue, but you will continue to be reactive. The most costly (and thorough) case is that you rebuild and ensure that foundation is strong, easy to build upon when designs change later and will grow with you.

Decide what data is important

Data is incredibly valuable, and that’s what makes it a liability too. Consider the data you are collecting and storing. Do customers require access to their data from any device or can the information be kept local? Does your business rely on analytics and can you reasonably anonymise this data for use in AI training models? Understanding the important data can help you put the right mitigations and segregations in place.

Involve Security in architecture and design

Understanding who needs access to your data can influence how a system is designed and implemented. The earlier security can highlight potential risks and threats, the easier it will be to mitigate them. One activity could be to threat model, but ensure you involve the right people. If you choose to use a third party to analyse and store your data, due diligence is best done before you engage in a contract with them. Ask for a Proof of Concept period, perform a robust and repeatable third party vendor security assessment and ensure they meet your regulation needs.

API Security at the application level

Cloud services have introduced convenient ways to secure which traffic can access certain information, but relying on network layer controls to authenticate and authorize who can access your data can leave you open to bypassed controls. In some scenarios, this may be the only option. Ensuring development considers and implements this level of authentication and authorization for each endpoint gives fine-grained control and ensures the business logic is kept close to the application. This defines clear security testing scenarios, and also means that code review has a better chance of highlighting implementation issues, while static analysis and tests can validate the behaviour you expect.

Secrets Management

How secrets are stored and accessed during development could create data leaks in production. I have found countless passwords in code, which have then been exposed in the application and network layers – or committed to public code repositories. Anybody who finds these could then attempt to access more information from the API, internal databases, endpoints, and more.

Authentication & Authorization

How consumers of your API will be authenticated and authorized is one of the most critical issues I find in implementations. There are various pros and cons with API keys, token-based authentication and using a centralized trust with claims and cryptographic signing. Each solution will increase complexity but will also increase security. This is why involving security early in the design phase could make a difference to the time you reach market and the cost it takes to get there.

2. Understand what your assets are (and who has access to them)

Assets can be defined as anything you think is valuable to your business. Generally, customer data and your Intellectual Property fall into this category. Over time, logic, applications and services accumulate. Mergers and acquisitions, increasing headcount quickly, and multi-tenancy can amplify the security issues this brings. Something I’ve seen in companies is that assets such as databases and API endpoints that are considered "legacy" (that is, a company thinks they’ve switched off an old endpoint and moved to the newer version) are still available. Some of the reasons this could happen:

• lack of standardisation in release management, way of working and continuous deployment can amplify the problem
• mergers and acquisitions haven’t or can’t be fully inventoried
• knowledge is lost when people leave the company
• no formal or business requirements that keep track of what and where infrastructure is deployed

Steps towards auditing your assets and endpoints:

Define what is important

Understand which digital assets are your most critical, and what would be a "game over" or calamity scenario for your company. Define the level of controls required for these assets. If you don’t know what these are, get multiple departments involved. Make sure engineering, data science, IT and more departments are involved in this audit – you might be surprised where some of your data is living.

Attack Surface Management

Invest in a tool or company that can keep track of where your digital assets live. Some of these companies will also probe for vulnerabilities and alert you if anything changes. Some may go the extra step and find assets that you don’t know about, but it can be difficult if something like a domain name and certificate are not registered as part of your company or within the internet protocol / CIDR ranges that you provide for them to scan.

Use controls to monitor for changes

Monitor the assets you care about, especially if something changes. A new deployment or change to controls could turn a private database or storage into something public – you want to know as soon as this happens. Different needs may suggest that this should be blocked from happening in the first place, but you can’t always be sure. Cloud companies provide multiple services that can also help with this.

API Developer Portals

Collating API documentation provides benefits in multiple ways. Knowledge is kept in one place, in case teams shift or people leave. Security testing is made easier, preventing precious time that is usually spent wading through code, multiple code repositories or undocumented endpoints. Only make public what is necessary.

3. Implement a secure by design and secure by default approach

More than buzz words (or phrases), the business plays a deeper role in defining what is "secure". Above, we outlined defining what is important. Now we must be clear about what to do about it and why. When an engineering team needs to make a choice about the next feature or piece of work they pick up, how and what informs this choice? Is it a deadline, customer satisfaction, or is it because "the security team" said so? Security team mandates and policies are not enough – instead, provide clear guidance and guardrails. For example, if you process medical data of millions of people, protecting that medical data is important. Without clear understanding of what the business finds critically important and why, and who should have access to that data, it can be difficult for teams to prioritise the right work. Why this might not be happening securely:

• Service and productivity teams are non-existent, overburdened or lack the skills to collaborate with teams
• A single developer may have too many responsibilities, from designing the software to understanding how to deploy and run it operationally
• The business needs teams to "go faster" because of the market demands, competition and arbitrary deadlines

Steps towards building up secure-first development:

Balance speed with Security and Quality

Once you know what is important to protect and how to protect it, get smarter about the controls in place. Automate what you can, but understand that manual intervention is needed too. Give enough time to do the right thinking upfront, design well, implement and test the robustness of an application. Simply pushing out features in a brittle application will mean constantly growing tech debt, which could bury long-standing security issues.

Paved Roads and Golden paths

Expert teams within developer experience, platform, and operations could help build in security controls so that development teams can do what they do best. When creating a new project, having to define the same security controls for that infrastructure is a cognitive load that can be reduced. Implementing secure guard rails could mean that any time a certain database or data store is deployed, it always has the most secure controls switched on by default, making it clearer to review, test and monitor. Exemptions can still be made with security involvement, depending on the context of the application deployed.

Avoid default options on installations

When implementing certain APIs or technologies, often they come with a default installation. Passwords should be changed, but also consider the default API and information pages that may also be deployed. Usually these are default paths that can be easily searched for, they give away more information about your system than is actually required.

4. Log, Monitor, Hunt, Learn

This post has focused on proactive measures that you can implement early, but inevitably there will be a time where you need to react to a security incident. It could be a critical patch, a breach of a system or a deployment oopsie. Knowing what to do in these situations and how to recover is important.

Monitoring your application traffic

Baseline what "normal" usage of your application is, and monitor for those outliers. The more information you can find out, such as geolocation, endpoints being accessed, amount of data consumed at any one time, will help you with various controls. You may find you need to limit the rate in which clients request data, important for cost and from slowing down those harvesting data. Simply monitoring isn’t enough – take this a step further and find which of these issues can be automated – such as throttling access dynamically based on anomalous behaviour.

Logging pitfalls

Logging and memory dumps could expose sensitive information, can be costly and may need further implementation guidance. This information is often needed to help troubleshoot a system, usually in development but it could also be in production. Do not expose this information to the end user unless it’s imperative for their own trouble shooting. There are countless times APIs have been overly helpful and I have been able to gather technology versions, implementation issues and more information about data I can access (that perhaps I shouldn’t have access to).

Post-incident no-blame reviews and learning culture

Once an incident happens, how quickly are you able to respond and recover? You may uncover gaps in your process, such as gathering the right people, information and responding as quickly as possible, or lack of a disaster recovery plan. Ensure you learn from these incidents by conducting a post-incident review, focusing on what went wrong and how it could be prevented in future. Removing the blame of "who" creates a more open, safe space to share, but more importantly, understand issues don’t happen in a silo. If someone was the cause of an incident, what process (or lack of controls) led to that?

Conclusion

This post took you on the journey of designing a new application and how it relates to API security, auditing what you already have, and considering the options available to secure your API(s) in various phases. Ultimately, we want to secure how consumers of your API access your data, reduce the misuse of your data, and mitigate security issues where we can. Acting and implementing solutions should be the last step – the first step is to understand the context and problem you are trying to solve. Many of the above points such as auditing your APIs, considering who and how they will be used, and involving different disciplines will help you explore this. The importance of education cannot be understated, as it will then help with choosing the best solutions. Education will also help with improving, evolving the application, and how you respond to security incidents. As a business, provide clear guidance, empower engineering to take ownership and give the time to make the best decisions possible.

This blog is part of our series "Holistic Horizons". Check out the previous entry – “I’m thinking about developer productivity” by Jochum Börger