Developer Portal Business Continuity Planning: Case Study Findings
Introduction
Our developer portal provides seamless interaction between developers and our departmental development standards. As disruptions become more prevalent and are often unexpected, ensuring the resilience of our developer portal is important, not just for our users but for our reputation in the developer community. This exploration aims not just to uncover best practices but also to contribute to the wider, ongoing business continuity planning (BCP) conversation, enhancing our understanding of how we can thrive amidst evolving disruptions.
Overview
We brainstormed possible scenarios that could trigger our BCP and how those scenarios might negatively affect the Developer Portal (DP). As a team, we determined what we would need to do to keep the DP afloat and who we might need to keep informed.
Scope and Objectives
This case study explores possible disruption scenarios, the common effects of those disruptions, and proposes robust response strategies. Our goal is to fortify our capacity to navigate disruptions so we can maintain our pivotal role in the department’s IT ecosystem. Given the DP is a site with no physical dependencies, the scope of our BCP investigation focused on the cloud-based services and infrastructure that supports the DP.
Critical Functions and Dependencies
Refer to the Key Vital Areas tab on the matrix (see appendices).
- Azure Static Web App: The site comprises both Jekyll and Blazor Mockaco.
- Azure Web API: Provides support for a range of client-side interactions such as generating colours for tags and interacting with the GitHub repo via Octokit API.
- Azure Active Directory B2C: Supports identity and access management for login functionality and role-based access.
- Azure Storage: Hosts graphic snippets for some posts and custom AAD B2C page layouts.
- Azure Cosmos DB: Supports API subscription/key management and provides support for semi-structured data storage.
- qed-developer-portal GitHub repository: Hosts the source for the Jekyll portion of the site (the blog engine).
- mockportal GitHub repository: Hosts the source for the Mockaco portion of the site (API management).
- qed-dp-content GitHub repository: Hosts the content from users in the form of posts.
- Toast UI Editor: Used as the input editor for posts, feedback, and help pages.
- Octokit API: Used to interact with GitHub repositories.
Response and Recovery Strategies
Our response and recovery strategies include regenerating API keys, client secrets, and passwords; rollbacks; moderating user input; reverting to defaults; and escalating issues to Microsoft. Further details and specifics are available in column E on the first two tabs of the BCP spreadsheet.
Communication Protocols
For problems directly affecting the site or user access, we will apply a notice to the site which alerts our users that they may notice some delays or reduced functionality while we respond to the problem. For problems directly relating to security or privacy, we will communicate with the CyberSecurity team and/or Infrastructure team (and affected users as required). For problems directly relating to our corporate subscriptions or accounts with GitHub and/or Microsoft, we will communicate with the Director of Application Development and Delivery and any contacts/representatives from those companies with whom we have a relationship.
Findings
Given the DP is a cloud-based solution, hosted using Azure SWA with GitHub CI/CD pipelines, we do not rely on physical infrastructure or DoE network/resources. These cloud-based tools offer some continuity protections by default:
- GitHub pipelines ensure that failed builds won’t impact the live site.
- AAD B2C authentication can easily be reverted to use default settings to keep functionality running.
- GitHub repo access issues won’t stop our users from accessing the site in read-only mode.
- Removing third-party tools like the Toast UI editor or Octokit API doesn’t throw obvious error pages to our users.
- The DP would only be entirely unavailable if there is no internet access or if a cybersecurity incident requires us to take the site offline temporarily.
We also found a handful of issues that need attention, that will be managed via new user stories in future sprints, including:
- Only one person currently has access to the generic developerportal@ and developer@ email accounts so we need to extend access to the rest of the team.
- We need to sanitise input fields to help prevent injection attacks.
- We need to add B2C layouts to the GitHub repository for ease of retrieval.
- We need to store the password for the ‘DP admin’ user in Azure Key Vault.
- We need to look into implementing a DDoS protection plan and anomaly detection mechanisms and set up DDoS protection metric alerts through Azure Portal.
- We need to look into how to secure our Azure storage container/content.
- We need to add a config item to use the ‘default’ API key instead of one from Cosmos DB, as a fallback.
- We need to add placeholder text to alert users when the Toast UI editor fails to load.
In summary, our cloud-based setup with Azure SWA and GitHub CI/CD not only enhances efficiency but also ensures continuity, demonstrated through resilient deployment processes and uninterrupted user access even during adjustments or disruptions.
Continuous Improvement
The matrix and workflow (see appendices) will be living documents and we will review and maintain them regularly. Actual frequency to be advised at a departmental level.
Appendices
BCP Findings Matrix:
DP_BCP_Notes.xlsx