Developer Portal
Overview
A developer portal is a centralized repository that stores and displays all the relevant information concerning the software a team is building. Having the data in one place allows developers at all levels to build a system that helps them keep track of the software ecosystem, get notifications on changes, and visualize related information.
Such a portal also makes it easy to keep an eye on various aspects of the software development process, such as the microservices and how they are connected, ownership status, dependencies, resources, and much more that we will touch on below.
An internal developer portal is generally composed of a few different tools. Each of them has a distinct function and is equally important in providing developers with a comprehensive picture of what they are building.
Service Catalog
At the center of the developer portal lies the software catalog. Its object is to give developers a clear single pane view into their systems at any given point. It includes everything from monoliths to microservices and a clear overview of what they do. It also includes information on ownership, environments, metrics, on-call, runbooks, as well as documentation. By offering an extensive overview, the service catalog helps developers and the organisation identify the baseline that they and it are at. Only with adequate data in this regard, i.e., by knowing what exists, can they reflect on what needs to change or improve and take the appropriate actions for optimization.
The service catalog should help engineering teams quickly answer the following questions:
- What’s the current running version in production for a given service?
- Who owns this microservice?
- How is it triggered?
- Does it expose API routes?
- Who is on-call?
- Is this version production-ready?
- DORA metrics for a given service or developer
Let’s look at a common case, which would require a unified view of (1) Service, (2) Environment, (3) Deployed Service and (4) Deployment. Taking these 4 elements, in this use case, will provide a clear understanding of each service’s maturity and readiness and a detailed view of every service’s lifecycle from the first commit to many deployments running across different environments. Let’s look at the definitions for this case.
Service
A service can be a microservice, software monolith or any other software architecture.
Environment
An environment is any production, staging (UAT), QA, DevEnv, on-demand or any other environment type.
Deployed Service
A deployed service is a representation of the current “live” version of a service running in a specific environment. It will include references to the service, environment and deployment, as well as real-time information such as status, uptime and any other relevant metadata.
Deployment
A deployment could be described as an object representing a CD job. It includes the version of the deployed service and a link to the job itself. Unlike other objects, the deployment is an immutable item in the software catalog. It is important to keep it immutable to ensure the catalog remains a consistent source of truth.
GitOps
GitOps lets developers get things done using code changes in git. Developers can deploy a microservice, provision cloud resources, manage environments, configurations and more. While this does offer a good developer experience, it doesn’t replace the need for a developer portal.
There are several reasons why:
- Files associated with GitOps are distributed across the codebase, making it difficult to determine what needs to be changed and creating a risk of significant outages.
- DevOps and developer properties exist in the same file, making it difficult for developers to tell what needs to change.
- A proliferation of tool requests which create many tickets.
- Cases where Git doesn’t reflect the state of the world, which may lead developers to make the wrong decisions.
- Lack of clarity when there are many repositories and configuration files.
Project Generation Tool
A developer portal usually also has a project generation tool. In Backstage this is called software templates, other third parties call this scaffolding. Under the hood these are usually run via Cookiecutter. This gives developers a unified way to spin up new services and infrastructure from within the portal iteself. This way, the developer portal operates the services from the moment of their creation. This is important because, to give developers a unified experience, it is best to start from the moment a project is conceived. When they can create new services and subsequently keep track of the same services in the same place via automatic discovery, you are maximizing the potential of a developer portal.
As a result, the developer portal is able to handle everything from getting information about on-call schedules and triggering incidents to deploying things and dealing with incident management. Following this, all the aspects of actual production use of a particular service belong in the developer portal as well.
Benefits
A unified experience
There are multiple reasons to set up a developer portal. The first is to give our developers a unified experience for all the different tasks that are part of the development lifecycle. This includes foundational activities such as spinning up new services and operating those services in real-time. Additionally, a developer portal is also designed to go one step further and incorporate tasks like API management, documenting services, running actions on them, deploying pieces of software, and rolling them back if needed.
By providing a comprehensive end-to-end experience, you make it easier for everyone to be on the same page and have eyes on the entire lifecycle. Built on the concept of self-service, a development portal also speeds up service creation and deployment processes, among others. Without a development portal, not only do you run the risk of losing things but it also becomes difficult for developers to communicate and collaborate effectively.
Drive ownership and accountability
Knowing who owns resources and services gives developers the clarity they need. We already have the issue with developers asking the wrong person around details on services.
Clarity and onboarding
Software development is a complex process, especially when it involves architecting a variety of microservices and assembling them to build an application. With the assortment of tools put to use to develop software today, the development lifecycle is more or less scattered across these tools. As a result, the developer experience is fragmented and discordant.
This is made more complicated by the fact that things are constantly changing in the development environment.
It becomes difficult to onboard and explain to a new team member how work is done within the team, whether they recently joined the company or have switched over from another team.
In this context, a developer portal pulls everything into a single place to give developers a single, unified, high-quality software development experience. They can continue to use the tools they prefer and follow certain best practices without having to worry about integrations or difficulties in communication with fellow team members or engineering leadership.
Documentation
A developer portal simplifies documentation practices as well as the process of looking up any docs. Its centralized nature ensures that developers need not embark on a wild goose chase every time they need to look up something specific. Clear and easy-to-find documentation also does away with the potential for discrepancies in the information. When we started our techdocs project this was an attempt to unify documentation. With this being baked into a developer portal we would bring the knowledge closer to the domain or the service.
Establishing best practices and track service maturity
The developer portal is also instrumental in establishing good development practices.
Consistency is key to simplifying and streamlining processes in teams. With the inclusion of certain best practices in the team member’s workflows, it is easier for developers to connect various microservices as well as collaborate with each other. One of the core ideas behind setting up a developer portal is to help you prioritize efficiency and bring standardization across the entire organization in addition to increasing visibility and improving velocity.
We can also track service maturity the way we see it: a life cycle flow of production readiness, quality, security, compliance and DORA metrics. You can then immediately score all services and resources, by teams, developers and more, with immediate knowledge about which services or resources are up to par and which aren’t. The more we scale the harder it will become to track both best practises and service maturity.
DORA metrics
DORA metrics have become the standard for gauging the efficacy of software development teams, and can provide crucial insights into areas for growth. These metrics are essential for organizations looking to modernize. The four metrics used are deployment frequency (DF), lead time for changes (MLT), mean time to recovery (MTTR), and change failure rate (CFR).
To Backstage or not to Backstage
A lot of the momentum behind the interest in platform engineering and internal developer portals comes from Spotify’s backstage io project. Initially, backstage was a project designed to bring order to engineering infrastructure through the use of an internal developer portal that provides access to devops tools, data, microservices, APIs and more. In 2020 Backstage was released under an open source license, and in 2022 it was accepted as an incubating project into the Cloud Native Computing foundation.
At the end of 2022, backstage.io announced that it will be offering five premium paid plugins, one for role-based access control to the internal developer platform, and others for user insights (analytics on backstage use) as well as “soundcheck” for KPI scorecards on developer portal entities and more.
After opting in to the idea of an internal developer portal, one of the first decisions is, as a result, whether to backstage or not to backstage. Below are a list of paid for plugins which are generally pieces of functionality that come as standard with hosted version.
My recommendation would be to go for a hosted developer portal rather than Backstage. All hosted versions remove the TCO or total cost of ownership. TCO is an estimation of the expenses associated with purchasing, deploying, using and retiring a product or piece of equipment. We have to think about the cost of needing UI / UX resources, as well as technical resources keeping the platform stood up and up to date. It also lacks serveral features which would be instrumental to us such as scorecards or DORA metrics.
Paid for Plugins
RBAC (Role-Based Access Control)
The RBAC plugin is a no-code management UI for restricting access to plugins, routes, and data within Backstage. Admins can quickly define roles, assign users and groups, and configure permissions to encode authorization decisions according to your organization’s evolving security and compliance needs.
Soundcheck
The Soundcheck plugin visualizes checks for security, testing, reliability, and other development and operational standards for your software components. Instead of asking your developers to file and respond to yet another ticket, streamline software quality through Soundcheck’s clear scorecards, actionable feedback, and positive reinforcement.
Insights
The Insights plugin is your window into Backstage usage at your organization. You can measure Backstage adoption, identify anomalous trends, and understand user behavior.