Roadie Catalog
Published on November 15th, 2022Overview
Roadie Catalog is designed to model all of your software assets, artefacts, teams, users, etc - everything and anything that contributes to software development.
It is based on, and API compatible with, the Backstage Software Catalog .
Catalog Entities
Each item in the catalog is referred to as an entity. Entities can be ingested from external sources or can be defined using YAML files (typically catalog-info.yaml) stored alongside your code.
Entity Kinds
Roadie supports several kinds of entities, organised into three categories:
Business Information - Organisational context:
| Kind | Description |
|---|---|
| Domain | A collection of entities and Systems that form a distinct part of your organisation |
| System | A collection of different entities that have a set function |
| Product | A collection of entities that constitute a product |
Software Artefacts - Individual elements of your software ecosystem:
| Kind | Description |
|---|---|
| Component | A piece of software, e.g. a website, service, or data pipeline |
| API | Defines the boundaries between Components |
| Resource | Infrastructure required by Components (databases, queues, etc.) |
| Repository | Source code representations, often created automatically from SCM tools |
| Template | Scaffolder templates for creating new software |
Ownership - People and teams:
| Kind | Description |
|---|---|
| User | An individual person |
| Group | A team, interest group, or organisational unit |
For detailed information and guidance, see Modeling Entities.
Relationships between Entities
Entities can be linked together using relationships to model dependencies, ownership, and organisational structure. For example:
- A Component can depend on other Components or Resources
- A Component can provide or consume APIs
- An API can be part of a System
- A System can belong to a Domain
Relationships can be visualised using the Catalog Graph plugin and only need to be defined on one side to appear on both entities.
For a complete list of available relationships, see Showing Dependencies.
Data Model
Entities in the catalog follow a consistent structure:
apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
name: my-service
description: A sample service
annotations:
github.com/project-slug: my-org/my-service
tags:
- python
- backend
spec:
type: service
lifecycle: production
owner: group:default/platform-team
system: core-platform
Key concepts:
- kind - The type of entity (Component, API, System, etc.)
- metadata - Common fields like name, description, annotations, tags, and links
- spec - Kind-specific fields including type, lifecycle, owner, and relationships
Syncing with Sources of Truth
Automatically creating entities in your Catalog provides a simple and effective way to create a useful and permanently in-sync Catalog easily.
There are a number of ways to do this;
- Auto-ingestion: Roadie can automatically discover and ingest entities from your existing systems via auto-ingestion.
- Auto-discovery: Roadie can also discover valid catalog-info files in your systems via auto-discovery.
- API-based ingestion: the Roadie API can be used for both push-based creation of entities and idempotent set-based creation of entities from a given source.
Auto-ingestion
Entities can be created in your Catalog automatically, without the need for any YAML files.
Available auto-ingestion sources include:
Auto-discovery
Configure patterns to automatically find and import catalog-info.yaml files from:
- GitHub - Discover entities across repositories and organisations
- GitLab - Import from GitLab projects and groups
- Bitbucket - Scan workspaces for catalog files
- Azure DevOps - Discover from Azure repos
- AWS S3 - Import from S3 buckets
API-Based Ingestion
Use the Roadie Entity API to programmatically create and manage entities from any source.
Syncing the Catalog with downstream services
Once created, the Catalog provides a useful reference point for other systems that require a catalogued version of your software development lifecycle.
Roadie provides the means to sync the Catalog with those systems, namely:
- The Roadie Entity API
- Plugins, like the DX or Incident.io plugin, which exfiltrate Catalog data from Roadie to be used in those systems.
Enriching Entities
Any metadata can be added to entities without updating source YAML files or modifying auto-ingestion sources using Decorators. This allows you to do things like:
- Add links to external tools and dashboards
- Apply annotations for plugin configuration
- Enrich entities with additional metadata
Further Reading
| Topic | Description |
|---|---|
| Adding to your catalog | Get started adding your first entities |
| Modeling Entities | Detailed guide to entity kinds, types, and fields |
| Showing Dependencies | Configure relationships between entities |
| Managing Locations | Set up autodiscovery and manage import sources |
| Decorating Components | Add metadata without editing source files |
| Troubleshooting | Debug common catalog issues |