Roadie Catalog

Published on November 15th, 2022

Overview

Roadie Catalog is designed to model all of your software assets, artefacts, teams, users, etc - everything and anything that contributes to software development.

It is based on, and API compatible with, the Backstage Software Catalog .

Catalog Entities

Each item in the catalog is referred to as an entity. Entities can be ingested from external sources or can be defined using YAML files (typically catalog-info.yaml) stored alongside your code.

Entity Kinds

Roadie supports several kinds of entities, organised into three categories:

Business Information - Organisational context:

KindDescription
DomainA collection of entities and Systems that form a distinct part of your organisation
SystemA collection of different entities that have a set function
ProductA collection of entities that constitute a product

Software Artefacts - Individual elements of your software ecosystem:

KindDescription
ComponentA piece of software, e.g. a website, service, or data pipeline
APIDefines the boundaries between Components
ResourceInfrastructure required by Components (databases, queues, etc.)
RepositorySource code representations, often created automatically from SCM tools
TemplateScaffolder templates for creating new software

Ownership - People and teams:

KindDescription
UserAn individual person
GroupA team, interest group, or organisational unit

For detailed information and guidance, see Modeling Entities.

Relationships between Entities

Entities can be linked together using relationships to model dependencies, ownership, and organisational structure. For example:

  • A Component can depend on other Components or Resources
  • A Component can provide or consume APIs
  • An API can be part of a System
  • A System can belong to a Domain

Relationships can be visualised using the Catalog Graph plugin and only need to be defined on one side to appear on both entities.

For a complete list of available relationships, see Showing Dependencies.

Data Model

Entities in the catalog follow a consistent structure:

yaml
apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
  name: my-service
  description: A sample service
  annotations:
    github.com/project-slug: my-org/my-service
  tags:
    - python
    - backend
spec:
  type: service
  lifecycle: production
  owner: group:default/platform-team
  system: core-platform

Key concepts:

  • kind - The type of entity (Component, API, System, etc.)
  • metadata - Common fields like name, description, annotations, tags, and links
  • spec - Kind-specific fields including type, lifecycle, owner, and relationships

Syncing with Sources of Truth

Automatically creating entities in your Catalog provides a simple and effective way to create a useful and permanently in-sync Catalog easily.

There are a number of ways to do this;

  • Auto-ingestion: Roadie can automatically discover and ingest entities from your existing systems via auto-ingestion.
  • Auto-discovery: Roadie can also discover valid catalog-info files in your systems via auto-discovery.
  • API-based ingestion: the Roadie API can be used for both push-based creation of entities and idempotent set-based creation of entities from a given source.

Auto-ingestion

Entities can be created in your Catalog automatically, without the need for any YAML files.

Available auto-ingestion sources include:

Auto-discovery

Configure patterns to automatically find and import catalog-info.yaml files from:

  • GitHub - Discover entities across repositories and organisations
  • GitLab - Import from GitLab projects and groups
  • Bitbucket - Scan workspaces for catalog files
  • Azure DevOps - Discover from Azure repos
  • AWS S3 - Import from S3 buckets

API-Based Ingestion

Use the Roadie Entity API to programmatically create and manage entities from any source.

Syncing the Catalog with downstream services

Once created, the Catalog provides a useful reference point for other systems that require a catalogued version of your software development lifecycle.

Roadie provides the means to sync the Catalog with those systems, namely:

Enriching Entities

Any metadata can be added to entities without updating source YAML files or modifying auto-ingestion sources using Decorators. This allows you to do things like:

  • Add links to external tools and dashboards
  • Apply annotations for plugin configuration
  • Enrich entities with additional metadata

Further Reading

TopicDescription
Adding to your catalogGet started adding your first entities
Modeling EntitiesDetailed guide to entity kinds, types, and fields
Showing DependenciesConfigure relationships between entities
Managing LocationsSet up autodiscovery and manage import sources
Decorating ComponentsAdd metadata without editing source files
TroubleshootingDebug common catalog issues