Transitioning from Self-Hosted Backstage to Roadie

Paddle helps software-as-a-service (SaaS) companies grow globally by taking care of payments, checkout, subscription management, and licensing.

For the last 4+ years Paddle self-hosted Backstage and used it as their Internal Developer Portal (IDP), before moving to Roadie in early 2024.

I met with Ioannis Georgoulas, Director of SRE, to discuss self-hosting Backstage and why the time was right for Paddle to move over to Roadie.

Ioannis has been at Paddle for 5 years and has seen the company grow through Series B to the current Series D round. He has also been part of developing a centralised Platform team, consisting of SRE and Application Excellence teams dedicated to solving problems and improving the efficiency of product engineering teams.

Self-hosting Backstage

Early journey

The Paddle team were early adopters of Backstage in 2020. It initially started as a hackathon project but quickly became part of the workflow for engineering teams.

After initial experimentation with (the then recently open-sourced) Backstage, the team built out their own backend in Go to speed up internal development and pushed hard to get a complete and accurate Catalog.

The dream of Backstage though, soon gave way to reality.

Initially the focus was getting a complete service catalog, but once that goal was achieved we realised it wasn’t driving change. It was solving problems for sure, but smaller ones.

TechDocs was a similar story: useful in a small way and good at the point in time when we brought it in, but after we moved to Notion, which became our knowledge base, TechDocs declined in usefulness.

Gaining traction with custom plugins

Undeterred, Paddle team moved on to developing custom plugins for testing checkout workflows and providing deployment information of components. That’s when they started to see real adoption and improved efficiency. These plugins pushed Backstage from a limited tool to something that could move the needle for Paddle.

This was a big success, but keeping Backstage up to date while developing custom plugins meant that the team would have effectively spent twice as much time and effort to get the value out of their custom plugins.

Backstage in theory is good, but in practice it’s challenging. You need to have people following up, implementing, making sure it’s up to date.

The additional workload was compounded by the Go backend which aided backend plugin development time and maintenance but dramatically slowed down the rate at which new open source plugins and upgrades could be adopted. This was work that Paddle was willing to prioritise, but it soon meant that effort was being expended on the tool than supporting the use-cases that engineering teams were interested in.

Enter Roadie…

Jump forward to 2024, and the Paddle team were interested in more than just keep their existing installation of Backstage up to date — they wanted to work on improving their custom plugins and start to change engineering behaviour. To do that they needed to know more about the current health of their software, not just document it in the Catalog.

We wanted to have more metrics - it was hard to understand how people were using our tools and our main driver was to improve that. We needed a good tool to do that job.

We went to a conference in Dublin and saw companies with scorecarding products and were like ‘this is cool’ - but we’re getting value from Backstage, custom plugins are important for us.

This triggered a move away from self-hosting Backstage though. Ioannis spent time with his team to come up with a checklist of key features that they were looking for in a new provider and functionality that they didn’t want to lose from their existing Backstage implementation.

A generalised version of this checklist can be found here.

Then they found Roadie.

After checking out our website and marketing materials they set up a demo so they could see how things worked and quickly moved into a one-month Proof of Value (PoV) trial.

We didn’t have a specific timelines in mind, but wanted to land it as quick as possible - we were excited after the demo.

So how did it go?

For Ioannis and the team, it was a pleasant surprise when they started the PoV. After self-hosting for many years, the improvements that Roadie has built on top of the open source Backstage shone through. Ease of use is something we prioritise and have that comes through when someone has moved from their own instance.

As Ioannis puts it, with Roadie “out of the box you have everything”.

With the core functionality taken care of, the Paddle team were able to refocus on what matters most: helping their engineering teams and improving practices.

Custom plugins

Part of the work that Paddle were keen to get back to was the development of Custom Plugins, so the custom plugin workflow was of particular interest during the PoV. They were able to self-serve and get up and running within a day:

When you develop Custom Plugins, moving from local to Roadie is very straightforward. We didn’t have to think about auth or connecting to x, y, z. When compared to our old Backstage instance, it felt easy to develop and work with. We’re looking to make more of these.

Support

While the trial was looking at specific tasks like importing items into the software catalog and setting up templates for task automation, one of the key overarching questions to be answered was about support. As Ioannis puts it, “Responsiveness is what we want in a partnership”:

We’ll be partners for years. It feels like you’re on top of things. You’ll help us go after things and find solutions. It’s reassuring you’re responsive and helpful”

Scorecards

The first step on that journey is locking in some of the engineering standards work that the team has built up over the years. Good practices like RFCs and Decision Records are in place, and good behaviours exist across Paddle engineering teams, but feedback to engineers on their work and automated checks against core standards were missing before Roadie.

With the TechInsights plugin can now create checks across their software catalog and automate the process of gathering information about services to improving standards.

And what’s next?

So now that that’s all in place, what next?

Roadie allows us to focus on what will drive change.

Now we have the tools, we can think about bigger initiatives - how we manage and provision resources in general, how people deploy their code, further standardisation and improvements of the developer experience, lessening context switching. This is years of work.

Paddle is rolling out the first wave of scorecards across their whole organisation to automate checks on adoption, production readiness, security compliance - driving the behavioural change through checks without having to worry about the maintenance of a Backstage installation.