Improving Search, some mobile UX tweaks, and Tech Insights historical data ingestion

The latest features and updates from Roadie.

Introducing OpenSearch for better discoverability

As the volume of data in the Catalog grows, we often see customers hit a discoverability problem: how do you find items when the Catatlog is tens of thousands of, often very similar seeming, entities?

Well: search. Duh.

But, how do you make sure search results are fast, accurate and scalable? Backstage supports different search engines, but the better ones (Elastic-like ones like OpenSearch) tend to get expensive at scale, especially when you run almost isolated stacks for each customer as we do.

This month we dedicated a bit of time to figuring out how to do it at scale, and voilà! Better search has arrived!

We stripped out the old search engine and brought in OpenSearch, an Elastic-search-like search engine (an open-source one, of course).

That means you get a few improvements out of the box:

• Casing is better handled: camelCase, kebab-case and snake_case are factored in, so the user doesn’t need to know which was used when they’re searching
• Partial search terms and descriptions are better handled: for example, skipping some-word in camel-some-word-case and just typing camel case will be returned as a result.
• Word fragments allow for partial word matches: We’re currently using 4-7 chars for the ngrams, so fragments of words should be well matched

We also have brought ourselves a lot more headroom here for optimisation - we’d reached the end of the line with the pg_search after many years of trying to force it to be the search engine that we’d have liked it to be. OpenSearch has a lot more we can tweak.

So what’s next? We’ll be rolling out OpenSearch progressively to customers, starting this month.

Mobile UX gets some love

Being able to search for something in the Catalog is one thing, but if you can’t readily view it or the experience is poor then that’s not much use.

Roadie is largely optimised for web (big tables of software == a laptop or desktop-based experience is always going to be best) but there’s a baseline for usability that should be met to make some key use-cases function.

This month we focused on the use of tech docs on mobile. If it’s 4am, you’ve just been woken up by an incident, and you need to quickly read a runbook to see what to do next: probably best to not have to get your laptop out when you have your phone right there.

Tech Insights historical data ingestion gets 12x more data-ness

Tech Insights push-based Data Sources allow for historical data ingestion for our customers when migrating from other systems to Roadie. That’s been in place for a while, but this month we bumped the volume of data you can ingest.

Over the last few months we’ve seen more and more customers want to ingest a lot of data. Part of that is the growing maturity of scorecards as a concept within the wider industry, but also part of the wider shift towrads managed Backstage as the IDP of choice for many orgs.

Down to brass tacts though: the push-based data source can now ingest one year of data. Previously it was 1 month.

DX Plugin

Last but not least, we’ve introduced the DX plugin to Roadie to help sync the Catalog between the two.

More details here for those that are interested: https://roadie.io/docs/integrations/dx/

This is something we’re expecting to see a lot of in the coming months: dev tools that need a software Catalog simply sync to a Catalog like Roadie/Backstage rather than build it themselves. Tools like DataDog and Sentry are dipping their toes into the software Catalog world but it’s a tricky business keeping up to date with all the different sources of truth and ingestion mechanisms: that’s why open-source tools like Backstage will always have the edge.

More on this trend on the New Stack for those that are interested.