Scaling model observability with Superwise & New Relic

Let’s skip the obvious, if you’re reading this it’s a safe bet that you already know that ML monitoring is a must; data integrity, model drift, performance degradation, etc., are already the basic standard of any MLOps monitoring tool. But as any ML practitioner will attest to, it’s one thing to monitor a single machine learning model, it’s another altogether to achieve automated model observability for dozens of live models all with immediate impact on daily predictions and business operations. Enter Superwise; high-scale model observability done right. What does that mean? Zeroing in on issues that lead to action, without alert fatigue and false alarms. The platform comes with built-in KPIs, automated issue detection and insights, and a self-service monitoring engine to deliver immediate value without sacrificing customization down the road. 

Model observability is all about context so it’s only natural for us to integrate our model KPIs and model insights into New Relic to take observability higher, further, faster. With the integration, Superwise and New Relic users will be able to explore model incidents within their New Relic workflow, as well as view Superwise’s model KPIs. 

What do you get?

The Superwise model observability dashboard gives you out-of-the-box information regarding your active models, their activity status, drift levels, and any open incidents detected for specific time intervals or filters. But we don’t stop there; you can configure any custom metric and incident you need to monitor for your specific use cases and monitor them in New Relic.

The basics

• The model activity overview gives you a quick view of your active models, their activity (predictions) over time, and the total number of predictions during the filtered timeframe.
• With drift detection and the model input drift chart, users can identify what models are drifting and may require retraining.
• Using incident widgets, users can easily see how many models currently have open incidents (violations of any monitoring policy configured), how incidents are being distributed among the different models, and drill down into the model incident details. 

The custom

Superwise’s flexible monitoring policy builder lets you configure various model monitoring policies and send detected incidents into one or more downstream channels including New Relic, PagerDuty, Slack, Email, and more. You have full control over what policies are sent to which channels to ensure that the right team gets the right alert at the right time. 

What do you need to do?

It takes only a few minutes to integrate Superwise and New Relic so you can access our model KPIs and incidents in New Relic One. Check out the integration documentation and we’ll walk you through it.  

Don’t have Superwise yet?

We can fix that.