Scaling model observability with Superwise & New Relic

Or Itzary

October 14th, 2021 min read

October 14th, 2021

min read

Superwise and New Relic Integration

Let’s skip the obvious, if you’re reading this it’s a safe bet that you already know that ML monitoring is a must; data integrity, model drift, performance degradation, etc., are already the basic standard of any MLOps monitoring tool. But as any ML practitioner will attest to, it’s one thing to monitor a single machine learning model, it’s another altogether to achieve automated model observability for dozens of live models all with immediate impact on daily predictions and business operations. Enter Superwise; high-scale model observability done right. What does that mean? Zeroing in on issues that lead to action, without alert fatigue and false alarms. The platform comes with built-in KPIs, automated issue detection and insights, and a self-service monitoring engine to deliver immediate value without sacrificing customization down the road. 

Model observability is all about context so it’s only natural for us to integrate our model KPIs and model insights into New Relic to take observability higher, further, faster. With the integration, Superwise and New Relic users will be able to explore model incidents within their New Relic workflow, as well as view Superwise’s model KPIs. 

What do you get?

The Superwise model observability dashboard gives you out-of-the-box information regarding your active models, their activity status, drift levels, and any open incidents detected for specific time intervals or filters. But we don’t stop there; you can configure any custom metric and incident you need to monitor for your specific use cases and monitor them in New Relic.

The basics

  • The model activity overview gives you a quick view of your active models, their activity (predictions) over time, and the total number of predictions during the filtered timeframe.
  • With drift detection and the model input drift chart, users can identify what models are drifting and may require retraining.
  • Using incident widgets, users can easily see how many models currently have open incidents (violations of any monitoring policy configured), how incidents are being distributed among the different models, and drill down into the model incident details. 

The custom

Superwise’s flexible monitoring policy builder lets you configure various model monitoring policies and send detected incidents into one or more downstream channels including New Relic, PagerDuty, Slack, Email, and more. You have full control over what policies are sent to which channels to ensure that the right team gets the right alert at the right time. 

What do you need to do?

It takes only a few minutes to integrate Superwise and New Relic so you can access our model KPIs and incidents in New Relic One. Check out the integration documentation and we’ll walk you through it.  

Don’t have Superwise yet?

We can fix that.

Everything you need to know about AI direct to your inbox

Superwise Newsletter

Superwise needs the contact information you provide to us to contact you about our products and services. You may unsubscribe from these communications at any time. For information on how to unsubscribe, as well as our privacy practices and commitment to protecting your privacy, please review our privacy policy.

Featured Posts

Drift in machine learning
May 5, 2022

Everything you need to know about drift in machine learning

What keeps you up at night? If you’re an ML engineer or data scientist, then drift is most likely right up there on the top of the list. But drift in machine learning comes in many forms and variations. Concept drift, data drift, and model drift all pop up on this list, but even they

Read now >
Everything you need to know about drift in machine learning
July 12, 2022

Concept drift detection basics

This article will illustrate how you can use Layer and Amazon SageMaker to deploy a machine learning model and track it using Superwise.

Read now >
Concept drift detection basics
Data Drift
August 31, 2022

Data drift detection basics

Drift in machine learning comes in many shapes and sizes. Although concept drift is the most widely discussed, data drift is the most frequent, also known as covariate shift. This post covers the basics of understanding, measuring, and monitoring data drift in ML systems. Data drift occurs when the data your model is running on

Read now >
Data drift detection basics