NLP monitoring

Easily enrich unstructured NLP data with the Elemeta open-source and log tabular metafeature representations to Superwise to monitor.

Easily extract NLP metafeatures

Elemeta

Elemeta is an open-source library in Python for metafeature extraction. With it, you will be able to explore, monitor, and extract features from unstructured data through enriched tabular representations. It provides a straightforward Python API for metadata extraction from unstructured data like text and images.

Star
Docs

Contextual metrics

I want to monitor average values for SentimentPolarity in all social pipelines across all segments & entire set

Monitor contextual metafeatures from Elemeta with Superwise to identify changes in the input or output behaviors of your end users and models.

Monitor for both behavior spikes and gradual drifts.
Build custom metafeatures to identify domain-specific, biased inputs/outputs.

Statistical metrics

I want to monitor max value for OutOfVocabularyCount in video search model across teens segment

Monitor statistical metafeatures from Elemeta with Superwise to identify changes in the distribution and representation of your NLP use case.

Identify where your model is underperforming.
Monitor for implied changes in population representation.

Try the community edition

No credit card required. FREE forever.

1

Sign up for Superwise

Easily get started with a free community edition account.

!pip install superwise

2

Build your project

import superwise as sw
project = sw.project(“Fraud detection”)

model = sw.model(project,“Customer a”)

policy = sw.policy(model,drift_template)

3

Start monitoring

Fraud detection

Entire population drift – high probability of concept drift. Open incident investigation →

Fraud detection

Segment “tablet shoppers” drifting. Split model and retrain.

FREE community edition

Featured resources

Blog

Monitoring NLP with Superwise & Elemeta

In this post, we’re going to show you an example of how to use Elemeta together with Superwise’s model observability community edition to supply visibility and monitoring of your NLP model’s input text.

Elemeta: Extract metafeatures from unstructured data

We’re excited to release into beta v1.0 of Elemeta, our open-source library for exploring, monitoring, and extracting features from unstructured data.

Challenges of NLP monitoring

Monitoring ML, in general, is not trivial – NLP monitoring, in particular, produces a few unique challenges that we’ll examine in this post.