Managing Fraud Score models

Go to Fraud Score > Models to see your labels and their models. Use the Web, iOS, and Android tabs to switch between platforms. The page has three sections:

• Awaiting your approval: new model versions ready for your review. Click Review to compare a version with the active model.
• Needs attention: issues that block training or reduce model quality, each with a suggested action.
• Labels: all your labels, with the latest model version, status, and the number of fraud and non-fraud events.

Show issues in Needs attention

Issue	Description
Uncategorized	The label needs a category before it can train
Requires data	The label needs more fraud or non-fraud events to train
Balance	Too few events are labeled as non-fraud
Same period	Your non-fraud events don’t cover enough of the period your fraud events cover
Stale	No new labels were submitted recently
Expiry	Many labeled events expire within a month

Show model statuses

Status	Description
Requires data	The label doesn’t have enough labeled events to train
Needs category	The label needs a category before it can train
Ready to train	The label has enough data to train a new version
Training	Fingerprint is training a new version
Under review	Fingerprint is reviewing the new version for quality
Pending approval	The new version is ready for your review
Active	The model is scoring your events

​

Model details

Click a label to open its model details. Use the Version selector to switch between model versions. The model details page shows:

• Balance: the number of fraud and non-fraud events the model trains on. Label at least as many non-fraud events as fraud events, and aim for about 10 non-fraud events for every fraud event.
• Performance: Precision, Recall, PR-AUC, ROC-AUC, and F1, measured on your labeled data. Expect lower precision on live traffic, where fraud is rarer than in your labeled data.
• Health: issues that affect this model, if any.
• Top contributing signals: the Smart Signals and device attributes that contributed most to the model’s predictions.
• Fraud score distribution: where your recent traffic lands on the fraud score, and roughly how many events your current threshold flags.
• Charts: the ROC curve, the precision-recall curve, and precision, recall, and F1 at each threshold.
• Version history: past versions with their performance metrics. Discarded versions are on the Discarded tab.

​

Choose a threshold

The threshold decides how high a score must be for Fraud Score to flag an event. A lower threshold catches more fraud but flags more legitimate events. A higher threshold flags fewer legitimate events but lets more fraud through. To change it, open a label and select a Threshold option: The threshold applies to all current and future models of the label, on all platforms. You can change it at any time.

​

Review and approve a model version

When a new version is ready, it appears in Awaiting your approval. The active version keeps running until you approve the new one.

1. Click Review next to the version, or Review changes on the model details page.
2. Compare the new version with the active one. The changelog shows the change in labeled events, recall, false positive rate, precision, ROC-AUC, and PR-AUC.
3. Click Approve changes to make the new version active, or Discard to reject it. You must enter a reason to discard a version.

When you approve a version, the previous version is archived and stays in the version history. A discarded version stays in the version history but can’t be used again.

​

Auto-apply

To skip manual approval, select Auto-apply safe models in the review panel. Fingerprint then automatically applies future versions of the label that pass its internal safety checks. The setting applies to all platforms of the label.