Principled Detection of OutofDistribution Examples in Neural Networks
17 Jul 2017Problem Statement

Given a pretrained neural network, which is trained using data from some distribution P (referred to as indistribution data), the task is to detect the examples coming from a distribution Q which is different from P (referred to as outofdistribution data).

For example, if a digit recognizer neural network is trained using MNIST images, an outofdistribution example would be images of animals.

Neural Networks can make high confidence predictions even in such cases where the input is unrecognisable or irrelevant.

The paper proposes ODIN which can detect such outofdistribution examples without changing the pretrained model itself.
ODIN

Uses 2 major techniques
 Temperature Scaling

Softmax classifier for the classification network can be written as:
p_{i}(x, T) = exp(f_{i}(x)/T) / sum(exp(f_{j}(x)/T))
where x is the input, p is the softmax probability and T is the temperature scaling parameter.
 Increasing T (up to some extent) boosts the performance in distinguishing indistribution and outofdistribution examples.

 Input Preprocessing

Add small perturbations to the input (image) before feeding it into the network.

x_perturbed = x  ε * sign(δ_{x}log(p_{y}(x, T)))
where ε is the perturbation magnitude
 The perturbations are such that softmax scores between indistribution and outofdistribution samples become separable.

 Temperature Scaling
 Given an input (image), first perturb the input.
 Feed the perturbed input to the network to get its softmax score.
 If the softmax score is greater than some threshold, mark the input as indistribution and feed in the unperturbed version of the input to the network for classification.
 Otherwise, mark the input as outofdistribution.
 For detailed mathematical treatment, refer section 6 and appendix in the paper
Experiments

Code available on github

Models
 DenseNet with depth L = 100 and growth rate k = 12
 Wide ResNet with depth = 28 and widen factor = 10

InDistribution Datasets
 CIFAR10
 CIFAR100

OutofDistribution Datasets
 TinyImageNet
 LSUN
 iSUN
 Gaussian Noise

Metrics
 False Positive Rate at 95% True Positive Rate
 Detection Error  minimum misclassification probability over all thresholds
 Area Under the Receiver Operating Characteristic Curve
 Area Under the PrecisionRecall Curve

ODIN outperforms the baseline across all datasets and all models by a good margin.
Notes
 Very simple and straightforward approach with theoretical justification under some conditions.
 Limited to examples from Vision so can not judge its applicability for NLP tasks.