`innsight`

- Get the Insights of your Neural Network`innsight`

is an R package that interprets the behavior and explains individual predictions of modern neural networks. Many methods for explaining individual predictions already exist, but hardly any of them are implemented or available in R. Most of these so-called *‘Feature Attribution’* methods are only implemented in Python and thus difficult to access or use for the R community. In this sense, the package `innsight`

provides a common interface for various methods for the interpretability of neural networks and can therefore be considered as an R analogue to iNNvestigate for Python.

This package implements several model-specific interpretability (Feature Attribution) methods based on neural networks in R, e.g.,

- Layer-wise Relevance Propagation (LRP)
- Including propagation rules:
*ε*-rule and*α*-*β*-rule

- Including propagation rules:
- Deep Learning Important Features (DeepLift)
- Including propagation rules for non-linearities: rescale rule and reveal-cancel rule

- Gradient-based methods:
- Vanilla Gradient, including Gradient x Input
- Smoothed gradients (SmoothGrad), including SmoothGrad x Input

- Connection Weights

Example results for these methods on ImageNet with pretrained network Vgg16:

The package `innsight`

aims to be as flexible as possible and independent of a specific deep learning package in which the passed network has been learned. Basically, a neural network of the libraries `torch`

, `keras`

and `neuralnet`

can be passed, which is internally converted into a `torch`

model with special insights needed for interpretation. Currently, the following model and layer types are accepted:

`torch::nn_sequential`

with layers`nn_linear`

,`nn_conv1d`

,`nn_conv2d`

,`nn_max_pool1d`

,`nn_max_pool2d`

,`nn_avg_pool1d`

,`nn_avg_pool2d`

,`nn_dropout`

,`nn_flatten`

(see torch issue #716 and use`classname = "nn_flatten"`

)`keras::keras_model_sequential`

or`keras::keras_model`

with layers`layer_dense`

,`layer_conv_1d`

,`layer_conv_2d`

,`layer_max_pooling_1d`

,`layer_max_pooling_2d`

,`layer_average_pooling_1d`

,`layer_average_pooling_2d`

,`layer_dropout`

,`layer_flatten`

But it is also possible to pass an arbitrary net in form of a named list (see vignette for details).

The package can be installed with the following command (successful installation of `devtools`

is required)

Internally, any passed model is converted to a `torch`

model, thus the correct functionality of this package relies on a complete and correct installation of `torch`

. For this reason, the following command must be run manually to install the missing libraries LibTorch and LibLantern:

**Note:** Currently this can lead to problems under Windows if the Visual Studio runtime is not pre-installed. See the issue on GitHub here or for more information and other problems with installing `torch`

see the official installation vignette of `torch`

.

You have a trained neural network `model`

on your model input data `data`

. Now you want to interpret individual datapoints or the overall behavior by using the methods from the package `innsight`

, then stick to the following pseudo code:

```
# --------------- Train your model -----------------
# 'model' has to be an instance of either torch::nn_sequential,
# keras::keras_model_sequential, keras::keras_model or neuralnet::neuralnet
model = ...
# -------------- Convert your model ----------------
# For keras and neuralnet
converter <- Converter$new(model)
# For a torch model the argument 'input_dim' is required
converter <- Converter$new(model, input_dim = model_input_dim)
# ----------------- Apply method -------------------
# Apply global method
result <- Method$new(converter) # no data argument is needed
# Plot the result
plot(result)
# Apply local methods
result <- Method$new(converter, data)
# Plot individual results
plot(result)
# Plot a boxplot of all given data points in argument 'data'
boxplot(result)
```

```
library(innsight)
library(torch)
data(iris)
# Prepare Data
x <- torch_tensor(as.matrix(iris[, -5]))
y <- torch_tensor(as.integer(iris[, 5]))
# Define Model
model <- nn_sequential(
nn_linear(4, 15),
nn_relu(),
nn_dropout(0.3),
nn_linear(15, 10),
nn_relu(),
nn_dropout(0.3),
nn_linear(10, 5),
nn_relu(),
nn_linear(5, 3),
nn_softmax(2)
)
# Train model
optimizer <- optim_adam(model$parameters, lr = 0.0002)
for (t in 1:2500) {
y_pred <- torch_log(model(x))
loss <- nnf_nll_loss(y_pred, y)
if (t %% 250 == 0) {
cat("Loss: ", as.numeric(loss), "\n")
}
optimizer$zero_grad()
loss$backward()
optimizer$step()
}
#> Loss: 1.034311
#> Loss: 0.8588966
#> Loss: 0.6706114
#> Loss: 0.5762339
#> Loss: 0.5493336
#> Loss: 0.5144928
#> Loss: 0.4707141
#> Loss: 0.3532465
#> Loss: 0.2839047
#> Loss: 0.2687521
```

```
# create a Converter for this model
converter <- Converter$new(model, input_dim = c(4),
input_names = list(names(iris[,-5])),
output_names = list(levels(iris[,5])))
# Apply local method LRP with epsilon rule
lrp_eps <- LRP$new(converter, iris[,-5], rule_name = "epsilon")
# Plot the individual result for two datapoints and all classes
p1 <- plot(lrp_eps, data_idx = c(1,102), output_idx = 1:3) +
ggplot2::labs(title = "LRP ('epsilon')")
# Plot the boxplot statistic for all datapoints and all classes
# without a preprocess function
p2 <- boxplot(lrp_eps, output_idx = 1:3, ref_data_idx = 1, preprocess_FUN = identity)
gridExtra::grid.arrange(p1,p2, ncol = 1, layout_matrix = matrix(c(1,1,1,2,2), ncol = 1))
```

```
library(keras)
# Load image
image <- image_load("man/images/imagenet_rooster.png", target_size = c(224,224))
image <- image_to_array(image)
p_image <- grid::rasterGrob(image / 255)
# Preprocess image
x <- array_reshape(image, c(1, dim(image)))
x <- imagenet_preprocess_input(x)
# Get pretrained Vgg16
model <- application_vgg16()
# Convert the model
converter <- Converter$new(model)
# Apply LRP with rule 'alpha_beta' with alpha = 1
lrp_ab <- LRP$new(converter, x,
output_idx = c(8), # 8 is index for class 'cock'
rule_name = "alpha_beta",
rule_param = 1,
channels_first = FALSE)
# We have to flip the y axis before plotting
lrp_ab$result <- torch::torch_flip(lrp_ab$result, c(2))
p_lrp_ab <- plot(lrp_ab)
gridExtra::grid.arrange(p_image, p_lrp_ab, layout_matrix = matrix(c(1,2,2), nrow = 1))
```

This work is funded by the German Research Foundation (DFG) in the context of the Emmy Noether Grant 437611051.