An important part of model building is the “proc eyeball” sanity check. It can also be a painful part of the process, when you are the data scientist tasked with creating and checking 10,000 or more near-identical plots. The `otvPlots`

package is designed to streamline this process. `otvPlots`

is an R package which takes a csv file as input and provides a pdf of VLM plots and csv files of summary statistics as output, optionally ordered so that any severely abnormal time series will be at the top of the pdf. The only strict requirement of the data scientist is to specify which column of the input data file contains the date variable.

`otvPlots`

is efficiently implemented using `data.table`

and `ggplot2`

packages in R. Plots are automatically labeled if a variable dictionary is provided. Important variables can be given a highlighted label. A custom fuzzy matching algorithm can be provided by the user.

Discrete and numeric variables are handled automatically and given separate treatment. All binary variables are treated as categorical.

For each numerical variable, the output plots include * side-by-side boxplots (left), * a trace plot of p1, p50, and p99 percentiles, * a trace plot of mean and +-1 SD control limits, and * a trace plot of missing and zero rates (bottom right).

For each categorical variable (including a numerical variable with no more than 2 unique levels not including NA), the output plots include * a frequency bar plot (left), and * a grid of trace plots on categories’ proportions over time (right).

The order of variables in the CSV files is the same as in the PDF file. * A CSV file for numerical variables, including the number of observations (counts), p1, p25, p50, p75, and p99 quantiles, mean, SD, missing and zero rates. * A CSV file for categorical variables, including the number of observations (counts) and categories’ proportions. Each row is a category of a categorical (or binary) variable. The row whose `category == 'NA'`

corresponds to missing. Categories among the same variable are ordered by global prevalence in a descending order.

Open an R (or RStudio) console and install the package from CRAN

`install.packages("otvPlots")`

Alternatively, if you prefer to install from GitHub:

- Install the
`devtools`

package if not yet. You only need to do this once, so feel free to skip this step if the`devtools`

is already installed. You will be asked to select a CRAN mirror.

`install.packages("devtools")`

- Install the
`otvPlots`

package

`devtools::install_github("capitalone/otvPlots")`

You can also build the package yourself by cloning the repo, setting your working directory to the otvPlots folder and running `devtools::build()`

in R, after installing the `devtools`

package.

Note that otvPlots does depend on R and several R packages to run. You can see a complete and up to date list of dependencies in the Imports field in the DESCRIPTION file.

Open an R console (or RStudio). Load the `otvPlots`

pacakge first (all its dependent packages should be loaded automatically).

`library(otvPlots)`

The main function of the package is `vlm`

. Before execute this function, input data need to be prepared using the `PrepData`

function. **Please check out the help files to see all options and many usage examples (highly recommended!)**

```
help(vlm)
help(PrepData)
```

The data `bankData`

and its labels `bankLables`

are built-in datasets in the `otvPlots`

package.

After running the following code, a pdf file named “bank.pdf” and two csv files named “bank_numerical_summary.csv” and “bank_categorical_summary.csv” will be generated in the current working directory.

```
## Load the datasets
data(bankData)
data(bankLabels)
## Prepare data and labels
bankData <- PrepData(bankData, dateNm = "date", dateGp = "months",
dateGpBp = "quarters")
bankLabels <- PrepLabels(bankLabels)
## Generate a pdf file of vlm plots, and csv files of summary statistics
vlm(dataFl = bankData, dateNm = "date", labelFl = bankLabels,
sortFn = "OrderByR2", dateGp = "months", dateGpBp = "quarters", outFl = "bank")
```

`bankData`

dataThe `PrepData`

function only needs to be run once on a dataset. After that `vlm`

can be run directly with the argument `dataNeedPrep = FALSE`

(the default).

- If csv files of summary statistics are not need, set
`genCSV = FALSE`

.

```
vlm(dataFl = bankData, dateNm = "date", labelFl = bankLabels, genCSV = FALSE,
sortFn = "OrderByR2", dateGp = "months", dateGpBp = "quarters", outFl = "bank2")
```

- If weights are provided, they will be used in all statistical calculations

```
bankData[, weight := rnorm(.N, 1, .1)]
bankData[, weight := weight / mean(weight)]
vlm(dataFl = bankData, dateNm = "date", labelFl = bankLabels,
dateGp = "months", dateGpBp = "quarters", weightNm = "weight", outFl = "bank3")
```

- Customize plotting order by passing a vector of variable names to argument
`sortVars`

, but the`"date"`

column must be excluded from`sortVars`

```
sortVars <- sort(bankLabels[varCol!="date", varCol])
vlm(dataFl = bankData, dateNm = "date", labelFl = bankLabels,
dateGp = "months", dateGpBp = "quarters", outFl = "bank4",
sortVars = sortVars)
```

- Create plots for a specific variable using the
`varNms`

argument

```
vlm(dataFl = bankData, dateNm = "date", labelFl = bankLabels,
dateGp = "months", dateGpBp = "quarters", outFl = "bank5",
varNms = "age", sortVars = NULL)
```

All examples for this package come from the Bank Marketing dataset available at the UCI Machine Learning Repository. The UCI repository maintains a free collection of datasets for researchers at its website.

Moro et al., S. Moro, P. Cortez, and P. Rita (2014). A Data-Driven Approach to Predict the Success of Bank Telemarketing. Decision Support Systems, Elsevier, 62:22-31, June 2014

Lichman, M. (2013). UCI Machine Learning Repository [http://archive.ics.uci.edu/ml]. Irvine, CA: University of California, School of Information and Computer Science.

Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Contributors: We welcome your interest in Capital One’s Open Source Projects (the “Project”).

Any Contributor to the project must accept and sign a CLA indicating agreement to the license terms. Except for the license granted in this CLA to Capital One and to recipients of software distributed by Capital One, you reserve all right, title, and interest in and to your contributions; this CLA does not impact your rights to use your own contributions for any other purpose.

This project adheres to the Open Source Code of Conduct. By participating, you are expected to honor this code.