2 Software

The current flavor1 of statistical software is either R or Python :

  • R is primarily used by data analysts, statisticians and in academia.
  • Python is a general purpose programming lanugage and is primarily favored by data scientists and programmers.

However, both R and Python can be successfully used in either data analysis or data science. Consequently, examples will be provided in both R and Python.

As of the writing of this book, some statistical, methods, which are readily availables in R, need to be manually implemented in Python. However, this provides a great example for cases, where specific methods may need to be manually implemented in a different environment in real-life scenarios. As such, another indirect aim of this book is to also provide a degree of autonomy from any specific libraries which may change or become obsolete in the future (this may also apply to programming/scripting languages as well!).

The software versions of R and Python used in this book are:

  • For R:
## [1] "R version 3.6.0 (2019-04-26)"
  • For Python:
## 3.8.1 (tags/v3.8.1:1b293b6, Dec 18 2019, 23:11:46) [MSC v.1916 64 bit (AMD64)]

Please note that some of the packages and libraries used might be early versions, which are subject to change. In case you encounter any errors, make sure to use the version you are using matches the ones used in this book. If you do not encounter any errors but do decide to use a different version, take note that you might encounter some differences in the output or results.

This book is compiled using bookdown and as such both R and Python code along with the output are also being executed when compiling each chapter. This ensures that the code and output are the same that you should get when running the code examples.

The package versions are provided below (note - the list may be updated along with any updates to the chapters in this book):

Especially take note of packages, with versions below 1.0.0 - those packages are the most likely to experience changes to functions and output with newer versions/updates.

For a complete list of packages installed on the current system, which was used for compiling this book, see below:

## abind==1.4-5
##  ADGofTest==0.3
##  AER==1.2-8
##  alphavantager==0.1.2
##  anytime==0.3.6
##  askpass==1.1
##  assertthat==0.2.1
##  astsa==1.9
##  aTSA==3.1.2
##  aweek==1.0.0
##  backports==1.1.5
##  base==3.6.0
##  base64enc==0.1-3
##  bayesm==3.1-4
##  bdsmatrix==1.3-3
##  betareg==3.1-2
##  BH==1.72.0-2
##  bibtex==0.4.2.1
##  BiocManager==1.30.10
##  BiocVersion==3.10.1
##  bit==1.1-14
##  bit64==0.9-7
##  bitops==1.0-6
##  bookdown==0.16
##  boot==1.3-22
##  broom==0.5.3
##  calculus==0.2.1
##  callr==3.4.0
##  car==3.0-6
##  carData==3.0-3
##  caret==6.0-86
##  caTools==1.17.1.3
##  cellranger==1.1.0
##  checkmate==1.9.4
##  chron==2.3-54
##  class==7.3-15
##  classInt==0.4-2
##  cli==2.0.0
##  clipr==0.7.0
##  clisymbols==1.2.0
##  clue==0.3-57
##  cluster==2.0.8
##  coarseDataTools==0.6-5
##  coda==0.19-3
##  codetools==0.2-16
##  collections==0.2.3
##  colorspace==1.4-1
##  combinat==0.0-8
##  compiler==3.6.0
##  copula==0.999-19.1
##  countrycode==1.1.0
##  cowplot==1.0.0
##  cpp11==0.2.2
##  crayon==1.3.4
##  crosstalk==1.0.0
##  curl==4.3
##  cyclocomp==1.1.0
##  DAAG==1.22.1
##  data.table==1.13.0
##  datasets==3.6.0
##  DBI==1.1.0
##  dbplyr==1.4.2
##  desc==1.2.0
##  deSolve==1.28
##  digest==0.6.23
##  DistributionUtils==0.6-0
##  doParallel==1.0.15
##  dplyr==1.0.2
##  DT==0.11
##  dyn==0.2-9.6
##  dynlm==0.3-6
##  e1071==1.7-3
##  Ecdat==0.3-4
##  Ecfun==0.2-0
##  ellipsis==0.3.0
##  EpiEstim==2.2-1
##  eurostat==3.6.1
##  evaluate==0.14
##  expm==0.999-4
##  expsmooth==2.3
##  extraDistr==1.8.11
##  fansi==0.4.0
##  farver==2.0.1
##  fastICA==1.2-2
##  fastmap==1.0.1
##  fBasics==3042.89
##  fda==2.4.8
##  fGarch==3042.83.1
##  FinancialInstrument==1.3.1
##  FinTS==0.4-6
##  fitdistrplus==1.0-14
##  flexmix==2.3-15
##  fma==2.3
##  FNN==1.1.3
##  forcats==0.4.0
##  foreach==1.4.7
##  forecast==8.10
##  foreign==0.8-71
##  Formula==1.2-3
##  fpp==0.5
##  fpp2==2.3
##  fracdiff==1.5-0
##  fs==1.3.1
##  fUnitRoots==3042.79
##  furrr==0.1.0
##  future==1.19.1
##  gbRd==0.4-11
##  gdata==2.18.0
##  GeneralizedHyperbolic==0.8-4
##  generics==0.0.2
##  ggfortify==0.4.8
##  ggimage==0.2.5
##  ggplot2==3.3.2
##  ggplotify==0.0.4
##  gh==1.0.1
##  globals==0.13.0
##  glue==1.4.2
##  goftest==1.2-2
##  gower==0.2.1
##  gplots==3.0.1.1
##  graphics==3.6.0
##  grDevices==3.6.0
##  grid==3.6.0
##  gridExtra==2.3
##  gridGraphics==0.4-1
##  grr==0.9.5
##  gsl==2.1-6
##  gss==2.1-10
##  gtable==0.3.0
##  gtools==3.8.1
##  gtrendsR==1.4.7
##  haven==2.2.0
##  hexbin==1.28.0
##  hexSticker==0.4.6
##  highr==0.8
##  HMMpa==1.0.1
##  hms==0.5.2
##  htmltools==0.4.0
##  htmlwidgets==1.5.1
##  httpuv==1.5.2
##  httr==1.4.1
##  igraph==1.2.4.2
##  incidence==1.7.1
##  Information==0.0.9
##  InformationValue==1.2.3
##  ini==0.3.1
##  ipred==0.9-9
##  IRdisplay==0.7.0
##  IRkernel==1.1
##  ISLR==1.2
##  isoband==0.2.2
##  iterators==1.0.12
##  janitor==2.0.1
##  jpeg==0.1-8.1
##  jsonlite==1.6
##  kableExtra==1.1.0
##  kernlab==0.9-29
##  KernSmooth==2.23-15
##  knitr==1.26
##  ks==1.11.6
##  labeling==0.3
##  languageserver==0.3.2
##  later==1.0.0
##  lattice==0.20-38
##  latticeExtra==0.6-29
##  lava==1.6.6
##  lazyeval==0.2.2
##  leaps==3.0
##  lifecycle==0.2.0
##  lintr==2.0.0
##  listenv==0.8.0
##  lme4==1.1-21
##  lmtest==0.9-37
##  locfit==1.5-9.1
##  lrmest==3.0
##  lsei==1.2-0
##  lubridate==1.7.4
##  maditr==0.6.3
##  magick==2.2
##  magrittr==1.5
##  manipulateWidget==0.10.0
##  maptools==0.9-9
##  markdown==1.1
##  MASS==7.3-51.4
##  Matrix.utils==0.9.7
##  Matrix==1.2-17
##  Matrix==1.2-18
##  MatrixModels==0.4-1
##  maxLik==1.3-6
##  mclust==5.4.5
##  mcmc==0.9-7
##  MCMCpack==1.4-6
##  methods==3.6.0
##  mfx==1.2-2
##  mgcv==1.8-28
##  mime==0.8
##  miniUI==0.1.1.1
##  minqa==1.2.4
##  miscTools==0.6-26
##  mlmRev==1.0-7
##  mnormt==1.5-5
##  modeest==2.4.0
##  modeldata==0.0.2
##  ModelMetrics==1.2.2.2
##  modelr==0.1.5
##  modeltools==0.2-22
##  multcomp==1.4-11
##  multicool==0.1-11
##  munsell==0.5.0
##  mvtnorm==1.0-11
##  nlme==3.1-139
##  nloptr==1.2.1
##  nnet==7.3-12
##  nortest==1.0-4
##  npsurv==0.4-0
##  numDeriv==2016.8-1.1
##  ocedata==0.1.6
##  OECD==0.2.4
##  olsrr==0.5.2
##  openssl==1.4.1
##  openxlsx==4.1.4
##  orcutt==2.3
##  pacman==0.5.1
##  padr==0.5.3
##  parallel==3.6.0
##  pbdZMQ==0.3-3
##  pbkrtest==0.4-7
##  pBrackets==1.0
##  pcaPP==1.9-73
##  PerformanceAnalytics==1.5.3
##  phaseR==2.1.3
##  pillar==1.4.3
##  pkgbuild==1.0.6
##  pkgconfig==2.0.3
##  pkgload==1.0.2
##  plm==2.2-0
##  plogr==0.2.0
##  plotly==4.9.1
##  plyr==1.8.5
##  png==0.1-7
##  praise==1.0.0
##  prettyunits==1.0.2
##  pROC==1.16.2
##  processx==3.4.1
##  prodlim==2019.11.13
##  progress==1.2.2
##  promises==1.1.0
##  ps==1.3.0
##  pspline==1.0-18
##  psych==1.9.12
##  purrr==0.3.3
##  quadprog==1.5-8
##  Quandl==2.10.0
##  quantmod==0.4-15
##  quantreg==5.54
##  R6==2.4.1
##  rappdirs==0.3.1
##  rattle.data==1.0.2
##  RColorBrewer==1.1-2
##  Rcpp==1.0.3
##  RcppArmadillo==0.9.800.3.0
##  RcppEigen==0.3.3.7.0
##  RCurl==1.95-4.12
##  Rdpack==0.11-1
##  readr==1.3.1
##  readxl==1.3.1
##  recipes==0.1.12
##  RefManageR==1.2.12
##  rematch==1.0.1
##  rematch2==2.1.0
##  remotes==2.1.0
##  repr==1.0.2
##  reprex==0.3.0
##  reshape2==1.4.3
##  reticulate==1.14
##  rex==1.1.2
##  rgl==0.100.30
##  riingo==0.3.1
##  rio==0.5.16
##  rJava==0.9-12
##  rjson==0.2.20
##  rJython==0.0-4
##  rlang==0.4.7
##  rmarkdown==2.0
##  rmutil==1.1.4
##  ROCR==1.0-7
##  rpart==4.1-15
##  rprojroot==1.3-2
##  rsample==0.0.8
##  rsdmx==0.5-13
##  Rsolnp==1.16
##  rstudioapi==0.10
##  rSymPy==0.2-1.2
##  rugarch==1.4-1
##  rvcheck==0.1.7
##  rvest==0.3.5
##  sandwich==2.5-1
##  scales==1.1.0
##  scatterplot3d==0.3-41
##  seasonal==1.7.0
##  selectr==0.4-2
##  servr==0.15
##  sf==0.8-0
##  shiny==1.4.0
##  shinyjs==1.1
##  showtext==0.7
##  showtextdb==2.0
##  SkewHyperbolic==0.4-0
##  slider==0.1.5
##  sm==2.2-5.6
##  smaa==0.3-0
##  snakecase==0.11.0
##  sourcetools==0.1.7
##  sp==1.3-2
##  SparseM==1.78
##  spatial==7.3-11
##  spd==2.0-1
##  splines==3.6.0
##  SQUAREM==2017.10-1
##  stable==1.1.4
##  stabledist==0.7-1
##  statip==0.2.3
##  stats==3.6.0
##  stats4==3.6.0
##  stochvol==2.0.4
##  stringdist==0.9.5.5
##  stringi==1.5.3
##  stringr==1.4.0
##  strucchange==1.5-2
##  styler==1.2.0
##  survival==2.44-1.1
##  swirl==2.4.4
##  sys==3.3
##  sysfonts==0.8
##  systemfit==1.1-24
##  tcltk==3.6.0
##  TeachingDemos==2.10
##  testthat==2.3.1
##  TH.data==1.0-10
##  tibble==3.0.3
##  tidyquant==1.0.1
##  tidyr==1.1.2
##  tidyselect==1.1.0
##  tidyverse==1.3.0
##  timeDate==3043.102
##  timeSeries==3042.102
##  tinytex==0.18
##  tis==1.37.1
##  tools==3.6.0
##  translations==3.6.0
##  truncnorm==1.0-8
##  TSA==1.2
##  tsDyn==0.9-48.1
##  tseries==0.10-47
##  tseriesChaos==0.1-13.1
##  tsinteger==0.1
##  TTR==0.23-6
##  units==0.6-5
##  urca==1.3-0
##  utf8==1.1.4
##  utils==3.6.0
##  uuid==0.1-2
##  vars==1.5-3
##  vctrs==0.3.4
##  viridisLite==0.3.0
##  warp==0.1.0
##  waveslim==1.8.2
##  webshot==0.5.2
##  whisker==0.4
##  withr==2.1.2
##  writexl==1.2
##  x13binary==1.1.39-2
##  xfun==0.11
##  XML==3.98-1.20
##  xml2==1.2.2
##  xmlparsedata==1.0.3
##  xptr==1.1.2
##  xtable==1.8-4
##  xts==0.11-2
##  yaml==2.2.0
##  zeallot==0.1.0
##  zip==2.0.4
##  zoo==1.8-6
## alabaster==0.7.12
## argh==0.26.2
## astroid==2.3.3
## atomicwrites==1.3.0
## attrs==19.3.0
## autopep8==1.4.4
## babel==2.8.0
## backcall==0.1.0
## bcrypt==3.1.7
## beautifulsoup4==4.9.1
## bleach==3.1.0
## boto3==1.11.5
## boto==2.49.0
## botocore==1.14.5
## bs4==0.0.1
## certifi==2019.11.28
## cffi==1.13.2
## chardet==3.0.4
## cloudpickle==1.2.2
## colorama==0.4.3
## cryptography==2.8
## cycler==0.10.0
## cython==0.29.14
## decorator==4.4.1
## defusedxml==0.6.0
## descartes==1.1.0
## diff-match-patch==20181111
## docutils==0.15.2
## entrypoints==0.3
## ez-setup==0.9
## flake8==3.7.9
## freetype-py==2.1.0.post1
## future==0.18.2
## gensim==3.8.1
## helpdev==0.6.10
## idna==2.8
## imagesize==1.2.0
## importlib-metadata==1.4.0
## intervaltree==3.0.2
## ipykernel==5.1.3
## ipython-genutils==0.2.0
## ipython==7.11.1
## ipywidgets==7.5.1
## isort==4.3.21
## jedi==0.14.1
## jinja2==2.10.3
## jmespath==0.9.4
## joblib==0.14.1
## js2xml==0.3.1
## json5==0.8.5
## jsonschema==3.2.0
## jupyter-client==5.3.4
## jupyter-console==6.1.0
## jupyter-contrib-core==0.3.3
## jupyter-contrib-nbextensions==0.5.1
## jupyter-core==4.6.1
## jupyter-highlight-selected-word==0.2.0
## jupyter-latex-envs==1.4.6
## jupyter-nbextensions-configurator==0.4.1
## jupyter==1.0.0
## jupyterlab-server==1.0.6
## jupyterlab==1.2.5
## keyring==21.1.0
## kiwisolver==1.1.0
## lazy-object-proxy==1.4.3
## lxml==4.4.2
## markupsafe==1.1.1
## matplotlib==3.1.2
## mccabe==0.6.1
## mistune==0.8.4
## mizani==0.6.0
## more-itertools==8.1.0
## mpmath==1.1.0
## nbconvert==5.6.1
## nbformat==5.0.3
## nltk==3.4.5
## nose==1.3.7
## notebook==6.0.2
## numpy==1.18.1
## numpydoc==0.9.2
## packaging==20.0
## palettable==3.3.0
## pandas-datareader==0.8.1
## pandas==0.25.3
## pandocfilters==1.4.2
## paramiko==2.7.1
## parso==0.5.2
## pathtools==0.1.2
## patsy==0.5.1
## pexpect==4.7.0
## pickleshare==0.7.5
## pip==20.1.1
## plotly==4.10.0
## plotnine==0.6.0
## pluggy==0.13.1
## ply==3.11
## prometheus-client==0.7.1
## prompt-toolkit==3.0.2
## psutil==5.6.7
## ptyprocess==0.6.0
## pycodestyle==2.5.0
## pycparser==2.19
## pydocstyle==5.0.2
## pyflakes==2.1.1
## pygments==2.5.2
## pylint==2.4.4
## pynacl==1.3.0
## pyparsing==2.4.6
## pypdf4==1.27.0
## pypng==0.0.20
## pyqt5-sip==12.7.0
## pyqt5==5.12.3
## pyqtwebengine==5.12.1
## pyrsistent==0.15.7
## python-dateutil==2.8.1
## python-jsonrpc-server==0.3.3
## python-language-server==0.31.6
## pytz==2019.3
## pywin32-ctypes==0.2.0
## pywin32==227
## pywinpty==0.5.7
## pyyaml==5.3
## pyzmq==18.1.1
## qdarkstyle==2.7
## qtawesome==0.6.1
## qtconsole==4.6.0
## qtpy==1.9.0
## requests==2.22.0
## retrying==1.3.3
## rope==0.16.0
## s3transfer==0.3.1
## scikit-gof==0.1.3
## scikit-learn==0.22.1
## scipy==1.4.1
## seaborn==0.10.1
## selenium==3.141.0
## send2trash==1.5.0
## setuptools==45.0.0
## sip==5.1.0
## six==1.14.0
## slimit==0.8.1
## smart-open==1.9.0
## snowballstemmer==2.0.0
## sortedcontainers==2.1.0
## soupsieve==2.0.1
## sphinx==2.3.1
## sphinxcontrib-applehelp==1.0.1
## sphinxcontrib-devhelp==1.0.1
## sphinxcontrib-htmlhelp==1.0.2
## sphinxcontrib-jsmath==1.0.1
## sphinxcontrib-qthelp==1.0.2
## sphinxcontrib-serializinghtml==1.1.3
## spyder-kernels==1.8.1
## spyder==4.0.1
## statsmodels==0.11.0
## sympy==1.5.1
## terminado==0.8.3
## testpath==0.4.4
## theano==1.0.4
## toml==0.10.0
## tornado==6.0.3
## traitlets==4.3.3
## urllib3==1.25.7
## watchdog==0.9.0
## wcwidth==0.1.8
## webencodings==0.5.1
## wheel==0.33.6
## widgetsnbextension==3.5.1
## wrapt==1.11.2
## yapf==0.29.0
## zipp==1.0.0

  1. Other popular statistical software include: Gretl, SAS, STATA, SPSS, EViews, MATLAB. Additionaly, gaining in popularity are: Octave and Julia