{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lecture 14 - Ensemble Methods <a id=\"section0\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[![View notebook on Github](https://img.shields.io/static/v1.svg?logo=github&label=Repo&message=View%20On%20Github&color=lightgrey)](https://github.com/avakanski/Fall-2024-Applied-Data-Science-with-Python/blob/main/docs/Lectures/Theme_3-Model_Engineering/Lecture_14-Ensemble_Methods/Lecture_14-Ensemble_Methods.ipynb)\n",
    "[![Open In Collab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/avakanski/Fall-2024-Applied-Data-Science-with-Python/blob/main/docs/Lectures/Theme_3-Model_Engineering/Lecture_14-Ensemble_Methods/Lecture_14-Ensemble_Methods.ipynb) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='top'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- [14.1 Ensemble Methods](#14.1-ensemble-methods)\n",
    "    - [14.1.1 Loading the Dataset](#14.1.1-loading-the-dataset)\n",
    "- [14.2 Voting Ensemble](#14.2-voting-ensemble)\n",
    "- [14.3 Bagging Ensemble](#14.3-bagging-ensemble)\n",
    "- [14.4 Boosting Ensemble](#14.4-boosting-ensemble)\n",
    "    - [14.4.1 Gradient Boosting Ensemble](#14.4.1-gradient-boosting-ensemble)\n",
    "    - [14.4.2 Recent Gradient Boosting Methods: XGBoost, LightGBM, CatBoost](#14.4.2-recent-gradient-boosting-methods:-xgboost,-lightgbm,-catboost)\n",
    "        - [14.4.2.1 XGBoost (Extreme Gradient Boosting)](#14.4.2.1-xgboost-(extreme-gradient-boosting))\n",
    "        - [14.4.2.2 LightGBM (Light Gradient Boosting Machine)](#14.4.2.2-lightgbm-(light-gradient-boosting-machine))\n",
    "        - [14.4.2.3 CatBoost (Categorical Boosting)](#14.4.2.3-catboost-(categorical-boosting))\n",
    "    - [14.4.3 AdaBoost Ensemble](#14.4.3-adaboost-ensemble)\n",
    "- [14.5 Stacking Ensemble](#14.5-stacking-ensemble)\n",
    "- [References](#references)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 14.1 Ensemble Methods <a id=\"14.1-ensemble-methods\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Ensemble methods** combine the predictions of several other machine learning models, where by aggregating the results of the other models trained on the same dataset, ensembles can typically achieve better performance than any individual model. The individual models are often referred to as *base models* or *base learners*. \n",
    "\n",
    "When we introduced scikit-learn we mentioned that Random Forest is an example of an ensemble model, since it combines a multitude of Decision Tree models to make predictions. \n",
    "\n",
    "Ensembles are one of the most powerful Machine Learning methods, and they are often used in the winning solutions in many Machine Learning competitions. \n",
    "\n",
    "Ensemble methods can be used for different learning tasks, including classification and regression. In this lecture, we will focus on ensemble methods for classification.\n",
    "\n",
    "Ensemble models are divided into four general groups:\n",
    "\n",
    "- **Voting Methods**: make predictions based on majority voting of the individual models.\n",
    "- **Bagging Methods**: train individual models on random subsets of the training data. \n",
    "- **Boosting Methods**: train individual models sequentially by learning from the errors. Examples are Gradient Boosting, XGBoost, LightGBM, CatBoost, and AdaBoost.\n",
    "- **Stacking Methods**: train individual models, and use another model to aggregate their predictions. \n",
    "\n",
    "Most of the above ensemble methods are implemented in scikit-learn, except for XGBoost, LightGBM, and CatBoost."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 14.1.1 Loading the Dataset <a id=\"14.1.1-loading-the-dataset\"/>\n",
    "\n",
    "To demonstrate the working principles of ensemble methods, we will use a dataset of electricity usage in New South Wales, Australia. In that market, electricity prices are not fixed and they are set every five minutes based on demand and supply of the market. \n",
    "\n",
    "We will train classifiers to predict whether the electricity price will go UP or DOWN the next time the price is set. \n",
    "\n",
    "The dataset contains 45,312 rows of data from May 7, 1996 to December 5, 1998. Each row of the dataset refers to a period of 30 minutes. It has information about the date and day of the week, and has other features related to electricity demand, schedule transfer, etc. \n",
    "\n",
    "Let's load the dataset `electricity_data` using pandas. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import seaborn as sns\n",
    "import sklearn\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Let's hide warning messages in the cells\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "elec_df = pd.read_csv('data/electricity_data.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's inspect the dataset. The `date` column does not show the actual date correctly, but we will use it as is, because the date has already been encoded into an adequate numerical format.\n",
    "\n",
    "The `class` column contains the target labels for the classification task, where we would like to predict whether the electricity price will go UP or DOWN."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>date</th>\n",
       "      <th>day</th>\n",
       "      <th>period</th>\n",
       "      <th>nswprice</th>\n",
       "      <th>nswdemand</th>\n",
       "      <th>vicprice</th>\n",
       "      <th>vicdemand</th>\n",
       "      <th>transfer</th>\n",
       "      <th>class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.0</td>\n",
       "      <td>2</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.056443</td>\n",
       "      <td>0.439155</td>\n",
       "      <td>0.003467</td>\n",
       "      <td>0.422915</td>\n",
       "      <td>0.414912</td>\n",
       "      <td>UP</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.0</td>\n",
       "      <td>2</td>\n",
       "      <td>0.021277</td>\n",
       "      <td>0.051699</td>\n",
       "      <td>0.415055</td>\n",
       "      <td>0.003467</td>\n",
       "      <td>0.422915</td>\n",
       "      <td>0.414912</td>\n",
       "      <td>UP</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.0</td>\n",
       "      <td>2</td>\n",
       "      <td>0.042553</td>\n",
       "      <td>0.051489</td>\n",
       "      <td>0.385004</td>\n",
       "      <td>0.003467</td>\n",
       "      <td>0.422915</td>\n",
       "      <td>0.414912</td>\n",
       "      <td>UP</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.0</td>\n",
       "      <td>2</td>\n",
       "      <td>0.063830</td>\n",
       "      <td>0.045485</td>\n",
       "      <td>0.314639</td>\n",
       "      <td>0.003467</td>\n",
       "      <td>0.422915</td>\n",
       "      <td>0.414912</td>\n",
       "      <td>UP</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.0</td>\n",
       "      <td>2</td>\n",
       "      <td>0.085106</td>\n",
       "      <td>0.042482</td>\n",
       "      <td>0.251116</td>\n",
       "      <td>0.003467</td>\n",
       "      <td>0.422915</td>\n",
       "      <td>0.414912</td>\n",
       "      <td>DOWN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0.0</td>\n",
       "      <td>2</td>\n",
       "      <td>0.106383</td>\n",
       "      <td>0.041161</td>\n",
       "      <td>0.207528</td>\n",
       "      <td>0.003467</td>\n",
       "      <td>0.422915</td>\n",
       "      <td>0.414912</td>\n",
       "      <td>DOWN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0.0</td>\n",
       "      <td>2</td>\n",
       "      <td>0.127660</td>\n",
       "      <td>0.041161</td>\n",
       "      <td>0.171824</td>\n",
       "      <td>0.003467</td>\n",
       "      <td>0.422915</td>\n",
       "      <td>0.414912</td>\n",
       "      <td>DOWN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>0.0</td>\n",
       "      <td>2</td>\n",
       "      <td>0.148936</td>\n",
       "      <td>0.041161</td>\n",
       "      <td>0.152782</td>\n",
       "      <td>0.003467</td>\n",
       "      <td>0.422915</td>\n",
       "      <td>0.414912</td>\n",
       "      <td>DOWN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>0.0</td>\n",
       "      <td>2</td>\n",
       "      <td>0.170213</td>\n",
       "      <td>0.041161</td>\n",
       "      <td>0.134930</td>\n",
       "      <td>0.003467</td>\n",
       "      <td>0.422915</td>\n",
       "      <td>0.414912</td>\n",
       "      <td>DOWN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>0.0</td>\n",
       "      <td>2</td>\n",
       "      <td>0.191489</td>\n",
       "      <td>0.041161</td>\n",
       "      <td>0.140583</td>\n",
       "      <td>0.003467</td>\n",
       "      <td>0.422915</td>\n",
       "      <td>0.414912</td>\n",
       "      <td>DOWN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   date  day    period  nswprice  nswdemand  vicprice  vicdemand  transfer  \\\n",
       "0   0.0    2  0.000000  0.056443   0.439155  0.003467   0.422915  0.414912   \n",
       "1   0.0    2  0.021277  0.051699   0.415055  0.003467   0.422915  0.414912   \n",
       "2   0.0    2  0.042553  0.051489   0.385004  0.003467   0.422915  0.414912   \n",
       "3   0.0    2  0.063830  0.045485   0.314639  0.003467   0.422915  0.414912   \n",
       "4   0.0    2  0.085106  0.042482   0.251116  0.003467   0.422915  0.414912   \n",
       "5   0.0    2  0.106383  0.041161   0.207528  0.003467   0.422915  0.414912   \n",
       "6   0.0    2  0.127660  0.041161   0.171824  0.003467   0.422915  0.414912   \n",
       "7   0.0    2  0.148936  0.041161   0.152782  0.003467   0.422915  0.414912   \n",
       "8   0.0    2  0.170213  0.041161   0.134930  0.003467   0.422915  0.414912   \n",
       "9   0.0    2  0.191489  0.041161   0.140583  0.003467   0.422915  0.414912   \n",
       "\n",
       "  class  \n",
       "0    UP  \n",
       "1    UP  \n",
       "2    UP  \n",
       "3    UP  \n",
       "4  DOWN  \n",
       "5  DOWN  \n",
       "6  DOWN  \n",
       "7  DOWN  \n",
       "8  DOWN  \n",
       "9  DOWN  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "elec_df.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exploratory Data Analysis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 45312 entries, 0 to 45311\n",
      "Data columns (total 9 columns):\n",
      " #   Column     Non-Null Count  Dtype  \n",
      "---  ------     --------------  -----  \n",
      " 0   date       45312 non-null  float64\n",
      " 1   day        45312 non-null  int64  \n",
      " 2   period     45312 non-null  float64\n",
      " 3   nswprice   45312 non-null  float64\n",
      " 4   nswdemand  45312 non-null  float64\n",
      " 5   vicprice   45312 non-null  float64\n",
      " 6   vicdemand  45312 non-null  float64\n",
      " 7   transfer   45312 non-null  float64\n",
      " 8   class      45312 non-null  object \n",
      "dtypes: float64(7), int64(1), object(1)\n",
      "memory usage: 3.1+ MB\n"
     ]
    }
   ],
   "source": [
    "elec_df.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From the summary statistics below, we can tell that the maximum value in each column is 1 and the minimum is 0, meaning that the features are already normalized, so we don't need to apply data scaling. This does not apply to the `day` column that has values between 1 and 7 (i.e., Monday to Sunday), but we can leave that column as is. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>date</th>\n",
       "      <th>day</th>\n",
       "      <th>period</th>\n",
       "      <th>nswprice</th>\n",
       "      <th>nswdemand</th>\n",
       "      <th>vicprice</th>\n",
       "      <th>vicdemand</th>\n",
       "      <th>transfer</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>45312.000000</td>\n",
       "      <td>45312.000000</td>\n",
       "      <td>45312.000000</td>\n",
       "      <td>45312.000000</td>\n",
       "      <td>45312.000000</td>\n",
       "      <td>45312.000000</td>\n",
       "      <td>45312.000000</td>\n",
       "      <td>45312.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.499080</td>\n",
       "      <td>4.003178</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.057868</td>\n",
       "      <td>0.425418</td>\n",
       "      <td>0.003467</td>\n",
       "      <td>0.422915</td>\n",
       "      <td>0.500526</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.340308</td>\n",
       "      <td>1.998695</td>\n",
       "      <td>0.294756</td>\n",
       "      <td>0.039991</td>\n",
       "      <td>0.163323</td>\n",
       "      <td>0.010213</td>\n",
       "      <td>0.120965</td>\n",
       "      <td>0.153373</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.031934</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>0.250000</td>\n",
       "      <td>0.035127</td>\n",
       "      <td>0.309134</td>\n",
       "      <td>0.002277</td>\n",
       "      <td>0.372346</td>\n",
       "      <td>0.414912</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.456329</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.048652</td>\n",
       "      <td>0.443693</td>\n",
       "      <td>0.003467</td>\n",
       "      <td>0.422915</td>\n",
       "      <td>0.414912</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.880547</td>\n",
       "      <td>6.000000</td>\n",
       "      <td>0.750000</td>\n",
       "      <td>0.074336</td>\n",
       "      <td>0.536001</td>\n",
       "      <td>0.003467</td>\n",
       "      <td>0.469252</td>\n",
       "      <td>0.605702</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>7.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               date           day        period      nswprice     nswdemand  \\\n",
       "count  45312.000000  45312.000000  45312.000000  45312.000000  45312.000000   \n",
       "mean       0.499080      4.003178      0.500000      0.057868      0.425418   \n",
       "std        0.340308      1.998695      0.294756      0.039991      0.163323   \n",
       "min        0.000000      1.000000      0.000000      0.000000      0.000000   \n",
       "25%        0.031934      2.000000      0.250000      0.035127      0.309134   \n",
       "50%        0.456329      4.000000      0.500000      0.048652      0.443693   \n",
       "75%        0.880547      6.000000      0.750000      0.074336      0.536001   \n",
       "max        1.000000      7.000000      1.000000      1.000000      1.000000   \n",
       "\n",
       "           vicprice     vicdemand      transfer  \n",
       "count  45312.000000  45312.000000  45312.000000  \n",
       "mean       0.003467      0.422915      0.500526  \n",
       "std        0.010213      0.120965      0.153373  \n",
       "min        0.000000      0.000000      0.000000  \n",
       "25%        0.002277      0.372346      0.414912  \n",
       "50%        0.003467      0.422915      0.414912  \n",
       "75%        0.003467      0.469252      0.605702  \n",
       "max        1.000000      1.000000      1.000000  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# summary statistics\n",
    "elec_df.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "date         0\n",
       "day          0\n",
       "period       0\n",
       "nswprice     0\n",
       "nswdemand    0\n",
       "vicprice     0\n",
       "vicdemand    0\n",
       "transfer     0\n",
       "class        0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Checking missing values\n",
    "elec_df.isnull().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's see how many UPs and DOWNs are in the `class` column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Axes: xlabel='class', ylabel='count'>"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZMAAAFzCAYAAAD7WOTjAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAosElEQVR4nO3df1RVZb7H8c8J5YgEJxH5NZLZSkmDW/eiF9EmRQ10RK5jkxWzSG+KNfhjGGX0Wqu01iRTpvbDNY65bEzTqJVazTIZLJXJFDVW3CR/5mDiFcQUDsrQAfHcP7ru2xE140EPP96vtfZa7uf57r2fh0V8evY+P2xut9stAAAM3OTtAQAAWj/CBABgjDABABgjTAAAxggTAIAxwgQAYIwwAQAYI0wAAMY6eHsAbcmFCxd04sQJBQQEyGazeXs4AGDM7Xbr7NmzioiI0E03XXn9QZg0oxMnTigyMtLbwwCAZldaWqru3btfsZ8waUYBAQGSvv+hBwYGenk0AGCuurpakZGR1t+3KyFMmtHFW1uBgYGECYA25cdu3fMAHgBgjDABABgjTAAAxggTAIAxwgQAYIwwAQAYI0wAAMYIEwCAMcIEAGCMMAEAGCNMAADG+GwuoB069lyMt4eAG+jWZ/Ze92uwMgEAGCNMAADGCBMAgDHCBABgjDABABgjTAAAxggTAIAxr4ZJdna2+vfvr4CAAIWEhGjMmDE6ePCgR82ECRNks9k8tgEDBnjUuFwuTZs2TcHBwfL391dKSoqOHz/uUVNZWam0tDQ5HA45HA6lpaWpqqrKo+bYsWMaPXq0/P39FRwcrOnTp6uuru66zB0A2hKvhkl+fr6mTJmigoICbd68WefPn1diYqJqamo86kaMGKGysjJr++ijjzz6MzMztWHDBuXk5Gj79u06d+6ckpOT1dDQYNWkpqaqqKhIubm5ys3NVVFRkdLS0qz+hoYGjRo1SjU1Ndq+fbtycnK0bt06zZw58/r+EACgDbC53W63twdx0alTpxQSEqL8/Hzdd999kr5fmVRVVen999+/7DFOp1PdunXT6tWr9dBDD0mSTpw4ocjISH300UdKSkrS/v371bdvXxUUFCguLk6SVFBQoPj4eB04cEBRUVHatGmTkpOTVVpaqoiICElSTk6OJkyYoIqKCgUGBv7o+Kurq+VwOOR0Oq+pHvAW3gHfvpi8A/5a/661qGcmTqdTkhQUFOTRvm3bNoWEhKh3795KT09XRUWF1VdYWKj6+nolJiZabREREYqOjtaOHTskSTt37pTD4bCCRJIGDBggh8PhURMdHW0FiSQlJSXJ5XKpsLDwsuN1uVyqrq722ACgPWoxYeJ2uzVjxgzde++9io6OttpHjhypNWvWaMuWLVq4cKH27NmjoUOHyuVySZLKy8vl6+urLl26eJwvNDRU5eXlVk1ISEija4aEhHjUhIaGevR36dJFvr6+Vs2lsrOzrWcwDodDkZGRTf8BAEAr1mI+6HHq1Kn68ssvtX37do/2i7euJCk6Olr9+vVTjx49tHHjRo0dO/aK53O73bLZbNb+D/9tUvNDc+bM0YwZM6z96upqAgVAu9QiVibTpk3Thx9+qK1bt6p79+5XrQ0PD1ePHj10+PBhSVJYWJjq6upUWVnpUVdRUWGtNMLCwnTy5MlG5zp16pRHzaUrkMrKStXX1zdasVxkt9sVGBjosQFAe+TVMHG73Zo6darWr1+vLVu2qGfPnj96zOnTp1VaWqrw8HBJUmxsrDp27KjNmzdbNWVlZSouLtbAgQMlSfHx8XI6ndq9e7dVs2vXLjmdTo+a4uJilZWVWTV5eXmy2+2KjY1tlvkCQFvl1dtcU6ZM0dq1a/XBBx8oICDAWhk4HA75+fnp3Llzmjdvnh544AGFh4fr6NGjevLJJxUcHKxf/vKXVu3EiRM1c+ZMde3aVUFBQcrKylJMTIyGDx8uSerTp49GjBih9PR0LVu2TJI0efJkJScnKyoqSpKUmJiovn37Ki0tTQsWLNCZM2eUlZWl9PR0VhwA8CO8ujJZunSpnE6nhgwZovDwcGt75513JEk+Pj7au3ev/uM//kO9e/fW+PHj1bt3b+3cuVMBAQHWeRYvXqwxY8Zo3LhxGjRokDp37qy//vWv8vHxsWrWrFmjmJgYJSYmKjExUf/yL/+i1atXW/0+Pj7auHGjOnXqpEGDBmncuHEaM2aMXnrppRv3AwGAVqpFvc+kteN9JmgteJ9J+9Lu3mcCAGidCBMAgDHCBABgjDABABgjTAAAxggTAIAxwgQAYIwwAQAYI0wAAMYIEwCAMcIEAGCMMAEAGCNMAADGCBMAgDHCBABgjDABABgjTAAAxggTAIAxwgQAYIwwAQAYI0wAAMYIEwCAMcIEAGCMMAEAGCNMAADGCBMAgDHCBABgjDABABgjTAAAxggTAIAxwgQAYIwwAQAYI0wAAMYIEwCAMcIEAGCMMAEAGCNMAADGCBMAgDHCBABgjDABABgjTAAAxggTAIAxwgQAYIwwAQAYI0wAAMYIEwCAMa+GSXZ2tvr376+AgACFhIRozJgxOnjwoEeN2+3WvHnzFBERIT8/Pw0ZMkRfffWVR43L5dK0adMUHBwsf39/paSk6Pjx4x41lZWVSktLk8PhkMPhUFpamqqqqjxqjh07ptGjR8vf31/BwcGaPn266urqrsvcAaAt8WqY5Ofna8qUKSooKNDmzZt1/vx5JSYmqqamxqp58cUXtWjRIi1ZskR79uxRWFiY7r//fp09e9aqyczM1IYNG5STk6Pt27fr3LlzSk5OVkNDg1WTmpqqoqIi5ebmKjc3V0VFRUpLS7P6GxoaNGrUKNXU1Gj79u3KycnRunXrNHPmzBvzwwCAVszmdrvd3h7ERadOnVJISIjy8/N13333ye12KyIiQpmZmZo9e7ak71choaGheuGFF/T444/L6XSqW7duWr16tR566CFJ0okTJxQZGamPPvpISUlJ2r9/v/r27auCggLFxcVJkgoKChQfH68DBw4oKipKmzZtUnJyskpLSxURESFJysnJ0YQJE1RRUaHAwMAfHX91dbUcDoecTuc11QPecuy5GG8PATfQrc/sbfKx1/p3rUU9M3E6nZKkoKAgSVJJSYnKy8uVmJho1djtdg0ePFg7duyQJBUWFqq+vt6jJiIiQtHR0VbNzp075XA4rCCRpAEDBsjhcHjUREdHW0EiSUlJSXK5XCosLLzseF0ul6qrqz02AGiPWkyYuN1uzZgxQ/fee6+io6MlSeXl5ZKk0NBQj9rQ0FCrr7y8XL6+vurSpctVa0JCQhpdMyQkxKPm0ut06dJFvr6+Vs2lsrOzrWcwDodDkZGRP3XaANAmtJgwmTp1qr788ku9/fbbjfpsNpvHvtvtbtR2qUtrLlfflJofmjNnjpxOp7WVlpZedUwA0Fa1iDCZNm2aPvzwQ23dulXdu3e32sPCwiSp0cqgoqLCWkWEhYWprq5OlZWVV605efJko+ueOnXKo+bS61RWVqq+vr7RiuUiu92uwMBAjw0A2iOvhonb7dbUqVO1fv16bdmyRT179vTo79mzp8LCwrR582arra6uTvn5+Ro4cKAkKTY2Vh07dvSoKSsrU3FxsVUTHx8vp9Op3bt3WzW7du2S0+n0qCkuLlZZWZlVk5eXJ7vdrtjY2OafPAC0IR28efEpU6Zo7dq1+uCDDxQQEGCtDBwOh/z8/GSz2ZSZman58+erV69e6tWrl+bPn6/OnTsrNTXVqp04caJmzpyprl27KigoSFlZWYqJidHw4cMlSX369NGIESOUnp6uZcuWSZImT56s5ORkRUVFSZISExPVt29fpaWlacGCBTpz5oyysrKUnp7OigMAfoRXw2Tp0qWSpCFDhni0/+Uvf9GECRMkSbNmzVJtba0yMjJUWVmpuLg45eXlKSAgwKpfvHixOnTooHHjxqm2tlbDhg3TypUr5ePjY9WsWbNG06dPt171lZKSoiVLllj9Pj4+2rhxozIyMjRo0CD5+fkpNTVVL7300nWaPQC0HS3qfSatHe8zQWvB+0zal3b3PhMAQOvk1dtc8BT7+1XeHgJuoMIFj3p7CECzYWUCADBGmAAAjBEmAABjhAkAwBhhAgAwRpgAAIwRJgAAY4QJAMAYYQIAMEaYAACMESYAAGOECQDAGGECADBGmAAAjBEmAABjhAkAwBhhAgAwRpgAAIwRJgAAY4QJAMAYYQIAMEaYAACMESYAAGOECQDAGGECADBGmAAAjBEmAABjhAkAwBhhAgAwRpgAAIwRJgAAY4QJAMAYYQIAMEaYAACMESYAAGOECQDAGGECADBGmAAAjBEmAABjhAkAwBhhAgAwRpgAAIwRJgAAY14Nk7///e8aPXq0IiIiZLPZ9P7773v0T5gwQTabzWMbMGCAR43L5dK0adMUHBwsf39/paSk6Pjx4x41lZWVSktLk8PhkMPhUFpamqqqqjxqjh07ptGjR8vf31/BwcGaPn266urqrse0AaDN8WqY1NTU6O6779aSJUuuWDNixAiVlZVZ20cffeTRn5mZqQ0bNignJ0fbt2/XuXPnlJycrIaGBqsmNTVVRUVFys3NVW5uroqKipSWlmb1NzQ0aNSoUaqpqdH27duVk5OjdevWaebMmc0/aQBogzo05aChQ4dq/fr1uuWWWzzaq6urNWbMGG3ZsuWazjNy5EiNHDnyqjV2u11hYWGX7XM6nVqxYoVWr16t4cOHS5LeeustRUZG6uOPP1ZSUpL279+v3NxcFRQUKC4uTpK0fPlyxcfH6+DBg4qKilJeXp727dun0tJSRURESJIWLlyoCRMm6Pnnn1dgYOA1zQcA2qsmrUy2bdt22VtA3333nT799FPjQV16rZCQEPXu3Vvp6emqqKiw+goLC1VfX6/ExESrLSIiQtHR0dqxY4ckaefOnXI4HFaQSNKAAQPkcDg8aqKjo60gkaSkpCS5XC4VFhZecWwul0vV1dUeGwC0Rz9pZfLll19a/963b5/Ky8ut/YaGBuXm5upnP/tZsw1u5MiRevDBB9WjRw+VlJTo6aef1tChQ1VYWCi73a7y8nL5+vqqS5cuHseFhoZaYysvL1dISEijc4eEhHjUhIaGevR36dJFvr6+HnO8VHZ2tp599lnTaQJAq/eTwuSee+6xHoQPHTq0Ub+fn59ee+21ZhvcQw89ZP07Ojpa/fr1U48ePbRx40aNHTv2ise53W7ZbDZr/4f/Nqm51Jw5czRjxgxrv7q6WpGRkVeeEAC0UT8pTEpKSuR2u3X77bdr9+7d6tatm9Xn6+urkJAQ+fj4NPsgLwoPD1ePHj10+PBhSVJYWJjq6upUWVnpsTqpqKjQwIEDrZqTJ082OtepU6es1UhYWJh27drl0V9ZWan6+vpGK5YfstvtstvtxvMCgNbuJz0z6dGjh2677TZduHDBWiVc3MLDw69rkEjS6dOnVVpaqvDwcElSbGysOnbsqM2bN1s1ZWVlKi4utsIkPj5eTqdTu3fvtmp27dolp9PpUVNcXKyysjKrJi8vT3a7XbGxsdd1TgDQFjTp1VySdOjQIW3btk0VFRW6cOGCR98zzzxzTec4d+6cvv76a2u/pKRERUVFCgoKUlBQkObNm6cHHnhA4eHhOnr0qJ588kkFBwfrl7/8pSTJ4XBo4sSJmjlzprp27aqgoCBlZWUpJibGenVXnz59NGLECKWnp2vZsmWSpMmTJys5OVlRUVGSpMTERPXt21dpaWlasGCBzpw5o6ysLKWnp/NKLgC4Bk0Kk+XLl+s3v/mNgoODFRYW1ujZw7WGyeeff66EhARr/+Lzh/Hjx2vp0qXau3evVq1apaqqKoWHhyshIUHvvPOOAgICrGMWL16sDh06aNy4caqtrdWwYcO0cuVKj1XSmjVrNH36dOtVXykpKR7vbfHx8dHGjRuVkZGhQYMGyc/PT6mpqXrppZea8uMBgHbH5na73T/1oB49eigjI0OzZ8++HmNqtaqrq+VwOOR0Opu0oon9/arrMCq0VIULHvXatY89F+O1a+PGu/WZvU0+9lr/rjXpfSaVlZV68MEHmzw4AEDb0qQwefDBB5WXl9fcYwEAtFJNemZyxx136Omnn1ZBQYFiYmLUsWNHj/7p06c3y+AAAK1Dk8Lk9ddf180336z8/Hzl5+d79NlsNsIEANqZJoVJSUlJc48DANCK8eVYAABjTVqZPPbYY1ftf+ONN5o0GABA69SkMKmsrPTYr6+vV3Fxsaqqqi77AZAAgLatSWGyYcOGRm0XLlxQRkaGbr/9duNBAQBal2Z7ZnLTTTfpd7/7nRYvXtxcpwQAtBLN+gD+yJEjOn/+fHOeEgDQCjTpNtcPvxBK+v5LpMrKyrRx40aNHz++WQYGAGg9mhQmX3zxhcf+TTfdpG7dumnhwoU/+kovAEDb06Qw2bp1a3OPAwDQijX5y7Gk77/69uDBg7LZbOrdu7fH1/gCANqPJj2Ar6mp0WOPPabw8HDdd999+vnPf66IiAhNnDhR//znP5t7jACAFq5JYTJjxgzl5+frr3/9q6qqqlRVVaUPPvhA+fn5mjlzZnOPEQDQwjXpNte6dev03nvvaciQIVbbL37xC/n5+WncuHFaunRpc40PANAKNGll8s9//lOhoaGN2kNCQrjNBQDtUJPCJD4+XnPnztV3331ntdXW1urZZ59VfHx8sw0OANA6NOk218svv6yRI0eqe/fuuvvuu2Wz2VRUVCS73c7X+QJAO9SkMImJidHhw4f11ltv6cCBA3K73Xr44Yf161//Wn5+fs09RgBAC9ekMMnOzlZoaKjS09M92t944w2dOnVKs2fPbpbBAQBahyY9M1m2bJnuvPPORu133XWX/vznPxsPCgDQujQpTMrLyxUeHt6ovVu3biorKzMeFACgdWlSmERGRuqzzz5r1P7ZZ58pIiLCeFAAgNalSc9MJk2apMzMTNXX11tf0/vJJ59o1qxZvAMeANqhJoXJrFmzdObMGWVkZKiurk6S1KlTJ82ePVtz5sxp1gECAFq+JoWJzWbTCy+8oKefflr79++Xn5+fevXqJbvd3tzjAwC0AkYfQX/zzTerf//+zTUWAEAr1azfAQ8AaJ8IEwCAMcIEAGCMMAEAGCNMAADGCBMAgDHCBABgjDABABgjTAAAxggTAIAxwgQAYIwwAQAYI0wAAMYIEwCAMcIEAGDMq2Hy97//XaNHj1ZERIRsNpvef/99j36326158+YpIiJCfn5+GjJkiL766iuPGpfLpWnTpik4OFj+/v5KSUnR8ePHPWoqKyuVlpYmh8Mhh8OhtLQ0VVVVedQcO3ZMo0ePlr+/v4KDgzV9+nTrWyQBAFfn1TCpqanR3XffrSVLlly2/8UXX9SiRYu0ZMkS7dmzR2FhYbr//vt19uxZqyYzM1MbNmxQTk6Otm/frnPnzik5OVkNDQ1WTWpqqoqKipSbm6vc3FwVFRUpLS3N6m9oaNCoUaNUU1Oj7du3KycnR+vWreP77AHgGhl906KpkSNHauTIkZftc7vdevnll/XUU09p7NixkqQ333xToaGhWrt2rR5//HE5nU6tWLFCq1ev1vDhwyVJb731liIjI/Xxxx8rKSlJ+/fvV25urgoKChQXFydJWr58ueLj43Xw4EFFRUUpLy9P+/btU2lpqSIiIiRJCxcu1IQJE/T8888rMDDwBvw0AKD1arHPTEpKSlReXq7ExESrzW63a/DgwdqxY4ckqbCwUPX19R41ERERio6Otmp27twph8NhBYkkDRgwQA6Hw6MmOjraChJJSkpKksvlUmFh4RXH6HK5VF1d7bEBQHvUYsOkvLxckhQaGurRHhoaavWVl5fL19dXXbp0uWpNSEhIo/OHhIR41Fx6nS5dusjX19equZzs7GzrOYzD4VBkZORPnCUAtA0tNkwustlsHvtut7tR26UurblcfVNqLjVnzhw5nU5rKy0tveq4AKCtarFhEhYWJkmNVgYVFRXWKiIsLEx1dXWqrKy8as3Jkycbnf/UqVMeNZdep7KyUvX19Y1WLD9kt9sVGBjosQFAe9Riw6Rnz54KCwvT5s2brba6ujrl5+dr4MCBkqTY2Fh17NjRo6asrEzFxcVWTXx8vJxOp3bv3m3V7Nq1S06n06OmuLhYZWVlVk1eXp7sdrtiY2Ov6zwBoC3w6qu5zp07p6+//traLykpUVFRkYKCgnTrrbcqMzNT8+fPV69evdSrVy/Nnz9fnTt3VmpqqiTJ4XBo4sSJmjlzprp27aqgoCBlZWUpJibGenVXnz59NGLECKWnp2vZsmWSpMmTJys5OVlRUVGSpMTERPXt21dpaWlasGCBzpw5o6ysLKWnp7PaAIBr4NUw+fzzz5WQkGDtz5gxQ5I0fvx4rVy5UrNmzVJtba0yMjJUWVmpuLg45eXlKSAgwDpm8eLF6tChg8aNG6fa2loNGzZMK1eulI+Pj1WzZs0aTZ8+3XrVV0pKisd7W3x8fLRx40ZlZGRo0KBB8vPzU2pqql566aXr/SMAgDbB5na73d4eRFtRXV0th8Mhp9PZpBVN7O9XXYdRoaUqXPCo16597LkYr10bN96tz+xt8rHX+netxT4zAQC0HoQJAMAYYQIAMEaYAACMESYAAGOECQDAGGECADBGmAAAjBEmAABjhAkAwBhhAgAwRpgAAIwRJgAAY4QJAMAYYQIAMEaYAACMESYAAGOECQDAGGECADBGmAAAjBEmAABjhAkAwBhhAgAwRpgAAIwRJgAAY4QJAMAYYQIAMEaYAACMESYAAGOECQDAGGECADBGmAAAjBEmAABjhAkAwBhhAgAwRpgAAIwRJgAAY4QJAMAYYQIAMEaYAACMESYAAGOECQDAGGECADBGmAAAjBEmAABjLTpM5s2bJ5vN5rGFhYVZ/W63W/PmzVNERIT8/Pw0ZMgQffXVVx7ncLlcmjZtmoKDg+Xv76+UlBQdP37co6ayslJpaWlyOBxyOBxKS0tTVVXVjZgiALQJLTpMJOmuu+5SWVmZte3du9fqe/HFF7Vo0SItWbJEe/bsUVhYmO6//36dPXvWqsnMzNSGDRuUk5Oj7du369y5c0pOTlZDQ4NVk5qaqqKiIuXm5io3N1dFRUVKS0u7ofMEgNasg7cH8GM6dOjgsRq5yO126+WXX9ZTTz2lsWPHSpLefPNNhYaGau3atXr88cfldDq1YsUKrV69WsOHD5ckvfXWW4qMjNTHH3+spKQk7d+/X7m5uSooKFBcXJwkafny5YqPj9fBgwcVFRV14yYLAK1Ui1+ZHD58WBEREerZs6cefvhh/eMf/5AklZSUqLy8XImJiVat3W7X4MGDtWPHDklSYWGh6uvrPWoiIiIUHR1t1ezcuVMOh8MKEkkaMGCAHA6HVXMlLpdL1dXVHhsAtEctOkzi4uK0atUq/e1vf9Py5ctVXl6ugQMH6vTp0yovL5ckhYaGehwTGhpq9ZWXl8vX11ddunS5ak1ISEija4eEhFg1V5KdnW09Z3E4HIqMjGzyXAGgNWvRYTJy5Eg98MADiomJ0fDhw7Vx40ZJ39/Oushms3kc43a7G7Vd6tKay9Vfy3nmzJkjp9NpbaWlpT86JwBoi1p0mFzK399fMTExOnz4sPUc5dLVQ0VFhbVaCQsLU11dnSorK69ac/LkyUbXOnXqVKNVz6XsdrsCAwM9NgBoj1pVmLhcLu3fv1/h4eHq2bOnwsLCtHnzZqu/rq5O+fn5GjhwoCQpNjZWHTt29KgpKytTcXGxVRMfHy+n06ndu3dbNbt27ZLT6bRqAABX16JfzZWVlaXRo0fr1ltvVUVFhf7whz+ourpa48ePl81mU2ZmpubPn69evXqpV69emj9/vjp37qzU1FRJksPh0MSJEzVz5kx17dpVQUFBysrKsm6bSVKfPn00YsQIpaena9myZZKkyZMnKzk5mVdyAcA1atFhcvz4cT3yyCP69ttv1a1bNw0YMEAFBQXq0aOHJGnWrFmqra1VRkaGKisrFRcXp7y8PAUEBFjnWLx4sTp06KBx48aptrZWw4YN08qVK+Xj42PVrFmzRtOnT7de9ZWSkqIlS5bc2MkCQCtmc7vdbm8Poq2orq6Ww+GQ0+ls0vOT2N+vug6jQktVuOBRr1372HMxXrs2brxbn9n740VXcK1/11rVMxMAQMtEmAAAjBEmAABjhAkAwBhhAgAwRpgAAIwRJgAAY4QJAMAYYQIAMEaYAACMESYAAGOECQDAGGECADBGmAAAjBEmAABjhAkAwBhhAgAwRpgAAIwRJgAAY4QJAMAYYQIAMEaYAACMESYAAGOECQDAGGECADBGmAAAjBEmAABjhAkAwBhhAgAwRpgAAIwRJgAAY4QJAMAYYQIAMEaYAACMESYAAGOECQDAGGECADBGmAAAjBEmAABjhAkAwBhhAgAwRpgAAIwRJgAAY4QJAMAYYQIAMEaYAACMESaX+NOf/qSePXuqU6dOio2N1aeffurtIQFAi0eY/MA777yjzMxMPfXUU/riiy/085//XCNHjtSxY8e8PTQAaNEIkx9YtGiRJk6cqEmTJqlPnz56+eWXFRkZqaVLl3p7aADQonXw9gBairq6OhUWFuq//uu/PNoTExO1Y8eOyx7jcrnkcrmsfafTKUmqrq5u0hgaXLVNOg6tU1N/T5rD2e8avHZt3Hgmv2sXj3W73VetI0z+z7fffquGhgaFhoZ6tIeGhqq8vPyyx2RnZ+vZZ59t1B4ZGXldxoi2xfHaE94eAtqLbIfxKc6ePSuH48rnIUwuYbPZPPbdbnejtovmzJmjGTNmWPsXLlzQmTNn1LVr1yseA0/V1dWKjIxUaWmpAgMDvT0ctGH8rjWN2+3W2bNnFRERcdU6wuT/BAcHy8fHp9EqpKKiotFq5SK73S673e7Rdsstt1yvIbZpgYGB/AeOG4LftZ/uaiuSi3gA/398fX0VGxurzZs3e7Rv3rxZAwcO9NKoAKB1YGXyAzNmzFBaWpr69eun+Ph4vf766zp27JieeIJ72wBwNYTJDzz00EM6ffq0nnvuOZWVlSk6OlofffSRevTo4e2htVl2u11z585tdLsQaG78rl1fNvePvd4LAIAfwTMTAIAxwgQAYIwwAQAYI0wAAMYIE1x3Q4YMUWZmZqP2999/3/qkgJUrV8pms1lbeHi4xo0bp5KSkhs8WrQUEyZMsH4fOnbsqNDQUN1///164403dOHCBY/aHTt26Be/+IW6dOmiTp06KSYmRgsXLlRDw/9/BtmAAQP0m9/8xuO4pUuXymazacWKFR7tEydOtN5ftm3bNtlsNkVHR3ucT/r+TcorV65sxlm3XoQJWozAwECVlZXpxIkTWrt2rYqKipSSktLoP2C0HyNGjFBZWZmOHj2qTZs2KSEhQb/97W+VnJys8+fPS5I2bNigwYMHq3v37tq6dasOHDig3/72t3r++ef18MMPWx9QmJCQoK1bt3qcf9u2bYqMjLxse0JCgkfbkSNHtGrVqus429aNMEGLYbPZFBYWpvDwcCUkJGju3LkqLi7W119/7e2hwUvsdrvCwsL0s5/9TP/2b/+mJ598Uh988IE2bdqklStXqqamRunp6UpJSdHrr7+ue+65R7fddpsmTZqkN998U++9957effddSd+HycGDB1VWVmadPz8/X3PmzNG2bdusttLSUv3jH/9oFCbTpk3T3Llz9d13392Qubc2hAlaLD8/P0lSfX29l0eClmTo0KG6++67tX79euXl5en06dPKyspqVDd69Gj17t1bb7/9tiRp0KBB6tixoxUc+/btU21trR577DFVV1fr8OHDkqStW7fK19e30ccoZWZm6vz581qyZMn1nWArRZigRTp+/LgWLFig7t27q3fv3t4eDlqYO++8U0ePHtWhQ4ckSX369Lli3cUaf39/9e/f3wqTbdu26d5775XdbtegQYM82uPi4tS5c2ePc3Xu3Flz585Vdna29d1F+H+ECVoMp9Opm2++Wf7+/oqMjFRdXZ3Wr18vX19fbw8NLcylXw1xpQ/yuLQuISHBIzSGDBkiSRo8eLBH+9ChQy97vokTJyo4OFgvvPCC+STaGMIE111gYOBl/0+uqqrK46PAAwICVFRUpL179+rcuXMqLCxU//79b+RQ0Urs379fPXv2tFat+/fvv2zdgQMH1KtXL2s/ISFBhw4d0v/8z/8oPz9fgwcPlvT/YXLs2DGVlJQ0el5yUYcOHfSHP/xBr7zyik6cONHMs2rdCBNcd3feeac+//zzRu179uxRVFSUtX/TTTfpjjvu0O233y5/f/8bOUS0Ilu2bNHevXv1wAMPKDExUUFBQVq4cGGjug8//FCHDx/WI488YrUNHDhQdrtdf/rTn1RbW6vY2FhJUr9+/eR0OrVs2TJ16tRJAwYMuOL1H3zwQd11112X/ZbV9oxPDcZ1l5GRoSVLlmjKlCmaPHmy/Pz8tHnzZq1YsUKrV6/29vDQgrlcLpWXl6uhoUEnT55Ubm6usrOzlZycrEcffVQ+Pj5atmyZHn74YU2ePFlTp05VYGCgPvnkE/3+97/Xr371K40bN846n5+fn+Li4vTaa69p0KBB8vHxkSR17NhR8fHxeu2116zAuZo//vGPSkpKuq5zb21YmeC6u+222/Tpp5/qyJEjSkxMVP/+/bVy5UqtXLlSDz74oLeHhxYsNzdX4eHhuu222zRixAht3bpVr776qj744AMrCH71q19p69atKi0t1X333aeoqCgtWrRITz31lHJychp9hXZCQoLOnj1rPS+5aPDgwTp79uwVb3H90NChQzV06FDrvS7gI+gBAM2AlQkAwBhhAgAwRpgAAIwRJgAAY4QJAMAYYQIAMEaYAACMESZAC3P06FHZbDYVFRV5eyjANSNMAADGCBMAgDHCBPCSCxcu6IUXXtAdd9whu92uW2+9Vc8//3yjuoaGBk2cOFE9e/aUn5+foqKi9Morr3jUbNu2Tf/+7/8uf39/3XLLLRo0aJC++eYbSdJ///d/KyEhQQEBAQoMDFRsbOxlP8UZMMGnBgNeMmfOHC1fvlyLFy/Wvffeq7KyMh04cKBR3YULF9S9e3e9++67Cg4O1o4dOzR58mSFh4dr3LhxOn/+vMaMGaP09HS9/fbbqqur0+7du60POPz1r3+tf/3Xf9XSpUvl4+OjoqIidezY8UZPF20cH/QIeMHZs2fVrVs3LVmyRJMmTfLoO3r0qHr27KkvvvhC99xzz2WPnzJlik6ePKn33ntPZ86cUdeuXbVt2zbry55+KDAwUK+99prGjx9/PaYCSOI2F+AV+/fvl8vl0rBhw66p/s9//rP69eunbt266eabb9by5ct17NgxSVJQUJAmTJigpKQkjR49Wq+88orKysqsY2fMmKFJkyZp+PDh+uMf/6gjR45clzmhfSNMAC/w8/O75tp3331Xv/vd7/TYY48pLy9PRUVF+s///E/V1dVZNX/5y1+0c+dODRw4UO+884569+6tgoICSdK8efP01VdfadSoUdqyZYv69u2rDRs2NPuc0L5xmwvwgu+++05BQUF69dVXf/Q217Rp07Rv3z598sknVs3w4cP17bffXvG9KPHx8erfv79effXVRn2PPPKIampq9OGHHzbrnNC+sTIBvKBTp06aPXu2Zs2apVWrVunIkSMqKCjQihUrGtXecccd+vzzz/W3v/1Nhw4d0tNPP609e/ZY/SUlJZozZ4527typb775Rnl5eTp06JD69Omj2tpaTZ06Vdu2bdM333yjzz77THv27FGfPn1u5HTRDvBqLsBLnn76aXXo0EHPPPOMTpw4ofDwcD3xxBON6p544gkVFRXpoYceks1m0yOPPKKMjAxt2rRJktS5c2cdOHBAb775pk6fPq3w8HBNnTpVjz/+uM6fP6/Tp0/r0Ucf1cmTJxUcHKyxY8fq2WefvdHTRRvHbS4AgDFucwEAjBEmAABjhAkAwBhhAgAwRpgAAIwRJgAAY4QJAMAYYQIAMEaYAACMESYAAGOECQDAGGECADD2v1V45jKnKnw4AAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 400x400 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(4,4))\n",
    "sns.countplot(data=elec_df, x='class')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's also inspect the UPs and DOWNs of the electricity price for each day of the week. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA/YAAAFzCAYAAACU+pIWAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABANElEQVR4nO3de1hVdd7//9eWswpbUTklGpNmGmoOGkKW5wMTUmOTlg1pmofRdEjNbuvXDM2YmE1pA/cw6ZgndKi5J6p7KjxUUoYoMjGpY2ZFiRNbHAc2YrhR3L8/+rrutqgpIoulz8d1retifdZ7L94fWtM1r9Zan21zu91uAQAAAAAAS2phdgMAAAAAAKDhCPYAAAAAAFgYwR4AAAAAAAsj2AMAAAAAYGEEewAAAAAALIxgDwAAAACAhRHsAQAAAACwMII9AAAAAAAW5m12A1Zx+vRpffPNNwoMDJTNZjO7HQAAAADAVc7tduvYsWOKiIhQixbnvy9PsL9I33zzjSIjI81uAwAAAABwjSktLVXHjh3Pe5xgf5ECAwMlffcHDQoKMrkbAAAAAMDVrqqqSpGRkUYePR+C/UU68/h9UFAQwR4AAAAA0GR+6HVwFs8DAAAAAMDCmk2wT0tLk81mU0pKijHmdruVmpqqiIgIBQQEaNCgQdq7d6/H51wul2bNmqX27durVatWSkpK0qFDhzxqKioqlJycLLvdLrvdruTkZFVWVjbBrAAAAAAAuLKaRbAvLCzU8uXL1atXL4/xJUuW6IUXXlBGRoYKCwsVFham4cOH69ixY0ZNSkqKcnJylJ2drW3btqm6ulqJiYmqq6szasaPH6/i4mLl5uYqNzdXxcXFSk5ObrL5AQAAAABwpdjcbrfbzAaqq6v14x//WH/4wx+0cOFC3XLLLVq2bJncbrciIiKUkpKixx9/XNJ3d+dDQ0P17LPPatq0aXI6nerQoYPWrVuncePGSfq/1evffvttjRw5Uvv27VOPHj1UUFCg2NhYSVJBQYHi4uL06aefqlu3bhfVZ1VVlex2u5xOJ+/YAwAAALjquN1unTp1yuMmKa4sLy8veXt7n/cd+ovNoaYvnjdz5kzdeeedGjZsmBYuXGiMl5SUyOFwaMSIEcaYn5+fBg4cqPz8fE2bNk1FRUU6efKkR01ERISio6OVn5+vkSNHavv27bLb7Uaol6T+/fvLbrcrPz//ooM9AAAAAFytamtrVVZWpm+//dbsVq45LVu2VHh4uHx9fRt8DlODfXZ2tv7+97+rsLCw3jGHwyFJCg0N9RgPDQ3V119/bdT4+vqqbdu29WrOfN7hcCgkJKTe+UNCQoyac3G5XHK5XMZ+VVXVRc4KAAAAAKzj9OnTKikpkZeXlyIiIuTr6/uDq7Dj8rndbtXW1urIkSMqKSlR165d1aJFw96WNy3Yl5aW6pe//KU2bdokf3//89adfUG53e4fvMjOrjlX/Q+dJy0tTU8//fQFfw8AAAAAWF1tba1Onz6tyMhItWzZ0ux2rikBAQHy8fHR119/rdra2gtm4wsxbfG8oqIilZeXKyYmRt7e3vL29lZeXp5+//vfy9vb27hTf/Zd9fLycuNYWFiYamtrVVFRccGaw4cP1/v9R44cqfc0wPctWLBATqfT2EpLSy9rvgAAAADQnDX0bjEuT2P83U37Jzd06FDt3r1bxcXFxta3b1898MADKi4u1o9+9COFhYVp8+bNxmdqa2uVl5en+Ph4SVJMTIx8fHw8asrKyrRnzx6jJi4uTk6nUzt37jRqduzYIafTadSci5+fn4KCgjw2AAAAAACaG9MexQ8MDFR0dLTHWKtWrdSuXTtjPCUlRYsWLVLXrl3VtWtXLVq0SC1bttT48eMlSXa7XZMnT9bcuXPVrl07BQcHa968eerZs6eGDRsmSerevbtGjRqlKVOm6KWXXpIkTZ06VYmJiSycBwAAAACwPNNXxb+Q+fPnq6amRjNmzFBFRYViY2O1adMmBQYGGjVLly6Vt7e3xo4dq5qaGg0dOlSrV6+Wl5eXUbN+/XrNnj3bWD0/KSlJGRkZTT4fAAAAAMCl+eqrrxQVFaWPP/5Yt9xyi9ntNEumf4+9VfA99gAAAMCVcfA3Pc1u4Yro9KvdZrdwUU6cOKGSkhJFRUU1ePG2K+lqD/YX+vtfbA5ldQQAAAAAACyMYA8AAAAAMN3p06f17LPPqkuXLvLz81OnTp30zDPP1Kurq6vT5MmTFRUVpYCAAHXr1k0vvviiR83WrVt16623qlWrVmrTpo1uu+02ff3115Kkf/zjHxo8eLACAwMVFBSkmJgY7dq1q0nmeKU063fsAQAAAADXhgULFmjFihVaunSpBgwYoLKyMn366af16k6fPq2OHTvq1VdfVfv27ZWfn6+pU6cqPDxcY8eO1alTp3T33XdrypQp+vOf/6za2lrt3LlTNptNkvTAAw+oT58+yszMlJeXl4qLi+Xj49PU021UBHsAAAA0uavxnWqrvE8NNEfHjh3Tiy++qIyMDE2YMEGSdMMNN2jAgAH66quvPGp9fHz09NNPG/tRUVHKz8/Xq6++qrFjx6qqqkpOp1OJiYm64YYbJH33bWlnHDx4UI899phuuukmSVLXrl2v8OyuPB7FBwAAAACYat++fXK5XBo6dOhF1f/xj39U37591aFDB7Vu3VorVqzQwYMHJUnBwcGaOHGiRo4cqdGjR+vFF19UWVmZ8dk5c+bo4Ycf1rBhw7R48WJ98cUXV2ROTYlgDwAAAAAwVUBAwEXXvvrqq3r00Uc1adIkbdq0ScXFxXrooYdUW1tr1KxatUrbt29XfHy8XnnlFd14440qKCiQJKWmpmrv3r2688479d5776lHjx7Kyclp9Dk1JYI9AAAAAMBUXbt2VUBAgN59990frP3www8VHx+vGTNmqE+fPurSpcs577r36dNHCxYsUH5+vqKjo7Vhwwbj2I033qhHH31UmzZt0pgxY7Rq1apGnU9T4x17AABwxfE+NQDgQvz9/fX4449r/vz58vX11W233aYjR45o79699R7P79Kli9auXauNGzcqKipK69atU2FhoaKioiRJJSUlWr58uZKSkhQREaH9+/frs88+04MPPqiamho99thj+tnPfqaoqCgdOnRIhYWFuueee8yYdqMh2AMAGh0hDgAAXKqnnnpK3t7e+tWvfqVvvvlG4eHhmj59er266dOnq7i4WOPGjZPNZtP999+vGTNm6J133pEktWzZUp9++qnWrFmjo0ePKjw8XI888oimTZumU6dO6ejRo3rwwQd1+PBhtW/fXmPGjPFYjM+KbG632212E1ZQVVUlu90up9OpoKAgs9sBgGaNYI+zcU3gbFwT+L6r8XqQrHNNnDhxQiUlJYqKipK/v7/Z7UiSXN/sNbuFRucXcfM5xy/097/YHMo79gAAAAAAWBjBHgAAAAAACyPYAwAAAABgYQR7AAAAAAAsjGAPAAAAAICFEewBAAAAALAwgj0AAAAAABZGsAcAAAAAwMII9gAAAAAAWJi32Q0AAAAAAJqnmMfWNtnvKnruwUv+zPCfTVTvHjfpd7/5L4/xN3Pf1djJv9SJf+3R2lde19Q5/59xLCykvW67NUYLn3xUUZ06XnbfzQF37AEAAAAAV7WgwNb66uOtKvn7+1qdsUT/2PupfvbQLNXV1ZndWqMg2AMAAAAArmo2m01hIe0VHtpBg267VU/O+YX2fnpAX5QcNLu1RkGwBwAAAABcUwL8/SRJJ0+dMrmTxkGwBwAAAABcMw5949DSzNW6LjxUXX90vdntNAoWzwMAAAAAXNWcVcfUrms/ud3StzU16tOzh1750zL5+vqY3VqjINgDAAAAACwpqHVrOY9V1xuvdB5TUGBrYz+wdSsV5P5FthY2hXZop1YtWzZlm1ccwR4AAAAAYEndukRp4/vb6o0X/WOPbrzhemO/RYsWuiGqUxN21rRMfcc+MzNTvXr1UlBQkIKCghQXF6d33nnHOD5x4kTZbDaPrX///h7ncLlcmjVrltq3b69WrVopKSlJhw4d8qipqKhQcnKy7Ha77Ha7kpOTVVlZ2RRTBAAAAABcIVMn3KcvvyrVL59YqE/2fqoDX3ylP67+s1Znv6ZHpz9kdntNxtRg37FjRy1evFi7du3Srl27NGTIEN11113au3evUTNq1CiVlZUZ29tvv+1xjpSUFOXk5Cg7O1vbtm1TdXW1EhMTPb6PcPz48SouLlZubq5yc3NVXFys5OTkJpsnAAAAAKDxXR95nd7NWaMvvy5V4vipuu3O+7Tu1de1YulC3TN6pNntNRlTH8UfPXq0x/4zzzyjzMxMFRQU6Oabb5Yk+fn5KSws7JyfdzqdWrlypdatW6dhw4ZJkrKyshQZGaktW7Zo5MiR2rdvn3Jzc1VQUKDY2FhJ0ooVKxQXF6f9+/erW7duV3CGAAAAAGBdRc89aHYLP+jHvW7W/65/6bzHHxx3tx4cd3fTNWSCZvOOfV1dnf7yl7/o+PHjiouLM8a3bt2qkJAQtWnTRgMHDtQzzzyjkJAQSVJRUZFOnjypESNGGPURERGKjo5Wfn6+Ro4cqe3bt8tutxuhXpL69+8vu92u/Pz88wZ7l8sll8tl7FdVVTX2lIGrxsHf9DS7hUbX6Ve7zW4BAAAAuCimf4/97t271bp1a/n5+Wn69OnKyclRjx49JEkJCQlav3693nvvPT3//PMqLCzUkCFDjMDtcDjk6+urtm3bepwzNDRUDofDqDnzHwK+LyQkxKg5l7S0NOOdfLvdrsjIyMaaMgAAAAAAjcb0O/bdunVTcXGxKisr9de//lUTJkxQXl6eevTooXHjxhl10dHR6tu3rzp37qy33npLY8aMOe853W63bDabsf/9n89Xc7YFCxZozpw5xn5VVRXhHgAAAADQ7Jge7H19fdWlSxdJUt++fVVYWKgXX3xRL71U/x2J8PBwde7cWQcOHJAkhYWFqba2VhUVFR537cvLyxUfH2/UHD58uN65jhw5otDQ0PP25efnJz8/v8uaGwAAAAAAV5rpj+Kfze12e7zb/n1Hjx5VaWmpwsPDJUkxMTHy8fHR5s2bjZqysjLt2bPHCPZxcXFyOp3auXOnUbNjxw45nU6jBgAAAAAAqzL1jv0TTzyhhIQERUZG6tixY8rOztbWrVuVm5ur6upqpaam6p577lF4eLi++uorPfHEE2rfvr1++tOfSpLsdrsmT56suXPnql27dgoODta8efPUs2dPY5X87t27a9SoUZoyZYrxFMDUqVOVmJjIivgAAAAAAMszNdgfPnxYycnJKisrk91uV69evZSbm6vhw4erpqZGu3fv1tq1a1VZWanw8HANHjxYr7zyigIDA41zLF26VN7e3ho7dqxqamo0dOhQrV69Wl5eXkbN+vXrNXv2bGP1/KSkJGVkZDT5fAEAAAAAaGymBvuVK1ee91hAQIA2btz4g+fw9/dXenq60tPTz1sTHBysrKysBvUIAAAAAEBz1uzesQcAAAAAABePYA8AAAAAgIWZ/nV3AAAAAIDm6eBvejbZ7+r0q92X/JmHU55U1l/ekCR5e3sruE2QorvfqLF3/0QPjr1bLVr8373s7YUfa/Hvl2tH0T9Uc+KEukR1VvK9d2nWlGRjjbY7Eserd/RNSl/8K+Nzy9dka/YTC5X5u6f10P33GOPT5j6l/Qe+1NY31ysvf6dG3jtJPbp1UeHmv3qs+RbaPU7PpT6uKY/efMnzu1jcsQcAAAAAWNaIwQP01cdbtb9go97I+qMGxt+qeb9arJ9OmKlTp05Jkt54Z4uG/+whXRceqo2vvqx/5P2vZk5+QM+mL1fyLx6T2+2WJN1xWz/l5Rd6nP+D7YXqGBGmvPyd9cbviL/VY+zLr0uV9T9vXsHZnhvBHgAAAABgWX6+vgoLaa/rwkPVp2cPPT57qv7ycro2vveh1r76uo5/+61mPJaqxBGD9IclqeodfZOuj7xOk8b/TH9aukivvbVJ//NmriRpYPyt+uyLEpUdPmKc/8OCXZr/yMP6MH+XMVb6rzKVfH1IA88K9jMeGq/f/u6/deKEq2km///wKD4uWVM+jtNUGvLYDwAAAIDmafCAWPXq0U1vvPOu2rVto6MVlUqZNrFe3Z0jBqnrj67Xq2+8o3vvSlB8vz7y8fHWB9sLNe7un2jfZ1+o5oRLE+4boycXLdXnX36tLj/qrLz8nfL19VFcv1s8zvfIw8n682t/U+bqDXp0+kNNM1lxxx4AAAAAcBXq1iVKX5f+Swe+/FqSdFPXH5237sCXX0mSWrVsqZje0frg/z2O/0F+oeL79ZGfn6/i+vVR3vb/N769UP369FLLgACPc7UM8NcTj/5Cz6X/Sc6qY1doZvUR7AEAAAAAVx232y2bzeaxfzF1A+Nv1QffC/B3xPeTJN3ev+/3xndp0G231j+ZpIfuH6N2wW31u/9e2SjzuBgEewAAAADAVefTz7/U9ZHXqeuPOn+3f+DLc9bt/7xEXaI6G/sD42/VgS+/0r/KDuvDgkLd3v//gv2H2wt18F9l+upg/ffrz/D29lbq/Fn675Xr9Y2jvJFndW4EewAAAADAVeX9bTu0Z98B3X3ncA0bGK/gNna9uHxNvbq/bXpfn5d8rbF3JRhjcX1vkZ+fr15ak62aEy79uFcPSVJM75vlrDqmP617Vf7+for9ce/z/v57Ro9Uj2436JkXMht/cufA4nkAAAAAAMty1dbKUf5v1dXVqfzfR7Xp/W16LuNP+smwgfr5z5Lk5eWljGd/reQZj2nG/FT9YuL9Cgxsrfe3FeiJhc9rzJ0j9LOkUcb5AgL81a9PL2Wu2qC4fn2M76T38fFRbExvZa7aoP4x34X/C/ntgkc1+oFpV3TuZxDsAQAAAACWten9bbq+zyB5e3urrT1IPXvcqOd/u0DJ996lFi2+e0h9TOIIhXRopyXpKzTsnomqOXFCN1zfSY/PmqpZU5I93rGXpIHx/bStYJfuiOvnMX57/35678OC8z6G/32DB8Rq0G23aktefqPN9XwI9gAAAACAc2ruXwv9p2XP6E/Lnrmo2gGxMRoQG3NRtU/Nnamn5s6sN74gZZoWpNS/Cz8w/lad+NeeeuN/27D8on7f5eIdewAAAAAALIxgDwAAAACAhRHsAQAAAACwMII9AAAAAAAWRrAHAAAAAMDCCPYAAAAAALndbrNbuCY1xt+dYA8AAAAA1zAfHx9J0rfffmtyJ9emM3/3M/8cGoLvsQcAAACAa5iXl5fatGmj8vJySVLLli1ls9lM7an21GlTf/+V4D5xwnPf7da3336r8vJytWnTRl5eXg0+N8EeAAAAAK5xYWFhkmSEe7OdqmwefTQm7+Pnjt9t2rQx/v4NPvdlfRoAAAAAYHk2m03h4eEKCQnRyZMnzW5H3/z3L81uodFFzHyz3piPj89l3ak/g2APAAAAAJD03WP5jRE0L5f38TKzW2h0/v7+V+zcBPsrLOaxtWa30OhyAs3uAAAAAABwBqviAwAAAABgYaYG+8zMTPXq1UtBQUEKCgpSXFyc3nnnHeO42+1WamqqIiIiFBAQoEGDBmnv3r0e53C5XJo1a5bat2+vVq1aKSkpSYcOHfKoqaioUHJysux2u+x2u5KTk1VZWdkUUwQAAAAA4IoyNdh37NhRixcv1q5du7Rr1y4NGTJEd911lxHelyxZohdeeEEZGRkqLCxUWFiYhg8frmPHjhnnSElJUU5OjrKzs7Vt2zZVV1crMTFRdXV1Rs348eNVXFys3Nxc5ebmqri4WMnJyU0+XwAAAAAAGpup79iPHj3aY/+ZZ55RZmamCgoK1KNHDy1btkxPPvmkxowZI0las2aNQkNDtWHDBk2bNk1Op1MrV67UunXrNGzYMElSVlaWIiMjtWXLFo0cOVL79u1Tbm6uCgoKFBsbK0lasWKF4uLitH//fnXr1q1pJw0AAAAAQCNqNu/Y19XVKTs7W8ePH1dcXJxKSkrkcDg0YsQIo8bPz08DBw5Ufn6+JKmoqEgnT570qImIiFB0dLRRs337dtntdiPUS1L//v1lt9uNmnNxuVyqqqry2AAAAAAAaG5MD/a7d+9W69at5efnp+nTpysnJ0c9evSQw+GQJIWGhnrUh4aGGsccDod8fX3Vtm3bC9aEhITU+70hISFGzbmkpaUZ7+Tb7XZFRkZe1jwBAAAAALgSTA/23bp1U3FxsQoKCvSLX/xCEyZM0D//+U/juM1m86h3u931xs52ds256n/oPAsWLJDT6TS20tLSi50SAAAAAABNxvRg7+vrqy5duqhv375KS0tT79699eKLLyosLEyS6t1VLy8vN+7ih4WFqba2VhUVFResOXz4cL3fe+TIkXpPA3yfn5+fsVr/mQ0AAAAAgObG1MXzzsXtdsvlcikqKkphYWHavHmz+vTpI0mqra1VXl6enn32WUlSTEyMfHx8tHnzZo0dO1aSVFZWpj179mjJkiWSpLi4ODmdTu3cuVO33nqrJGnHjh1yOp2Kj483YYa41sU8ttbsFhpdTqDZHQAAAADXLlOD/RNPPKGEhARFRkbq2LFjys7O1tatW5WbmyubzaaUlBQtWrRIXbt2VdeuXbVo0SK1bNlS48ePlyTZ7XZNnjxZc+fOVbt27RQcHKx58+apZ8+exir53bt316hRozRlyhS99NJLkqSpU6cqMTGRFfEBAAAAAJZnarA/fPiwkpOTVVZWJrvdrl69eik3N1fDhw+XJM2fP181NTWaMWOGKioqFBsbq02bNikw8P9uDy5dulTe3t4aO3asampqNHToUK1evVpeXl5Gzfr16zV79mxj9fykpCRlZGQ07WQBAAAAALgCTA32K1euvOBxm82m1NRUpaamnrfG399f6enpSk9PP29NcHCwsrKyGtomAAAAAADNlumL5wEAAAAAgIYj2AMAAAAAYGEEewAAAAAALIxgDwAAAACAhRHsAQAAAACwMII9AAAAAAAWRrAHAAAAAMDCCPYAAAAAAFgYwR4AAAAAAAsj2AMAAAAAYGEEewAAAAAALIxgDwAAAACAhRHsAQAAAACwMII9AAAAAAAWRrAHAAAAAMDCCPYAAAAAAFgYwR4AAAAAAAvzNrsBALjWxTy21uwWGl1OoNkdAABw7eD/S4A79gAAAAAAWBh37AEAaGa48wIAAC4Fd+wBAAAAALAwgj0AAAAAABZGsAcAAAAAwMJ4xx4AAKAZuxrXXJBYdwEAGhN37AEAAAAAsDCCPQAAAAAAFkawBwAAAADAwkx9xz4tLU2vvfaaPv30UwUEBCg+Pl7PPvusunXrZtRMnDhRa9as8fhcbGysCgoKjH2Xy6V58+bpz3/+s2pqajR06FD94Q9/UMeOHY2aiooKzZ49W2+++aYkKSkpSenp6WrTps2VnSQAAADQiK7GdRdYcwG4PKbesc/Ly9PMmTNVUFCgzZs369SpUxoxYoSOHz/uUTdq1CiVlZUZ29tvv+1xPCUlRTk5OcrOzta2bdtUXV2txMRE1dXVGTXjx49XcXGxcnNzlZubq+LiYiUnJzfJPAEAAAAAuFJMvWOfm5vrsb9q1SqFhISoqKhId9xxhzHu5+ensLCwc57D6XRq5cqVWrdunYYNGyZJysrKUmRkpLZs2aKRI0dq3759ys3NVUFBgWJjYyVJK1asUFxcnPbv3+/xhAAAAAAAAFbSrN6xdzqdkqTg4GCP8a1btyokJEQ33nijpkyZovLycuNYUVGRTp48qREjRhhjERERio6OVn5+viRp+/btstvtRqiXpP79+8tutxs1Z3O5XKqqqvLYAAAAAABobppNsHe73ZozZ44GDBig6OhoYzwhIUHr16/Xe++9p+eff16FhYUaMmSIXC6XJMnhcMjX11dt27b1OF9oaKgcDodRExISUu93hoSEGDVnS0tLk91uN7bIyMjGmioAAAAAAI3G1Efxv++RRx7RJ598om3btnmMjxs3zvg5Ojpaffv2VefOnfXWW29pzJgx5z2f2+2WzWYz9r//8/lqvm/BggWaM2eOsV9VVUW4BwAAAAA0O83ijv2sWbP05ptv6v333/dYyf5cwsPD1blzZx04cECSFBYWptraWlVUVHjUlZeXKzQ01Kg5fPhwvXMdOXLEqDmbn5+fgoKCPDYAAAAAAJobU4O92+3WI488otdee03vvfeeoqKifvAzR48eVWlpqcLDwyVJMTEx8vHx0ebNm42asrIy7dmzR/Hx8ZKkuLg4OZ1O7dy506jZsWOHnE6nUQMAAAAAgBWZ+ij+zJkztWHDBr3xxhsKDAw03ne32+0KCAhQdXW1UlNTdc899yg8PFxfffWVnnjiCbVv314//elPjdrJkydr7ty5ateunYKDgzVv3jz17NnTWCW/e/fuGjVqlKZMmaKXXnpJkjR16lQlJiayIj4AAAAAwNJMDfaZmZmSpEGDBnmMr1q1ShMnTpSXl5d2796ttWvXqrKyUuHh4Ro8eLBeeeUVBQYGGvVLly6Vt7e3xo4dq5qaGg0dOlSrV6+Wl5eXUbN+/XrNnj3bWD0/KSlJGRkZV36SAAAAAABcQaYGe7fbfcHjAQEB2rhx4w+ex9/fX+np6UpPTz9vTXBwsLKysi65RwAAAAAAmrNmsXgeAAAAAABoGII9AAAAAAAWRrAHAAAAAMDCCPYAAAAAAFgYwR4AAAAAAAsj2AMAAAAAYGEEewAAAAAALIxgDwAAAACAhRHsAQAAAACwsAYF+yFDhqiysrLeeFVVlYYMGXK5PQEAAAAAgIvUoGC/detW1dbW1hs/ceKEPvzww8tuCgAAAAAAXBzvSyn+5JNPjJ//+c9/yuFwGPt1dXXKzc3Vdddd13jdAQAAAACAC7qkYH/LLbfIZrPJZrOd85H7gIAApaenN1pzAAAAAADgwi4p2JeUlMjtdutHP/qRdu7cqQ4dOhjHfH19FRISIi8vr0ZvEgAAAAAAnNslBfvOnTtLkk6fPn1FmgEAAAAAAJfmkoL993322WfaunWrysvL6wX9X/3qV5fdGAAAAAAA+GENCvYrVqzQL37xC7Vv315hYWGy2WzGMZvNRrAHAAAAAKCJNCjYL1y4UM8884wef/zxxu4HAAAAAABcggZ9j31FRYXuvffexu4FAAAAAABcogYF+3vvvVebNm1q7F4AAAAAAMAlatCj+F26dNFTTz2lgoIC9ezZUz4+Ph7HZ8+e3SjNAQAAAACAC2tQsF++fLlat26tvLw85eXleRyz2WwEewAAAAAAmkiDgn1JSUlj9wEAAAAAABqgQe/YAwAAAACA5qFBd+wnTZp0weMvv/xyg5oBAAAAAACXpkHBvqKiwmP/5MmT2rNnjyorKzVkyJBGaQwAAAAAAPywBj2Kn5OT47H97W9/05dffqn77rtP/fv3v+jzpKWlqV+/fgoMDFRISIjuvvtu7d+/36PG7XYrNTVVERERCggI0KBBg7R3716PGpfLpVmzZql9+/Zq1aqVkpKSdOjQIY+aiooKJScny263y263Kzk5WZWVlQ2ZPgAAAAAAzUajvWPfokULPfroo1q6dOlFfyYvL08zZ85UQUGBNm/erFOnTmnEiBE6fvy4UbNkyRK98MILysjIUGFhocLCwjR8+HAdO3bMqElJSVFOTo6ys7O1bds2VVdXKzExUXV1dUbN+PHjVVxcrNzcXOXm5qq4uFjJycmNM3kAAAAAAEzSoEfxz+eLL77QqVOnLro+NzfXY3/VqlUKCQlRUVGR7rjjDrndbi1btkxPPvmkxowZI0las2aNQkNDtWHDBk2bNk1Op1MrV67UunXrNGzYMElSVlaWIiMjtWXLFo0cOVL79u1Tbm6uCgoKFBsbK0lasWKF4uLitH//fnXr1q2R/gIAAAAAADStBgX7OXPmeOy73W6VlZXprbfe0oQJExrcjNPplCQFBwdL+u5r9RwOh0aMGGHU+Pn5aeDAgcrPz9e0adNUVFSkkydPetREREQoOjpa+fn5GjlypLZv3y673W6Eeknq37+/7Ha78vPzzxnsXS6XXC6XsV9VVdXgeQEAAAAAcKU0KNh//PHHHvstWrRQhw4d9Pzzz//givnn43a7NWfOHA0YMEDR0dGSJIfDIUkKDQ31qA0NDdXXX39t1Pj6+qpt27b1as583uFwKCQkpN7vDAkJMWrOlpaWpqeffrpBcwEAAAAAoKk0KNi///77jd2HHnnkEX3yySfatm1bvWM2m81j3+121xs729k156q/0HkWLFjg8WRCVVWVIiMjL/g7AQAAAABoapf1jv2RI0e0f/9+2Ww23XjjjerQoUODzjNr1iy9+eab+uCDD9SxY0djPCwsTNJ3d9zDw8ON8fLycuMuflhYmGpra1VRUeFx1768vFzx8fFGzeHDh8/Z/9lPA5zh5+cnPz+/Bs0HAAAAAICm0qBV8Y8fP65JkyYpPDxcd9xxh26//XZFRERo8uTJ+vbbby/6PG63W4888ohee+01vffee4qKivI4HhUVpbCwMG3evNkYq62tVV5enhHaY2Ji5OPj41FTVlamPXv2GDVxcXFyOp3auXOnUbNjxw45nU6jBgAAAAAAK2pQsJ8zZ47y8vL0v//7v6qsrFRlZaXeeOMN5eXlae7cuRd9npkzZyorK0sbNmxQYGCgHA6HHA6HampqJH33+HxKSooWLVqknJwc7dmzRxMnTlTLli01fvx4SZLdbtfkyZM1d+5cvfvuu/r444/185//XD179jRWye/evbtGjRqlKVOmqKCgQAUFBZoyZYoSExNZER8AAAAAYGkNehT/r3/9q/7nf/5HgwYNMsZ+8pOfKCAgQGPHjlVmZuZFnedM3ffPI333tXcTJ06UJM2fP181NTWaMWOGKioqFBsbq02bNikwMNCoX7p0qby9vTV27FjV1NRo6NChWr16tby8vIya9evXa/bs2cbq+UlJScrIyGjA7AEAAAAAaD4aFOy//fbbc76bHhIScsmP4v8Qm82m1NRUpaamnrfG399f6enpSk9PP29NcHCwsrKyLro3AAAAAACsoEGP4sfFxenXv/61Tpw4YYzV1NTo6aefVlxcXKM1BwAAAAAALqxBd+yXLVumhIQEdezYUb1795bNZlNxcbH8/Py0adOmxu4RAAAAAACcR4OCfc+ePXXgwAFlZWXp008/ldvt1n333acHHnhAAQEBjd0jAAAAAAA4jwYF+7S0NIWGhmrKlCke4y+//LKOHDmixx9/vFGaAwAAAAAAF9agd+xfeukl3XTTTfXGb775Zv3xj3+87KYAAAAAAMDFaVCwdzgcCg8PrzfeoUMHlZWVXXZTAAAAAADg4jQo2EdGRuqjjz6qN/7RRx8pIiLispsCAAAAAAAXp0Hv2D/88MNKSUnRyZMnNWTIEEnSu+++q/nz52vu3LmN2iAAAAAAADi/BgX7+fPn6z//+Y9mzJih2tpaSZK/v78ef/xxLViwoFEbBAAAAAAA59egYG+z2fTss8/qqaee0r59+xQQEKCuXbvKz8+vsfsDAAAAAAAX0KBgf0br1q3Vr1+/xuoFAAAAAABcogYtngcAAAAAAJoHgj0AAAAAABZGsAcAAAAAwMII9gAAAAAAWBjBHgAAAAAACyPYAwAAAABgYQR7AAAAAAAsjGAPAAAAAICFEewBAAAAALAwgj0AAAAAABZGsAcAAAAAwMII9gAAAAAAWBjBHgAAAAAACyPYAwAAAABgYQR7AAAAAAAszNRg/8EHH2j06NGKiIiQzWbT66+/7nF84sSJstlsHlv//v09alwul2bNmqX27durVatWSkpK0qFDhzxqKioqlJycLLvdLrvdruTkZFVWVl7h2QEAAAAAcOWZGuyPHz+u3r17KyMj47w1o0aNUllZmbG9/fbbHsdTUlKUk5Oj7Oxsbdu2TdXV1UpMTFRdXZ1RM378eBUXFys3N1e5ubkqLi5WcnLyFZsXAAAAAABNxdvMX56QkKCEhIQL1vj5+SksLOycx5xOp1auXKl169Zp2LBhkqSsrCxFRkZqy5YtGjlypPbt26fc3FwVFBQoNjZWkrRixQrFxcVp//796tatW+NOCgAAAACAJtTs37HfunWrQkJCdOONN2rKlCkqLy83jhUVFenkyZMaMWKEMRYREaHo6Gjl5+dLkrZv3y673W6Eeknq37+/7Ha7UQMAAAAAgFWZesf+hyQkJOjee+9V586dVVJSoqeeekpDhgxRUVGR/Pz85HA45Ovrq7Zt23p8LjQ0VA6HQ5LkcDgUEhJS79whISFGzbm4XC65XC5jv6qqqpFmBQAAAABA42nWwX7cuHHGz9HR0erbt686d+6st956S2PGjDnv59xut2w2m7H//Z/PV3O2tLQ0Pf300w3sHAAAAACAptHsH8X/vvDwcHXu3FkHDhyQJIWFham2tlYVFRUedeXl5QoNDTVqDh8+XO9cR44cMWrOZcGCBXI6ncZWWlraiDMBAAAAAKBxWCrYHz16VKWlpQoPD5ckxcTEyMfHR5s3bzZqysrKtGfPHsXHx0uS4uLi5HQ6tXPnTqNmx44dcjqdRs25+Pn5KSgoyGMDAAAAAKC5MfVR/Orqan3++efGfklJiYqLixUcHKzg4GClpqbqnnvuUXh4uL766is98cQTat++vX76059Kkux2uyZPnqy5c+eqXbt2Cg4O1rx589SzZ09jlfzu3btr1KhRmjJlil566SVJ0tSpU5WYmMiK+AAAAAAAyzM12O/atUuDBw829ufMmSNJmjBhgjIzM7V7926tXbtWlZWVCg8P1+DBg/XKK68oMDDQ+MzSpUvl7e2tsWPHqqamRkOHDtXq1avl5eVl1Kxfv16zZ882Vs9PSkpSRkZGE80SAAAAAIArx9RgP2jQILnd7vMe37hx4w+ew9/fX+np6UpPTz9vTXBwsLKyshrUIwAAAAAAzZml3rEHAAAAAACeCPYAAAAAAFgYwR4AAAAAAAsj2AMAAAAAYGEEewAAAAAALIxgDwAAAACAhRHsAQAAAACwMII9AAAAAAAWRrAHAAAAAMDCCPYAAAAAAFgYwR4AAAAAAAsj2AMAAAAAYGEEewAAAAAALIxgDwAAAACAhRHsAQAAAACwMII9AAAAAAAWRrAHAAAAAMDCCPYAAAAAAFgYwR4AAAAAAAsj2AMAAAAAYGEEewAAAAAALIxgDwAAAACAhRHsAQAAAACwMII9AAAAAAAWRrAHAAAAAMDCCPYAAAAAAFgYwR4AAAAAAAszNdh/8MEHGj16tCIiImSz2fT66697HHe73UpNTVVERIQCAgI0aNAg7d2716PG5XJp1qxZat++vVq1aqWkpCQdOnTIo6aiokLJycmy2+2y2+1KTk5WZWXlFZ4dAAAAAABXnqnB/vjx4+rdu7cyMjLOeXzJkiV64YUXlJGRocLCQoWFhWn48OE6duyYUZOSkqKcnBxlZ2dr27Ztqq6uVmJiourq6oya8ePHq7i4WLm5ucrNzVVxcbGSk5Ov+PwAAAAAALjSvM385QkJCUpISDjnMbfbrWXLlunJJ5/UmDFjJElr1qxRaGioNmzYoGnTpsnpdGrlypVat26dhg0bJknKyspSZGSktmzZopEjR2rfvn3Kzc1VQUGBYmNjJUkrVqxQXFyc9u/fr27dujXNZAEAAAAAuAKa7Tv2JSUlcjgcGjFihDHm5+engQMHKj8/X5JUVFSkkydPetREREQoOjraqNm+fbvsdrsR6iWpf//+stvtRs25uFwuVVVVeWwAAAAAADQ3zTbYOxwOSVJoaKjHeGhoqHHM4XDI19dXbdu2vWBNSEhIvfOHhIQYNeeSlpZmvJNvt9sVGRl5WfMBAAAAAOBKaLbB/gybzeax73a7642d7eyac9X/0HkWLFggp9NpbKWlpZfYOQAAAAAAV16zDfZhYWGSVO+uenl5uXEXPywsTLW1taqoqLhgzeHDh+ud/8iRI/WeBvg+Pz8/BQUFeWwAAAAAADQ3zTbYR0VFKSwsTJs3bzbGamtrlZeXp/j4eElSTEyMfHx8PGrKysq0Z88eoyYuLk5Op1M7d+40anbs2CGn02nUAAAAAABgVaauil9dXa3PP//c2C8pKVFxcbGCg4PVqVMnpaSkaNGiReratau6du2qRYsWqWXLlho/frwkyW63a/LkyZo7d67atWun4OBgzZs3Tz179jRWye/evbtGjRqlKVOm6KWXXpIkTZ06VYmJiayIDwAAAACwPFOD/a5duzR48GBjf86cOZKkCRMmaPXq1Zo/f75qamo0Y8YMVVRUKDY2Vps2bVJgYKDxmaVLl8rb21tjx45VTU2Nhg4dqtWrV8vLy8uoWb9+vWbPnm2snp+UlKSMjIwmmiUAAAAAAFeOqcF+0KBBcrvd5z1us9mUmpqq1NTU89b4+/srPT1d6enp560JDg5WVlbW5bQKAAAAAECz1GzfsQcAAAAAAD+MYA8AAAAAgIUR7AEAAAAAsDCCPQAAAAAAFkawBwAAAADAwgj2AAAAAABYGMEeAAAAAAALI9gDAAAAAGBhBHsAAAAAACyMYA8AAAAAgIUR7AEAAAAAsDCCPQAAAAAAFkawBwAAAADAwgj2AAAAAABYGMEeAAAAAAALI9gDAAAAAGBhBHsAAAAAACyMYA8AAAAAgIUR7AEAAAAAsDCCPQAAAAAAFkawBwAAAADAwgj2AAAAAABYGMEeAAAAAAALI9gDAAAAAGBhBHsAAAAAACyMYA8AAAAAgIU162Cfmpoqm83msYWFhRnH3W63UlNTFRERoYCAAA0aNEh79+71OIfL5dKsWbPUvn17tWrVSklJSTp06FBTTwUAAAAAgCuiWQd7Sbr55ptVVlZmbLt37zaOLVmyRC+88IIyMjJUWFiosLAwDR8+XMeOHTNqUlJSlJOTo+zsbG3btk3V1dVKTExUXV2dGdMBAAAAAKBReZvdwA/x9vb2uEt/htvt1rJly/Tkk09qzJgxkqQ1a9YoNDRUGzZs0LRp0+R0OrVy5UqtW7dOw4YNkyRlZWUpMjJSW7Zs0ciRI5t0LgAAAAAANLZmf8f+wIEDioiIUFRUlO677z59+eWXkqSSkhI5HA6NGDHCqPXz89PAgQOVn58vSSoqKtLJkyc9aiIiIhQdHW3UnI/L5VJVVZXHBgAAAABAc9Osg31sbKzWrl2rjRs3asWKFXI4HIqPj9fRo0flcDgkSaGhoR6fCQ0NNY45HA75+vqqbdu25605n7S0NNntdmOLjIxsxJkBAAAAANA4mnWwT0hI0D333KOePXtq2LBheuuttyR998j9GTabzeMzbre73tjZLqZmwYIFcjqdxlZaWtrAWQAAAAAAcOU062B/tlatWqlnz546cOCA8d792Xfey8vLjbv4YWFhqq2tVUVFxXlrzsfPz09BQUEeGwAAAAAAzY2lgr3L5dK+ffsUHh6uqKgohYWFafPmzcbx2tpa5eXlKT4+XpIUExMjHx8fj5qysjLt2bPHqAEAAAAAwMqa9ar48+bN0+jRo9WpUyeVl5dr4cKFqqqq0oQJE2Sz2ZSSkqJFixapa9eu6tq1qxYtWqSWLVtq/PjxkiS73a7Jkydr7ty5ateunYKDgzVv3jzj0X4AAAAAAKyuWQf7Q4cO6f7779e///1vdejQQf3791dBQYE6d+4sSZo/f75qamo0Y8YMVVRUKDY2Vps2bVJgYKBxjqVLl8rb21tjx45VTU2Nhg4dqtWrV8vLy8usaQEAAAAA0GiadbDPzs6+4HGbzabU1FSlpqaet8bf31/p6elKT09v5O4AAAAAADCfpd6xBwAAAAAAngj2AAAAAABYGMEeAAAAAAALI9gDAAAAAGBhBHsAAAAAACyMYA8AAAAAgIUR7AEAAAAAsDCCPQAAAAAAFkawBwAAAADAwgj2AAAAAABYGMEeAAAAAAALI9gDAAAAAGBhBHsAAAAAACyMYA8AAAAAgIUR7AEAAAAAsDCCPQAAAAAAFkawBwAAAADAwgj2AAAAAABYGMEeAAAAAAALI9gDAAAAAGBhBHsAAAAAACyMYA8AAAAAgIUR7AEAAAAAsDCCPQAAAAAAFkawBwAAAADAwgj2AAAAAABYGMEeAAAAAAALu6aC/R/+8AdFRUXJ399fMTEx+vDDD81uCQAAAACAy3LNBPtXXnlFKSkpevLJJ/Xxxx/r9ttvV0JCgg4ePGh2awAAAAAANNg1E+xfeOEFTZ48WQ8//LC6d++uZcuWKTIyUpmZmWa3BgAAAABAg3mb3UBTqK2tVVFRkf7rv/7LY3zEiBHKz88/52dcLpdcLpex73Q6JUlVVVWX9LvrXDWX2G3zd8ynzuwWGt2l/nO9HFwT1sA1cXm4Ji4P14Q1NNU1cTVeDxLXxOW4Gq+Jq/F6kLgmLsfVeE005Ho48xm3233BOpv7hyquAt98842uu+46ffTRR4qPjzfGFy1apDVr1mj//v31PpOamqqnn366KdsEAAAAAKCe0tJSdezY8bzHr4k79mfYbDaPfbfbXW/sjAULFmjOnDnG/unTp/Wf//xH7dq1O+9nrgVVVVWKjIxUaWmpgoKCzG4HzQDXBM7GNYGzcU3gbFwT+D6uB5yNa+L/uN1uHTt2TBEREResuyaCffv27eXl5SWHw+ExXl5ertDQ0HN+xs/PT35+fh5jbdq0uVItWk5QUNA1/z8yeOKawNm4JnA2rgmcjWsC38f1gLNxTXzHbrf/YM01sXier6+vYmJitHnzZo/xzZs3ezyaDwAAAACA1VwTd+wlac6cOUpOTlbfvn0VFxen5cuX6+DBg5o+fbrZrQEAAAAA0GDXTLAfN26cjh49qt/85jcqKytTdHS03n77bXXu3Nns1izFz89Pv/71r+u9poBrF9cEzsY1gbNxTeBsXBP4Pq4HnI1r4tJdE6viAwAAAABwtbom3rEHAAAAAOBqRbAHAAAAAMDCCPYAAAAAAFgYwR4AAAAAAAsj2OOifPDBBxo9erQiIiJks9n0+uuvm90STJaWlqZ+/fopMDBQISEhuvvuu7V//36z24KJMjMz1atXLwUFBSkoKEhxcXF65513zG4LzURaWppsNptSUlLMbgUmSU1Nlc1m89jCwsLMbgsm+9e//qWf//znateunVq2bKlbbrlFRUVFZrcFk1x//fX1/j1hs9k0c+ZMs1tr9gj2uCjHjx9X7969lZGRYXYraCby8vI0c+ZMFRQUaPPmzTp16pRGjBih48ePm90aTNKxY0ctXrxYu3bt0q5duzRkyBDddddd2rt3r9mtwWSFhYVavny5evXqZXYrMNnNN9+ssrIyY9u9e7fZLcFEFRUVuu222+Tj46N33nlH//znP/X888+rTZs2ZrcGkxQWFnr8O2Lz5s2SpHvvvdfkzpq/a+Z77HF5EhISlJCQYHYbaEZyc3M99letWqWQkBAVFRXpjjvuMKkrmGn06NEe+88884wyMzNVUFCgm2++2aSuYLbq6mo98MADWrFihRYuXGh2OzCZt7c3d+lhePbZZxUZGalVq1YZY9dff715DcF0HTp08NhfvHixbrjhBg0cONCkjqyDO/YAGoXT6ZQkBQcHm9wJmoO6ujplZ2fr+PHjiouLM7sdmGjmzJm68847NWzYMLNbQTNw4MABRUREKCoqSvfdd5++/PJLs1uCid5880317dtX9957r0JCQtSnTx+tWLHC7LbQTNTW1iorK0uTJk2SzWYzu51mj2AP4LK53W7NmTNHAwYMUHR0tNntwES7d+9W69at5efnp+nTpysnJ0c9evQwuy2YJDs7W3//+9+VlpZmditoBmJjY7V27Vpt3LhRK1askMPhUHx8vI4ePWp2azDJl19+qczMTHXt2lUbN27U9OnTNXv2bK1du9bs1tAMvP7666qsrNTEiRPNbsUSeBQfwGV75JFH9Mknn2jbtm1mtwKTdevWTcXFxaqsrNRf//pXTZgwQXl5eYT7a1Bpaal++ctfatOmTfL39ze7HTQD33+lr2fPnoqLi9MNN9ygNWvWaM6cOSZ2BrOcPn1affv21aJFiyRJffr00d69e5WZmakHH3zQ5O5gtpUrVyohIUERERFmt2IJ3LEHcFlmzZqlN998U++//746duxodjswma+vr7p06aK+ffsqLS1NvXv31osvvmh2WzBBUVGRysvLFRMTI29vb3l7eysvL0+///3v5e3trbq6OrNbhMlatWqlnj176sCBA2a3ApOEh4fX+w+/3bt318GDB03qCM3F119/rS1btujhhx82uxXL4I49gAZxu92aNWuWcnJytHXrVkVFRZndEpoht9stl8tldhswwdChQ+uteP7QQw/ppptu0uOPPy4vLy+TOkNz4XK5tG/fPt1+++1mtwKT3HbbbfW+Kvezzz5T586dTeoIzcWZRZnvvPNOs1uxDII9Lkp1dbU+//xzY7+kpETFxcUKDg5Wp06dTOwMZpk5c6Y2bNigN954Q4GBgXI4HJIku92ugIAAk7uDGZ544gklJCQoMjJSx44dU3Z2trZu3VrvGxRwbQgMDKy35karVq3Url071uK4Rs2bN0+jR49Wp06dVF5eroULF6qqqkoTJkwwuzWY5NFHH1V8fLwWLVqksWPHaufOnVq+fLmWL19udmsw0enTp7Vq1SpNmDBB3t7E1YvFXwoXZdeuXRo8eLCxf+ZduAkTJmj16tUmdQUzZWZmSpIGDRrkMb5q1SoWOblGHT58WMnJySorK5PdblevXr2Um5ur4cOHm90agGbg0KFDuv/++/Xvf/9bHTp0UP/+/VVQUMDd2WtYv379lJOTowULFug3v/mNoqKitGzZMj3wwANmtwYTbdmyRQcPHtSkSZPMbsVSbG632212EwAAAAAAoGFYPA8AAAAAAAsj2AMAAAAAYGEEewAAAAAALIxgDwAAAACAhRHsAQAAAACwMII9AAAAAAAWRrAHAAAAAMDCCPYAAKDRDBo0SCkpKWa3AQDANYVgDwAAAACAhRHsAQAAAACwMII9AABokOPHj+vBBx9U69atFR4erueff97jeFZWlvr27avAwECFhYVp/PjxKi8vlyS53W516dJFv/vd7zw+s2fPHrVo0UJffPFFk80DAACrI9gDAIAGeeyxx/T+++8rJydHmzZt0tatW1VUVGQcr62t1W9/+1v94x//0Ouvv66SkhJNnDhRkmSz2TRp0iStWrXK45wvv/yybr/9dt1www1NORUAACzN5na73WY3AQAArKW6ulrt2rXT2rVrNW7cOEnSf/7zH3Xs2FFTp07VsmXL6n2msLBQt956q44dO6bWrVurrKxMkZGRys/P16233qqTJ0/quuuu03PPPacJEyY08YwAALAu7tgDAIBL9sUXX6i2tlZxcXHGWHBwsLp162bsf/zxx7rrrrvUuXNnBQYGatCgQZKkgwcPSpLCw8N155136uWXX5Yk/e1vf9OJEyd07733Nt1EAAC4ChDsAQDAJfuhB/6OHz+uESNGqHXr1srKylJhYaFycnIkffeI/hkPP/ywsrOzVVNTo1WrVmncuHFq2bLlFe0dAICrDcEeAABcsi5dusjHx0cFBQXGWEVFhT777DNJ0qeffqp///vfWrx4sW6//XbddNNNxsJ53/eTn/xErVq1UmZmpt555x1NmjSpyeYAAMDVwtvsBgAAgPW0bt1akydP1mOPPaZ27dopNDRUTz75pFq0+O6eQadOneTr66v09HRNnz5de/bs0W9/+9t65/Hy8tLEiRO1YMECdenSxePRfgAAcHG4Yw8AABrkueee0x133KGkpCQNGzZMAwYMUExMjCSpQ4cOWr16tf7yl7+oR48eWrx4cb2vtjtj8uTJqq2t5W49AAANxKr4AADAVB999JEGDRqkQ4cOKTQ01Ox2AACwHII9AAAwhcvlUmlpqaZOnarw8HCtX7/e7JYAALAkHsUHAACm+POf/6xu3brJ6XRqyZIlZrcDAIBlccceAAAAAAAL4449AAAAAAAWRrAHAAAAAMDCCPYAAAAAAFgYwR4AAAAAAAsj2AMAAAAAYGEEewAAAAAALIxgDwAAAACAhRHsAQAAAACwMII9AAAAAAAW9v8DSGTt65L8bwIAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 1200x400 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(12,4))\n",
    "sns.countplot(data=elec_df, x='day', hue='class')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also plot all histograms of the numerical features, and quickly check the plots."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABNQAAANCCAYAAACu0JFZAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAADvF0lEQVR4nOzdf1yV9f3/8ecRDkcgPAnGryS1Zmah5UQRbelSwCZRcxsVRtqcuWkaUz+W+akwTactc8PlyjltobnPZ2XTMgK3svzgT4ol6of6LLNoIs4Q/NXhCNf3D7/nyiOgHOTX4Tzutxu3Ou/zuq7zenFdnavrxfu6LothGIYAAAAAAAAANEqntk4AAAAAAAAA8CY01AAAAAAAAAAP0FADAAAAAAAAPEBDDQAAAAAAAPAADTUAAAAAAADAAzTUAAAAAAAAAA/QUAMAAAAAAAA8QEMNAAAAAAAA8AANNQAAAAAAAMADNNSAFpKVlSWLxeLxcuvWrdOyZcuaPyEAgM9r6rEJAIDWNmHCBPXs2bNZ19mzZ09NmDChWdcJ30VDDWhnaKgBAAAA8HVPPPGENmzY0NZpAA3yb+sEAAAAAAAAJOn06dMKCgrSdddd19apABfFDDWgGbz11lu65ZZbZLPZ1KtXL/3617+uE/O73/1Ot912m8LDwxUcHKx+/fppyZIlcjqdZsyIESP01ltv6dChQ7JYLOaPS3V1tRYsWKAbbrhBNptNV111lR588EEdPXq0VeoEAHiP5jo2zZ8/X/7+/vryyy/rLP/Tn/5UYWFh+uabb1q0FgBA++C6dcBHH32ksWPHqkuXLrLb7br//vvrnJP8+c9/VkJCgoKDg3XFFVcoOTlZH330kVvMhAkTdMUVV2jv3r1KSkpSSEiIRo4cab534SWf33zzjebMmaNevXopICBAV199taZOnarjx4+7xTmdTs2ePVuRkZEKCgrSrbfeql27djX77wO+jRlqwGX629/+prvuuksJCQlav369ampqtGTJEh05csQt7p///KfS09PNL/9//OMfeuaZZ/S///u/+uMf/yhJeuGFF/TQQw/pn//8Z53pzbW1tbrrrrv0wQcfaPbs2Ro6dKgOHTqkp556SiNGjNCePXsUGBjYanUDANqv5jw2TZ48Wc8884xefPFFLViwwFz266+/1vr16/Xwww+rc+fOrVofAKBt/fCHP1RaWpp+/vOfa9++fXriiSe0f/9+7dy5U1arVQsXLtR//ud/6sEHH9R//ud/qrq6Ws8++6y+973vadeuXbrxxhvNdVVXVys1NVWTJ0/WY489prNnz9b7mYZh6O6779bf/vY3zZkzR9/73vf08ccf66mnntL27du1fft22Ww2SdKkSZP0pz/9SbNmzVJiYqKKi4s1duxYnThxolV+P/ARBoDLEh8fb0RHRxtnzpwxx6qqqozQ0FCjof/EampqDKfTafzpT38y/Pz8jK+//tp8b8yYMUaPHj3qLPPqq68akozXXnvNbXz37t2GJOOFF15onoIAAF6vuY9N48ePN8LDww2Hw2GOLV682OjUqZNx8ODBFqsDANC+PPXUU4Yk45e//KXb+Nq1aw1JRk5OjvHFF18Y/v7+xrRp09xiTpw4YURGRhppaWnm2Pjx4w1Jxh//+Mc6nzV+/Hi386Lc3FxDkrFkyRK3uD//+c+GJOOll14yDMMwDhw4cNEcx48f35TSgTq45BO4DKdOndLu3bs1duxYt7/Oh4SE6M4773SL/eijj5SamqqwsDD5+fnJarXqgQceUE1NjT755JNLftabb76pK6+8UnfeeafOnj1r/txyyy2KjIzUe++919zlAQC8UEscmx555BGVl5frv//7vyWdmzW9YsUKjRkzptmfwAYAaP/GjRvn9jotLU3+/v5699139c477+js2bN64IEH3M5bOnfurOHDh9d73vKjH/3okp/597//XZLqPKXzJz/5iYKDg/W3v/1NkvTuu+9eNEegubA3AZehoqJCtbW1ioyMrPPe+WNffPGFvve976lPnz76zW9+o549e6pz587atWuXpk6dqjNnzlzys44cOaLjx48rICCg3vf//e9/N70QAECH0RLHpgEDBuh73/uefve732ncuHF688039fnnn+vFF19slZoAAO3LhccYf39/hYWF6dixY+btBQYNGlTvsp06uc/rCQoKUpcuXS75mceOHZO/v7+uuuoqt3GLxaLIyEgdO3bMjLtYjkBzoaEGXIauXbvKYrGorKysznvnj73xxhs6deqUXn/9dfXo0cMcLyoqavRndevWTWFhYcrNza33/ZCQkMYnDgDosFrq2DR9+nT95Cc/0Ycffqjly5fr+uuvV2JiYrPnDwBo/8rKynT11Vebr8+ePatjx44pLCxM3bp1kyT95S9/cTu+NOT8h7BdTFhYmM6ePaujR4+6NdUMw1BZWZnZwHM1zRrKEWguXPIJXIbg4GANHjxYr7/+utsTzk6cOKFNmzaZr10HCddNMqVzX/wrV66ss06bzVbvjLWUlBQdO3ZMNTU1iouLq/PTp0+f5iwNAOClWuLYJJ27AfU111yjmTNnasuWLZoyZUqjT4IAAB3L2rVr3V7/13/9l86ePasRI0YoOTlZ/v7++uc//1nveUtcXFyTPtP19M+cnBy38ddee02nTp0y3x8xYsRFcwSaCzPUgMs0f/58jR49WomJiZo5c6Zqamq0ePFiBQcH6+uvv5YkJSYmKiAgQPfdd59mz56tb775RitWrFBFRUWd9fXr10+vv/66VqxYoYEDB6pTp06Ki4vTvffeq7Vr1+oHP/iBHnnkEQ0ePFhWq1WlpaV69913ddddd+mHP/xha5cPAGiHmvvYJEl+fn6aOnWqHn30UQUHB9e5hw0AwHe8/vrr8vf3V2JiovmUz5tvvllpaWkKCAjQ008/rblz5+qzzz7T6NGj1bVrVx05ckS7du1ScHCw5s2b5/FnJiYmKjk5WY8++qiqqqo0bNgw8ymfAwYMUEZGhiSpb9++uv/++7Vs2TJZrVaNGjVKxcXF+vWvf92oS0uBRmvjhyIAHcLGjRuN/v37GwEBAcY111xj/OpXvzKfgOOyadMm4+abbzY6d+5sXH311cZ//Md/GG+//bYhyXj33XfNuK+//tr48Y9/bFx55ZWGxWJxW4fT6TR+/etfm+u54oorjBtuuMGYPHmy8emnn7ZmyQCAdq45j00un3/+uSHJ+PnPf96KlQAA2gvXcaSwsNC48847jSuuuMIICQkx7rvvPuPIkSNusW+88Ybx/e9/3+jSpYths9mMHj16GD/+8Y+NLVu2mDHjx483goOD6/2sC5/yaRiGcebMGePRRx81evToYVitViMqKsr4xS9+YVRUVLjFORwOY+bMmUZ4eLjRuXNnY8iQIcb27duNHj168JRPNBuLYRhGG/bzAAAA4CWys7M1ffp0FRcX66abbmrrdAAArSwrK0vz5s3T0aNHzXulAb6KSz4BAABwUR999JEOHjyop59+WnfddRfNNAAA4PNoqAEAAOCifvjDH6qsrEzf+9739Pvf/76t0wEAAGhzXPIJAAAAAAAAeKBTWycAAAAAAAAAeBMaagAAAAAAAIAHaKgBAAAAAAAAHvDphxLU1tbqX//6l0JCQmSxWNo6HQBoE4Zh6MSJE4qOjlanTvydpS1wPAIAjkftAccjAGj88cinG2r/+te/FBMT09ZpAEC78OWXX6p79+5tnYZP4ngEAN/ieNR2OB4BwLcudTzy6YZaSEiIpHO/pC5duni0rNPpVF5enpKSkmS1WlsivXbBV+qUqLWj8pVaL6fOqqoqxcTEmN+JaH2+fDzy9vwlamgvqKF94Hjk3Zp6POoI+25jUWvH4yt1StTaWI09Hvl0Q801jblLly5NOoEJCgpSly5dOvSO6Ct1StTaUflKrc1RJ5d2tB1fPh55e/4SNbQX1NA+cDzybk09HnWEfbexqLXj8ZU6JWr11KWOR9ycAAAAAAAAAPAADTUAAAAAAADAAzTUAAAAAAAAAA/QUAMAAAAAAAA8QEMNAAAAAAAA8AANNQAAAAAAAMADNNQAAAAAAAAAD/i3dQLeLjbrHTlqLG5jn/9qTBtlAwCAd4nNekdLBrsfT73lONrzsbckSTY/w6yh5JmUNs7q8njr/9f0fOwtt+3gTfuSaz+SvG9fOj93F1cN8E31fYd0NPV917Slhr7n6vvv01PtrdaW0px1tuT2aA6+sk2l1jkeMUMNAAAAAAAA8AAz1AAA6AAu/EujN8zMAQAAALwVM9QAAAAAAAAAD9BQAwAAAAAAADzAJZ8AcBkausEol9sBAAAAQMfFDDUAAAAAAADAAzTUAAAAAAAAAA/QUAMAAAAAAAA8QEMNAAAAAAAA8AANNQAAAAAAAMADNNQAAB3WV199pfvvv19hYWEKCgrSLbfcosLCQvN9wzCUlZWl6OhoBQYGasSIEdq3b5/bOhwOh6ZNm6Zu3bopODhYqampKi0tdYupqKhQRkaG7Ha77Ha7MjIydPz48dYoEQAAAEAboKEGAOiQKioqNGzYMFmtVr399tvav3+/nnvuOV155ZVmzJIlS7R06VItX75cu3fvVmRkpBITE3XixAkzJjMzUxs2bND69eu1bds2nTx5UikpKaqpqTFj0tPTVVRUpNzcXOXm5qqoqEgZGRmtWS4AAACAVuTf1gkAANASFi9erJiYGK1evdoc69mzp/nvhmFo2bJlmjt3rsaOHStJevnllxUREaF169Zp8uTJqqys1KpVq/TKK69o1KhRkqScnBzFxMRoy5YtSk5O1oEDB5Sbm6sdO3YoPj5ekrRy5UolJCSopKREffr0ab2iAQAAALQKZqgBADqkjRs3Ki4uTj/5yU8UHh6uAQMGaOXKleb7Bw8eVFlZmZKSkswxm82m4cOHq6CgQJJUWFgop9PpFhMdHa3Y2FgzZvv27bLb7WYzTZKGDBkiu91uxgAAAADoWJihBgDokD777DOtWLFCM2bM0OOPP65du3Zp+vTpstlseuCBB1RWViZJioiIcFsuIiJChw4dkiSVlZUpICBAXbt2rRPjWr6srEzh4eF1Pj88PNyMuZDD4ZDD4TBfV1VVSZKcTqecTqdHdbribZ2MesfbO1fe5+fvNbn7uedu62R4Te4Xamg/Ov+99szmZ3jtvuTajyTv25fOz90c+/81NCV/b6gZAAAXGmoAgA6ptrZWcXFxWrhwoSRpwIAB2rdvn1asWKEHHnjAjLNYLG7LGYZRZ+xCF8bUF3+x9SxatEjz5s2rM56Xl6egoKCLfnZD5sfVur3evHlzk9bT2ubHuf75bf7ekvuSwe6v58fVek3uDblwP5K8Y3ucvy28bV+6cD+SvGdfqi93l/z8fI/Xd/r06cvIBgCA1kVDDQDQIUVFRenGG290G+vbt69ee+01SVJkZKSkczPMoqKizJjy8nJz1lpkZKSqq6tVUVHhNkutvLxcQ4cONWOOHDlS5/OPHj1aZ/aby5w5czRjxgzzdVVVlWJiYpSUlKQuXbp4VKfT6VR+fr6e2NNJjtpvG3jFWckeraetDHw6V/Pjat3y95bcY7PekXRuRo6rhsInR7dxVk3T0H4kecf2iM16x207eNO+5NqPJO/bl87P3cVVQ2JioqxWq0frc83WBQDAG9BQAwB0SMOGDVNJSYnb2CeffKIePXpIknr16qXIyEjl5+drwIABkqTq6mpt3bpVixcvliQNHDhQVqtV+fn5SktLkyQdPnxYxcXFWrJkiSQpISFBlZWV2rVrlwYPPjddY+fOnaqsrDSbbhey2Wyy2Wx1xq1Wq8cnoC6OWoscNd82Qpq6ntbmanycn7/X5F7j3nhy1Fq8JveGXLgfSd6xPc7P2dv2pQt/35L37Ev15e7SlO8zb6gZAAAXGmoAgA7pl7/8pYYOHaqFCxcqLS1Nu3bt0ksvvaSXXnpJ0rnLNDMzM7Vw4UL17t1bvXv31sKFCxUUFKT09HRJkt1u18SJEzVz5kyFhYUpNDRUs2bNUr9+/cynfvbt21ejR4/WpEmT9OKLL0qSHnroIaWkpPCETwAAAKCDoqEGAOiQBg0apA0bNmjOnDl6+umn1atXLy1btkzjxo0zY2bPnq0zZ85oypQpqqioUHx8vPLy8hQSEmLGPP/88/L391daWprOnDmjkSNHas2aNfLz8zNj1q5dq+nTp5tPA01NTdXy5ctbr1gAAAAArYqGGgCgw0pJSVFKSkqD71ssFmVlZSkrK6vBmM6dOys7O1vZ2dkNxoSGhionJ+dyUgUAAADgRTq1dQIAAAAAAACAN6GhBgAAAAAAAHiAhhoAAAAAAADgARpqAAAAAAAAgAdoqAEAAAAAAAAeaJGG2ldffaX7779fYWFhCgoK0i233KLCwkLzfcMwlJWVpejoaAUGBmrEiBHat2+f2zocDoemTZumbt26KTg4WKmpqSotLXWLqaioUEZGhux2u+x2uzIyMnT8+PGWKAkAAAAAmt3Zs2f1n//5n+rVq5cCAwN17bXX6umnn1Ztba0Z01znTwCA5tPsDbWKigoNGzZMVqtVb7/9tvbv36/nnntOV155pRmzZMkSLV26VMuXL9fu3bsVGRmpxMREnThxwozJzMzUhg0btH79em3btk0nT55USkqKampqzJj09HQVFRUpNzdXubm5KioqUkZGRnOXBDSo52Nv1fsDAAAANMbixYv1+9//XsuXL9eBAwe0ZMkSPfvss8rOzjZjmuv8CQDQfPybe4WLFy9WTEyMVq9ebY717NnT/HfDMLRs2TLNnTtXY8eOlSS9/PLLioiI0Lp16zR58mRVVlZq1apVeuWVVzRq1ChJUk5OjmJiYrRlyxYlJyfrwIEDys3N1Y4dOxQfHy9JWrlypRISElRSUqI+ffo0d2kAAAAA0Ky2b9+uu+66S2PGjJF07tzp1Vdf1Z49eyQ13/kTAKB5NXtDbePGjUpOTtZPfvITbd26VVdffbWmTJmiSZMmSZIOHjyosrIyJSUlmcvYbDYNHz5cBQUFmjx5sgoLC+V0Ot1ioqOjFRsbq4KCAiUnJ2v79u2y2+1mM02ShgwZIrvdroKCgnobag6HQw6Hw3xdVVUlSXI6nXI6nR7V6Yq3dTIafK8jcNXSkWpqSFNqtfnV3f6erqMtsF2bT3vZBy6nTl/YDwAAaK9uvfVW/f73v9cnn3yi66+/Xv/4xz+0bds2LVu2TFLznT8BAJpXszfUPvvsM61YsUIzZszQ448/rl27dmn69Omy2Wx64IEHVFZWJkmKiIhwWy4iIkKHDh2SJJWVlSkgIEBdu3atE+NavqysTOHh4XU+Pzw83Iy50KJFizRv3rw643l5eQoKCvK8WEnz42rrjG3evLlJ62rP8vPz2zqFVuNJrUsG1z/uLfsA2/Xytbd9oCl1nj59ugUyAQAAjfHoo4+qsrJSN9xwg/z8/FRTU6NnnnlG9913nyQ12/lTfZprwsHFJht0NK4a20utDW2nhv7o64n2VmtLac46W3J7NAdf2abStzW25ISDZm+o1dbWKi4uTgsXLpQkDRgwQPv27dOKFSv0wAMPmHEWi8VtOcMw6oxd6MKY+uIvtp45c+ZoxowZ5uuqqirFxMQoKSlJXbp0uXRx53E6ncrPz9cTezrJUev+ecVZHecvQK46ExMTZbVa2zqdFtWUWmOz3ql3vL3vA2zX5tNe9oHLqdP1P88AAKD1/fnPf1ZOTo7WrVunm266SUVFRcrMzFR0dLTGjx9vxjXH+dOFmnvCQX2TDTqq9lJrQ3/EbeiPvk3RXmptac1RZ2tsj+bgK9tUatkJB83eUIuKitKNN97oNta3b1+99tprkqTIyEhJ5/6KEhUVZcaUl5ebf3WJjIxUdXW1Kioq3P7KUl5erqFDh5oxR44cqfP5R48erfPXGxebzSabzVZn3Gq1NvlE21FrkaPG/SDVERsUl/M78jae1Hrhtj9/Hd6A7Xr52ts+0JQ6fWUfAACgPfqP//gPPfbYY7r33nslSf369dOhQ4e0aNEijR8/vtnOn+rTXBMOLjbZoKOxdTI0P6623dTa0B9xG/qjryfaW60tpTnrbMnt0Rx8ZZtK39bakhMOmr2hNmzYMJWUlLiNffLJJ+rRo4ckqVevXoqMjFR+fr4GDBggSaqurtbWrVu1ePFiSdLAgQNltVqVn5+vtLQ0SdLhw4dVXFysJUuWSJISEhJUWVmpXbt2afDgc+3enTt3qrKy8qIHDQAAAABoL06fPq1OnTq5jfn5+am29twMkuY6f6pPc084qG+yQUfVXmptaDs1Z27tpdaW1hx1tsb2aA6+sk2llp1w0OwNtV/+8pcaOnSoFi5cqLS0NO3atUsvvfSSXnrpJUnnpipnZmZq4cKF6t27t3r37q2FCxcqKChI6enpkiS73a6JEydq5syZCgsLU2hoqGbNmqV+/fqZT63p27evRo8erUmTJunFF1+UJD300ENKSUnhCZ8AAAAAvMKdd96pZ555Rtdcc41uuukmffTRR1q6dKl++tOfSmq+8ycAQPNq9obaoEGDtGHDBs2ZM0dPP/20evXqpWXLlmncuHFmzOzZs3XmzBlNmTJFFRUVio+PV15enkJCQsyY559/Xv7+/kpLS9OZM2c0cuRIrVmzRn5+fmbM2rVrNX36dPNpNqmpqVq+fHlzlwQAAAAALSI7O1tPPPGEpkyZovLyckVHR2vy5Ml68sknzZjmOn8CADSfZm+oSVJKSopSUlIafN9isSgrK0tZWVkNxnTu3FnZ2dnKzs5uMCY0NFQ5OTmXkyoAAAAAtJmQkBAtW7ZMy5YtazCmuc6fAADNp0UaakBH0/Oxt9o6BQAAAAAA0E50unQIAAAAAAAAABcaagAAAAAAAIAHaKgBAAAAAAAAHqChBgAAAAAAAHiAhhoAoEPKysqSxWJx+4mMjDTfNwxDWVlZio6OVmBgoEaMGKF9+/a5rcPhcGjatGnq1q2bgoODlZqaqtLSUreYiooKZWRkyG63y263KyMjQ8ePH2+NEgEAAAC0ERpqAIAO66abbtLhw4fNn71795rvLVmyREuXLtXy5cu1e/duRUZGKjExUSdOnDBjMjMztWHDBq1fv17btm3TyZMnlZKSopqaGjMmPT1dRUVFys3NVW5uroqKipSRkdGqdQIAAABoXf5tnQAAAC3F39/fbVaai2EYWrZsmebOnauxY8dKkl5++WVFRERo3bp1mjx5siorK7Vq1Sq98sorGjVqlCQpJydHMTEx2rJli5KTk3XgwAHl5uZqx44dio+PlyStXLlSCQkJKikpUZ8+fVqvWAAAAACthoYa0AJ6PvZWveOf/2pMK2cC+LZPP/1U0dHRstlsio+P18KFC3Xttdfq4MGDKisrU1JSkhlrs9k0fPhwFRQUaPLkySosLJTT6XSLiY6OVmxsrAoKCpScnKzt27fLbrebzTRJGjJkiOx2uwoKChpsqDkcDjkcDvN1VVWVJMnpdMrpdHpUoyve1smod7y9c+V9fv5ek7ufe+62TobX5H6hhvaj899rz2x+htfuS679SPK+fen83M2x/19DU/L3hpoBAHChoQYA6JDi4+P1pz/9Sddff72OHDmiBQsWaOjQodq3b5/KysokSREREW7LRERE6NChQ5KksrIyBQQEqGvXrnViXMuXlZUpPDy8zmeHh4ebMfVZtGiR5s2bV2c8Ly9PQUFBnhX6/82Pq3V7vXnz5iatp7XNj3P989v8vSX3JYPdX8+Pq/Wa3Bty4X4kecf2OH9beNu+dOF+JHnPvlRf7i75+fker+/06dOXkQ0AAK2LhhoAoEO64447zH/v16+fEhISdN111+nll1/WkCFDJEkWi8VtGcMw6oxd6MKY+uIvtZ45c+ZoxowZ5uuqqirFxMQoKSlJXbp0uXhhF3A6ncrPz9cTezrJUfvtZxZnJXu0nrYy8OlczY+rdcvfW3KPzXpH0rkZOa4aCp8c3cZZNU1D+5HkHdsjNusdt+3gTfuSaz+SvG9fOj93F1cNiYmJslqtHq3PNVsXAABvQEMNAOATgoOD1a9fP3366ae6++67JZ2bYRYVFWXGlJeXm7PWIiMjVV1drYqKCrdZauXl5Ro6dKgZc+TIkTqfdfTo0Tqz385ns9lks9nqjFutVo9PQF0ctRY5ar5thDR1Pa3N1fg4P3+vyb3GvfHkqLV4Te4NuXA/krxje5yfs7ftSxf+viXv2Zfqy92lKd9n3lAzAAAuPOUTAOATHA6HDhw4oKioKPXq1UuRkZFulyRVV1dr69atZrNs4MCBslqtbjGHDx9WcXGxGZOQkKDKykrt2rXLjNm5c6cqKyvNGAAAAAAdDzPUAAAd0qxZs3TnnXfqmmuuUXl5uRYsWKCqqiqNHz9eFotFmZmZWrhwoXr37q3evXtr4cKFCgoKUnp6uiTJbrdr4sSJmjlzpsLCwhQaGqpZs2apX79+5lM/+/btq9GjR2vSpEl68cUXJUkPPfSQUlJSeMInAAAA0IHRUAMAdEilpaW677779O9//1tXXXWVhgwZoh07dqhHjx6SpNmzZ+vMmTOaMmWKKioqFB8fr7y8PIWEhJjreP755+Xv76+0tDSdOXNGI0eO1Jo1a+Tn52fGrF27VtOnTzefBpqamqrly5e3brEAAAAAWhUNNQBAh7R+/fqLvm+xWJSVlaWsrKwGYzp37qzs7GxlZ2c3GBMaGqqcnJympgkAAADAC3EPNQAAAAAAAMADNNQAAAAAAAAAD9BQAwAAAAAAADxAQw0AAAAAAADwAA01AAAAAAAAwAM01AAAAAAAAAAP0FADAAAAAAAAPEBDDQAAAAAAAPAADTUAAAAAAADAAzTUAAAAAAAAAA/QUAMAAAAAAAA8QEMNAAAAAAAA8AANNQAAAAAAAMADNNQAAAAAAAAAD9BQAwAAAAAAADxAQw0AAAAAAADwAA01AAAAAGhDX331le6//36FhYUpKChIt9xyiwoLC833DcNQVlaWoqOjFRgYqBEjRmjfvn1u63A4HJo2bZq6deum4OBgpaamqrS0tLVLAQCfQUMNAAAAANpIRUWFhg0bJqvVqrffflv79+/Xc889pyuvvNKMWbJkiZYuXarly5dr9+7dioyMVGJiok6cOGHGZGZmasOGDVq/fr22bdumkydPKiUlRTU1NW1QFQB0fP5tnQAAAAAA+KrFixcrJiZGq1evNsd69uxp/rthGFq2bJnmzp2rsWPHSpJefvllRUREaN26dZo8ebIqKyu1atUqvfLKKxo1apQkKScnRzExMdqyZYuSk5NbtSYA8AXMUAMAAACANrJx40bFxcXpJz/5icLDwzVgwACtXLnSfP/gwYMqKytTUlKSOWaz2TR8+HAVFBRIkgoLC+V0Ot1ioqOjFRsba8YAAJoXM9QAoBF6PvZWW6eAy7Bo0SI9/vjjeuSRR7Rs2TJJ5/7iP2/ePL300kuqqKhQfHy8fve73+mmm24yl3M4HJo1a5ZeffVVnTlzRiNHjtQLL7yg7t27mzEVFRWaPn26Nm7cKElKTU1Vdna226U6AAA05LPPPtOKFSs0Y8YMPf7449q1a5emT58um82mBx54QGVlZZKkiIgIt+UiIiJ06NAhSVJZWZkCAgLUtWvXOjGu5evjcDjkcDjM11VVVZIkp9Mpp9PZ6BpcsbZORqOX8VauGttLrQ1tJ5vf5efX3mptKc1ZZ0tuj+bgK9tU+rZGT77LXBq7DA01AECHtnv3br300kvq37+/27jrfjRr1qzR9ddfrwULFigxMVElJSUKCQmRdO5+NJs2bdL69esVFhammTNnKiUlRYWFhfLz85Mkpaenq7S0VLm5uZKkhx56SBkZGdq0aVPrFgoA8Eq1tbWKi4vTwoULJUkDBgzQvn37tGLFCj3wwANmnMVicVvOMIw6Yxe6VMyiRYs0b968OuN5eXkKCgrypAxJ0vy4Wo+X8VbtpdbNmzfXO75kcPN9RnuptaU1R52tsT2ag69sU0nKz8/3eJnTp083Ko6GGgCgwzp58qTGjRunlStXasGCBeZ4c92P5sCBA8rNzdWOHTsUHx8vSVq5cqUSEhJUUlKiPn36tH7RAACvEhUVpRtvvNFtrG/fvnrttdckSZGRkZLOzUKLiooyY8rLy81Za5GRkaqurlZFRYXbLLXy8nINHTq0wc+eM2eOZsyYYb6uqqpSTEyMkpKS1KVLl0bX4HQ6lZ+fryf2dJKj9uJNPm9n62Roflxtu6m1OKv+++PFZr1z2etub7W2lOassyW3R3PwlW0qfVtrYmKirFarR8u6ZuteCg21FtDQpWGf/2pMK2cCoK3wPdA+TJ06VWPGjNGoUaPcGmqXuh/N5MmTL3k/muTkZG3fvl12u91spknSkCFDZLfbVVBQ0GBDrbkusXEtI9Wdut+U6e1tob5LD7wmdz/33G2dDK/J/UIXu1zLG2qy+Rleuy+dfxmQt+1L9V3C1BqX2HQ0w4YNU0lJidvYJ598oh49ekiSevXqpcjISOXn52vAgAGSpOrqam3dulWLFy+WJA0cOFBWq1X5+flKS0uTJB0+fFjFxcVasmRJg59ts9lks9nqjFutVo9PQCXJUWuRo6Zjn6S7tJdaG9pOzZlbe6m1pTVHna2xPZqDr2xTqWnfZ42Np6EGAOiQ1q9frw8//FC7d++u815z3Y+mrKxM4eHhddYfHh5+0XvWNPclNlLdqfsNXXLQ3syPc/3z2/y9JfcLL9+YH1frNbk3pL5LQLyhpvO3hbftS/VdBuQt+9LFLmFqyUtsOppf/vKXGjp0qBYuXKi0tDTt2rVLL730kl566SVJ5y71zMzM1MKFC9W7d2/17t1bCxcuVFBQkNLT0yVJdrtdEydO1MyZMxUWFqbQ0FDNmjVL/fr1M2dZAwCaFw01AECH8+WXX+qRRx5RXl6eOnfu3GBcc9yPpr74S62nuS6xkRq+zKahSw7am4FP59a59MBbcnddvnH+5ROFT45u46ya5mKXa3nD9ojNeqfey1i8JXcXb9uX6ruEqTUuseloBg0apA0bNmjOnDl6+umn1atXLy1btkzjxo0zY2bPnq0zZ85oypQp5oN08vLyzHt+StLzzz8vf39/paWlmQ/SWbNmjXnPTwBA86KhBgDocAoLC1VeXq6BAweaYzU1NXr//fe1fPly89Kay70fTWRkpI4cOVLn848ePVpn9tv5mvsSG6nu1P2mrqe1uRof5+fvNblfcKmEo9biNbk3pL5LQLyhpvNz9rZ9qb5LbrxlX7rY5UIteYlNR5SSkqKUlJQG37dYLMrKylJWVlaDMZ07d1Z2drays7NbIEMAwIU6tXUCAAA0t5EjR2rv3r0qKioyf+Li4jRu3DgVFRXp2muvNe9H4+K6H42rWXb+/WhcXPejccUkJCSosrJSu3btMmN27typysrKi94EGgAAAIB3Y4YaAKDDCQkJUWxsrNtYcHCwwsLCzPHmuB9N3759NXr0aE2aNEkvvviiJOmhhx5SSkoKT/gEAAAAOjAaagAAn9Rc96NZu3atpk+fbj4NNDU1VcuXL2/1egAAAAC0nha/5HPRokXmk2lcDMNQVlaWoqOjFRgYqBEjRmjfvn1uyzkcDk2bNk3dunVTcHCwUlNTVVpa6hZTUVGhjIwM2e122e12ZWRk6Pjx4y1dEgDAC7333ntatmyZ+dp1P5rDhw/rm2++0datW+vManPdj+bYsWM6ffq0Nm3apJiYGLeY0NBQ5eTkqKqqSlVVVcrJydGVV17ZChUBAAAAaCst2lDbvXu3XnrpJfXv399tfMmSJVq6dKmWL1+u3bt3KzIyUomJiTpx4oQZk5mZqQ0bNmj9+vXatm2bTp48qZSUFNXU1Jgx6enpKioqUm5urnJzc1VUVKSMjIyWLAkAAAAAAAA+rsUaaidPntS4ceO0cuVKt6ejGYahZcuWae7cuRo7dqxiY2P18ssv6/Tp01q3bp0kqbKyUqtWrdJzzz2nUaNGacCAAcrJydHevXu1ZcsWSdKBAweUm5urP/zhD0pISFBCQoJWrlypN99803x6GwAAAAAAANDcWqyhNnXqVI0ZM8a8cbPLwYMHVVZWZt5rRpJsNpuGDx+ugoICSVJhYaGcTqdbTHR0tGJjY82Y7du3y263Kz4+3owZMmSI7Ha7GQMAAAAAAAA0txZ5KMH69ev14Ycfavfu3XXeKysrkyRFRES4jUdEROjQoUNmTEBAgNvMNleMa/mysjKFh4fXWX94eLgZcyGHwyGHw2G+rqqqkiQ5nU45nc7GlmcuI0m2TobHy3gTV87emLunLlarza/x27kxn9HW2K6ea+/7wOXU6Qv7AQAAAAA0p2ZvqH355Zd65JFHlJeXp86dOzcYZ7FY3F4bhlFn7EIXxtQXf7H1LFq0SPPmzasznpeXp6CgoIt+dkPmx9U2Onbz5s1N+oz2ID8/v61TaDX11bpkcPOsu73tA76+XT3hLftAU+o8ffp0C2QCAAAAAB1XszfUCgsLVV5eroEDB5pjNTU1ev/997V8+XLz/mZlZWWKiooyY8rLy81Za5GRkaqurlZFRYXbLLXy8nINHTrUjDly5Eidzz969Gid2W8uc+bM0YwZM8zXVVVViomJUVJSkrp06eJRnU6nU/n5+XpiTyc5ai/eCHQpzkr26DPaA1ediYmJslqtbZ1Oi7pYrbFZ7zTLZ7SXfYDt6rn2vg9cTp2u2boAAAAAgMZp9obayJEjtXfvXrexBx98UDfccIMeffRRXXvttYqMjFR+fr4GDBggSaqurtbWrVu1ePFiSdLAgQNltVqVn5+vtLQ0SdLhw4dVXFysJUuWSJISEhJUWVmpXbt2afDgc1NHdu7cqcrKSrPpdiGbzSabzVZn3Gq1NvlE21FrkaOmcQ01b25cXM7vyNvUV2tjt3Fj1t2e+Pp29YS37ANNqdNX9gEAAAAAaC7N3lALCQlRbGys21hwcLDCwsLM8czMTC1cuFC9e/dW7969tXDhQgUFBSk9PV2SZLfbNXHiRM2cOVNhYWEKDQ3VrFmz1K9fP/MhB3379tXo0aM1adIkvfjii5Kkhx56SCkpKerTp09zlwUAAAAAAABIaqGHElzK7NmzdebMGU2ZMkUVFRWKj49XXl6eQkJCzJjnn39e/v7+SktL05kzZzRy5EitWbNGfn5+ZszatWs1ffp082mgqampWr58eavXAwAAAAAAAN/RKg219957z+21xWJRVlaWsrKyGlymc+fOys7OVnZ2doMxoaGhysnJaaYsAQAAAAAAgEvr1NYJAAAAAAAAAN6EhhoAAAAAAADgARpqAAAAAAAAgAdoqAEAAAAAAAAeoKEGAAAAAAAAeICGGgAAAAAAAOABGmoAAAAAAACAB2ioAQAAAAAAAB6goQYAAAAAAAB4gIYaAAAAAAAA4AEaagCADmnFihXq37+/unTpoi5duighIUFvv/22+b5hGMrKylJ0dLQCAwM1YsQI7du3z20dDodD06ZNU7du3RQcHKzU1FSVlpa6xVRUVCgjI0N2u112u10ZGRk6fvx4a5QIAAAAoI3QUAMAdEjdu3fXr371K+3Zs0d79uzR7bffrrvuustsmi1ZskRLly7V8uXLtXv3bkVGRioxMVEnTpww15GZmakNGzZo/fr12rZtm06ePKmUlBTV1NSYMenp6SoqKlJubq5yc3NVVFSkjIyMVq8XAAAAQOvxb+sEAABoCXfeeafb62eeeUYrVqzQjh07dOONN2rZsmWaO3euxo4dK0l6+eWXFRERoXXr1mny5MmqrKzUqlWr9Morr2jUqFGSpJycHMXExGjLli1KTk7WgQMHlJubqx07dig+Pl6StHLlSiUkJKikpER9+vRp3aIBAAAAtApmqAEAOryamhqtX79ep06dUkJCgg4ePKiysjIlJSWZMTabTcOHD1dBQYEkqbCwUE6n0y0mOjpasbGxZsz27dtlt9vNZpokDRkyRHa73YwBAAAA0PEwQw0A0GHt3btXCQkJ+uabb3TFFVdow4YNuvHGG81mV0REhFt8RESEDh06JEkqKytTQECAunbtWiemrKzMjAkPD6/zueHh4WZMfRwOhxwOh/m6qqpKkuR0OuV0Oj2q0RVv62TUO97eufI+P3+vyd3PPXdbJ8Nrcr9QQ/vR+e+1ZzY/w2v3Jdd+JHnfvnR+7ubY/6+hKfl7Q80AALjQUAMAdFh9+vRRUVGRjh8/rtdee03jx4/X1q1bzfctFotbvGEYdcYudGFMffGXWs+iRYs0b968OuN5eXkKCgq66Oc3ZH5crdvrzZs3N2k9rW1+nOuf3+bvLbkvGez+en5crdfk3pAL9yPJO7bH+dvC2/alC/cjyXv2pfpyd8nPz/d4fadPn76MbAAAaF001AAAHVZAQIC+853vSJLi4uK0e/du/eY3v9Gjjz4q6dwMs6ioKDO+vLzcnLUWGRmp6upqVVRUuM1SKy8v19ChQ82YI0eO1Pnco0eP1pn9dr45c+ZoxowZ5uuqqirFxMQoKSlJXbp08ahGp9Op/Px8PbGnkxy13zbxirOSPVpPWxn4dK7mx9W65e8tucdmvSPp3IwcVw2FT45u46yapqH9SPKO7RGb9Y7bdvCmfcm1H0nety+dn7uLq4bExERZrVaP1uearQsAgDegoQYA8BmGYcjhcKhXr16KjIxUfn6+BgwYIEmqrq7W1q1btXjxYknSwIEDZbValZ+fr7S0NEnS4cOHVVxcrCVLlkiSEhISVFlZqV27dmnw4HNTNXbu3KnKykqz6VYfm80mm81WZ9xqtXp8AuriqLXIUfNtI6Sp62ltrsbH+fl7Te417o0nR63Fa3JvyIX7keQd2+P8nL1tX7rw9y15z75UX+4uTfk+84aaAQBw4aEEAIAO6fHHH9cHH3ygzz//XHv37tXcuXP13nvvady4cbJYLMrMzNTChQu1YcMGFRcXa8KECQoKClJ6erokyW63a+LEiZo5c6b+9re/6aOPPtL999+vfv36mU/97Nu3r0aPHq1JkyZpx44d2rFjhyZNmqSUlBSe8AkAaJJFixaZxykXwzCUlZWl6OhoBQYGasSIEdq3b5/bcg6HQ9OmTVO3bt0UHBys1NRUlZaWtnL2AOA7aKgBADqkI0eOKCMjQ3369NHIkSO1c+dO5ebmKjExUZI0e/ZsZWZmasqUKYqLi9NXX32lvLw8hYSEmOt4/vnndffddystLU3Dhg1TUFCQNm3aJD8/PzNm7dq16tevn5KSkpSUlKT+/fvrlVdeafV6AQDeb/fu3XrppZfUv39/t/ElS5Zo6dKlWr58uXbv3q3IyEglJibqxIkTZkxmZqY2bNig9evXa9u2bTp58qRSUlJUU1PT2mUAgE/gkk8AQIe0atWqi75vsViUlZWlrKysBmM6d+6s7OxsZWdnNxgTGhqqnJycpqYJAIAk6eTJkxo3bpxWrlypBQsWmOOGYWjZsmWaO3euxo4dK0l6+eWXFRERoXXr1mny5MmqrKzUqlWr9Morr5izqHNychQTE6MtW7YoObn9308QALwNM9QAAAAAoI1NnTpVY8aMMRtiLgcPHlRZWZmSkpLMMZvNpuHDh6ugoECSVFhYKKfT6RYTHR2t2NhYMwYA0LyYoQYAAAAAbWj9+vX68MMPtXv37jrvlZWVSVKdp0dHRETo0KFDZkxAQIDbU6ldMa7l6+NwOORwOMzXrietOp1OOZ3ORufvirV1Mhq9jLdy1dheam1oO9n8Lj+/9lZrS2nOOltyezQHX9mm0rc1evJd5tLYZWioAQAAAEAb+fLLL/XII48oLy9PnTt3bjDOYnF/qqphGHXGLnSpmEWLFmnevHl1xvPy8hQUFHSJzOuaH1fr8TLeqr3Uunnz5nrHlwxuvs9oL7W2tOaoszW2R3PwlW0qSfn5+R4vc/r06UbF0VADAAAAgDZSWFio8vJyDRw40ByrqanR+++/r+XLl6ukpETSuVloUVFRZkx5ebk5ay0yMlLV1dWqqKhwm6VWXl6uoUOHNvjZc+bM0YwZM8zXVVVViomJUVJSkrp06dLoGpxOp/Lz8/XEnk5y1F68yeftbJ0MzY+rbTe1FmfVf3+82Kx3Lnvd7a3WltKcdbbk9mgOvrJNpW9rTUxMlNVq9WhZ12zdS6GhBgAAAABtZOTIkdq7d6/b2IMPPqgbbrhBjz76qK699lpFRkYqPz9fAwYMkCRVV1dr69atWrx4sSRp4MCBslqtys/PV1pamiTp8OHDKi4u1pIlSxr8bJvNJpvNVmfcarV6fAIqSY5aixw1Hfsk3aW91NrQdmrO3NpLrS2tOepsje3RHHxlm0pN+z5rbDwNNQAAAABoIyEhIYqNjXUbCw4OVlhYmDmemZmphQsXqnfv3urdu7cWLlyooKAgpaenS5LsdrsmTpyomTNnKiwsTKGhoZo1a5b69etX5yEHAIDmQUMNAAAAANqx2bNn68yZM5oyZYoqKioUHx+vvLw8hYSEmDHPP/+8/P39lZaWpjNnzmjkyJFas2aN/Pz82jBzAOi4aKgBAAAAQDvy3nvvub22WCzKyspSVlZWg8t07txZ2dnZys7ObtnkAACSpE5tnQAAAAAAAADgTWioAQAAAAAAAB6goQYAAAAAAAB4gIYaAAAAAAAA4AEaagAAAAAAAIAHaKgBAAAAAAAAHqChBgAAAAAAAHiAhhoAAAAAAADgARpqAAAAAAAAgAdoqAEAAAAAAAAeoKEGAAAAAAAAeICGGgAAAAAAAOABGmoAAAAAAACAB2ioAQA6pEWLFmnQoEEKCQlReHi47r77bpWUlLjFGIahrKwsRUdHKzAwUCNGjNC+ffvcYhwOh6ZNm6Zu3bopODhYqampKi0tdYupqKhQRkaG7Ha77Ha7MjIydPz48ZYuEQAAAEAboaEGAOiQtm7dqqlTp2rHjh3Kz8/X2bNnlZSUpFOnTpkxS5Ys0dKlS7V8+XLt3r1bkZGRSkxM1IkTJ8yYzMxMbdiwQevXr9e2bdt08uRJpaSkqKamxoxJT09XUVGRcnNzlZubq6KiImVkZLRqvQAAAABaj39bJwAAQEvIzc11e7169WqFh4ersLBQt912mwzD0LJlyzR37lyNHTtWkvTyyy8rIiJC69at0+TJk1VZWalVq1bplVde0ahRoyRJOTk5iomJ0ZYtW5ScnKwDBw4oNzdXO3bsUHx8vCRp5cqVSkhIUElJifr06dO6hQMAAABocTTUAAA+obKyUpIUGhoqSTp48KDKysqUlJRkxthsNg0fPlwFBQWaPHmyCgsL5XQ63WKio6MVGxurgoICJScna/v27bLb7WYzTZKGDBkiu92ugoKCehtqDodDDofDfF1VVSVJcjqdcjqdHtXlird1Muodb+9ceZ+fv9fk7ueeu62T4TW5X6ih/ej899ozm5/htfuSaz+SvG9fOj93c+z/19CU/L2hZgAAXGioAQA6PMMwNGPGDN16662KjY2VJJWVlUmSIiIi3GIjIiJ06NAhMyYgIEBdu3atE+NavqysTOHh4XU+Mzw83Iy50KJFizRv3rw643l5eQoKCvKwunPmx9W6vd68eXOT1tPa5se5/vlt/t6S+5LB7q/nx9V6Te4NuXA/krxje5y/LbxtX7pwP5K8Z1+qL3eX/Px8j9d3+vTpy8gGAIDW1ewNtUWLFun111/X//7v/yowMFBDhw7V4sWL3f5CbxiG5s2bp5deekkVFRWKj4/X7373O910001mjMPh0KxZs/Tqq6/qzJkzGjlypF544QV1797djKmoqND06dO1ceNGSVJqaqqys7N15ZVXNndZAAAv9vDDD+vjjz/Wtm3b6rxnsVjcXhuGUWfsQhfG1Bd/sfXMmTNHM2bMMF9XVVUpJiZGSUlJ6tKly0U/+0JOp1P5+fl6Yk8nOWq//bzirGSP1tNWBj6dq/lxtW75e0vusVnvSDo3I8dVQ+GTo9s4q6ZpaD+SvGN7xGa947YdvGlfcu1HkvftS+fn7uKqITExUVar1aP1uWbrAgDgDZq9oea6CfSgQYN09uxZzZ07V0lJSdq/f7+Cg4MlfXsT6DVr1uj666/XggULlJiYqJKSEoWEhEg6dxPoTZs2af369QoLC9PMmTOVkpKiwsJC+fn5STp3E+jS0lLzPjkPPfSQMjIytGnTpuYuCwDgpaZNm6aNGzfq/fffd/ujTGRkpKRzM8yioqLM8fLycnPWWmRkpKqrq1VRUeE2S628vFxDhw41Y44cOVLnc48ePVpn9puLzWaTzWarM261Wj0+AXVx1FrkqPm2EdLU9bQ2V+Pj/Py9Jvca98aTo9biNbk35ML9SPKO7XF+zt62L134+5a8Z1+qL3eXpnyfeUPNAAC4NPtTPnNzczVhwgTddNNNuvnmm7V69Wp98cUXKiwslKQ6N4GOjY3Vyy+/rNOnT2vdunWSZN4E+rnnntOoUaM0YMAA5eTkaO/evdqyZYskmTeB/sMf/qCEhAQlJCRo5cqVevPNN1VSUtLcZQEAvIxhGHr44Yf1+uuv6+9//7t69erl9n6vXr0UGRnpdllSdXW1tm7dajbLBg4cKKvV6hZz+PBhFRcXmzEJCQmqrKzUrl27zJidO3eqsrLSjAEAAADQsbT4PdR87SbQjVnGm7hy9sbcPXWxWuu76e7lfEZbY7t6rr3vA5dTZ0fdD6ZOnap169bpr3/9q0JCQsz7mdntdgUGBspisSgzM1MLFy5U79691bt3by1cuFBBQUFKT083YydOnKiZM2cqLCxMoaGhmjVrlvr162c+9bNv374aPXq0Jk2apBdffFHSuRnTKSkpPOETAAAA6KBatKHmizeBvhhvuLlsQ5pyY1lvVV+tF7vprifa2z7g69vVE96yD3AT6G+tWLFCkjRixAi38dWrV2vChAmSpNmzZ+vMmTOaMmWKeU/PvLw88/YDkvT888/L399faWlp5j0916xZY95+QJLWrl2r6dOnm38ISk1N1fLly1u2QAAAAABtpkUbar54E+iL8YYb417IVWdTbizrbS5Wa3033W2K9rIPsF091973gcups6PeBNowLj2r0GKxKCsrS1lZWQ3GdO7cWdnZ2crOzm4wJjQ0VDk5OU1JEwAAAIAXarGGmq/eBPpivLlxcTm/I29TX62N3caNWXd74uvb1RPesg9wE2gAAAAAaHnN/lACbgINAAAAAACAjqzZZ6hxE2gAAAAAAAB0ZM3eUOMm0AAAAAAAAOjImr2hxk2gAQAAAAAA0JE1+z3UAAAAAAAAgI6MhhoAAAAAAADgARpqAAAAAAAAgAdoqAEAAAAAAAAeoKEGAAAAAG1k0aJFGjRokEJCQhQeHq67775bJSUlbjGGYSgrK0vR0dEKDAzUiBEjtG/fPrcYh8OhadOmqVu3bgoODlZqaqpKS0tbsxQA8Ck01AAAAACgjWzdulVTp07Vjh07lJ+fr7NnzyopKUmnTp0yY5YsWaKlS5dq+fLl2r17tyIjI5WYmKgTJ06YMZmZmdqwYYPWr1+vbdu26eTJk0pJSVFNTU1blAUAHZ5/WycAAAAAAL4qNzfX7fXq1asVHh6uwsJC3XbbbTIMQ8uWLdPcuXM1duxYSdLLL7+siIgIrVu3TpMnT1ZlZaVWrVqlV155RaNGjZIk5eTkKCYmRlu2bFFycnKr1wUAHR0NNQAAAABoJyorKyVJoaGhkqSDBw+qrKxMSUlJZozNZtPw4cNVUFCgyZMnq7CwUE6n0y0mOjpasbGxKigoaLCh5nA45HA4zNdVVVWSJKfTKafT2eicXbG2Tkajl/FWrhrbS60NbSeb3+Xn195qbSnNWWdLbo/m4CvbVPq2Rk++y1wauwwNNQAAAABoBwzD0IwZM3TrrbcqNjZWklRWViZJioiIcIuNiIjQoUOHzJiAgAB17dq1Toxr+fosWrRI8+bNqzOel5enoKAgj/OfH1fr8TLeqr3Uunnz5nrHlwxuvs9oL7W2tOaoszW2R3PwlW0qSfn5+R4vc/r06UbF0VADAAAAgHbg4Ycf1scff6xt27bVec9isbi9NgyjztiFLhUzZ84czZgxw3xdVVWlmJgYJSUlqUuXLo3O2+l0Kj8/X0/s6SRH7cVz8na2Tobmx9W2m1qLs+qffRib9c5lr7u91dpSmrPOltwezcFXtqn0ba2JiYmyWq0eLeuarXspNNQAAAAAoI1NmzZNGzdu1Pvvv6/u3bub45GRkZLOzUKLiooyx8vLy81Za5GRkaqurlZFRYXbLLXy8nINHTq0wc+02Wyy2Wx1xq1Wq8cnoJLkqLXIUdOxT9Jd2kutDW2n5sytvdTa0pqjztbYHs3BV7ap1LTvs8bG85RPAAAAAGgjhmHo4Ycf1uuvv66///3v6tWrl9v7vXr1UmRkpNtlS9XV1dq6davZLBs4cKCsVqtbzOHDh1VcXHzRhhoAoOmYoQYAAAAAbWTq1Klat26d/vrXvyokJMS855ndbldgYKAsFosyMzO1cOFC9e7dW71799bChQsVFBSk9PR0M3bixImaOXOmwsLCFBoaqlmzZqlfv37mUz8BAM2LhhoAAAAAtJEVK1ZIkkaMGOE2vnr1ak2YMEGSNHv2bJ05c0ZTpkxRRUWF4uPjlZeXp5CQEDP++eefl7+/v9LS0nTmzBmNHDlSa9askZ+fX2uVAgA+hYYaAAAAALQRwzAuGWOxWJSVlaWsrKwGYzp37qzs7GxlZ2c3Y3YAgIZwDzUAQIf0/vvv684771R0dLQsFoveeOMNt/cNw1BWVpaio6MVGBioESNGaN++fW4xDodD06ZNU7du3RQcHKzU1FSVlpa6xVRUVCgjI0N2u112u10ZGRk6fvx4C1cHAAAAoC3RUAMAdEinTp3SzTffrOXLl9f7/pIlS7R06VItX75cu3fvVmRkpBITE3XixAkzJjMzUxs2bND69eu1bds2nTx5UikpKaqpqTFj0tPTVVRUpNzcXOXm5qqoqEgZGRktXh8AAACAtsMlnwCADumOO+7QHXfcUe97hmFo2bJlmjt3rsaOHStJevnllxUREaF169Zp8uTJqqys1KpVq/TKK6+YN3TOyclRTEyMtmzZouTkZB04cEC5ubnasWOH4uPjJUkrV65UQkKCSkpK1KdPn9YpFgAAAECrYoYaAMDnHDx4UGVlZUpKSjLHbDabhg8froKCAklSYWGhnE6nW0x0dLRiY2PNmO3bt8tut5vNNEkaMmSI7Ha7GQMAAACg42GGGgDA55SVlUmSIiIi3MYjIiJ06NAhMyYgIEBdu3atE+NavqysTOHh4XXWHx4ebsbUx+FwyOFwmK+rqqokSU6nU06n06NaXPG2Tka94+2dK+/z8/ea3P3cc7d1Mrwm9ws1tB+d/157ZvMzvHZfcu1HkvftS+fnbo79/xqakr831AwAgAsNNQCAz7JYLG6vDcOoM3ahC2Pqi7/UehYtWqR58+bVGc/Ly1NQUNCl0q7X/Lhat9ebN29u0npa2/w41z+/zd9bcl8y2P31/Lhar8m9IRfuR5J3bI/zt4W37UsX7keS9+xL9eXukp+f7/H6Tp8+fRnZAADQumioAQB8TmRkpKRzM8yioqLM8fLycnPWWmRkpKqrq1VRUeE2S628vFxDhw41Y44cOVJn/UePHq0z++18c+bM0YwZM8zXVVVViomJUVJSkrp06eJRLU6nU/n5+XpiTyc5ar9t4hVnJXu0nrYy8OlczY+rdcvfW3KPzXpH0rkZOa4aCp8c3cZZNU1D+5HkHdsjNusdt+3gTfuSaz+SvG9fOj93F1cNiYmJslqtHq3PNVsXAABvQEMNAOBzevXqpcjISOXn52vAgAGSpOrqam3dulWLFy+WJA0cOFBWq1X5+flKS0uTJB0+fFjFxcVasmSJJCkhIUGVlZXatWuXBg8+N1Vj586dqqysNJtu9bHZbLLZbHXGrVarxyegLo5aixw13zZCmrqe1uZqfJyfv9fkXuPeeHLUWrwm94ZcuB9J3rE9zs/Z2/alC3/fkvfsS/Xl7tKU7zNvqBkAABcaagCADunkyZP6v//7P/P1wYMHVVRUpNDQUF1zzTXKzMzUwoUL1bt3b/Xu3VsLFy5UUFCQ0tPTJUl2u10TJ07UzJkzFRYWptDQUM2aNUv9+vUzn/rZt29fjR49WpMmTdKLL74oSXrooYeUkpLCEz4BAACADoyGGgCgQ9qzZ4++//3vm69dl1iOHz9ea9as0ezZs3XmzBlNmTJFFRUVio+PV15enkJCQsxlnn/+efn7+ystLU1nzpzRyJEjtWbNGvn5+Zkxa9eu1fTp082ngaampmr58uWtVCUAAACAtkBDDQDQIY0YMUKGUfcJdC4Wi0VZWVnKyspqMKZz587Kzs5WdnZ2gzGhoaHKycm5nFQBAAAAeJlObZ0AAAAAAAAA4E1oqAEAAAAAAAAeoKEGAAAAAAAAeICGGgAAAAAAAOABGmoAAAAAAACAB2ioAQAAAAAAAB6goQYAAAAAAAB4gIYaAAAAAAAA4AEaagAAAAAAAIAHaKgBAAAAAAAAHqChBgAAAAAAAHiAhhoAAAAAAADgARpqAAAAAAAAgAdoqAEAAAAAAAAeoKEGAAAAAAAAeICGGgAAAAAAAOABGmoAAAAAAACAB/zbOgEAgGd6PvZWnTGbn6Elg9sgGQAAAADwQcxQAwAAAAAAADxAQw0AAAAAAADwAJd8Aq2ovkv1JOnzX41p5UwAAAAAAEBTef0MtRdeeEG9evVS586dNXDgQH3wwQdtnRIANJuej71V5wftE8cjAEB7wPEIAFqHVzfU/vznPyszM1Nz587VRx99pO9973u644479MUXX7R1agAAH8LxCADQHnA8AoDW49UNtaVLl2rixIn62c9+pr59+2rZsmWKiYnRihUr2jo1AIAP4XgEAGgPOB4BQOvx2nuoVVdXq7CwUI899pjbeFJSkgoKCupdxuFwyOFwmK8rKyslSV9//bWcTqdHn+90OnX69Gn5OzupptbSqGWOHTvm0We0B646jx07JqvV2tbptKiL1ep/9lSLfvZ3Zv1XnbGdc0a22OexXT3XXPuAp98Djf1c/1pDp0/XNqnOEydOSJIMw/BoOZzTXo9H3nLM8Xee0unTtW75e03u//+/T9d/f/7OTl6T+4Uu9v813lCT/9lTbtvBm/al87/nvW1fqu8YxfGo7bTl8agp50beqr7vmrbU0HdFc/y/a3urtaU0Z50tuT2ag69sU6mVjkeGl/rqq68MScb//M//uI0/88wzxvXXX1/vMk899ZQhiR9++OGHn3p+vvzyy9b4+u5wOB7xww8//DTvD8ejpuF4xA8//PDTvD+XOh557Qw1F4vFvatqGEadMZc5c+ZoxowZ5uva2lp9/fXXCgsLa3CZhlRVVSkmJkZffvmlunTp4nniXsJX6pSotaPylVovp07DMHTixAlFR0e3UHa+geNR03h7/hI1tBfU0D5wPGp7bXE86gj7bmNRa8fjK3VK1NpYjT0eeW1DrVu3bvLz81NZWZnbeHl5uSIiIupdxmazyWazuY1deeWVl5VHly5dOvyOKPlOnRK1dlS+UmtT67Tb7S2QjW/geNQ8vD1/iRraC2poHzgetb72cDzqCPtuY1Frx+MrdUrU2hiNOR557UMJAgICNHDgQOXn57uN5+fna+jQoW2UFQDA13A8AgC0BxyPAKB1ee0MNUmaMWOGMjIyFBcXp4SEBL300kv64osv9POf/7ytUwMA+BCORwCA9oDjEQC0Hq9uqN1zzz06duyYnn76aR0+fFixsbHavHmzevTo0eKfbbPZ9NRTT9WZIt3R+EqdErV2VL5Sq6/U2V5xPGo6b89foob2ghrah45Qgzdrq+ORL213au14fKVOiVqbm8UweC41AAAAAAAA0Fheew81AAAAAAAAoC3QUAMAAAAAAAA8QEMNAAAAAAAA8AANNQAAAAAAAMADNNQu4oUXXlCvXr3UuXNnDRw4UB988MFF47du3aqBAweqc+fOuvbaa/X73/++lTK9PJ7U+frrrysxMVFXXXWVunTpooSEBL3zzjutmO3l8XSbuvzP//yP/P39dcstt7Rsgs3I01odDofmzp2rHj16yGaz6brrrtMf//jHVsq26Tytc+3atbr55psVFBSkqKgoPfjggzp27FgrZdt077//vu68805FR0fLYrHojTfeuOQy3vqdhMZpyj7RnixatEiDBg1SSEiIwsPDdffdd6ukpKSt0/LIihUr1L9/f3Xp0sU8Jr799tttndZlWbRokSwWizIzM9s6lUbLysqSxWJx+4mMjGzrtDz21Vdf6f7771dYWJiCgoJ0yy23qLCwsK3TarSePXvW2Q4Wi0VTp05t69TQTHzl3Eji/KgxvO38yFfOjSTfOD9qN+dGBuq1fv16w2q1GitXrjT2799vPPLII0ZwcLBx6NCheuM/++wzIygoyHjkkUeM/fv3GytXrjSsVqvxl7/8pZUz94yndT7yyCPG4sWLjV27dhmffPKJMWfOHMNqtRoffvhhK2fuOU9rdTl+/Lhx7bXXGklJScbNN9/cOslepqbUmpqaasTHxxv5+fnGwYMHjZ07dxr/8z//04pZe87TOj/44AOjU6dOxm9+8xvjs88+Mz744APjpptuMu6+++5WztxzmzdvNubOnWu89tprhiRjw4YNF4331u8kNJ6n+0R7k5ycbKxevdooLi42ioqKjDFjxhjXXHONcfLkybZOrdE2btxovPXWW0ZJSYlRUlJiPP7444bVajWKi4vbOrUm2bVrl9GzZ0+jf//+xiOPPNLW6TTaU089Zdx0003G4cOHzZ/y8vK2TssjX3/9tdGjRw9jwoQJxs6dO42DBw8aW7ZsMf7v//6vrVNrtPLycrdtkJ+fb0gy3n333bZODc3AV86NDIPzo454fuQr50aG4TvnR+3l3IiGWgMGDx5s/PznP3cbu+GGG4zHHnus3vjZs2cbN9xwg9vY5MmTjSFDhrRYjs3B0zrrc+ONNxrz5s1r7tSaXVNrveeee4z//M//NJ566imvOGAYhue1vv3224bdbjeOHTvWGuk1G0/rfPbZZ41rr73Wbey3v/2t0b179xbLsSU05qDhrd9JaBpvbKhdqLy83JBkbN26ta1TuSxdu3Y1/vCHP7R1Gh47ceKE0bt3byM/P98YPny41zXUvOX43JBHH33UuPXWW9s6jWb1yCOPGNddd51RW1vb1qmgGfjKuZFhcH7UEc+PfOXcyDB88/yoLc+NuOSzHtXV1SosLFRSUpLbeFJSkgoKCupdZvv27XXik5OTtWfPHjmdzhbL9XI0pc4L1dbW6sSJEwoNDW2JFJtNU2tdvXq1/vnPf+qpp55q6RSbTVNq3bhxo+Li4rRkyRJdffXVuv766zVr1iydOXOmNVJukqbUOXToUJWWlmrz5s0yDENHjhzRX/7yF40ZM6Y1Um5V3vidBN9WWVkpSe3+eNKQmpoarV+/XqdOnVJCQkJbp+OxqVOnasyYMRo1alRbp9Ikn376qaKjo9WrVy/de++9+uyzz9o6JY+4jsM/+clPFB4ergEDBmjlypVtnVaTVVdXKycnRz/96U9lsVjaOh1cJl85N5I4P5I63vmRr5wbSZwfXUxLfSf5X25iHdG///1v1dTUKCIiwm08IiJCZWVl9S5TVlZWb/zZs2f173//W1FRUS2Wb1M1pc4LPffcczp16pTS0tJaIsVm05RaP/30Uz322GP64IMP5O/vPf+pNKXWzz77TNu2bVPnzp21YcMG/fvf/9aUKVP09ddft9t7BTSlzqFDh2rt2rW655579M033+js2bNKTU1VdnZ2a6TcqrzxOwm+yzAMzZgxQ7feeqtiY2PbOh2P7N27VwkJCfrmm290xRVXaMOGDbrxxhvbOi2PrF+/Xh9++KF2797d1qk0SXx8vP70pz/p+uuv15EjR7RgwQINHTpU+/btU1hYWFun1yifffaZVqxYoRkzZujxxx/Xrl27NH36dNlsNj3wwANtnZ7H3njjDR0/flwTJkxo61TQDHzl3Eji/EjqeOdHvnJuJHF+dDEt9Z3EDLWLuPAvaoZhXPSvbPXF1zfe3nhap8urr76qrKws/fnPf1Z4eHhLpdesGltrTU2N0tPTNW/ePF1//fWtlV6z8mS71tbWymKxaO3atRo8eLB+8IMfaOnSpVqzZk27/0uMJ3Xu379f06dP15NPPqnCwkLl5ubq4MGD+vnPf94aqbY6b/1Ogu95+OGH9fHHH+vVV19t61Q81qdPHxUVFWnHjh36xS9+ofHjx2v//v1tnVajffnll3rkkUeUk5Ojzp07t3U6TXLHHXfoRz/6kfr166dRo0bprbfekiS9/PLLbZxZ49XW1uq73/2uFi5cqAEDBmjy5MmaNGmSVqxY0dapNcmqVat0xx13KDo6uq1TQTPylXMjifOjjnh+5CvnRhLnRw1pie8k72grt7Ju3brJz8+vThe3vLy8TlfTJTIyst54f3//dvvX0abU6fLnP/9ZEydO1H//9397xeUhntZ64sQJ7dmzRx999JEefvhhSee+WA3DkL+/v/Ly8nT77be3Su6easp2jYqK0tVXXy273W6O9e3bV4ZhqLS0VL17927RnJuiKXUuWrRIw4YN03/8x39Ikvr376/g4GB973vf04IFC9rtX0ubwhu/k+Cbpk2bpo0bN+r9999X9+7d2zodjwUEBOg73/mOJCkuLk67d+/Wb37zG7344ottnFnjFBYWqry8XAMHDjTHampq9P7772v58uVyOBzy8/Nrwww9FxwcrH79+unTTz9t61QaLSoqqs7Mxr59++q1115ro4ya7tChQ9qyZYtef/31tk4FzcRXzo0kzo+kjnd+5CvnRhLnRxfTUt9JzFCrR0BAgAYOHKj8/Hy38fz8fA0dOrTeZRISEurE5+XlKS4uTlartcVyvRxNqVM695eXCRMmaN26dV5zbbWntXbp0kV79+5VUVGR+fPzn//cnIkQHx/fWql7rCnbddiwYfrXv/6lkydPmmOffPKJOnXq1G5PcJtS5+nTp9Wpk/vXnutE0fUXio7CG7+T4FsMw9DDDz+s119/XX//+9/Vq1evtk6pWRiGIYfD0dZpNNrIkSPrHO/i4uI0btw4FRUVeV0zTZIcDocOHDjgVScBw4YNU0lJidvYJ598oh49erRRRk23evVqhYeHe83/I+LSfOXcSOL8SOp450e+cm4kcX50MS32nXRZjzTowFyPm121apWxf/9+IzMz0wgODjY+//xzwzAM47HHHjMyMjLMeNdjWH/5y18a+/fvN1atWuUVj4b2tM5169YZ/v7+xu9+9zu3R6MfP368rUpoNE9rvZC3PMXGMDyv9cSJE0b37t2NH//4x8a+ffuMrVu3Gr179zZ+9rOftVUJjeJpnatXrzb8/f2NF154wfjnP/9pbNu2zYiLizMGDx7cViU02okTJ4yPPvrI+OijjwxJxtKlS42PPvrIfAR2R/lOQuNdap9o737xi18YdrvdeO+999yOJ6dPn27r1Bptzpw5xvvvv28cPHjQ+Pjjj43HH3/c6NSpk5GXl9fWqV0Wb3vK58yZM4333nvP+Oyzz4wdO3YYKSkpRkhIiHks8Aa7du0y/P39jWeeecb49NNPjbVr1xpBQUFGTk5OW6fmkZqaGuOaa64xHn300bZOBc3MV86NDIPzo454fuQr50aG4TvnR+3l3IiG2kX87ne/M3r06GEEBAQY3/3ud42tW7ea740fP94YPny4W/x7771nDBgwwAgICDB69uxprFixopUzbhpP6hw+fLghqc7P+PHjWz/xJvB0m57PWw4YLp7WeuDAAWPUqFFGYGCg0b17d2PGjBlecWLraZ2//e1vjRtvvNEIDAw0oqKijHHjxhmlpaWtnLXn3n333Yv+t9eRvpPQOJfaJ9q7+nKXZKxevbqtU2u0n/70p+b3z1VXXWWMHDnS65tphuF9DbV77rnHiIqKMqxWqxEdHW2MHTvW2LdvX1un5bFNmzYZsbGxhs1mM2644QbjpZdeauuUPPbOO+8YkoySkpK2TgUtwFfOjQyD8yOXjnR+5CvnRobhG+dH7eXcyGIYHWgeHwAAAAAAANDCuIcaAAAAAAAA4AEaagAAAAAAAIAHaKgBAAAAAAAAHqChBgAAAAAAAHiAhhoAAAAAAADgARpqAAAAAAAAgAdoqAEAAAAAAAAeoKEGAAAAAAAAeICGGgAAAAAAAOABGmoAAAAAAACAB2ioAQAAAAAAAB6goQYAAAAAAAB4gIYaAAAAAAAA4AEaagAAAAAAAIAHaKgBAAAAAAAAHqChBgAAAAAAAHiAhhoAAAAAAADgARpqAAAAAAAAgAdoqAEAAAAAAAAeoKEGAAAAAAAAeICGGgAAAAAAAOABGmoAAAAAAACAB2ioAV7MYrEoKyurrdMAALRTEyZMUM+ePds6jRbXs2dPTZgwoa3TAACfkpWVJYvF0iaf/fnnn8tisWjNmjVt8vmAJPm3dQIAmm779u3q3r17W6cBAAAAwMf87Gc/0+jRo9vks6OiorR9+3Zdd911bfL5gERDDfA6hmHom2++UWBgoIYMGdLW6QAAAADwQd27d2/1P+7X1NTo7NmzstlsnAuhzXHJJ3ySa3ryvn37dN9998lutysiIkI//elPVVlZacb993//t+Lj42W32xUUFKRrr71WP/3pTyWda2xFRERo6tSpZnxNTY26du2qTp066ciRI+b40qVL5e/vr+PHj0s6dwnOFVdcoX379mnkyJEKDg7WVVddpYcfflinT592y9Visejhhx/W73//e/Xt21c2m00vv/yy+d6Fl3x+9dVXeuihhxQTE6OAgABFR0frxz/+sVs+VVVVmjVrlnr16qWAgABdffXVyszM1KlTp5rl9wsAvqatjyuStGbNGvXp00c2m019+/bVn/70p3pzra6u1oIFC3TDDTfIZrPpqquu0oMPPqijR4+6xfXs2VMpKSl68803NWDAAAUGBqpv37568803zc/r27evgoODNXjwYO3Zs8dt+T179ujee+9Vz549FRgYqJ49e+q+++7ToUOH3OLWrFkji8Wid999V7/4xS/UrVs3hYWFaezYsfrXv/7lFut0OjV79mxFRkYqKChIt956q3bt2nWpzQMA8MAbb7whi8Wiv/3tb3XeW7FihSwWiz7++OMGL/lct26dEhISdMUVV+iKK67QLbfcolWrVpnvjxgxQrGxsfrggw80ZMgQBQYG6uqrr9YTTzyhmpoaM851WeeSJUu0YMEC9erVSzabTe+++26Dl3z+7//+r+677z5FRETIZrPpmmuu0QMPPCCHw2HGlJWVafLkyerevbsCAgLUq1cvzZs3T2fPnm2G3x58CTPU4NN+9KMf6Z577tHEiRO1d+9ezZkzR5L0xz/+Udu3b9c999yje+65R1lZWercubMOHTqkv//975LONbNuv/12bdmyxVzfnj17dPz4cQUGBupvf/ub0tPTJUlbtmzRwIEDdeWVV5qxTqdTP/jBDzR58mQ99thjKigo0IIFC3To0CFt2rTJLc833nhDH3zwgZ588klFRkYqPDy83nq++uorDRo0SE6nU48//rj69++vY8eO6Z133lFFRYUiIiJ0+vRpDR8+XKWlpWbMvn379OSTT2rv3r3asmVLm90LAQC8XVsdV9asWaMHH3xQd911l5577jlVVlYqKytLDodDnTp9+/fT2tpa3XXXXfrggw80e/ZsDR06VIcOHdJTTz2lESNGaM+ePQoMDDTj//GPf2jOnDmaO3eu7Ha75s2bp7Fjx2rOnDn629/+poULF8pisejRRx9VSkqKDh48aC7/+eefq0+fPrr33nsVGhqqw4cPa8WKFRo0aJD279+vbt26uf3ufvazn2nMmDFat26dvvzyS/3Hf/yH7r//fvP3I0mTJk3Sn/70J82aNUuJiYkqLi7W2LFjdeLEiWbcigDg21JSUhQeHq7Vq1dr5MiRbu+tWbNG3/3ud9W/f3+9/vrrdZZ98sknNX/+fI0dO1YzZ86U3W5XcXFxnT+mlJWV6d5779Vjjz2mp59+Wm+99ZYWLFigiooKLV++3C32t7/9ra6//nr9+te/VpcuXdS7d+968/7HP/6hW2+9Vd26ddPTTz+t3r176/Dhw9q4caOqq6tls9lUVlamwYMHq1OnTnryySd13XXXafv27VqwYIE+//xzrV69+jJ/e/ApBuCDnnrqKUOSsWTJErfxKVOmGJ07dzZqa2uNX//614Yk4/jx4w2u5w9/+IMhyfjiiy8MwzCMBQsWGDfccIORmppqPPjgg4ZhGEZ1dbURHBxsPP744+Zy48ePNyQZv/nNb9zW98wzzxiSjG3btpljkgy73W58/fXXdT5fkvHUU0+Zr3/6058aVqvV2L9/f4M5L1q0yOjUqZOxe/dut/G//OUvhiRj8+bNDS4LAKhfWx5XampqjOjoaOO73/2uUVtba67r888/N6xWq9GjRw9z7NVXXzUkGa+99prb5+7evduQZLzwwgvmWI8ePYzAwECjtLTUHCsqKjIkGVFRUcapU6fM8TfeeMOQZGzcuLHB2s6ePWucPHnSCA4Odjv+rV692pBkTJkyxS1+yZIlhiTj8OHDhmEYxoEDBwxJxi9/+Uu3uLVr1xqSjPHjxzf42QAAz8yYMcMIDAx0O2bt37/fkGRkZ2cbhvHtsc/ls88+M/z8/Ixx48ZddN3Dhw83JBl//etf3cYnTZpkdOrUyTh06JBhGIZx8OBBQ5Jx3XXXGdXV1W6xrvdWr15tjt1+++3GlVdeaZSXlzf42ZMnTzauuOIK8zNcXMfoffv2XTR34Hxc8gmflpqa6va6f//++uabb1ReXq5BgwZJktLS0vRf//Vf+uqrr+osP2rUKEkyZxPk5+crMTFRo0aNUn5+vqRzDw44deqUGXu+cePGub12zTx499133cZvv/12de3a9ZL1vP322/r+97+vvn37Nhjz5ptvKjY2VrfccovOnj1r/iQnJ8tisei999675OcAAOrXFseVkpIS/etf/1J6errbDOMePXpo6NChbut/8803deWVV+rOO+90OwbccsstioyMrHMMuOWWW3T11Vebr13HlxEjRigoKKjO+PkzEE6ePKlHH31U3/nOd+Tv7y9/f39dccUVOnXqlA4cONCo393563QdGy88dqalpcnfn4suAKA5/fSnP9WZM2f05z//2RxbvXq1bDabec5yofz8fNXU1LjduqAhISEhdb7309PTVVtbq/fff99tPDU1VVar9aLrO336tLZu3aq0tDRdddVVDca9+eab+v73v6/o6Gi34+Add9whSdq6deslcwdcaKjBp4WFhbm9ttlskqQzZ87otttu0xtvvKGzZ8/qgQceUPfu3RUbG6tXX33VjO/Ro4euu+46bdmyRadPn9b27dvNE5/S0lKVlJRoy5YtCgwMrHNS4+/vX+fzIyMjJUnHjh1zG4+KimpUPUePHr3kjUGPHDmijz/+WFar1e0nJCREhmHo3//+d6M+CwBQV1scV1zHDNcx5HwXjh05ckTHjx9XQEBAneNAWVlZnWNAaGio2+uAgICLjn/zzTfmWHp6upYvX66f/exneuedd7Rr1y7t3r1bV111lc6cOePR7+5iddZ3PAUAXJ6bbrpJgwYNMi+BrKmpUU5Oju666646xwAX1704G/OggoiIiDpjl3MuVFFRoZqamkadC23atKnOMfCmm26SJM6F4BH+nAdcxF133aW77rpLDodDO3bs0KJFi5Senq6ePXsqISFBkjRy5Ej99a9/1datW1VbW6sRI0YoJCRE0dHRys/P15YtW/S9733PPDFwOXv2rI4dO+Z2ElBWViap7klFY+9pdtVVV6m0tPSiMd26dVNgYKD++Mc/Nvg+AKBltMRxxXXMcB1DznfhmOuG/7m5ufXmFxIS0ix1VlZW6s0339RTTz2lxx57zBx3OBz6+uuvm7TO8+s8f9ac63gKAGheDz74oKZMmaIDBw7os88+0+HDh/Xggw82GO+aGVZaWqqYmJiLrvv8B+24XM65UGhoqPz8/Bp1LtS/f38988wz9b4fHR19yc8CXJihBjSCzWbT8OHDtXjxYknSRx99ZL43atQoHTlyRMuWLdOQIUPMk5GRI0dqw4YN2r17d72Xe0rS2rVr3V6vW7dO0rlLaZrijjvu0LvvvquSkpIGY1JSUvTPf/5TYWFhiouLq/PTs2fPJn02AKDxmvO40qdPH0VFRenVV1+VYRjm+KFDh1RQUOD2uSkpKTp27JhqamrqPQb06dOnWeqzWCwyDKPOH5P+8Ic/uD3BzROuY+OFx87/+q//4slsANAC7rvvPnXu3Flr1qzRmjVrdPXVVyspKanB+KSkJPn5+WnFihWXXPeJEye0ceNGt7F169apU6dOuu222zzONTAwUMOHD9d///d/X3SWWUpKioqLi3XdddfVexykoQZPMEMNaMCTTz6p0tJSjRw5Ut27d9fx48f1m9/8RlarVcOHDzfjbr/9dlksFuXl5WnevHnm+KhRozR+/Hjz3y8UEBCg5557TidPntSgQYPMp3zecccduvXWW5uU89NPP623335bt912mx5//HH169dPx48fV25urmbMmKEbbrhBmZmZeu2113Tbbbfpl7/8pfr376/a2lp98cUXysvL08yZMxUfH9+kzwcANKyljiudOnXS/Pnz9bOf/Uw//OEPNWnSJB0/flxZWVl1Lo+89957tXbtWv3gBz/QI488osGDB8tqtaq0tFTvvvuu7rrrLv3whz+87Fq7dOmi2267Tc8++6y6deumnj17auvWrVq1apXbE6890bdvX91///1atmyZrFarRo0apeLiYvOpbwCA5nXllVfqhz/8odasWaPjx49r1qxZbk+OvlDPnj31+OOPa/78+Tpz5ozuu+8+2e127d+/X//+97/djmlhYWH6xS9+oS+++ELXX3+9Nm/erJUrV+oXv/iFrrnmmiblu3TpUt16662Kj4/XY489pu985zs6cuSINm7cqBdffFEhISF6+umnlZ+fr6FDh2r69Onq06ePvvnmG33++efavHmzfv/73zfqklVAoqEGNCg+Pl579uzRo48+qqNHj+rKK69UXFyc/v73v5vX2EvnDga33HKLPvroI7cTHNe/u96/kNVq1Ztvvqnp06drwYIFCgwM1KRJk/Tss882Oeerr75au3bt0lNPPaVf/epXOnbsmK666irdeuut5r0OgoOD9cEHH+hXv/qVXnrpJR08eFCBgYG65pprNGrUKGaoAUALacnjysSJEyVJixcv1tixY82Tmq1bt7o9aMDPz08bN27Ub37zG73yyitatGiR/P391b17dw0fPlz9+vVrtnrXrVunRx55RLNnz9bZs2c1bNgw5efna8yYMU1e56pVqxQREaE1a9bot7/9rW655Ra99tpruvfee5stbwDAtx588EHzXp8TJky4ZPzTTz+t3r17Kzs7W+PGjZO/v7969+6t6dOnu8VFRkbqd7/7nWbNmqW9e/cqNDRUjz/+uFvTzVM333yzeS40Z84cnThxQpGRkbr99tvNe31GRUVpz549mj9/vp599lmVlpYqJCREvXr10ujRoxv1IDjAxWKcf20AgFYxYcIE/eUvf9HJkyfbOhUAAAAAaDUjRozQv//9bxUXF7d1KsBl4R5qAAAAAAAAgAdoqAEAAAAAAAAe4JJPAAAAAAAAwAPMUAMAAAAAAAA8QEMNAAAAAAAA8AANNQAAAAAAAMAD/m2dQFuqra3Vv/71L4WEhMhisbR1OgDQJgzD0IkTJxQdHa1Onfg7S1vgeAQAHI/aA45HAND445FPN9T+9a9/KSYmpq3TAIB24csvv1T37t3bOg2fxPEIAL7F8ajtcDwCgG9d6njk0w21kJAQSed+SV26dPFoWafTqby8PCUlJclqtbZEeu2Cr9QpUWtH5Su1Xk6dVVVViomJMb8T0fo4Hl2ar9QpUWtH5Ct1ShyPvF1Tj0fs4x2Tr9TqK3VK1NpYjT0e+XRDzTWNuUuXLk06gQkKClKXLl069I7oK3VK1NpR+UqtzVEnl3a0HY5Hl+YrdUrU2hH5Sp0SxyNv19TjEft4x+QrtfpKnRK1eupSxyNuTgAAAAAAAAB4gIYaAAAAAAAA4AEaagAAAAAAAIAHPG6ovf/++7rzzjsVHR0ti8WiN954w+39CRMmyGKxuP0MGTLELcbhcGjatGnq1q2bgoODlZqaqtLSUreYiooKZWRkyG63y263KyMjQ8ePH3eL+eKLL3TnnXcqODhY3bp10/Tp01VdXe1pSQAAAAAAAECjedxQO3XqlG6++WYtX768wZjRo0fr8OHD5s/mzZvd3s/MzNSGDRu0fv16bdu2TSdPnlRKSopqamrMmPT0dBUVFSk3N1e5ubkqKipSRkaG+X5NTY3GjBmjU6dOadu2bVq/fr1ee+01zZw509OSAAAAAAAAgEbz+Cmfd9xxh+64446LxthsNkVGRtb7XmVlpVatWqVXXnlFo0aNkiTl5OQoJiZGW7ZsUXJysg4cOKDc3Fzt2LFD8fHxkqSVK1cqISFBJSUl6tOnj/Ly8rR//359+eWXio6OliQ999xzmjBhgp555hmPn5IGAAAAAAAANIbHDbXGeO+99xQeHq4rr7xSw4cP1zPPPKPw8HBJUmFhoZxOp5KSksz46OhoxcbGqqCgQMnJydq+fbvsdrvZTJOkIUOGyG63q6CgQH369NH27dsVGxtrNtMkKTk5WQ6HQ4WFhfr+979fJy+HwyGHw2G+rqqqknTucapOp9OjGl3xni7nbXylTolaOypfqfVy6uzovxsAAAAAaG7N3lC744479JOf/EQ9evTQwYMH9cQTT+j2229XYWGhbDabysrKFBAQoK5du7otFxERobKyMklSWVmZ2YA7X3h4uFtMRESE2/tdu3ZVQECAGXOhRYsWad68eXXG8/LyFBQU1KR68/Pzm7Sct/GVOiVq7ah8pdam1Hn69OkWyAQAAAAAOq5mb6jdc8895r/HxsYqLi5OPXr00FtvvaWxY8c2uJxhGLJYLObr8//9cmLON2fOHM2YMcN8XVVVpZiYGCUlJXl8iajT6VR+fr4SExNltVobtUxs1jt1xoqzkj363NbWlDq9FbV2TL5S6+XU6ZqtC8C39XzsrXrHP//VmFbOBIC3is16R44a93MxvkMAdFQtcsnn+aKiotSjRw99+umnkqTIyEhVV1eroqLCbZZaeXm5hg4dasYcOXKkzrqOHj1qzkqLjIzUzp073d6vqKiQ0+msM3PNxWazyWaz1Rm3Wq1NPtH2ZNkLDy6u5b3B5fyOvA21dky+UmtT6vSF3wsAAAAANCePn/LpqWPHjunLL79UVFSUJGngwIGyWq1ulyUdPnxYxcXFZkMtISFBlZWV2rVrlxmzc+dOVVZWusUUFxfr8OHDZkxeXp5sNpsGDhzY0mUBAAAAAADAR3k8Q+3kyZP6v//7P/P1wYMHVVRUpNDQUIWGhiorK0s/+tGPFBUVpc8//1yPP/64unXrph/+8IeSJLvdrokTJ2rmzJkKCwtTaGioZs2apX79+plP/ezbt69Gjx6tSZMm6cUXX5QkPfTQQ0pJSVGfPn0kSUlJSbrxxhuVkZGhZ599Vl9//bVmzZqlSZMm8YRPAAAAAAAAtBiPZ6jt2bNHAwYM0IABAyRJM2bM0IABA/Tkk0/Kz89Pe/fu1V133aXrr79e48eP1/XXX6/t27crJCTEXMfzzz+vu+++W2lpaRo2bJiCgoK0adMm+fn5mTFr165Vv379lJSUpKSkJPXv31+vvPKK+b6fn5/eeustde7cWcOGDVNaWpruvvtu/frXv76c3wcAoINYtGiRBg0apJCQEIWHh+vuu+9WSUmJW8yECRNksVjcfoYMGeIW43A4NG3aNHXr1k3BwcFKTU1VaWmpW0xFRYUyMjJkt9tlt9uVkZGh48ePt3SJAAAAANqIxzPURowYIcMwGnz/nXfq3nj/Qp07d1Z2drays7MbjAkNDVVOTs5F13PNNdfozTffvOTnAQB8z9atWzV16lQNGjRIZ8+e1dy5c5WUlKT9+/crODjYjBs9erRWr15tvg4ICHBbT2ZmpjZt2qT169crLCxMM2fOVEpKigoLC80/BKWnp6u0tFS5ubmSzs2qzsjI0KZNm1qhUgAAAACtrcUfSgAAQFtwNbdcVq9erfDwcBUWFuq2224zx202myIjI+tdR2VlpVatWqVXXnnFvC1BTk6OYmJitGXLFiUnJ+vAgQPKzc3Vjh07FB8fL0lauXKlEhISVFJSYt6qAGgvGnqaJwAAABqPhhoAwCdUVlZKOjcD+nzvvfeewsPDdeWVV2r48OF65plnFB4eLkkqLCyU0+lUUlKSGR8dHa3Y2FgVFBQoOTlZ27dvl91uN5tpkjRkyBDZ7XYVFBTU21BzOBxyOBzm66qqKkmS0+mU0+n0qC5XvKfLeRtfqVNq+Vptfg1faVCflvyd+8p29ZU6pcur1Rd+PwCAjoOGGgCgwzMMQzNmzNCtt96q2NhYc/yOO+7QT37yE/Xo0UMHDx7UE088odtvv12FhYWy2WwqKytTQECAunbt6ra+iIgIlZWVSZLKysrMBtz5wsPDzZgLLVq0SPPmzasznpeXp6CgoCbVeP7TszsyX6lTarlalwz2LH7z5s0tksf5fGW7+kqdUtNqPX36dAtkAgBAy6ChBgDo8B5++GF9/PHH2rZtm9v4PffcY/57bGys4uLi1KNHD7311lsaO3Zsg+szDEMWi8V8ff6/NxRzvjlz5mjGjBnm66qqKsXExCgpKcnjJ1U7nU7l5+crMTFRVqvVo2W9ia/UKbV8rbFZl77f7fmKs5KbPQcXX9muvlKndHm1umbrAgDgDWioAQA6tGnTpmnjxo16//331b1794vGRkVFqUePHvr0008lSZGRkaqurlZFRYXbLLXy8nINHTrUjDly5EiddR09elQRERH1fo7NZpPNZqszbrVam3yyfTnLehNfqVNquVodNfU3ei+WR0vzle3qK3VKTavVV343AICOoVNbJwAAQEswDEMPP/ywXn/9df39739Xr169LrnMsWPH9OWXXyoqKkqSNHDgQFmtVrdLlw4fPqzi4mKzoZaQkKDKykrt2rXLjNm5c6cqKyvNGAAAAAAdCzPUAAAd0tSpU7Vu3Tr99a9/VUhIiHk/M7vdrsDAQJ08eVJZWVn60Y9+pKioKH3++ed6/PHH1a1bN/3whz80YydOnKiZM2cqLCxMoaGhmjVrlvr162c+9bNv374aPXq0Jk2apBdffFGS9NBDDyklJYUnfAIAAAAdFA01AECHtGLFCknSiBEj3MZXr16tCRMmyM/PT3v37tWf/vQnHT9+XFFRUfr+97+vP//5zwoJCTHjn3/+efn7+ystLU1nzpzRyJEjtWbNGvn5+Zkxa9eu1fTp082ngaampmr58uUtXyQAAACANkFDDQDQIRmGcdH3AwMD9c47l745e+fOnZWdna3s7OwGY0JDQ5WTk+NxjkBL6/nYW22dAgAAQIfEPdQAAAAAAAAAD9BQAwAAAAAAADxAQw0AAAAAAADwAPdQawHcrwQAAAAAAKDjYoYaAAAAAFymRYsWyWKxKDMz0xwzDENZWVmKjo5WYGCgRowYoX379rkt53A4NG3aNHXr1k3BwcFKTU1VaWmpW0xFRYUyMjJkt9tlt9uVkZGh48ePu8V88cUXuvPOOxUcHKxu3bpp+vTpqq6ubqlyAcDn0VADAAAAgMuwe/duvfTSS+rfv7/b+JIlS7R06VItX75cu3fvVmRkpBITE3XixAkzJjMzUxs2bND69eu1bds2nTx5UikpKaqpqTFj0tPTVVRUpNzcXOXm5qqoqEgZGRnm+zU1NRozZoxOnTqlbdu2af369Xrttdc0c+bMli8eAHwUDTUAAAAAaKKTJ09q3LhxWrlypbp27WqOG4ahZcuWae7cuRo7dqxiY2P18ssv6/Tp01q3bp0kqbKyUqtWrdJzzz2nUaNGacCAAcrJydHevXu1ZcsWSdKBAweUm5urP/zhD0pISFBCQoJWrlypN998UyUlJZKkvLw87d+/Xzk5ORowYIBGjRql5557TitXrlRVVVXr/1IAwAdwDzUAAAAAaKKpU6dqzJgxGjVqlBYsWGCOHzx4UGVlZUpKSjLHbDabhg8froKCAk2ePFmFhYVyOp1uMdHR0YqNjVVBQYGSk5O1fft22e12xcfHmzFDhgyR3W5XQUGB+vTpo+3btys2NlbR0dFmTHJyshwOhwoLC/X973+/3twdDoccDof52tV8czqdcjqdjf4duGJtnYwG3+soXPV0tLrq4yu1+kqdErV6uuyl0FADAADwcjwQCWgb69ev14cffqjdu3fXea+srEySFBER4TYeERGhQ4cOmTEBAQFuM9tcMa7ly8rKFB4eXmf94eHhbjEXfk7Xrl0VEBBgxtRn0aJFmjdvXp3xvLw8BQUFNbhcQ+bH1dYZ27x5s8fr8Qb5+fltnUKr8ZVafaVOiVov5fTp042Ko6EGAACABjXUrPv8V2NaOROgffnyyy/1yCOPKC8vT507d24wzmKxuL02DKPO2IUujKkvvikxF5ozZ45mzJhhvq6qqlJMTIySkpLUpUuXi+Z4PqfTqfz8fD2xp5Mcte6fV5yV3Oj1eANXrYmJibJarW2dTovylVp9pU6JWhursZfK01ADAAAAAA8VFhaqvLxcAwcONMdqamr0/vvva/ny5eb9zcrKyhQVFWXGlJeXm7PJIiMjVV1drYqKCrdZauXl5Ro6dKgZc+TIkTqff/ToUbf17Ny50+39iooKOZ3OOjPXzmez2WSz2eqMW63WJp1sO2otctS4N9Q66kl7U39H3shXavWVOiVqbcwyjcFDCQAAAADAQyNHjtTevXtVVFRk/sTFxWncuHEqKirStddeq8jISLfLjaqrq7V161azWTZw4EBZrVa3mMOHD6u4uNiMSUhIUGVlpXbt2mXG7Ny5U5WVlW4xxcXFOnz4sBmTl5cnm83m1vADADQfZqgBAAAAgIdCQkIUGxvrNhYcHKywsDBzPDMzUwsXLlTv3r3Vu3dvLVy4UEFBQUpPT5ck2e12TZw4UTNnzlRYWJhCQ0M1a9Ys9evXT6NGjZIk9e3bV6NHj9akSZP04osvSpIeeughpaSkqE+fPpKkpKQk3XjjjcrIyNCzzz6rr7/+WrNmzdKkSZM8unQTANB4NNQAAAAAoAXMnj1bZ86c0ZQpU1RRUaH4+Hjl5eUpJCTEjHn++efl7++vtLQ0nTlzRiNHjtSaNWvk5+dnxqxdu1bTp083nwaampqq5cuXm+/7+fnprbfe0pQpUzRs2DAFBgYqPT1dv/71r1uvWADwMR5f8vn+++/rzjvvVHR0tCwWi9544w3zPafTqUcffVT9+vVTcHCwoqOj9cADD+hf//qX2zpGjBghi8Xi9nPvvfe6xVRUVCgjI0N2u112u10ZGRk6fvy4W8wXX3yhO++8U8HBwerWrZumT5+u6upqT0sCAAAAgMv23nvvadmyZeZri8WirKwsHT58WN988422bt1aZ1Zb586dlZ2drWPHjun06dPatGmTYmJi3GJCQ0OVk5OjqqoqVVVVKScnR1deeaVbzDXXXKM333xTp0+f1rFjx5SdnV3v/dEAAM3D44baqVOndPPNN7v9RcTl9OnT+vDDD/XEE0/oww8/1Ouvv65PPvlEqampdWInTZqkw4cPmz+u6csu6enpKioqUm5urnJzc1VUVKSMjAzz/ZqaGo0ZM0anTp3Stm3btH79er322muaOXOmpyUBAAAAAAAAjebxJZ933HGH7rjjjnrfs9vtbjfUlKTs7GwNHjxYX3zxha655hpzPCgoSJGRkfWu58CBA8rNzdWOHTsUHx8vSVq5cqUSEhJUUlKiPn36KC8vT/v379eXX36p6OhoSdJzzz2nCRMm6JlnnuFeAQAAAAAAAGgRLX4PtcrKSlksljpTkteuXaucnBxFRETojjvu0FNPPWXeS2D79u2y2+1mM02ShgwZIrvdroKCAvXp00fbt29XbGys2UyTpOTkZDkcDhUWFur73/9+nVwcDoccDof5uqqqStK5S1WdTqdHdbni61vO5md4vJ726mJ1djTU2jH5Sq2XU2dH/90AAAAAQHNr0YbaN998o8cee0zp6eluM8bGjRunXr16KTIyUsXFxZozZ47+8Y9/mLPbysrKFB4eXmd94eHhKisrM2MiIiLc3u/atasCAgLMmAstWrRI8+bNqzOel5enoKCgJtV44Yw8SVoyuPHLb968uUmf29rqq7OjotaOyVdqbUqdp0+fboFMAAAAAKDjarGGmtPp1L333qva2lq98MILbu9NmjTJ/PfY2Fj17t1bcXFx+vDDD/Xd735X0rkbeF7IMAy38cbEnG/OnDmaMWOG+bqqqkoxMTFKSkry+BJRp9Op/Px8JSYmymq1ur0Xm/VOo9dTnJXs0ee2tovV2dFQa8fkK7VeTp2u2boAAAAAgMZpkYaa0+lUWlqaDh48qL///e+XbFZ997vfldVq1aeffqrvfve7ioyM1JEjR+rEHT161JyVFhkZqZ07d7q9X1FRIafTWWfmmovNZqv3STdWq7XJJ9r1Leuoqb+h19Dy3uByfkfehlo7Jl+ptSl1+sLvBQAAAACak8dP+bwUVzPt008/1ZYtWxQWFnbJZfbt2yen06moqChJUkJCgiorK7Vr1y4zZufOnaqsrNTQoUPNmOLiYh0+fNiMycvLk81m08CBA5u5KgAAAAAAAOAcj2eonTx5Uv/3f/9nvj548KCKiooUGhqq6Oho/fjHP9aHH36oN998UzU1Neb9zEJDQxUQEKB//vOfWrt2rX7wgx+oW7du2r9/v2bOnKkBAwZo2LBhkqS+fftq9OjRmjRpkl588UVJ0kMPPaSUlBT16dNHkpSUlKQbb7xRGRkZevbZZ/X1119r1qxZmjRpEk/4BAAAAAAAQIvxeIbanj17NGDAAA0YMECSNGPGDA0YMEBPPvmkSktLtXHjRpWWluqWW25RVFSU+VNQUCBJCggI0N/+9jclJyerT58+mj59upKSkrRlyxb5+fmZn7N27Vr169dPSUlJSkpKUv/+/fXKK6+Y7/v5+emtt95S586dNWzYMKWlpenuu+/Wr3/968v9nQAAAAAAAAAN8niG2ogRI2QYRoPvX+w9SYqJidHWrVsv+TmhoaHKycm5aMw111yjN99885LrAgAAAAAAAJpLs99DDQAAAAAAAOjIaKgBAAAAAAAAHqChBgAAAAAAAHiAhhoAoENatGiRBg0apJCQEIWHh+vuu+9WSUmJW4xhGMrKylJ0dLQCAwM1YsQI7du3zy3G4XBo2rRp6tatm4KDg5WamqrS0lK3mIqKCmVkZMhut8tutysjI0PHjx9v6RIBAAAAtBGPH0oAAIA32Lp1q6ZOnapBgwbp7Nmzmjt3rpKSkrR//34FBwdLkpYsWaKlS5dqzZo1uv7667VgwQIlJiaqpKREISEhkqTMzExt2rRJ69evV1hYmGbOnKmUlBQVFhaaT6dOT09XaWmpcnNzJUkPPfSQMjIytGnTprYpHh1Wz8feausUAAAAIBpqAIAOytXcclm9erXCw8NVWFio2267TYZhaNmyZZo7d67Gjh0rSXr55ZcVERGhdevWafLkyaqsrNSqVav0yiuvaNSoUZKknJwcxcTEaMuWLUpOTtaBAweUm5urHTt2KD4+XpK0cuVKJSQkqKSkRH369GndwgEAAAC0OBpqAACfUFlZKUkKDQ2VJB08eFBlZWVKSkoyY2w2m4YPH66CggJNnjxZhYWFcjqdbjHR0dGKjY1VQUGBkpOTtX37dtntdrOZJklDhgyR3W5XQUFBvQ01h8Mhh8Nhvq6qqpIkOZ1OOZ1Oj+pyxXu6nLfxlTqli9dq8zNaO50GNce28JXt6it1SpdXqy/8fgAAHQcNNQBAh2cYhmbMmKFbb71VsbGxkqSysjJJUkREhFtsRESEDh06ZMYEBASoa9eudWJcy5eVlSk8PLzOZ4aHh5sxF1q0aJHmzZtXZzwvL09BQUEeVndOfn5+k5bzNr5Sp1R/rUsGt0EiDdi8eXOzrctXtquv1Ck1rdbTp0+3QCYAALQMGmoAgA7v4Ycf1scff6xt27bVec9isbi9NgyjztiFLoypL/5i65kzZ45mzJhhvq6qqlJMTIySkpLUpUuXi372hZxOp/Lz85WYmCir1erRst7EV+qULl5rbNY7bZRVXcVZyZe9Dl/Zrr5Sp3R5tbpm6wIA/l979x4dVX3v//815DK5FEYC5iYB0YMRDCqFEoJVQCEhh5BjsYWf6cmBFgEXCuZAjgek1tAKKFbUhkophwNooHhaxarQmNBWFMNFUtJyK9UKCKsJ4RISLnEyJvv3B99sGTIBdsgkmZnnY60smM+8957Pewbmnf2ez+wNX0BDDQDg12bMmKF33nlHH374oXr06GGOx8bGSrq4wiwuLs4cr6ysNFetxcbGqq6uTlVVVW6r1CorKzV06FAz5vjx400e98SJE01WvzWy2+2y2+1NxkNCQlp8sH092/qSQMlT8pyrs/7Kzd621JqvQ6C8roGSp9SyXAPluQEA+IdO7T0BAAC8wTAMPf7443rrrbf0xz/+Ub1793a7v3fv3oqNjXX7WlJdXZ22bNliNssGDhyokJAQt5jy8nLt3bvXjElJSVF1dbV27txpxuzYsUPV1dVmDAAAAAD/wgo1AIBfeuyxx7Ru3Tr97ne/U+fOnc3zmTkcDoWHh8tmsyknJ0cLFy5Unz591KdPHy1cuFARERHKysoyYydPnqzZs2erW7duioqKUm5urvr3729e9bNv374aPXq0pkyZouXLl0uSpk6dqoyMDK7wCQAAAPgpGmoAAL+0bNkySdLw4cPdxletWqVJkyZJkp588knV1tZq+vTpqqqqUnJysoqKitS5c2cz/qWXXlJwcLDGjx+v2tpaPfDAA1q9erWCgoLMmLVr12rmzJnm1UAzMzO1dOlS7yYIAAAAoN3QUAMA+CXDMK4aY7PZlJeXp7y8vGZjwsLClJ+fr/z8/GZjoqKiVFBQ0JJpAj7r5jkbPY4ffm5MG88EAACg7XEONQAAAAAAAMACGmoAAAAAAACABTTUAAAAAAAAAAtoqAEAAAAAAAAW0FADAAAAAAAALOAqnwAAAB1QUt77ctbb2nsaAAAA8IAVagAAAAAAAIAFNNQAAAAAAAAAC2ioAQAAAAAAABZYbqh9+OGHGjt2rOLj42Wz2fT222+73W8YhvLy8hQfH6/w8HANHz5c+/btc4txOp2aMWOGunfvrsjISGVmZurYsWNuMVVVVcrOzpbD4ZDD4VB2drbOnDnjFvPFF19o7NixioyMVPfu3TVz5kzV1dVZTQkAAAAAAAC4ZpYbaufPn9ddd92lpUuXerx/8eLFWrJkiZYuXapPPvlEsbGxGjVqlM6ePWvG5OTkaMOGDVq/fr22bt2qc+fOKSMjQ/X19WZMVlaWysrKVFhYqMLCQpWVlSk7O9u8v76+XmPGjNH58+e1detWrV+/Xm+++aZmz55tNSUAAAAAsGzZsmW688471aVLF3Xp0kUpKSn6/e9/b97PYgMA8F+WG2rp6el69tlnNW7cuCb3GYahl19+WfPmzdO4ceOUlJSkNWvW6MKFC1q3bp0kqbq6WitXrtSLL76okSNHasCAASooKNCePXu0efNmSdKBAwdUWFio//mf/1FKSopSUlK0YsUKvffeezp48KAkqaioSPv371dBQYEGDBigkSNH6sUXX9SKFStUU1NzPc8JAAAAAFxVjx499Nxzz2nXrl3atWuX7r//fv3bv/2b2TRjsQEA+K/g1tzZoUOHVFFRodTUVHPMbrdr2LBhKikp0bRp01RaWiqXy+UWEx8fr6SkJJWUlCgtLU3btm2Tw+FQcnKyGTNkyBA5HA6VlJQoMTFR27ZtU1JSkuLj482YtLQ0OZ1OlZaWasSIEU3m53Q65XQ6zduNjTeXyyWXy2Up18Z4T9vZgwzL++morpSnvyFX/xQouV5Pnv7+3AAA4C1jx451u71gwQItW7ZM27dvV79+/dwWG0jSmjVrFBMTo3Xr1mnatGnmYoPXX39dI0eOlCQVFBQoISFBmzdvVlpamrnYYPv27ebx0YoVK5SSkqKDBw8qMTHRXGxw9OhR8/joxRdf1KRJk7RgwQJ16dKlDZ8VAAgMrdpQq6iokCTFxMS4jcfExOjIkSNmTGhoqLp27dokpnH7iooKRUdHN9l/dHS0W8zlj9O1a1eFhoaaMZdbtGiR5s+f32S8qKhIERER15JiE8XFxU3GFg++9u03bdrUosdta57y9Ffk6p8CJdeW5HnhwgUvzAQAgMBSX1+v3/zmNzp//rxSUlI6/GIDqfUWHDTG2js1XVjgbx/cBcqHtVLg5BooeUrkanXbq2nVhlojm83mdtswjCZjl7s8xlN8S2IuNXfuXM2aNcu8XVNTo4SEBKWmplr+1Mblcqm4uFijRo1SSEiI231Jee9f83725qVZety2dqU8/Q25+qdAyfV68uRr8gAAtNyePXuUkpKiL7/8Ut/4xje0YcMG9evXTyUlJZI67mIDqfUXHPx0UEOTMV9ZQGBVoHxYKwVOroGSp0SuV3OtCw5ataEWGxsr6eIbelxcnDleWVlpvsHHxsaqrq5OVVVVboWjsrJSQ4cONWOOHz/eZP8nTpxw28+OHTvc7q+qqpLL5WpSTBrZ7XbZ7fYm4yEhIS0+0Pa0rbP+ys3Dy7f3BdfzHPkacvVPgZJrS/IMhOcFAABvSUxMVFlZmc6cOaM333xTEydO1JYtW8z7O+piA6n1Fhw0frD39K5Ocja4P15HX0BgVaB8WCsFTq6BkqdErtfqWhcctGpDrXfv3oqNjVVxcbEGDBggSaqrq9OWLVv0/PPPS5IGDhyokJAQFRcXa/z48ZKk8vJy7d27V4sXL5YkpaSkqLq6Wjt37tTgwRe/P7ljxw5VV1ebTbeUlBQtWLBA5eXlZvOuqKhIdrtdAwcObM20AAAAAMCj0NBQ/cu//IskadCgQfrkk0/0yiuv6L//+78lddzFBlLrLzhwNtiaLC7w14P2QPmwVgqcXAMlT4lcr2Wba2H5Kp/nzp1TWVmZysrKJF28EEFZWZm++OIL2Ww25eTkaOHChdqwYYP27t2rSZMmKSIiQllZWZIkh8OhyZMna/bs2frDH/6g3bt369///d/Vv39/80Scffv21ejRozVlyhRt375d27dv15QpU5SRkaHExERJUmpqqvr166fs7Gzt3r1bf/jDH5Sbm6spU6Zw0k0AAAAA7cIwDDmdTrfFBo0aFxs0NssuXWzQqHGxwaULCRoXGzTytNhg7969Ki8vN2NYbAAA3mV5hdquXbvcTmrZuER44sSJWr16tZ588knV1tZq+vTpqqqqUnJysoqKitS5c2dzm5deeknBwcEaP368amtr9cADD2j16tUKCgoyY9auXauZM2eaJ+jMzMzU0qVLzfuDgoK0ceNGTZ8+Xffcc4/Cw8OVlZWln/3sZ9afBQAAAACw6KmnnlJ6eroSEhJ09uxZrV+/Xh988IEKCwvdFhv06dNHffr00cKFC5tdbNCtWzdFRUUpNze32cUGy5cvlyRNnTq12cUGL7zwgk6fPs1iAwDwMssNteHDh8swml69pZHNZlNeXp7y8vKajQkLC1N+fr7y8/ObjYmKilJBQcEV59KzZ0+99957V50zAAAAALS248ePKzs7W+Xl5XI4HLrzzjtVWFioUaNGSRKLDQDAj3nlKp8AAAAITDfP2ehx/PBzY9p4JoD3rVy58or3s9gAAPyX5XOoAQAAAAAAAIGMFWoAAADtxNNqLnuQocWD22EyAAAAuGasUAMAAAAAAAAsoKEGAAAAAAAAWEBDDQAAAAAAALCAhhoAAAAAAABgAQ01AAAAAAAAwAIaagAAv/Thhx9q7Nixio+Pl81m09tvv+12/6RJk2Sz2dx+hgwZ4hbjdDo1Y8YMde/eXZGRkcrMzNSxY8fcYqqqqpSdnS2HwyGHw6Hs7GydOXPGy9kBAAAAaE801AAAfun8+fO66667tHTp0mZjRo8erfLycvNn06ZNbvfn5ORow4YNWr9+vbZu3apz584pIyND9fX1ZkxWVpbKyspUWFiowsJClZWVKTs722t5AQAAAGh/we09AQAAvCE9PV3p6elXjLHb7YqNjfV4X3V1tVauXKnXX39dI0eOlCQVFBQoISFBmzdvVlpamg4cOKDCwkJt375dycnJkqQVK1YoJSVFBw8eVGJiYusmhQ7l5jkbrzn28HNjvDgTAAAAtDUaagCAgPXBBx8oOjpaN9xwg4YNG6YFCxYoOjpaklRaWiqXy6XU1FQzPj4+XklJSSopKVFaWpq2bdsmh8NhNtMkaciQIXI4HCopKWm2oeZ0OuV0Os3bNTU1kiSXyyWXy2Uph8Z4q9v5mo6Ypz3IuObYxHnvNbMPD2OdDLc//YWn164jvq7eECh5SteXayA8PwAA/0FDDQAQkNLT0/W9731PvXr10qFDh/T000/r/vvvV2lpqex2uyoqKhQaGqquXbu6bRcTE6OKigpJUkVFhdmAu1R0dLQZ48miRYs0f/78JuNFRUWKiIhoUT7FxcUt2s7XdKQ8Fw/27v5/OqjBuw/Qxi7/SvWlOtLr6k2BkqfUslwvXLjghZkAAOAdNNQAAAFpwoQJ5t+TkpI0aNAg9erVSxs3btS4ceOa3c4wDNlsNvP2pX9vLuZyc+fO1axZs8zbNTU1SkhIUGpqqrp06WIpD5fLpeLiYo0aNUohISGWtvUlHTHPpLz3vbJfeydDPx3UoKd3dZKzofl/R75mb15ak7GO+Lp6Q6DkKV1fro2rdQEA8AU01AAAkBQXF6devXrp008/lSTFxsaqrq5OVVVVbqvUKisrNXToUDPm+PHjTfZ14sQJxcTENPtYdrtddru9yXhISEiLD7avZ1tf0h55Nn+uNO82u5wNNjnr/aehdqXXjX+//qcluQbKcwMA8A9c5RMAAEmnTp3S0aNHFRcXJ0kaOHCgQkJC3L62VF5err1795oNtZSUFFVXV2vnzp1mzI4dO1RdXW3GAAAAAPA/rFADAPilc+fO6bPPPjNvHzp0SGVlZYqKilJUVJTy8vL00EMPKS4uTocPH9ZTTz2l7t276zvf+Y4kyeFwaPLkyZo9e7a6deumqKgo5ebmqn///uZVP/v27avRo0drypQpWr58uSRp6tSpysjI4AqfAAAAgB+joQYA8Eu7du3SiBEjzNuN5yybOHGili1bpj179ui1117TmTNnFBcXpxEjRuiNN95Q586dzW1eeuklBQcHa/z48aqtrdUDDzyg1atXKyjo60szrl27VjNnzjSvBpqZmamlS5e2UZYAAAAA2gMNNQCAXxo+fLgMw2j2/vffv/oJ5cPCwpSfn6/8/PxmY6KiolRQUNCiOQIAAADwTTTUAAAA4HWeLu5gDzK0eHA7TAYAAOA60VBrZ81dOezwc2PaeCYAAAAAAAC4FlzlEwAAAAAAALCAFWoAAABqftU4AAAAcLlWX6F28803y2azNfl57LHHJEmTJk1qct+QIUPc9uF0OjVjxgx1795dkZGRyszM1LFjx9xiqqqqlJ2dLYfDIYfDoezsbJ05c6a10wEAAAAAAADctHpD7ZNPPlF5ebn5U1xcLEn63ve+Z8aMHj3aLWbTpk1u+8jJydGGDRu0fv16bd26VefOnVNGRobq6+vNmKysLJWVlamwsFCFhYUqKytTdnZ2a6cDAAAAAAAAuGn1r3zeeOONbrefe+453XrrrRo2bJg5ZrfbFRsb63H76upqrVy5Uq+//rpGjhwpSSooKFBCQoI2b96stLQ0HThwQIWFhdq+fbuSk5MlSStWrFBKSooOHjyoxMTE1k4LAAAAAAAAkOTlc6jV1dWpoKBAs2bNks1mM8c/+OADRUdH64YbbtCwYcO0YMECRUdHS5JKS0vlcrmUmppqxsfHxyspKUklJSVKS0vTtm3b5HA4zGaaJA0ZMkQOh0MlJSXNNtScTqecTqd5u6amRpLkcrnkcrks5dYY72k7e5BhaV9X2n97u1Ke/oZc/VOg5Ho9efr7cwMAAAAArc2rDbW3335bZ86c0aRJk8yx9PR0fe9731OvXr106NAhPf3007r//vtVWloqu92uiooKhYaGqmvXrm77iomJUUVFhSSpoqLCbMBdKjo62ozxZNGiRZo/f36T8aKiIkVERLQox8avtF5q8eAW7crN5V+DbW+e8vRX5OqfAiXXluR54cIFL8wEAAAAAPyXVxtqK1euVHp6uuLj482xCRMmmH9PSkrSoEGD1KtXL23cuFHjxo1rdl+GYbitcrv0783FXG7u3LmaNWuWebumpkYJCQlKTU1Vly5drjkv6eKKjuLiYo0aNUohISFu9yXlvW9pX57szUu77n20hivl6W/I1T8FSq7Xk2fjal0AAAAAwLXxWkPtyJEj2rx5s956660rxsXFxalXr1769NNPJUmxsbGqq6tTVVWV2yq1yspKDR061Iw5fvx4k32dOHFCMTExzT6W3W6X3W5vMh4SEtLiA21P2zrrm2/qWdlvR3I9z5GvIVf/FCi5tiTPQHheAAAAAKA1tfpVPhutWrVK0dHRGjNmzBXjTp06paNHjyouLk6SNHDgQIWEhLh9bam8vFx79+41G2opKSmqrq7Wzp07zZgdO3aourrajAEAAAAAAAC8wSsr1BoaGrRq1SpNnDhRwcFfP8S5c+eUl5enhx56SHFxcTp8+LCeeuopde/eXd/5znckSQ6HQ5MnT9bs2bPVrVs3RUVFKTc3V/379zev+tm3b1+NHj1aU6ZM0fLlyyVJU6dOVUZGBlf4BAAAAAAAgFd5paG2efNmffHFF/rhD3/oNh4UFKQ9e/botdde05kzZxQXF6cRI0bojTfeUOfOnc24l156ScHBwRo/frxqa2v1wAMPaPXq1QoKCjJj1q5dq5kzZ5pXA83MzNTSpUu9kQ4AAAAAAABg8kpDLTU1VYZhNBkPDw/X++9f/YT9YWFhys/PV35+frMxUVFRKigouK55AgAAAAAAAFZ57RxqAAAAAAAAgD+ioQYAAAAALbBo0SJ961vfUufOnRUdHa0HH3xQBw8edIsxDEN5eXmKj49XeHi4hg8frn379rnFOJ1OzZgxQ927d1dkZKQyMzN17Ngxt5iqqiplZ2fL4XDI4XAoOztbZ86ccYv54osvNHbsWEVGRqp79+6aOXOm6urqvJI7AAQ6GmoAAAAA0AJbtmzRY489pu3bt6u4uFhfffWVUlNTdf78eTNm8eLFWrJkiZYuXapPPvlEsbGxGjVqlM6ePWvG5OTkaMOGDVq/fr22bt2qc+fOKSMjQ/X19WZMVlaWysrKVFhYqMLCQpWVlSk7O9u8v76+XmPGjNH58+e1detWrV+/Xm+++aZmz57dNk8GAAQYr5xDDQAAAAD8XWFhodvtVatWKTo6WqWlpbrvvvtkGIZefvllzZs3T+PGjZMkrVmzRjExMVq3bp2mTZum6upqrVy5Uq+//rpGjhwpSSooKFBCQoI2b96stLQ0HThwQIWFhdq+fbuSk5MlSStWrFBKSooOHjyoxMREFRUVaf/+/Tp69Kji4+MlSS+++KImTZqkBQsWqEuXLm34zACA/6OhBgAAgHaVlPe+nPU2t7HDz41pp9kALVddXS3p4gXUJOnQoUOqqKhQamqqGWO32zVs2DCVlJRo2rRpKi0tlcvlcouJj49XUlKSSkpKlJaWpm3btsnhcJjNNEkaMmSIHA6HSkpKlJiYqG3btikpKclspklSWlqanE6nSktLNWLEiCbzdTqdcjqd5u2amhpJksvlksvluua8G2PtnZpemM7KfnxBYz7+lpcngZJroOQpkavVba+GhhoAAAAAXCfDMDRr1ix9+9vfVlJSkiSpoqJCkhQTE+MWGxMToyNHjpgxoaGh6tq1a5OYxu0rKioUHR3d5DGjo6PdYi5/nK5duyo0NNSMudyiRYs0f/78JuNFRUWKiIi4as6X++mghiZjmzZtsrwfX1BcXNzeU2gzgZJroOQpkevVXLhw4ZriaKgBAAAAwHV6/PHH9de//lVbt25tcp/N5r4C0zCMJmOXuzzGU3xLYi41d+5czZo1y7xdU1OjhIQEpaamWvqKqMvlUnFxsZ7e1UnOBvfH2puXds378QWNuY4aNUohISHtPR2vCpRcAyVPiVyvVeNq3auhoQYAAAAA12HGjBl655139OGHH6pHjx7meGxsrKSLq8fi4uLM8crKSnM1WWxsrOrq6lRVVeW2Sq2yslJDhw41Y44fP97kcU+cOOG2nx07drjdX1VVJZfL1WTlWiO73S673d5kPCQkpEUH284GW5Ovb/vrQXtLnyNfFCi5BkqeErleyzbXgqt8AgAAAEALGIahxx9/XG+99Zb++Mc/qnfv3m739+7dW7GxsW5fOaqrq9OWLVvMZtnAgQMVEhLiFlNeXq69e/eaMSkpKaqurtbOnTvNmB07dqi6utotZu/evSovLzdjioqKZLfbNXDgwNZPHgACHCvUAABAQLl5zsb2ngIAP/HYY49p3bp1+t3vfqfOnTub5ypzOBwKDw+XzWZTTk6OFi5cqD59+qhPnz5auHChIiIilJWVZcZOnjxZs2fPVrdu3RQVFaXc3Fz179/fvOpn3759NXr0aE2ZMkXLly+XJE2dOlUZGRlKTEyUJKWmpqpfv37Kzs7WCy+8oNOnTys3N1dTpkzhCp8A4AU01AAAAACgBZYtWyZJGj58uNv4qlWrNGnSJEnSk08+qdraWk2fPl1VVVVKTk5WUVGROnfubMa/9NJLCg4O1vjx41VbW6sHHnhAq1evVlBQkBmzdu1azZw507waaGZmppYuXWreHxQUpI0bN2r69Om65557FB4erqysLP3sZz/zUvYAENhoqAEAgHbjabXY4efGtMNMAMA6wzCuGmOz2ZSXl6e8vLxmY8LCwpSfn6/8/PxmY6KiolRQUHDFx+rZs6fee++9q84JAHD9OIcaAAAAAAAAYAENNQCAX/rwww81duxYxcfHy2az6e2333a73zAM5eXlKT4+XuHh4Ro+fLj27dvnFuN0OjVjxgx1795dkZGRyszM1LFjx9xiqqqqlJ2dLYfDIYfDoezsbJ05c8bL2QEAAABoTzTUAAB+6fz587rrrrvczi9zqcWLF2vJkiVaunSpPvnkE8XGxmrUqFE6e/asGZOTk6MNGzZo/fr12rp1q86dO6eMjAzV19ebMVlZWSorK1NhYaEKCwtVVlam7Oxsr+cHAAAAoP1wDjUAgF9KT09Xenq6x/sMw9DLL7+sefPmady4cZKkNWvWKCYmRuvWrdO0adNUXV2tlStX6vXXXzevslZQUKCEhARt3rxZaWlpOnDggAoLC7V9+3YlJydLklasWKGUlBQdPHjQvPIaAAAAAP9CQw0AEHAOHTqkiooK80ppkmS32zVs2DCVlJRo2rRpKi0tlcvlcouJj49XUlKSSkpKlJaWpm3btsnhcJjNNEkaMmSIHA6HSkpKmm2oOZ1OOZ1O83ZNTY0kyeVyyeVyWcqlMd7qdh2FPajpCb0T5zU9oba9k6GfDvKcZ1Le+xYf01J4m7N3Mtz+9GdXytVX/0174uv/T624nlwD4fkBAPgPGmoAgIBTUVEhSYqJiXEbj4mJ0ZEjR8yY0NBQde3atUlM4/YVFRWKjo5usv/o6GgzxpNFixZp/vz5TcaLiooUERFhLZn/p7i4uEXbtbfFg63Fe8rT6j58xU8HNbT3FNqMp1w3bdrUDjPxLl/9f9oSLcn1woULXpgJAADeQUMNABCwbDab223DMJqMXe7yGE/xV9vP3LlzNWvWLPN2TU2NEhISlJqaqi5dulzr9CVdXNFRXFysUaNGKSQkxNK2HcG1ri67uEKtwWOeVleodXSNuT69q5OcDVf+9+jrrpTr3ry0dppV6/P1/6dWXE+ujat1AQDwBTTUAAABJzY2VtLFFWZxcXHmeGVlpblqLTY2VnV1daqqqnJbpVZZWamhQ4eaMcePH2+y/xMnTjRZ/XYpu90uu93eZDwkJKTFB9vXs21buHnOxmbusdYwGrDgj3LWX76NfzadnA02D7n6J0+5duR/zy3V0f+ftqaW5Boozw0AwD9wlU8AQMDp3bu3YmNj3b6SVFdXpy1btpjNsoEDByokJMQtpry8XHv37jVjUlJSVF1drZ07d5oxO3bsUHV1tRkDoGVunrOxyQ8AAEBHwQo1AIBfOnfunD777DPz9qFDh1RWVqaoqCj17NlTOTk5Wrhwofr06aM+ffpo4cKFioiIUFZWliTJ4XBo8uTJmj17trp166aoqCjl5uaqf//+5lU/+/btq9GjR2vKlClavny5JGnq1KnKyMjgCp8AAACAH6OhBgDwS7t27dKIESPM243nLJs4caJWr16tJ598UrW1tZo+fbqqqqqUnJysoqIide7c2dzmpZdeUnBwsMaPH6/a2lo98MADWr16tYKCvr5M5Nq1azVz5kzzaqCZmZlaunRpG2UJAAAAoD20+lc+8/LyZLPZ3H4az1UjXTxRc15enuLj4xUeHq7hw4dr3759bvtwOp2aMWOGunfvrsjISGVmZurYsWNuMVVVVcrOzpbD4ZDD4VB2drbOnDnT2ukAAHzU8OHDZRhGk5/Vq1dLungxgby8PJWXl+vLL7/Uli1blJSU5LaPsLAw5efn69SpU7pw4YLeffddJSQkuMVERUWpoKBANTU1qqmpUUFBgW644YY2yhIAAABAe/DKCrU77rhDmzdvNm9f+kn+4sWLtWTJEq1evVq33Xabnn32WY0aNUoHDx40VwXk5OTo3Xff1fr169WtWzfNnj1bGRkZKi0tNfeVlZWlY8eOqbCwUNLFr9hkZ2fr3Xff9UZKba6584Qcfm5MG88EAAAAAAAAl/JKQy04ONhtVVojwzD08ssva968eRo3bpwkac2aNYqJidG6des0bdo0VVdXa+XKlXr99dfNc9QUFBQoISFBmzdvVlpamg4cOKDCwkJt375dycnJkqQVK1YoJSVFBw8e5Lw1AAAAAAAA8BqvNNQ+/fRTxcfHy263Kzk5WQsXLtQtt9yiQ4cOqaKiwjzPjCTZ7XYNGzZMJSUlmjZtmkpLS+Vyudxi4uPjlZSUpJKSEqWlpWnbtm1yOBxmM02ShgwZIofDoZKSkmYbak6nU06n07xdU1MjSXK5XHK5XJZybIz3tJ09yLC0r5Y8blu5Up7+hlz9U6Dkej15+vtzAwAAAACtrdUbasnJyXrttdd022236fjx43r22Wc1dOhQ7du3TxUVFZKkmJgYt21iYmJ05MgRSVJFRYVCQ0PVtWvXJjGN21dUVCg6OrrJY0dHR5sxnixatEjz589vMl5UVKSIiAhrif4/xcXFTcYWD27Rrq7Jpk2bvLfzK/CUp78iV/8UKLm2JM8LFy54YSYAAAAA4L9avaGWnp5u/r1///5KSUnRrbfeqjVr1mjIkCGSLp4I+lKGYTQZu9zlMZ7ir7afuXPnmld5ky6uUEtISFBqaqq6dOly5cQu43K5VFxcrFGjRikkJMTtvqS89y3ty4q9eWle27cnV8rT35CrfwqUXK8nz8bVugAAAACAa+OVr3xeKjIyUv3799enn36qBx98UNLFFWZxcXFmTGVlpblqLTY2VnV1daqqqnJbpVZZWamhQ4eaMcePH2/yWCdOnGiy+u1Sdrtddru9yXhISEiLD7Q9beusv3Jz8Hq0V0Pgep4jX0Ou/ilQcm1JnoHwvAAAAABAa+rk7QdwOp06cOCA4uLi1Lt3b8XGxrp9Jamurk5btmwxm2UDBw5USEiIW0x5ebn27t1rxqSkpKi6ulo7d+40Y3bs2KHq6mozBgAAAAAAAPCGVl+hlpubq7Fjx6pnz56qrKzUs88+q5qaGk2cOFE2m005OTlauHCh+vTpoz59+mjhwoWKiIhQVlaWJMnhcGjy5MmaPXu2unXrpqioKOXm5qp///7mVT/79u2r0aNHa8qUKVq+fLkkaerUqcrIyOAKnwAAAAAAAPCqVm+oHTt2TA8//LBOnjypG2+8UUOGDNH27dvVq1cvSdKTTz6p2tpaTZ8+XVVVVUpOTlZRUZE6d+5s7uOll15ScHCwxo8fr9raWj3wwANavXq1goKCzJi1a9dq5syZ5tVAMzMztXTp0tZOBwAAAAAAAHDT6g219evXX/F+m82mvLw85eXlNRsTFham/Px85efnNxsTFRWlgoKClk4TAAAAAAAAaBGvn0MNAAAAAAAA8Cc01AAAAAAAAAALWv0rnwAAAIA33Dxno8fxw8+NaeOZAACAQEdDDQAAtJrmGh4AAACAP+ErnwAAAAAAAIAFNNQAAAAAAAAAC2ioAQAAAAAAABbQUAMAAAAAAAAsoKEGAAAAAAAAWEBDDQAAAAAAALCAhhoAAAAAAABgAQ01AAAAAAAAwAIaagAAAADQAh9++KHGjh2r+Ph42Ww2vf322273G4ahvLw8xcfHKzw8XMOHD9e+ffvcYpxOp2bMmKHu3bsrMjJSmZmZOnbsmFtMVVWVsrOz5XA45HA4lJ2drTNnzrjFfPHFFxo7dqwiIyPVvXt3zZw5U3V1dd5IGwAgGmoAAAAA0CLnz5/XXXfdpaVLl3q8f/HixVqyZImWLl2qTz75RLGxsRo1apTOnj1rxuTk5GjDhg1av369tm7dqnPnzikjI0P19fVmTFZWlsrKylRYWKjCwkKVlZUpOzvbvL++vl5jxozR+fPntXXrVq1fv15vvvmmZs+e7b3kASDABbf3BAAAAIDrcfOcjR7HDz83po1ngkCTnp6u9PR0j/cZhqGXX35Z8+bN07hx4yRJa9asUUxMjNatW6dp06apurpaK1eu1Ouvv66RI0dKkgoKCpSQkKDNmzcrLS1NBw4cUGFhobZv367k5GRJ0ooVK5SSkqKDBw8qMTFRRUVF2r9/v44ePar4+HhJ0osvvqhJkyZpwYIF6tKlSxs8GwAQWGioAQAAAEArO3TokCoqKpSammqO2e12DRs2TCUlJZo2bZpKS0vlcrncYuLj45WUlKSSkhKlpaVp27ZtcjgcZjNNkoYMGSKHw6GSkhIlJiZq27ZtSkpKMptpkpSWlian06nS0lKNGDHC4xydTqecTqd5u6amRpLkcrnkcrmuOdfGWHsno9n7/EVjPv6WlyeBkmug5CmRq9Vtr4aGGgAAAAC0soqKCklSTEyM23hMTIyOHDlixoSGhqpr165NYhq3r6ioUHR0dJP9R0dHu8Vc/jhdu3ZVaGioGePJokWLNH/+/CbjRUVFioiIuFqKTfx0UEOTsU2bNlnejy8oLi5u7ym0mUDJNVDylMj1ai5cuHBNcTTUAAAAAMBLbDab223DMJqMXe7yGE/xLYm53Ny5czVr1izzdk1NjRISEpSammrpa6Iul0vFxcV6elcnORvcH29vXto178cXNOY6atQohYSEtPd0vCpQcg2UPCVyvVaNq3WvhoYaACBg5eXlNflk/tJVAYZhaP78+frVr36lqqoqJScn6xe/+IXuuOMOM97pdCo3N1e//vWvVVtbqwceeECvvvqqevTo0aa5tLXmzlkFdCScWw3tKTY2VtLF1WNxcXHmeGVlpbmaLDY2VnV1daqqqnJbpVZZWamhQ4eaMcePH2+y/xMnTrjtZ8eOHW73V1VVyeVyNVm5dim73S673d5kPCQkpEUH284Gm5z17g01fz1ob+lz5IsCJddAyVMi12vZ5lpwlU8AQEC74447VF5ebv7s2bPHvK+1rs4GAAg8vXv3VmxsrNvXjerq6rRlyxazWTZw4ECFhIS4xZSXl2vv3r1mTEpKiqqrq7Vz504zZseOHaqurnaL2bt3r8rLy82YoqIi2e12DRw40Kt5AkCgYoUaACCgBQcHm6sILtVaV2cDAPivc+fO6bPPPjNvHzp0SGVlZYqKilLPnj2Vk5OjhQsXqk+fPurTp48WLlyoiIgIZWVlSZIcDocmT56s2bNnq1u3boqKilJubq769+9v1pW+fftq9OjRmjJlipYvXy5Jmjp1qjIyMpSYmChJSk1NVb9+/ZSdna0XXnhBp0+fVm5urqZMmcIVPgHAS2ioAQAC2qeffqr4+HjZ7XYlJydr4cKFuuWWW1rt6myetNZV1Rq3ufTPtmIPanolN68+3v+7cpynK8j5G3L1vrb+/8JV1axt60t27drldgXNxvORTZw4UatXr9aTTz6p2tpaTZ8+3Tx1QFFRkTp37mxu89JLLyk4OFjjx483Tx2wevVqBQUFmTFr167VzJkzzXqTmZmppUuXmvcHBQVp48aNmj59uu655x6Fh4crKytLP/vZz7z9FABAwKKhBgAIWMnJyXrttdd022236fjx43r22Wc1dOhQ7du3r9WuzuZJa19VTWr7qzUtHtymD2fydAU5f0Wu3tNeVx3kqmpXdq1XVetIhg8fLsNoviFss9mUl5envLy8ZmPCwsKUn5+v/Pz8ZmOioqJUUFBwxbn07NlT77333lXnDABoHa3eUFu0aJHeeust/e1vf1N4eLiGDh2q559/3lyOLEmTJk3SmjVr3LZLTk7W9u3bzdvXcpLnqqoqzZw5U++8846ki5/U5Ofn64YbbmjttAAAfig9Pd38e//+/ZWSkqJbb71Va9as0ZAhQyS1ztXZLtdaV1WT2u9qTUl577fZY0kXVzD9dFCDxyvI+Rty9b62vuogV1W7Ntd6VTUAADqCVm+obdmyRY899pi+9a1v6auvvtK8efOUmpqq/fv3KzIy0owbPXq0Vq1aZd4ODQ11209OTo7effddrV+/Xt26ddPs2bOVkZGh0tJSc/lzVlaWjh07psLCQkkXzyWQnZ2td999t7XTAgAEgMjISPXv31+ffvqpHnzwQUnXf3U2T1r7qmrXu21LXH4VtzZ7XA9XkPNX5Oo97dXU4qpqV98GAABf0eoNtcbmVqNVq1YpOjpapaWluu+++8xxu93u8STQkq7pJM8HDhxQYWGhtm/fruTkZEnSihUrlJKSooMHD7qtiAMA4Fo4nU4dOHBA9957r9vV2QYMGCDp66uzPf/885Lcr842fvx4SV9fnW3x4sXtlgcAAAAA7/L6OdSqq6slXfze/6U++OADRUdH64YbbtCwYcO0YMECRUdHS9I1neR527ZtcjgcZjNNkoYMGSKHw6GSkhKPDbW2Ogm0N0/UzEl0vYdc/VOg5BpoJ4FuLbm5uRo7dqx69uypyspKPfvss6qpqdHEiRNls9la5epsAAAAAPyPVxtqhmFo1qxZ+va3v62kpCRzPD09Xd/73vfUq1cvHTp0SE8//bTuv/9+lZaWym63X9NJnisqKswG3KWio6ObPRF0W50E2psnauYkut5Hrv4pUHINlJNAt5Zjx47p4Ycf1smTJ3XjjTdqyJAh2r59u3r16iVJrXZ1NgAdy81zNnocP/zcmDaeCQAA8FVebag9/vjj+utf/6qtW7e6jU+YMMH8e1JSkgYNGqRevXpp48aNGjduXLP7u/wkz55O+HylE0G31UmgvXmiZk6i6z3k6p8CJVdOAt0y69evv+L9rXV1NgAAAAD+xWsNtRkzZuidd97Rhx9+6HZlTk/i4uLUq1cvffrpp5Ku7STPsbGxOn78eJN9nThxwjxZ9OXa6iTQ3jypLifR9T5y9U+BkisngQYAAAAA7+vU2js0DEOPP/643nrrLf3xj39U7969r7rNqVOndPToUfMqapee5LlR40meGxtqKSkpqq6u1s6dO82YHTt2qLq6+opXVgMAAAAAAACuR6uvUHvssce0bt06/e53v1Pnzp3N85k5HA6Fh4fr3LlzysvL00MPPaS4uDgdPnxYTz31lLp3767vfOc7ZuzVTvLct29fjR49WlOmTNHy5cslSVOnTlVGRgZX+AQAoJU0d64pAAAAIJC1ekNt2bJlkqThw4e7ja9atUqTJk1SUFCQ9uzZo9dee01nzpxRXFycRowYoTfeeMPySZ7Xrl2rmTNnmlcDzczM1NKlS1s7JQAAAAAAAMDU6g01wzCueH94eLjef//qJ+2/lpM8R0VFqaCgwPIcAQAAAAAAgJby6lU+AQAAAF/R3FecDz83po1nAgAAOrpWvygBAAAAAAAA4M9oqAEAAAAAAAAW0FADAAAAAAAALKChBgAAAAAAAFjARQl8DCfLBQAAaFv8/gUAAC7HCjUAAAAAAADAAhpqAAAAAAAAgAU01AAAAAAAAAALOIcaAABo9hxRAAAAAJqioQYAAAC0ABcrAAAgcPGVTwAAAAAAAMACGmoAAAAAAACABTTUAAAAAAAAAAtoqAEAAAAAAAAWcFECAAAAoBVdfrECe5ChxYPbaTIAAMArWKEGAAAAAAAAWMAKNQAAAsjlK2cAtJ2kvPflrLe5jR1+bkw7zQYAAFwPGmp+orkDJH5JAwAAAAAAaF185RMAAAAAAACwgIYaAAAAAAAAYAENNQAAAAAAAMACzqF2nTydXBYAAAC4FlYvFML5cQEA6Bh8foXaq6++qt69eyssLEwDBw7URx991N5T6lBunrPR4w8AoHVRjwAAHQH1CADahk+vUHvjjTeUk5OjV199Vffcc4+WL1+u9PR07d+/Xz179mzv6QEAAkRHrEd8eAL4J67sjivpiPUIAPyVTzfUlixZosmTJ+uRRx6RJL388st6//33tWzZMi1atKidZ9ex8csYALQe6hGA9sZXRyFRjwCgLflsQ62urk6lpaWaM2eO23hqaqpKSko8buN0OuV0Os3b1dXVkqTTp0/L5XJZenyXy6ULFy4o2NVJ9Q3+cw61f8n9P7fb9k6GfjSgQadOnVJISEg7zaptNL6m5OpfAiXX68nz7NmzkiTDMLwxNb/XUerR3fPekvOSeuSzBb4ZwQ2GLlxo8Lu66wm5+p+OmOflv/M12jH3gevaL/Wo/bRnPbrSsdGpU6eueT++IFB+t5QCJ9dAyVMi12t1rfXIZ3/fPnnypOrr6xUTE+M2HhMTo4qKCo/bLFq0SPPnz28y3rt3b6/M0V9ktfcEALSJs2fPyuFwtPc0fA71qO0EUj0iV//jK3l2f7G9Z0A9aqmOWo86wr8pAGiJq9Ujn22oNbLZ3D8BMQyjyVijuXPnatasWebthoYGnT59Wt26dWt2m+bU1NQoISFBR48eVZcuXaxP3EcESp4SufqrQMn1evI0DENnz55VfHy8l2YXGKhH3hUoeUrk6o8CJU+JetQRtEc94t+4fwqUXAMlT4lcr9W11iOfbah1795dQUFBTT5tqaysbPKpTCO73S673e42dsMNN1zXPLp06eL3/xClwMlTIld/FSi5tjRPVgK0HPWobQVKnhK5+qNAyVOiHrWHjlCP+DfunwIl10DJUyLXa3Et9ahTSybUEYSGhmrgwIEqLi52Gy8uLtbQoUPbaVYAgEBDPQIAdATUIwBoWz67Qk2SZs2apezsbA0aNEgpKSn61a9+pS+++EKPPvpoe08NABBAqEcAgI6AegQAbcenG2oTJkzQqVOn9JOf/ETl5eVKSkrSpk2b1KtXL68/tt1u1zPPPNNkibS/CZQ8JXL1V4GSa6Dk2VFRj7wvUPKUyNUfBUqeUmDl2hG1Vz0KpNedXP1PoOQpkWtrsxlclxoAAAAAAAC4Zj57DjUAAAAAAACgPdBQAwAAAAAAACygoQYAAAAAAABYQEMNAAAAAAAAsICG2hW8+uqr6t27t8LCwjRw4EB99NFHV4zfsmWLBg4cqLCwMN1yyy365S9/2UYzvT5W8nzrrbc0atQo3XjjjerSpYtSUlL0/vvvt+Fsr4/V17TRxx9/rODgYN19993enWArspqr0+nUvHnz1KtXL9ntdt1666363//93zaabctZzXPt2rW66667FBERobi4OP3gBz/QqVOn2mi2Lffhhx9q7Nixio+Pl81m09tvv33VbXz1PQnuAqUWSdSja0E96rioR83z5fclfI165Bn16G7vTrCVBEotkgKjHnWYWmTAo/Xr1xshISHGihUrjP379xtPPPGEERkZaRw5csRj/Oeff25EREQYTzzxhLF//35jxYoVRkhIiPHb3/62jWdujdU8n3jiCeP55583du7cafz973835s6da4SEhBh//vOf23jm1lnNtdGZM2eMW265xUhNTTXuuuuutpnsdWpJrpmZmUZycrJRXFxsHDp0yNixY4fx8ccft+GsrbOa50cffWR06tTJeOWVV4zPP//c+Oijj4w77rjDePDBB9t45tZt2rTJmDdvnvHmm28akowNGzZcMd5X35PgLlBqkWFQj6hHX6MedWzUo8BEPaIeXcrX6lGg1CLDCJx61FFqEQ21ZgwePNh49NFH3cZuv/12Y86cOR7jn3zySeP22293G5s2bZoxZMgQr82xNVjN05N+/foZ8+fPb+2ptbqW5jphwgTjRz/6kfHMM8/4RMEwDOu5/v73vzccDodx6tSptpheq7Ga5wsvvGDccsstbmM///nPjR49enhtjt5wLUXDV9+T4C5QapFhUI+oRxdRj6hH6JioR9SjS/laPQqUWmQYgVmP2rMW8ZVPD+rq6lRaWqrU1FS38dTUVJWUlHjcZtu2bU3i09LStGvXLrlcLq/N9Xq0JM/LNTQ06OzZs4qKivLGFFtNS3NdtWqV/vGPf+iZZ57x9hRbTUtyfeeddzRo0CAtXrxYN910k2677Tbl5uaqtra2LabcIi3Jc+jQoTp27Jg2bdokwzB0/Phx/fa3v9WYMWPaYsptyhffk+AuUGqRRD2SqEeNqEfUI3Q81CPq0aV8rR4FSi2SqEdX4q33pODrnZg/OnnypOrr6xUTE+M2HhMTo4qKCo/bVFRUeIz/6quvdPLkScXFxXltvi3Vkjwv9+KLL+r8+fMaP368N6bYalqS66effqo5c+boo48+UnCw7/xXaUmun3/+ubZu3aqwsDBt2LBBJ0+e1PTp03X69OkOe66AluQ5dOhQrV27VhMmTNCXX36pr776SpmZmcrPz2+LKbcpX3xPgrtAqUUS9UiiHjWiHlGP0PFQj6hHjXyxHgVKLZKoR1firfckVqhdgc1mc7ttGEaTsavFexrvaKzm2ejXv/618vLy9MYbbyg6Otpb02tV15prfX29srKyNH/+fN12221tNb1WZeV1bWhokM1m09q1azV48GD967/+q5YsWaLVq1d3+E9irOS5f/9+zZw5Uz/+8Y9VWlqqwsJCHTp0SI8++mhbTLXN+ep7EtwFSi2SqEfUI+oR9QgdGfWIeuTL9ShQapFEPWqON96TfKOt3Ma6d++uoKCgJl3cysrKJl3NRrGxsR7jg4OD1a1bN6/N9Xq0JM9Gb7zxhiZPnqzf/OY3GjlypDen2Sqs5nr27Fnt2rVLu3fv1uOPPy7p4hurYRgKDg5WUVGR7r///jaZu1UteV3j4uJ00003yeFwmGN9+/aVYRg6duyY+vTp49U5t0RL8ly0aJHuuece/dd//Zck6c4771RkZKTuvfdePfvssx3209KW8MX3JLgLlFokUY8k6lEj6hH1CB0P9Yh6JPluPQqUWiRRj67EW+9JrFDzIDQ0VAMHDlRxcbHbeHFxsYYOHepxm5SUlCbxRUVFGjRokEJCQrw21+vRkjyli5+8TJo0SevWrfOZ71ZbzbVLly7as2ePysrKzJ9HH31UiYmJKisrU3JycltN3bKWvK733HOP/vnPf+rcuXPm2N///nd16tRJPXr08Op8W6oleV64cEGdOrm/7QUFBUn6+hMKf+GL70lwFyi1SKIeSdSjRtQj6hE6HuoR9Ujy3XoUKLVIoh5didfek67rkgZ+rPFysytXrjT2799v5OTkGJGRkcbhw4cNwzCMOXPmGNnZ2WZ842VY//M//9PYv3+/sXLlSp+4NLTVPNetW2cEBwcbv/jFL4zy8nLz58yZM+2VwjWzmuvlfOUqNoZhPdezZ88aPXr0ML773e8a+/btM7Zs2WL06dPHeOSRR9orhWtiNc9Vq1YZwcHBxquvvmr84x//MLZu3WoMGjTIGDx4cHulcM3Onj1r7N6929i9e7chyViyZImxe/du8xLY/vKeBHeBUosMg3pEPbqIekQ9QsdEPaIeeeIr9ShQapFhBE496ii1iIbaFfziF78wevXqZYSGhhrf/OY3jS1btpj3TZw40Rg2bJhb/AcffGAMGDDACA0NNW6++WZj2bJlbTzjlrGS57BhwwxJTX4mTpzY9hNvAauv6aV8pWA0sprrgQMHjJEjRxrh4eFGjx49jFmzZhkXLlxo41lbZzXPn//850a/fv2M8PBwIy4uzvj+979vHDt2rI1nbd2f/vSnK/7f86f3JLgLlFpkGNSjRtQj6lFHRj0KXNSji6hHX/OlehQotcgwAqMedZRaZDMMP1rHBwAAAAAAAHgZ51ADAAAAAAAALKChBgAAAAAAAFhAQw0AAAAAAACwgIYaAAAAAAAAYAENNQAAAAAAAMACGmoAAAAAAACABTTUAAAAAAAAAAtoqAEAAAAAAAAW0FADAAAAAAAALKChBgAAAAAAAFhAQw0AAAAAAACwgIYaAAAAAAAAYAENNQAAAAAAAMACGmoAAAAAAACABTTUAAAAAAAAAAtoqAEAAAAAAAAW0FADAAAAAAAALKChBgAAAAAAAFhAQw0AAAAAAACwgIYaAAAAAAAAYAENNQAAAAAAAMACGmoAAAAAAACABTTUEJDy8vJks9labX/Dhw/X8OHDW21/HZXNZlNeXl57TwMA/FZJSYny8vJ05syZ9p7KVb3xxhu64447FB4eLpvNprKysvaeEgAAQJuhoYaA9Mgjj2jbtm3tPQ0AANyUlJRo/vz5Hb6hduLECWVnZ+vWW29VYWGhtm3bpttuu629pwUAANBmgtt7AkB76NGjh3r06NHe0wAAoMVqa2sVHh7eLo/997//XS6XS//+7/+uYcOGtco+a2trFRYW1qoryAEAALyFFWrwK2+//bZsNpv+8Ic/NLlv2bJlstls+utf/9rsVz7XrVunlJQUfeMb39A3vvEN3X333Vq5cqV5v2EYWrx4sXr16qWwsDB985vf1O9//3uPc6mpqVFubq569+6t0NBQ3XTTTcrJydH58+fd4mw2mx5//HGtWrVKiYmJCg8P16BBg7R9+3YZhqEXXnhBvXv31je+8Q3df//9+uyzz9y2Ly4u1r/927+pR48eCgsL07/8y79o2rRpOnnypFtcY8779u3Tww8/LIfDoZiYGP3whz9UdXV1k7lPmTJF3bp10ze+8Q2NHj1af//736/85AMArkteXp7+67/+S5LUu3dv2Ww22Ww2ffDBB7r55puVkZGht956SwMGDFBYWJjmz58vSfrFL36h++67T9HR0YqMjFT//v21ePFiuVwut/0PHz5cSUlJ+uSTT3TvvfcqIiJCt9xyi5577jk1NDSYcQ0NDXr22WfNmnTDDTfozjvv1CuvvCJJmjRpkr797W9LkiZMmCCbzeZ22oNdu3YpMzNTUVFRCgsL04ABA/R///d/bnNZvXq1bDabioqK9MMf/lA33nijIiIi5HQ6W/15BQAA8AZWqMGvZGRkKDo6WqtWrdIDDzzgdt/q1av1zW9+U3feeafeeuutJtv++Mc/1k9/+lONGzdOs2fPlsPh0N69e3XkyBEzZv78+Zo/f74mT56s7373uzp69KimTJmi+vp6JSYmmnEXLlzQsGHDdOzYMT311FO68847tW/fPv34xz/Wnj17tHnzZreG3nvvvafdu3frueeek81m03//939rzJgxmjhxoj7//HMtXbpU1dXVmjVrlh566CGVlZWZ2//jH/9QSkqKHnnkETkcDh0+fFhLlizRt7/9be3Zs0chISFueT700EOaMGGCJk+erD179mju3LmSpP/93/+VdLFp+OCDD6qkpEQ//vGP9a1vfUsff/yx0tPTr/PVAQBcySOPPKLTp08rPz9fb731luLi4iRJ/fr1kyT9+c9/1oEDB/SjH/1IvXv3VmRkpKSLdSArK8v8AOcvf/mLFixYoL/97W/me3ujiooKff/739fs2bP1zDPPaMOGDZo7d67i4+P1H//xH5KkxYsXKy8vTz/60Y903333yeVy6W9/+5v5NdSnn35agwcP1mOPPaaFCxdqxIgR6tKliyTpT3/6k0aPHq3k5GT98pe/lMPh0Pr16zVhwgRduHBBkyZNcpvPD3/4Q40ZM0avv/66zp8/36RmAQAAdFgG4GdmzZplhIeHG2fOnDHH9u/fb0gy8vPzDcMwjGeeeca49J//559/bgQFBRnf//73m91vVVWVERYWZnznO99xG//4448NScawYcPMsUWLFhmdOnUyPvnkE7fY3/72t4YkY9OmTeaYJCM2NtY4d+6cOfb2228bkoy7777baGhoMMdffvllQ5Lx17/+1eMcGxoaDJfLZRw5csSQZPzud78z72vMefHixW7bTJ8+3QgLCzMf5/e//70hyXjllVfc4hYsWGBIMp555plmnyMAwPV54YUXDEnGoUOH3MZ79eplBAUFGQcPHrzi9vX19YbL5TJee+01IygoyDh9+rR537BhwwxJxo4dO9y26devn5GWlmbezsjIMO6+++4rPs6f/vQnQ5Lxm9/8xm389ttvNwYMGGC4XC638YyMDCMuLs6or683DMMwVq1aZUgy/uM//uOKjwMAANBR8ZVP+J0f/vCHqq2t1RtvvGGOrVq1Sna7XVlZWR63KS4uVn19vR577LFm97tt2zZ9+eWX+v73v+82PnToUPXq1ctt7L333lNSUpLuvvtuffXVV+ZPWlqa+fWdS40YMcJcaSBJffv2lSSlp6e7rWRrHL901VxlZaUeffRRJSQkKDg4WCEhIeZ8Dhw40CSPzMxMt9t33nmnvvzyS1VWVkq6uLpAUpM8m3vuAABt48477/R44v/du3crMzNT3bp1U1BQkEJCQvQf//Efqq+vb/J1/djYWA0ePLjJfi+tK4MHD9Zf/vIXTZ8+Xe+//75qamquaX6fffaZ/va3v5n149L696//+q8qLy/XwYMH3bZ56KGHrmnfAAAAHQ1f+YTfueOOO/Stb31Lq1at0tSpU1VfX6+CggL927/9m6Kiojxuc+LECUm64oUKTp06JeniwcjlLh87fvy4Pvvss2a/unL5+c0un1doaOgVx7/88ktJF89zk5qaqn/+8596+umn1b9/f0VGRqqhoUFDhgxRbW1tk8fu1q2b22273S5JZuypU6cUHBzcJM5T3gCAttP4FdBLffHFF7r33nuVmJioV155RTfffLPCwsK0c+dOPfbYY03qwOXv7dLFOnBp3Ny5cxUZGamCggL98pe/VFBQkO677z49//zzGjRoULPzO378uCQpNzdXubm5HmMur3+ecgIAAPAFNNTgl37wgx9o+vTpOnDggD7//HOVl5frBz/4QbPxN954oyTp2LFjSkhI8BjTeBBSUVHR5L6KigrdfPPN5u3u3bsrPDy8yblrLr2/Nezdu1d/+ctftHr1ak2cONEcv/zCBVZ069ZNX331lU6dOuV24OUpbwBA2/F0MZ23335b58+f11tvveW2WrqsrKzFjxMcHKxZs2Zp1qxZOnPmjDZv3qynnnpKaWlpOnr0qCIiIjxu11jb5s6dq3HjxnmMufR8o5LnnAAAAHwBX/mEX3r44YcVFham1atXa/Xq1brpppuUmprabHxqaqqCgoK0bNmyZmOGDBmisLAwrV271m28pKTE7asy0sWLI/zjH/9Qt27dNGjQoCY/lzbfrkfjgUjjKrNGy5cvb/E+R4wYIUlN8ly3bl2L9wkAuDaXrxq+Gk91wDAMrVixolXmc8MNN+i73/2uHnvsMZ0+fVqHDx9uNjYxMVF9+vTRX/7yF4+1b9CgQercuXOrzAsAAKC9sUINfumGG27Qd77zHa1evVpnzpxRbm6uOnVqvn98880366mnntJPf/pT1dbW6uGHH5bD4dD+/ft18uRJzZ8/X127dlVubq6effZZPfLII/re976no0ePKi8vr8nXIXNycvTmm2/qvvvu03/+53/qzjvvVENDg7744gsVFRVp9uzZSk5Ovu48b7/9dt16662aM2eODMNQVFSU3n33XRUXF7d4n6mpqbrvvvv05JNP6vz58xo0aJA+/vhjvf7669c9XwDAlfXv31+S9Morr2jixIkKCQlpsqrrUqNGjVJoaKgefvhhPfnkk/ryyy+1bNkyVVVVtXgOY8eOVVJSkgYNGqQbb7xRR44c0csvv6xevXqpT58+V9x2+fLlSk9PV1pamiZNmqSbbrpJp0+f1oEDB/TnP/9Zv/nNb1o8LwAAgI6Ehhr81g9+8AP9+te/liRNmjTpqvE/+clP1KdPH+Xn5+v73/++goOD1adPH82cOdMtJjIyUq+++qpef/113X777frlL3+pn/3sZ277ioyM1EcffaTnnntOv/rVr3To0CGFh4erZ8+eGjlyZKutUAsJCdG7776rJ554QtOmTVNwcLBGjhypzZs3q2fPni3aZ6dOnfTOO+9o1qxZWrx4serq6nTPPfdo06ZNuv3221tl3gAAz4YPH665c+dqzZo1WrFihRoaGsyLxXhy++23680339SPfvQjjRs3Tt26dVNWVpZmzZql9PT0Fs1hxIgRevPNN/U///M/qqmpUWxsrEaNGqWnn3662XODXrrtzp07tWDBAuXk5KiqqkrdunVTv379NH78+BbNBwAAoCOyGYZhtPckAAAAAAAAAF/BOdQAAAAAAAAAC2ioAQAAAAAAABbQUAMAAAAAAAAsoKEGAAAAAAAAWEBDDQAAAAAAALCAhhoAAAAAAABgQXB7T6A9NTQ06J///Kc6d+4sm83W3tMBgHZhGIbOnj2r+Ph4derE5yztgXoEANQjAIBvCeiG2j//+U8lJCS09zQAoEM4evSoevTo0d7TCEjUIwD4GvUIAOALArqh1rlzZ0kXi3aXLl0sbetyuVRUVKTU1FSFhIR4Y3odQqDkKZGrvwqUXK8nz5qaGiUkJJjviWh71KOrC5Q8JXL1R4GSp0Q9AgAEjoBuqDV+raZLly4tOoCJiIhQly5d/PoXo0DJUyJXfxUoubZGnnzVsP1Qj64uUPKUyNUfBUqeEvUIABA4ODkBAAAAAAAAYAENNQAAAAAAAMACGmoAAAAAAACABTTUAAAAAAAAAAtoqAEAAAAAAAAW0FADAAAAAAAALKChBgAAAAAAAFhAQw0AAAAAAACwILi9JwD4gpvnbPQ4fvi5MW08EwBAoEjKe1/OepvbGHUHAACgY2CFGgAAAAAAAGABDTUAAAAAAADAAhpqAAAAAAAAgAU01AAAAAAAAAALaKgBAAAAAAAAFtBQAwAAAAAAACygoQYAAAAAAABYQEMNAAAAAAAAsICGGgAAAAAAAGABDTUAAAAAAADAAhpqAAAAAAAAgAU01AAAAAAAAAALaKgBAAAAAAAAFtBQAwAAAAAAACygoQYAAAAAAABYQEMNAAAAAAAAsICGGgAAAAAAAGABDTUAAAAAAADAAhpqAAAAAAAAgAU01AAAAAAAAAALaKgBAAAAAAAAFtBQAwAAAAAAACygoQYAAAAAAABYQEMNAAAAAAAAsICGGgAAAAAAAGABDTUAAAAAAADAAhpqAAAAAAAAgAU01AAAAAAAAAALaKgBAAAAAAAAFtBQAwAAAAAAACygoQYAAAAAAABYQEMNAOBzPvzwQ40dO1bx8fGy2Wx6++233e6fNGmSbDab28+QIUPcYpxOp2bMmKHu3bsrMjJSmZmZOnbsmFtMVVWVsrOz5XA45HA4lJ2drTNnzrjFfPHFFxo7dqwiIyPVvXt3zZw5U3V1dd5IGwAAAEAHYbmhxkEMAKC9nT9/XnfddZeWLl3abMzo0aNVXl5u/mzatMnt/pycHG3YsEHr16/X1q1bde7cOWVkZKi+vt6MycrKUllZmQoLC1VYWKiysjJlZ2eb99fX12vMmDE6f/68tm7dqvXr1+vNN9/U7NmzWz9pAAAAAB1GsNUNGg9ifvCDH+ihhx7yGDN69GitWrXKvB0aGup2f05Ojt59912tX79e3bp10+zZs5WRkaHS0lIFBQVJungQc+zYMRUWFkqSpk6dquzsbL377ruSvj6IufHGG7V161adOnVKEydOlGEYys/Pt5oWAMCHpKenKz09/YoxdrtdsbGxHu+rrq7WypUr9frrr2vkyJGSpIKCAiUkJGjz5s1KS0vTgQMHVFhYqO3btys5OVmStGLFCqWkpOjgwYNKTExUUVGR9u/fr6NHjyo+Pl6S9OKLL2rSpElasGCBunTp0opZAwAAAOgoLDfUOIgBAPiCDz74QNHR0brhhhs0bNgwLViwQNHR0ZKk0tJSuVwupaammvHx8fFKSkpSSUmJ0tLStG3bNjkcDrMOSdKQIUPkcDhUUlKixMREbdu2TUlJSWYdkqS0tDQ5nU6VlpZqxIgRHufmdDrldDrN2zU1NZIkl8sll8tlKc/GeKvb+ZpAyVP6Okd7J6PZ+/xFoLyugZKndH25BsLzAwDwH5YbateiIx/EAAD8X3p6ur73ve+pV69eOnTokJ5++mndf//9Ki0tld1uV0VFhUJDQ9W1a1e37WJiYlRRUSFJqqioMGvXpaKjo91iYmJi3O7v2rWrQkNDzRhPFi1apPnz5zcZLyoqUkREhOV8Jam4uLhF2/maQMlTkn46qKHJ2OVfXfYXgfK6BkqeUstyvXDhghdmAgCAd7R6Q60jH8SwIsC6QMlTunKu9qCmqwSai/UFvK7+hxUB7iZMmGD+PSkpSYMGDVKvXr20ceNGjRs3rtntDMOQzWYzb1/69+uJudzcuXM1a9Ys83ZNTY0SEhKUmppqeYW1y+VScXGxRo0apZCQEEvb+pJAyVP6Otend3WSs8H939HevLR2mpV3BMrrGih5SteXa+Pv5gAA+IJWb6h15IMYVgS0XKDkKXnOdfFgz7G+vlIg0F9Xf8SKAM/i4uLUq1cvffrpp5Kk2NhY1dXVqaqqyu0DnsrKSg0dOtSMOX78eJN9nThxwvxAJzY2Vjt27HC7v6qqSi6Xq8mHPpey2+2y2+1NxkNCQlp8sH092/qSQMlTkpwNNjnr3X+n8dfcA+V1DZQ8pZblGijPDQDAP3jlK5+X6kgHMawIsC5Q8pSunGtS3vset/HVlQK8rv6HFQFXdurUKR09elRxcXGSpIEDByokJETFxcUaP368JKm8vFx79+7V4sWLJUkpKSmqrq7Wzp07NXjwxa76jh07VF1dbdarlJQULViwQOXl5ea+i4qKZLfbNXDgwLZOEwAAAEAb8XpDrSMdxLAioOUCJU/Jc66XrxC4NNaXBfrr6o8CZUXAuXPn9Nlnn5m3Dx06pLKyMkVFRSkqKkp5eXl66KGHFBcXp8OHD+upp55S9+7d9Z3vfEeS5HA4NHnyZM2ePVvdunVTVFSUcnNz1b9/f/OCOX379tXo0aM1ZcoULV++XNLFK05nZGQoMTFRkpSamqp+/fopOztbL7zwgk6fPq3c3FxNmTKFi+MAAAAAfsxyQ42DGABAe9u1a5fbxWcaVx9PnDhRy5Yt0549e/Taa6/pzJkziouL04gRI/TGG2+oc+fO5jYvvfSSgoODNX78eNXW1uqBBx7Q6tWrFRQUZMasXbtWM2fONC+kk5mZqaVLl5r3BwUFaePGjZo+fbruuecehYeHKysrSz/72c+8/RQAAAAAaEeWG2ocxAAA2tvw4cNlGJ4vFiJJ77/v+WvalwoLC1N+fr7y8/ObjYmKilJBQcEV99OzZ0+99957V308AAAAAP7DckONgxgAAAAAAAAEsk7tPQEAAAAAAADAl9BQAwAAAAAAACygoQYAAAAAAABYQEMNAAAAAAAAsICGGgAAAAAAAGABDTUAAAAAAADAAhpqAAAAAAAAgAU01AAAAAAAAAALaKgBAAAAAAAAFtBQAwAAAAAAACygoQYAAAAAAABYQEMNAAAAAAAAsICGGgAAAAAAAGABDTUAAAAAAADAAhpqAAAAAAAAgAU01AAAAAAAAAALaKgBAAAAAAAAFtBQAwAAAAAAACygoQYAAAAAAABYQEMNAAAAAAAAsICGGgAAAAAAAGABDTUAAAAAAADAAhpqAAAAAAAAgAU01AAAAAAAAAALaKgBAAAAAAAAFtBQAwAAAAAAACygoQYAAAAAAABYQEMNAAAAAAAAsICGGgAAAAAAAGABDTUAAAAAAADAAhpqAAAAAAAAgAU01AAAAAAAAAALaKgBAAAAAAAAFtBQAwAAAAAAACygoQYAAAAAAABYQEMNAAAAAAAAsICGGgAAAAAAAGABDTUAAAAAAADAAhpqAAAAAAAAgAU01AAAAAAAAAALaKgBAAAAAAAAFtBQAwAAAAAAACygoQYAAAAAAABYQEMNAAAAAAAAsMByQ+3DDz/U2LFjFR8fL5vNprffftvtfsMwlJeXp/j4eIWHh2v48OHat2+fW4zT6dSMGTPUvXt3RUZGKjMzU8eOHXOLqaqqUnZ2thwOhxwOh7Kzs3XmzBm3mC+++EJjx45VZGSkunfvrpkzZ6qurs5qSgAAH0MtAgAAANCeLDfUzp8/r7vuuktLly71eP/ixYu1ZMkSLV26VJ988oliY2M1atQonT171ozJycnRhg0btH79em3dulXnzp1TRkaG6uvrzZisrCyVlZWpsLBQhYWFKisrU3Z2tnl/fX29xowZo/Pnz2vr1q1av3693nzzTc2ePdtqSgAAH0MtAgAAANCegq1ukJ6ervT0dI/3GYahl19+WfPmzdO4ceMkSWvWrFFMTIzWrVunadOmqbq6WitXrtTrr7+ukSNHSpIKCgqUkJCgzZs3Ky0tTQcOHFBhYaG2b9+u5ORkSdKKFSuUkpKigwcPKjExUUVFRdq/f7+OHj2q+Ph4SdKLL76oSZMmacGCBerSpUuLnhAAQMdHLQIAAADQniw31K7k0KFDqqioUGpqqjlmt9s1bNgwlZSUaNq0aSotLZXL5XKLiY+PV1JSkkpKSpSWlqZt27bJ4XCYBzCSNGTIEDkcDpWUlCgxMVHbtm1TUlKSeQAjSWlpaXI6nSotLdWIESNaMzUAgI/whVrkdDrldDrN2zU1NZIkl8sll8tlKd/GeKvb+ZpAyVP6Okd7J6PZ+/xFoLyugZKndH25BsLzAwDwH63aUKuoqJAkxcTEuI3HxMToyJEjZkxoaKi6du3aJKZx+4qKCkVHRzfZf3R0tFvM5Y/TtWtXhYaGmjGX4wDGukDJU7pyrvagpgc1zcX6Al5X/8MBzNc6ei2SpEWLFmn+/PlNxouKihQREXG1FD0qLi5u0Xa+JlDylKSfDmpoMrZp06Z2mIn3BcrrGih5Si3L9cKFC16YCQAA3tGqDbVGNpvN7bZhGE3GLnd5jKf4lsRcigOYlguUPCXPuS4e7DnW1w9sAv119UccwHyto9YiSZo7d65mzZpl3q6pqVFCQoJSU1Mtf03U5XKpuLhYo0aNUkhIiKVtfUmg5Cl9nevTuzrJ2eD+72hvXlo7zco7AuV1DZQ8pevLtfHDbgAAfEGrNtRiY2MlXfzEPi4uzhyvrKw0P8GPjY1VXV2dqqqq3FYGVFZWaujQoWbM8ePHm+z/xIkTbvvZsWOH2/1VVVVyuVxNVgs04gDGukDJU7pyrkl573vcxlcPbHhd/Q8HMF/r6LVIuvgVVLvd3mQ8JCSkxf9Or2dbXxIoeUqSs8EmZ717Q81fcw+U1zVQ8pRalmugPDcAAP/Qqg213r17KzY2VsXFxRowYIAkqa6uTlu2bNHzzz8vSRo4cKBCQkJUXFys8ePHS5LKy8u1d+9eLV68WJKUkpKi6upq7dy5U4MHX1watGPHDlVXV5sHOikpKVqwYIHKy8vNA6aioiLZ7XYNHDjQ4/w4gGm5QMlT8pzr5Qc0l8b6skB/Xf0RBzAdvxYBAAAA8H2WG2rnzp3TZ599Zt4+dOiQysrKFBUVpZ49eyonJ0cLFy5Unz591KdPHy1cuFARERHKysqSJDkcDk2ePFmzZ89Wt27dFBUVpdzcXPXv39+80lrfvn01evRoTZkyRcuXL5ckTZ06VRkZGUpMTJQkpaamql+/fsrOztYLL7yg06dPKzc3V1OmTOGqagDg56hFAAAAANqT5Ybarl273K5a1vgVyokTJ2r16tV68sknVVtbq+nTp6uqqkrJyckqKipS586dzW1eeuklBQcHa/z48aqtrdUDDzyg1atXKygoyIxZu3atZs6caV6BLTMzU0uXLjXvDwoK0saNGzV9+nTdc889Cg8PV1ZWln72s59ZfxYAAD6FWgQAAACgPVluqA0fPlyG4fmKh9LFkzPn5eUpLy+v2ZiwsDDl5+crPz+/2ZioqCgVFBRccS49e/bUe++9d9U5AwD8C7UIAAAAQHvq1N4TAAAAAAAAAHwJDTUAAAAAAADAAhpqAAAAAAAAgAU01AAAAAAAAAALaKgBAAAAAAAAFtBQAwAAAAAAACygoQYAAAAAAABYQEMNAAAAAAAAsICGGgAAAAAAAGABDTUAAAAAAADAAhpqAAAAAAAAgAU01AAAAAAAAAALaKgBAAAAAAAAFtBQAwAAAAAAACygoQYAAAAAAABYQEMNAAAAAAAAsICGGgAAAAAAAGABDTUAAAAAAADAAhpqAAAAAAAAgAU01AAAAAAAAAALaKgBAAAAAAAAFtBQAwAAAAAAACygoQYAAAAAAABYQEMNAAAAAAAAsICGGgAAAAAAAGABDTUAAAAAAADAAhpqAAAAAAAAgAU01AAAAAAAAAALaKgBAAAAAAAAFtBQAwAAAAAAACygoQYAAAAAAABYQEMNAAAAAAAAsICGGgAAAAAAAGABDTUAAAAAAADAAhpqAAAAAAAAgAU01AAAAAAAAAALaKgBAAAAAAAAFtBQAwAAAAAAACygoQYAAAAAAABYQEMNAAAAAAAAsICGGgAAAAAAAGABDTUAAAAAAADAAhpqAAAAAAAAgAU01AAAAAAAAAALaKgBAPxSXl6ebDab209sbKx5v2EYysvLU3x8vMLDwzV8+HDt27fPbR9Op1MzZsxQ9+7dFRkZqczMTB07dswtpqqqStnZ2XI4HHI4HMrOztaZM2faIkUAAAAA7aTVG2ocwAAAOoo77rhD5eXl5s+ePXvM+xYvXqwlS5Zo6dKl+uSTTxQbG6tRo0bp7NmzZkxOTo42bNig9evXa+vWrTp37pwyMjJUX19vxmRlZamsrEyFhYUqLCxUWVmZsrOz2zRPAAAAAG3LKyvUOIABAHQEwcHBio2NNX9uvPFGSRc/3Hn55Zc1b948jRs3TklJSVqzZo0uXLigdevWSZKqq6u1cuVKvfjiixo5cqQGDBiggoIC7dmzR5s3b5YkHThwQIWFhfqf//kfpaSkKCUlRStWrNB7772ngwcPtlveAAAAALzLKw01DmAAAB3Bp59+qvj4ePXu3Vv/3//3/+nzzz+XJB06dEgVFRVKTU01Y+12u4YNG6aSkhJJUmlpqVwul1tMfHy8kpKSzJht27bJ4XAoOTnZjBkyZIgcDocZAwAAAMD/BHtjp40HMHa7XcnJyVq4cKFuueWWqx7ATJs27aoHMGlpaVc9gElMTPQ4L6fTKafTad6uqamRJLlcLrlcLks5NsZb3c7XBEqe0pVztQcZV9zG1/C6+p/rydNfn5vk5GS99tpruu2223T8+HE9++yzGjp0qPbt26eKigpJUkxMjNs2MTExOnLkiCSpoqJCoaGh6tq1a5OYxu0rKioUHR3d5LGjo6PNGE+oR9YFSp7S1znaOzWtPf6Wf6C8roGSp0Q9AgAEjlZvqHXkA5hFixZp/vz5TcaLiooUERFhLdH/p7i4uEXb+ZpAyVPynOviwZ5jN23a5OXZeFegv67+qCV5XrhwwQszaX/p6enm3/v376+UlBTdeuutWrNmjYYMGSJJstlsbtsYhtFk7HKXx3iKv9p+qEctFyh5StJPBzU0GfP1utOcQHldAyVPiXoEAPB/rd5Q68gHMHPnztWsWbPM2zU1NUpISFBqaqq6dOly5cQu43K5VFxcrFGjRikkJMTStr4kUPKUrpxrUt77HrfZm5fWFlNrdbyu/ud68mxcHeXvIiMj1b9/f3366ad68MEHJV38gCYuLs6MqaysND/0iY2NVV1dnaqqqtw+5KmsrNTQoUPNmOPHjzd5rBMnTjT58OhS1CPrAiVP6etcn97VSc4G999rfLXuNCdQXtdAyVOiHgEAAodXvvJ5qY50AGO322W325uMh4SEtPiXm+vZ1pcESp6S51yd9Z4btb7+nAT66+qPWpJnIDwv0sWvWR44cED33nuvevfurdjYWBUXF2vAgAGSpLq6Om3ZskXPP/+8JGngwIEKCQlRcXGxxo8fL0kqLy/X3r17tXjxYklSSkqKqqurtXPnTg0efHEp644dO1RdXW3WLE+oRy0XKHlKkrPB1qT++GvugfK6BkqeEvUIAOD/vHJRgks1HsDExcW5HcA0ajyAaTzwuPQAplHjAUxjzKUHMI2u5QAGABA4cnNztWXLFh06dEg7duzQd7/7XdXU1GjixImy2WzKycnRwoULtWHDBu3du1eTJk1SRESEsrKyJEkOh0OTJ0/W7Nmz9Yc//EG7d+/Wv//7v6t///4aOXKkJKlv374aPXq0pkyZou3bt2v79u2aMmWKMjIymj2fJwAAAADf1+or1HJzczV27Fj17NlTlZWVevbZZz0ewPTp00d9+vTRwoULmz2A6datm6KiopSbm9vsAczy5cslSVOnTuUABgBgOnbsmB5++GGdPHlSN954o4YMGaLt27erV69ekqQnn3xStbW1mj59uqqqqpScnKyioiJ17tzZ3MdLL72k4OBgjR8/XrW1tXrggQe0evVqBQUFmTFr167VzJkzzYvpZGZmaunSpW2bLAAAAIA21eoNNQ5gAAAdwfr16694v81mU15envLy8pqNCQsLU35+vvLz85uNiYqKUkFBQUunCQAAAMAHtXpDjQMYAAAAAAAA+DOvn0MNAAAAAAAA8Cc01AAAAAAAAAALaKgBAAAAAAAAFtBQAwAAAAAAACygoQYAAAAAAABYQEMNAAAAAAAAsICGGgAAAAAAAGABDTUAAAAAAADAAhpqAAAAAAAAgAU01AAAAAAAAAALaKgBAAAAAAAAFtBQAwAAAAAAACygoQYAAAAAAABYQEMNAAAAAAAAsICGGgAAAAAAAGABDTUAAAAAAADAAhpqAAAAAAAAgAU01AAAAAAAAAALaKgBAAAAAAAAFtBQAwAAAAAAACygoQYAAAAAAABYQEMNAAAAAAAAsICGGgAAAAAAAGABDTUAAAAAAADAAhpqAAAAAAAAgAU01AAAAAAAAAALaKgBAAAAAAAAFtBQAwAAAAAAACygoQYAAAAAAABYQEMNAAAAAAAAsICGGgAAAAAAAGABDTUAAAAAAADAAhpqAAAAAAAAgAXB7T0BAAAAtL+b52z0OH74uTFtPBMAAICOj4Ya0AE0dxBjBQc8AAAAAAC0DRpqaDOt0TRqLe3VfOpIzwEAAAAAAGgZGmoISJ4aW/YgQ4sHS0l578tZb2vxftoLX9UBAFwLb9eu1qpHnvZDTQMAAB0FDTW0uo7UZAIAIJC1Rk1urbreGh9mWZ0LDTgAAOAtNNTQYjTOfMOVDmAAAL6F2mtNaz1fzTXmWB0OAEDgoqEGAADgZVYaO3zo0fHQyAQAAJejoYZrYuW8YvAdnl5XPlUHAAAAAODKaKjBzeWfwPIpeeDh6ysAcG1YtYTm8PsUAAD+j4YagGtCow2Av6NBBm+zsuKf+goAQMdGQw3AdfF0AMpBAAAA14cPsgAA6Nh8vqH26quv6oUXXlB5ebnuuOMOvfzyy7r33nvbe1odHp/Cw5s4CEAgoh51PNQ6AAAAeItPN9TeeOMN5eTk6NVXX9U999yj5cuXKz09Xfv371fPnj3be3odAgcT6EhotMFfUY/aF7UOgYRaCgBAx+DTDbUlS5Zo8uTJeuSRRyRJL7/8st5//30tW7ZMixYtaufZtS0OJuDLODiAr6MetS5P7wmNJ3XnqtOAZ1Z/F6TGAgBwfXy2oVZXV6fS0lLNmTPHbTw1NVUlJSUet3E6nXI6nebt6upqSdLp06flcrksPb7L5dKFCxd06tQphYSEWJz91SUv+oOleG+9kMENhi5caFCwq5PqG/z7AIZcO55/yf2/696HvZOhHw1o8Nr/1Y7iet6Tzp49K0kyDMMbU/N7/l6PvM1TvfNU03zlfas1kKv/6Yh5njp1yiv7pR4BAAKFzzbUTp48qfr6esXExLiNx8TEqKKiwuM2ixYt0vz585uM9+7d2ytz9BdZ7T2BNkSu/imQcr0eZ8+elcPhaO9p+BzqUdsJpP/L5Op/Olqe3V9s7xk0j3oEAPAFPttQa2SzuX/KZxhGk7FGc+fO1axZs8zbDQ0NOn36tLp169bsNs2pqalRQkKCjh49qi5dulifuI8IlDwlcvVXgZLr9eRpGIbOnj2r+Ph4L80uMFCPvCtQ8pTI1R8FSp4S9QgAEDh8tqHWvXt3BQUFNfn0v7KysskqgUZ2u112u91t7IYbbriueXTp0sXvfzGSAidPiVz9VaDk2tI8WQnQctSjthUoeUrk6o8CJU+JegQA8H+d2nsCLRUaGqqBAwequLjYbby4uFhDhw5tp1kBAAIN9QgAAAAIPD67Qk2SZs2apezsbA0aNEgpKSn61a9+pS+++EKPPvpoe08NABBAqEcAAABAYPHphtqECRN06tQp/eQnP1F5ebmSkpK0adMm9erVy+uPbbfb9cwzzzT5yo6/CZQ8JXL1V4GSa6Dk2VFRj7wvUPKUyNUfBUqeUmDlCgAIbDaD61IDAAAAAAAA18xnz6EGAAAAAAAAtAcaagAAAAAAAIAFNNQAAAAAAAAAC2ioAQAAAAAAABbQULuCV199Vb1791ZYWJgGDhyojz766IrxW7Zs0cCBAxUWFqZbbrlFv/zlL9toptfHSp5vvfWWRo0apRtvvFFdunRRSkqK3n///Tac7fWx+po2+vjjjxUcHKy7777buxNsRVZzdTqdmjdvnnr16iW73a5bb71V//u//9tGs205q3muXbtWd911lyIiIhQXF6cf/OAHOnXqVBvNtuU+/PBDjR07VvHx8bLZbHr77bevuo2vvifBXaDUIol6dC2oRx0X9ah5vvy+BABAswx4tH79eiMkJMRYsWKFsX//fuOJJ54wIiMjjSNHjniM//zzz42IiAjjiSeeMPbv32+sWLHCCAkJMX7729+28cytsZrnE088YTz//PPGzp07jb///e/G3LlzjZCQEOPPf/5zG8/cOqu5Njpz5oxxyy23GKmpqcZdd93VNpO9Ti3JNTMz00hOTjaKi4uNQ4cOGTt27DA+/vjjNpy1dVbz/Oijj4xOnToZr7zyivH5558bH330kXHHHXcYDz74YBvP3LpNmzYZ8+bNM958801DkrFhw4YrxvvqexLcBUotMgzqEfXoa9Sjjo16BADARTTUmjF48GDj0UcfdRu7/fbbjTlz5niMf/LJJ43bb7/dbWzatGnGkCFDvDbH1mA1T0/69etnzJ8/v7Wn1upamuuECROMH/3oR8YzzzzjMwcwVnP9/e9/bzgcDuPUqVNtMb1WYzXPF154wbjlllvcxn7+858bPXr08NocveFaDmB89T0J7gKlFhkG9Yh6dBH1iHoEAICv4CufHtTV1am0tFSpqalu46mpqSopKfG4zbZt25rEp6WladeuXXK5XF6b6/VoSZ6Xa2ho0NmzZxUVFeWNKbaalua6atUq/eMf/9Azzzzj7Sm2mpbk+s4772jQoEFavHixbrrpJt12223Kzc1VbW1tW0y5RVqS59ChQ3Xs2DFt2rRJhmHo+PHj+u1vf6sxY8a0xZTblC++J8FdoNQiiXokUY8aUY+oRwAA+Irg9p5AR3Ty5EnV19crJibGbTwmJkYVFRUet6moqPAY/9VXX+nkyZOKi4vz2nxbqiV5Xu7FF1/U+fPnNX78eG9MsdW0JNdPP/1Uc+bM0UcffaTgYN/5r9KSXD///HNt3bpVYWFh2rBhg06ePKnp06fr9OnTHfa8NS3Jc+jQoVq7dq0mTJigL7/8Ul999ZUyMzOVn5/fFlNuU774ngR3gVKLJOqRRD1qRD2iHgEA4CtYoXYFNpvN7bZhGE3GrhbvabyjsZpno1//+tfKy8vTG2+8oejoaG9Nr1Vda6719fXKysrS/Pnzddttt7XV9FqVlde1oaFBNptNa9eu1eDBg/Wv//qvWrJkiVavXt2hVwVI1vLcv3+/Zs6cqR//+McqLS1VYWGhDh06pEcffbQtptrmfPU9Ce4CpRZJ1CPqEfWIegQAgO/wnY8521D37t0VFBTU5FPFysrKJp+wNYqNjfUYHxwcrG7dunltrtejJXk2euONNzR58mT95je/0ciRI705zVZhNdezZ89q165d2r17tx5//HFJF3/JNwxDwcHBKioq0v33398mc7eqJa9rXFycbrrpJjkcDnOsb9++MgxDx44dU58+fbw655ZoSZ6LFi3SPffco//6r/+SJN15552KjIzUvffeq2effdavPiX3xfckuAuUWiRRjyTqUSPqEfUIAABfwQo1D0JDQzVw4EAVFxe7jRcXF2vo0KEet0lJSWkSX1RUpEGDBikkJMRrc70eLclTurgSYNKkSVq3bp3PnOvDaq5dunTRnj17VFZWZv48+uijSkxMVFlZmZKTk9tq6pa15HW955579M9//lPnzp0zx/7+97+rU6dO6tGjh1fn21ItyfPChQvq1Mn9bS8oKEjS15+W+wtffE+Cu0CpRRL1SKIeNaIeUY8AAPAZbXkFBF/SePnzlStXGvv37zdycnKMyMhI4/Dhw4ZhGMacOXOM7OxsM77xkuD/+Z//aezfv99YuXKlT1wS3Gqe69atM4KDg41f/OIXRnl5uflz5syZ9krhmlnN9XK+dFU1q7mePXvW6NGjh/Hd737X2Ldvn7FlyxajT58+xiOPPNJeKVwTq3muWrXKCA4ONl599VXjH//4h7F161Zj0KBBxuDBg9srhWt29uxZY/fu3cbu3bsNScaSJUuM3bt3G0eOHDEMw3/ek+AuUGqRYVCPqEcXUY+oRwAA+Aoaalfwi1/8wujVq5cRGhpqfPOb3zS2bNli3jdx4kRj2LBhbvEffPCBMWDAACM0NNS4+eabjWXLlrXxjFvGSp7Dhg0zJDX5mThxYttPvAWsvqaX8qUDGMOwnuuBAweMkSNHGuHh4UaPHj2MWbNmGRcuXGjjWVtnNc+f//znRr9+/Yzw8HAjLi7O+P73v28cO3asjWdt3Z/+9Kcr/t/zp/ckuAuUWmQY1KNG1CPqUUdGPQIA4CKbYfjZunIAAAAAAADAiziHGgAAAAAAAGABDTUAAAAAAADAAhpqAAAAAAAAgAU01AAAAAAAAAALaKgBAAAAAAAAFtBQAwAAAAAAACygoQYAAAAAAABYQEMNAAAAAAAAsICGGgAAAAAAAGABDTUAAAAAAADAAhpqAAAAAAAAgAU01AAAAAAAAAAL/n8WRvkJwwd+0wAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 1500x1000 with 9 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "elec_df.hist(bins=50, figsize=(15,10))\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Data Preprocessing \n",
    "\n",
    "We will assign the data to the variable `X` and the targets (the column `class`) to `y`.\n",
    "\n",
    "Recall again from the lecture on Data Exploration and Preprocessing that if the dataset contained an `index` column it would be important to remove that column from the training dataset `X`, to avoid associating the target labels with the indices of the rows by the model. This dataset does not have an `index` column, therefore we don't need to worry about that. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = elec_df.drop('class', axis=1)\n",
    "y = elec_df['class']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since the target feature `class` is categorical, let's encode it to numerical values using Label Encoder. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import LabelEncoder\n",
    "\n",
    "label_enc = LabelEncoder()\n",
    "y = label_enc.fit_transform(y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 1, 1, ..., 0, 1, 0])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And let's split the data into training and test sets. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=20)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training data inputs (33984, 8)\n",
      "Training labels (33984,)\n",
      "Testing data inputs (11328, 8)\n",
      "Testing labels (11328,)\n"
     ]
    }
   ],
   "source": [
    "print('Training data inputs', X_train.shape)\n",
    "print('Training labels', y_train.shape)\n",
    "print('Testing data inputs', X_test.shape)\n",
    "print('Testing labels', y_test.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we mentioned previously, the data values are already scaled to the range [0, 1], therefore we don't need to apply feature scaling. If that was not the case, we would have applied one of the Scaler functions in scikit-learn."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 14.2 Voting Ensemble <a id=\"14.2-voting-ensemble\"/>\n",
    "\n",
    "**Voting ensemble** is the simplest ensemble technique, which aggregates the predictions of several individual classifiers and assigns the predicted class labels based on the majority votes of the individual classifiers. \n",
    "\n",
    "For example, if we have a dataset with two classes A and B, and we train three classifiers. Then, for one particular test sample:\n",
    "\n",
    "- Classifier 1 predicts class A\n",
    "- Classifier 2 predicts class B\n",
    "- Classifier 3 predicts class B\n",
    "  \n",
    "Ensemble majority voting is: class B (2 votes), class A (1 vote). The prediction by the Ensemble Method is: class B.\n",
    "\n",
    "Let's first train 3 classifiers on the training data for `electricity_data`, including: Logistic Regression, Support Vector Machines, and SGD. Afterward, we will apply a voting ensemble that uses their results to make predictions. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Logistic Regression accuracy is 75.5297 %\n"
     ]
    }
   ],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.metrics import accuracy_score\n",
    "\n",
    "lr_model = LogisticRegression()\n",
    "\n",
    "# train\n",
    "lr_model.fit(X_train, y_train)\n",
    "\n",
    "# evaluate\n",
    "lr_preds = lr_model.predict(X_test)\n",
    "lr_acc = accuracy_score(y_test, lr_preds)\n",
    "print('Logistic Regression accuracy is {0:7.4f} %'.format(lr_acc*100))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SVM accuracy is 73.6935 %\n"
     ]
    }
   ],
   "source": [
    "from sklearn.svm import SVC\n",
    "\n",
    "svm_model = SVC(random_state=1)\n",
    "\n",
    "# train\n",
    "svm_model.fit(X_train, y_train)\n",
    "\n",
    "# evaluate\n",
    "svm_preds = svm_model.predict(X_test)\n",
    "svm_acc = accuracy_score(y_test, svm_preds)\n",
    "print('SVM accuracy is {0:7.4f} %'.format(svm_acc*100))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SGD accuracy is 75.3708 %\n"
     ]
    }
   ],
   "source": [
    "from sklearn.linear_model import SGDClassifier\n",
    "\n",
    "sgd_model = SGDClassifier(random_state=1)\n",
    "\n",
    "# train\n",
    "sgd_model.fit(X_train, y_train)\n",
    "\n",
    "# evaluate\n",
    "sgd_preds = sgd_model.predict(X_test)\n",
    "sgd_acc = accuracy_score(y_test, sgd_preds)\n",
    "print('SGD accuracy is {0:7.4f} %'.format(sgd_acc*100))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy: Logistic Regression 75.5297; SVM 73.6935; SGD 75.3708\n"
     ]
    }
   ],
   "source": [
    "print('Accuracy: Logistic Regression {0:7.4f}; SVM {1:7.4f}; SGD {2:7.4f}'.format(lr_acc*100, svm_acc*100, sgd_acc*100))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In scikit-learn we will import `Voting Classifier` to aggregate the results of those 3 classifiers. In the cell below, the models are listed in the `estimators` argument of the `Voting Classifier`, and afterward the `Voting Classifier` model is fit on the training data. The voting argument `hard` means that the ensemble algorithm uses the predicted class labels for majority voting."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-1 {color: black;}#sk-container-id-1 pre{padding: 0;}#sk-container-id-1 div.sk-toggleable {background-color: white;}#sk-container-id-1 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-1 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-1 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-1 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-1 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-1 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-1 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-1 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-1 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-1 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-1 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-1 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-1 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-1 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-1 div.sk-item {position: relative;z-index: 1;}#sk-container-id-1 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-1 div.sk-item::before, #sk-container-id-1 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-1 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-1 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-1 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-1 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-1 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-1 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-1 div.sk-label-container {text-align: center;}#sk-container-id-1 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-1 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-1\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>VotingClassifier(estimators=[(&#x27;log_reg&#x27;, LogisticRegression()),\n",
       "                             (&#x27;svc&#x27;, SVC(random_state=1)),\n",
       "                             (&#x27;sgd&#x27;, SGDClassifier(random_state=1))])</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-1\" type=\"checkbox\" ><label for=\"sk-estimator-id-1\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">VotingClassifier</label><div class=\"sk-toggleable__content\"><pre>VotingClassifier(estimators=[(&#x27;log_reg&#x27;, LogisticRegression()),\n",
       "                             (&#x27;svc&#x27;, SVC(random_state=1)),\n",
       "                             (&#x27;sgd&#x27;, SGDClassifier(random_state=1))])</pre></div></div></div><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><label>log_reg</label></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-2\" type=\"checkbox\" ><label for=\"sk-estimator-id-2\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">LogisticRegression</label><div class=\"sk-toggleable__content\"><pre>LogisticRegression()</pre></div></div></div></div></div></div><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><label>svc</label></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-3\" type=\"checkbox\" ><label for=\"sk-estimator-id-3\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">SVC</label><div class=\"sk-toggleable__content\"><pre>SVC(random_state=1)</pre></div></div></div></div></div></div><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><label>sgd</label></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-4\" type=\"checkbox\" ><label for=\"sk-estimator-id-4\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">SGDClassifier</label><div class=\"sk-toggleable__content\"><pre>SGDClassifier(random_state=1)</pre></div></div></div></div></div></div></div></div></div></div>"
      ],
      "text/plain": [
       "VotingClassifier(estimators=[('log_reg', LogisticRegression()),\n",
       "                             ('svc', SVC(random_state=1)),\n",
       "                             ('sgd', SGDClassifier(random_state=1))])"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.ensemble import VotingClassifier\n",
    "\n",
    "voting_classifier = VotingClassifier(\n",
    "    \n",
    "    estimators=[('log_reg', lr_model),\n",
    "                ('svc', svm_model),\n",
    "                ('sgd', sgd_model)], \n",
    "    voting='hard')\n",
    "\n",
    "voting_classifier.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Voting Ensemble accuracy is 75.6532 %\n"
     ]
    }
   ],
   "source": [
    "# evaluate\n",
    "voting_preds = voting_classifier.predict(X_test)\n",
    "voting_acc = accuracy_score(y_test, voting_preds)\n",
    "print('Voting Ensemble accuracy is {0:7.4f} %'.format(voting_acc*100))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The Voting Ensemble achieved slightly higher accuracy than the three models. \n",
    "\n",
    "Beside `hard` voting, another alternative for the voting ensemble classifier is to change the argument to `voting='soft'`. In this case, the decision by the ensemble is made based on the average of the predicted class with the highest probabilities of all individual classifiers. \n",
    "\n",
    "Example of soft voting:\n",
    "\n",
    "- Classifier 1 predicts class A with probability 90%, class B with probability 10%\n",
    "- Classifier 2 predicts class A with probability 45%, class B with probability 55%\n",
    "- Classifier 3 predicts class A with probability 45%, class B with probability 55%\n",
    "- Ensemble probabilities: class A ((90% + 45% + 45%) / 3 = 60%), class B ((10% + 55% + 55%) / 3 = 40%). Prediction: class A.\n",
    "\n",
    "Note in the above example that if hard voting based on the class labels was applied, the prediction would have been class B, because both Classifiers 2 and 3 predicted higher probabilities for class B.\n",
    "\n",
    "Soft voting generally achieves better performance than hard voting, because it assigns higher weight to the predictions  with high confidence by the individual models.\n",
    "\n",
    "In the cell below, to apply soft voting with SVC we need to change the argument `probability=True`, as well as this approach requires to modify the outputs of the SGD model, and instead of SGD we will use a k-Nearest Neighbors model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-2 {color: black;}#sk-container-id-2 pre{padding: 0;}#sk-container-id-2 div.sk-toggleable {background-color: white;}#sk-container-id-2 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-2 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-2 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-2 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-2 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-2 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-2 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-2 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-2 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-2 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-2 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-2 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-2 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-2 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-2 div.sk-item {position: relative;z-index: 1;}#sk-container-id-2 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-2 div.sk-item::before, #sk-container-id-2 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-2 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-2 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-2 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-2 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-2 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-2 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-2 div.sk-label-container {text-align: center;}#sk-container-id-2 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-2 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-2\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>VotingClassifier(estimators=[(&#x27;log_reg&#x27;, LogisticRegression()),\n",
       "                             (&#x27;svc&#x27;, SVC(gamma=&#x27;auto&#x27;, probability=True)),\n",
       "                             (&#x27;knn&#x27;, KNeighborsClassifier())],\n",
       "                 voting=&#x27;soft&#x27;)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-5\" type=\"checkbox\" ><label for=\"sk-estimator-id-5\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">VotingClassifier</label><div class=\"sk-toggleable__content\"><pre>VotingClassifier(estimators=[(&#x27;log_reg&#x27;, LogisticRegression()),\n",
       "                             (&#x27;svc&#x27;, SVC(gamma=&#x27;auto&#x27;, probability=True)),\n",
       "                             (&#x27;knn&#x27;, KNeighborsClassifier())],\n",
       "                 voting=&#x27;soft&#x27;)</pre></div></div></div><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><label>log_reg</label></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-6\" type=\"checkbox\" ><label for=\"sk-estimator-id-6\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">LogisticRegression</label><div class=\"sk-toggleable__content\"><pre>LogisticRegression()</pre></div></div></div></div></div></div><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><label>svc</label></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-7\" type=\"checkbox\" ><label for=\"sk-estimator-id-7\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">SVC</label><div class=\"sk-toggleable__content\"><pre>SVC(gamma=&#x27;auto&#x27;, probability=True)</pre></div></div></div></div></div></div><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><label>knn</label></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-8\" type=\"checkbox\" ><label for=\"sk-estimator-id-8\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">KNeighborsClassifier</label><div class=\"sk-toggleable__content\"><pre>KNeighborsClassifier()</pre></div></div></div></div></div></div></div></div></div></div>"
      ],
      "text/plain": [
       "VotingClassifier(estimators=[('log_reg', LogisticRegression()),\n",
       "                             ('svc', SVC(gamma='auto', probability=True)),\n",
       "                             ('knn', KNeighborsClassifier())],\n",
       "                 voting='soft')"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn import neighbors\n",
    "\n",
    "knn_model = neighbors.KNeighborsClassifier()\n",
    "svm_classifier = SVC(gamma=\"auto\", probability=True)\n",
    "\n",
    "voting_classifier_soft = VotingClassifier(\n",
    "    \n",
    "    estimators=[('log_reg', lr_model),\n",
    "                ('svc', svm_classifier),\n",
    "                 ('knn', knn_model)], \n",
    "     voting='soft')\n",
    "\n",
    "voting_classifier_soft.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Voting Ensemble accuracy is 80.7998 %\n"
     ]
    }
   ],
   "source": [
    "# evaluate\n",
    "voting_preds = voting_classifier_soft.predict(X_test)\n",
    "voting_acc = accuracy_score(y_test, voting_preds)\n",
    "print('Voting Ensemble accuracy is {0:7.4f} %'.format(voting_acc*100))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ensemble Methods are generally well-performing. Even in the cases when the individual models are *weak learners* (which means that they predict slightly better than random guessing), ensemble learning can result in a *strong learner* (meaning that it has high prediction accuracy), as long as a large number of diverse individual models are used. \n",
    "\n",
    "In general, ensemble methods produce the best results when the individual models are independent of each other, e.g., they use different types of learning algorithms. This way, the individual models can produce diverse results, which can increase the chance that the ensembling will be more successful. Conversely, if the individual models use the same learning algorithm, they will probably make the same type of errors, which will reduce the accuracy when an ensemble method is used. \n",
    "\n",
    "Diversity in individual learners can be achieved by selecting models that use different learning algorithms, or by training models on different subsets of the training data, or different data features, etc. Also, many ensemble methods employ Decision Trees as individual models, where diversity can be achieved by varying the strategies for tree splitting, by varying the different hyperparameters, or by using techniques based on data diversity, feature diversity, and other techniques. Specific approaches are explained next. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 14.3 Bagging Ensemble <a id=\"14.3-bagging-ensemble\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we mentioned, one way to create diverse base models is to train them on different portions of the training data. Hence, instead of training different models on the same data and averaging their results with a Voting Ensemble, **Bagging Ensemble** method trains the base models on different subsets of the training data and aggregates the results. \n",
    "\n",
    "The term **bagging** is short for *bootstrap aggregating*, where bootstrapping refers to sampling subsets from the training data with replacement. In other words, the subsets of the training data used by different models can contain the same data points. When the subsets are sampled without replacement, it is called *pasting*.\n",
    "\n",
    "The aggregation of the predictions by the individual models is typically based on the most frequent prediction, similar to hard voting. The individual models used with bagging are typically Decision Trees.\n",
    "\n",
    "In scikit-learn we can import a Bagging Classifier, and fit it to the training data as any other model. Let's use bagging to train  Decision Trees on different subsets of the data and then average the predictions. In the cell below, `max_samples=0.5` means that each model will use a random subset containing 50% of the training data, and `max_features=0.5` means that each model will use a random subset of 50% of the input features. If bootstrap is `True` the training samples will be sampled with replacement, and if it is `False` there is no replacement (i.e., pasting). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-3 {color: black;}#sk-container-id-3 pre{padding: 0;}#sk-container-id-3 div.sk-toggleable {background-color: white;}#sk-container-id-3 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-3 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-3 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-3 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-3 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-3 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-3 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-3 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-3 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-3 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-3 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-3 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-3 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-3 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-3 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-3 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-3 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-3 div.sk-item {position: relative;z-index: 1;}#sk-container-id-3 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-3 div.sk-item::before, #sk-container-id-3 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-3 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-3 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-3 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-3 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-3 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-3 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-3 div.sk-label-container {text-align: center;}#sk-container-id-3 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-3 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-3\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>BaggingClassifier(estimator=DecisionTreeClassifier(class_weight=&#x27;balanced&#x27;),\n",
       "                  max_features=0.5, max_samples=0.5)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-9\" type=\"checkbox\" ><label for=\"sk-estimator-id-9\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">BaggingClassifier</label><div class=\"sk-toggleable__content\"><pre>BaggingClassifier(estimator=DecisionTreeClassifier(class_weight=&#x27;balanced&#x27;),\n",
       "                  max_features=0.5, max_samples=0.5)</pre></div></div></div><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-10\" type=\"checkbox\" ><label for=\"sk-estimator-id-10\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">estimator: DecisionTreeClassifier</label><div class=\"sk-toggleable__content\"><pre>DecisionTreeClassifier(class_weight=&#x27;balanced&#x27;)</pre></div></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-11\" type=\"checkbox\" ><label for=\"sk-estimator-id-11\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">DecisionTreeClassifier</label><div class=\"sk-toggleable__content\"><pre>DecisionTreeClassifier(class_weight=&#x27;balanced&#x27;)</pre></div></div></div></div></div></div></div></div></div></div>"
      ],
      "text/plain": [
       "BaggingClassifier(estimator=DecisionTreeClassifier(class_weight='balanced'),\n",
       "                  max_features=0.5, max_samples=0.5)"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.ensemble import BaggingClassifier\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "\n",
    "bagging_classifier = BaggingClassifier(\n",
    "      DecisionTreeClassifier(class_weight='balanced'),\n",
    "    max_samples=0.5, max_features=0.5, bootstrap=True\n",
    ")\n",
    "\n",
    "bagging_classifier.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Bagging Ensemble accuracy is 83.6335 %\n"
     ]
    }
   ],
   "source": [
    "# evaluate\n",
    "bagging_preds = bagging_classifier.predict(X_test)\n",
    "bagging_acc = accuracy_score(y_test, bagging_preds)\n",
    "print('Bagging Ensemble accuracy is {0:7.4f} %'.format(bagging_acc*100))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Bagging Ensembles often outperform Voting Ensembles. As well as, Bagging Ensembles can be trained in parallel using different CPU or GPU cores, which can reduce the processing time.  \n",
    "\n",
    "In general, Bagging Ensembles can acccept other classifiers as base learners, such as SVMs, Logistic Regression, etc."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 14.4 Boosting Ensemble <a id=\"14.4-boosting-ensemble\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Boosting Ensemble** trains the individual base models sequentially, where each next model has access to the predictions of the previous models and attempts to improve the performance. The term *boosting* refers to incremental improvement by reducing the errors of the predecessors, in order to gradually improve the performance.\n",
    "\n",
    "The most popular boosting ensembles are Gradient Boosting, XGBoost, LightGBM, CatBoost, and AdaBoost. \n",
    "\n",
    "Although Boosting Ensembles can employ various classifiers as base models, Decision Trees are most commonly used as base models. The reasons are because Decision Trees are easily diversified by varying the splitting of the trees or by other means, they can capture complex nonlinear relationships in the data, they are fast and simple to implement, and can also provide information about the feature importance."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 14.4.1 Gradient Boosting Ensemble <a id=\"14.4.1-gradient-boosting-ensemble\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Gradient Boosting Ensemble** fits the individual models sequentially, where each new model is fit to minimize a loss function based on the error (gradient) of a loss function made by the previous models. The individual models in gradient boosting are Decision Trees. \n",
    "\n",
    "Specifically, the initial model is trained on the full dataset, and the errors (gradients) of the loss function are obtained based on the difference between the predicted class of the model and the target label. The next model is trained by using the negative gradient of the loss function in an attempt to correct the mistakes made by the previous model. These steps are repeated for each single model. Ideally, as more models are trained the errors should become smaller, that is, the predictions by the models should better match the target labels. \n",
    "\n",
    "The residual errors represent the gradient of the loss function with respect to the predicted values. This is the reason why the method is called Gradient Boosting.\n",
    "\n",
    "In scikit-learn, Gradient Boosting Classifier has hyperparameters related to the number of estimators, i.e., Decision Trees (`n_estimators`), learning rate (`learning_rate`), maximum depth of the trees (`max_depth`), and others. The learning rate is typically set to values between 0.01 and 1, and it defines how quickly the models are updated. If the learning rate is too high, the models will be updated more quickly, which can cause overfitting."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-4 {color: black;}#sk-container-id-4 pre{padding: 0;}#sk-container-id-4 div.sk-toggleable {background-color: white;}#sk-container-id-4 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-4 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-4 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-4 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-4 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-4 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-4 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-4 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-4 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-4 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-4 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-4 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-4 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-4 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-4 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-4 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-4 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-4 div.sk-item {position: relative;z-index: 1;}#sk-container-id-4 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-4 div.sk-item::before, #sk-container-id-4 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-4 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-4 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-4 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-4 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-4 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-4 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-4 div.sk-label-container {text-align: center;}#sk-container-id-4 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-4 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-4\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>GradientBoostingClassifier(learning_rate=0.8, max_depth=2, n_estimators=500,\n",
       "                           random_state=42)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-12\" type=\"checkbox\" checked><label for=\"sk-estimator-id-12\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">GradientBoostingClassifier</label><div class=\"sk-toggleable__content\"><pre>GradientBoostingClassifier(learning_rate=0.8, max_depth=2, n_estimators=500,\n",
       "                           random_state=42)</pre></div></div></div></div></div>"
      ],
      "text/plain": [
       "GradientBoostingClassifier(learning_rate=0.8, max_depth=2, n_estimators=500,\n",
       "                           random_state=42)"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.ensemble import GradientBoostingClassifier\n",
    "\n",
    "grad_boost_classifier = GradientBoostingClassifier(\n",
    "                        n_estimators=500, \n",
    "                        learning_rate=0.8, \n",
    "                        random_state=42,\n",
    "                        max_depth=2)\n",
    "\n",
    "grad_boost_classifier.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Gradient Boosting Ensemble accuracy is 89.6893 %\n"
     ]
    }
   ],
   "source": [
    "# evaluate\n",
    "gboost_preds = grad_boost_classifier.predict(X_test)\n",
    "gboost_acc = accuracy_score(y_test, gboost_preds)\n",
    "print('Gradient Boosting Ensemble accuracy is {0:7.4f} %'.format(gboost_acc*100))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 14.4.2 Recent Gradient Boosting Methods: XGBoost, LightGBM, CatBoost <a id=\"14.4.2-recent-gradient-boosting-methods:-xgboost,-lightgbm,-catboost\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are several optimized variants of Gradient Boosting Ensembles, which include XGBoost, LightGBM, and CatBoost. These three ensemble methods are among the most powerful and robust Machine Learning models for processing tabular data at present time. Their performance is often superior to other conventional ML methods, including other ensemble methods. Also, they are very fast and scalable to large datasets. Because of these properties, XGBoost, LightGBM, and CatBoost are often the winning algorithms in many Kaggle competitions and other ML competitions where the datasets are in tabular format. \n",
    "\n",
    "Note that XGBoost, LightGBM, and Catboost are not implemented in Scikit-Learn, and they need to be installed as separate libraries, e.g., via:\n",
    "    \n",
    "    pip install xgboost catboost lightgbm\n",
    "    \n",
    "On the other hand, the API for working with these methods is the same as scikit-learn, and they use the `fit` and `predict` methods for model training and evaluation.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 14.4.2.1 XGBoost (Extreme Gradient Boosting) <a id=\"14.4.2.1-xgboost-(extreme-gradient-boosting)\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**XGBoost** or **eXtreme Gradient Boosting Ensemble** introduces several modifications and optimizations in the original Gradient Boosting algorithm, which speed up the training process and improve the predictive performance. The main modifications in XGBoost encompass the introduction of L1 and L2 regularization terms into the objective function to reduce overfitting, and using a more efficient tree-building algorithm that relies on histogram-based methods and pruning techniques to optimize the tree splitting. XGBoost can also handle missing data, either by treating the data samples with missing values as a separate category, or can apply techniques for filling in the missing values. Therefore, this ensemble method can work without significant exploratory data analysis and preprocessing. \n",
    "\n",
    "As shown in the next cells, applying these variants of Gradient Boosting is as simple as fitting and evaluating other scikit-learn models. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-5 {color: black;}#sk-container-id-5 pre{padding: 0;}#sk-container-id-5 div.sk-toggleable {background-color: white;}#sk-container-id-5 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-5 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-5 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-5 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-5 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-5 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-5 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-5 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-5 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-5 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-5 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-5 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-5 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-5 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-5 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-5 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-5 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-5 div.sk-item {position: relative;z-index: 1;}#sk-container-id-5 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-5 div.sk-item::before, #sk-container-id-5 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-5 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-5 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-5 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-5 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-5 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-5 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-5 div.sk-label-container {text-align: center;}#sk-container-id-5 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-5 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-5\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>XGBClassifier(base_score=None, booster=None, callbacks=None,\n",
       "              colsample_bylevel=None, colsample_bynode=None,\n",
       "              colsample_bytree=None, device=None, early_stopping_rounds=None,\n",
       "              enable_categorical=False, eval_metric=None, feature_types=None,\n",
       "              gamma=None, grow_policy=None, importance_type=None,\n",
       "              interaction_constraints=None, learning_rate=None, max_bin=None,\n",
       "              max_cat_threshold=None, max_cat_to_onehot=None,\n",
       "              max_delta_step=None, max_depth=None, max_leaves=None,\n",
       "              min_child_weight=None, missing=nan, monotone_constraints=None,\n",
       "              multi_strategy=None, n_estimators=None, n_jobs=None,\n",
       "              num_parallel_tree=None, random_state=None, ...)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-13\" type=\"checkbox\" checked><label for=\"sk-estimator-id-13\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">XGBClassifier</label><div class=\"sk-toggleable__content\"><pre>XGBClassifier(base_score=None, booster=None, callbacks=None,\n",
       "              colsample_bylevel=None, colsample_bynode=None,\n",
       "              colsample_bytree=None, device=None, early_stopping_rounds=None,\n",
       "              enable_categorical=False, eval_metric=None, feature_types=None,\n",
       "              gamma=None, grow_policy=None, importance_type=None,\n",
       "              interaction_constraints=None, learning_rate=None, max_bin=None,\n",
       "              max_cat_threshold=None, max_cat_to_onehot=None,\n",
       "              max_delta_step=None, max_depth=None, max_leaves=None,\n",
       "              min_child_weight=None, missing=nan, monotone_constraints=None,\n",
       "              multi_strategy=None, n_estimators=None, n_jobs=None,\n",
       "              num_parallel_tree=None, random_state=None, ...)</pre></div></div></div></div></div>"
      ],
      "text/plain": [
       "XGBClassifier(base_score=None, booster=None, callbacks=None,\n",
       "              colsample_bylevel=None, colsample_bynode=None,\n",
       "              colsample_bytree=None, device=None, early_stopping_rounds=None,\n",
       "              enable_categorical=False, eval_metric=None, feature_types=None,\n",
       "              gamma=None, grow_policy=None, importance_type=None,\n",
       "              interaction_constraints=None, learning_rate=None, max_bin=None,\n",
       "              max_cat_threshold=None, max_cat_to_onehot=None,\n",
       "              max_delta_step=None, max_depth=None, max_leaves=None,\n",
       "              min_child_weight=None, missing=nan, monotone_constraints=None,\n",
       "              multi_strategy=None, n_estimators=None, n_jobs=None,\n",
       "              num_parallel_tree=None, random_state=None, ...)"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import xgboost as xgb\n",
    "\n",
    "xgb_classifier = xgb.XGBClassifier()\n",
    "xgb_classifier.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "XGBoost Ensemble accuracy is 90.1660 %\n"
     ]
    }
   ],
   "source": [
    "# evaluate\n",
    "xgboost_preds = xgb_classifier.predict(X_test)\n",
    "xgboost_acc = accuracy_score(y_test, xgboost_preds)\n",
    "print('XGBoost Ensemble accuracy is {0:7.4f} %'.format(xgboost_acc*100))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 14.4.2.2 LightGBM (Light Gradient Boosting Machine) <a id=\"14.4.2.2-lightgbm-(light-gradient-boosting-machine)\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**LightGBM** or **Light Gradient Boosting Machine** is another optimized version of Gradient Boosting Ensemble, which was developed by Microsoft and focuses on efficiency and scalability of the approach. It uses a histogram-based approach for finding optimal splitting of the decision trees. As well as, LightGBM employs a leaf-wise tree growth strategy that can result in deeper trees compared to other ensemble algorithms. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[LightGBM] [Info] Number of positive: 14443, number of negative: 19541\n",
      "[LightGBM] [Info] Auto-choosing col-wise multi-threading, the overhead of testing was 0.001957 seconds.\n",
      "You can set `force_col_wise=true` to remove the overhead.\n",
      "[LightGBM] [Info] Total Bins 1586\n",
      "[LightGBM] [Info] Number of data points in the train set: 33984, number of used features: 8\n",
      "[LightGBM] [Info] [binary:BoostFromScore]: pavg=0.424994 -> initscore=-0.302305\n",
      "[LightGBM] [Info] Start training from score -0.302305\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-6 {color: black;}#sk-container-id-6 pre{padding: 0;}#sk-container-id-6 div.sk-toggleable {background-color: white;}#sk-container-id-6 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-6 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-6 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-6 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-6 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-6 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-6 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-6 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-6 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-6 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-6 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-6 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-6 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-6 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-6 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-6 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-6 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-6 div.sk-item {position: relative;z-index: 1;}#sk-container-id-6 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-6 div.sk-item::before, #sk-container-id-6 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-6 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-6 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-6 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-6 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-6 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-6 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-6 div.sk-label-container {text-align: center;}#sk-container-id-6 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-6 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-6\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>LGBMClassifier()</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-14\" type=\"checkbox\" checked><label for=\"sk-estimator-id-14\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">LGBMClassifier</label><div class=\"sk-toggleable__content\"><pre>LGBMClassifier()</pre></div></div></div></div></div>"
      ],
      "text/plain": [
       "LGBMClassifier()"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import lightgbm\n",
    "from lightgbm.sklearn import LGBMClassifier\n",
    "\n",
    "lightgbm_classifier = LGBMClassifier()\n",
    "lightgbm_classifier.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "XGBoost Ensemble accuracy is 87.9237 %\n"
     ]
    }
   ],
   "source": [
    "# evaluate\n",
    "lightgbm_preds = lightgbm_classifier.predict(X_test)\n",
    "lightgbm_acc = accuracy_score(y_test, lightgbm_preds)\n",
    "print('XGBoost Ensemble accuracy is {0:7.4f} %'.format(lightgbm_acc*100))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Optimizing the hyperparameters of ensemble methods can result in significant improvement in performance. The next cells demonstrate this, by training another LightGBM model with different hyperparameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[LightGBM] [Info] Number of positive: 14443, number of negative: 19541\n",
      "[LightGBM] [Info] Auto-choosing col-wise multi-threading, the overhead of testing was 0.002078 seconds.\n",
      "You can set `force_col_wise=true` to remove the overhead.\n",
      "[LightGBM] [Info] Total Bins 1586\n",
      "[LightGBM] [Info] Number of data points in the train set: 33984, number of used features: 8\n",
      "[LightGBM] [Info] [binary:BoostFromScore]: pavg=0.424994 -> initscore=-0.302305\n",
      "[LightGBM] [Info] Start training from score -0.302305\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-7 {color: black;}#sk-container-id-7 pre{padding: 0;}#sk-container-id-7 div.sk-toggleable {background-color: white;}#sk-container-id-7 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-7 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-7 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-7 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-7 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-7 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-7 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-7 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-7 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-7 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-7 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-7 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-7 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-7 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-7 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-7 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-7 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-7 div.sk-item {position: relative;z-index: 1;}#sk-container-id-7 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-7 div.sk-item::before, #sk-container-id-7 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-7 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-7 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-7 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-7 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-7 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-7 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-7 div.sk-label-container {text-align: center;}#sk-container-id-7 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-7 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-7\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>LGBMClassifier(num_leaves=100)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-15\" type=\"checkbox\" checked><label for=\"sk-estimator-id-15\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">LGBMClassifier</label><div class=\"sk-toggleable__content\"><pre>LGBMClassifier(num_leaves=100)</pre></div></div></div></div></div>"
      ],
      "text/plain": [
       "LGBMClassifier(num_leaves=100)"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lightgbm_classifier_2 = LGBMClassifier(num_leaves=100, learning_rate=0.1, n_estimators=100)\n",
    "lightgbm_classifier_2.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "XGBoost Ensemble accuracy is 91.6314 %\n"
     ]
    }
   ],
   "source": [
    "# evaluate\n",
    "lightgbm_2_preds = lightgbm_classifier_2.predict(X_test)\n",
    "lightgbm_2_acc = accuracy_score(y_test, lightgbm_2_preds)\n",
    "print('XGBoost Ensemble accuracy is {0:7.4f} %'.format(lightgbm_2_acc*100))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In general, tuning the hyperparameters of Gradient Boosting variants can be non-trivial, and requires to get familiar with the various hyperparameters and understand their impact on the performance. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 14.4.2.3 CatBoost (Categorical Boosting) <a id=\"14.4.2.3-catboost-(categorical-boosting)\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**CatBoost** or **Categorical Boosting** is specifically designed for handling categorical features, and it can operate directly on data with categorical features, without the requirement for applying ordinal or label encoding or other preprocessing techniques. It uses an ordered gradient boosting technique for direct processing of categorical features. Other modifications in CatBoost include the use of specialized regularization techniques (such as depth regularization and feature permutation-based regularization) and implementation of specialized symmetric trees structure to prevent overfitting.\n",
    "\n",
    "Similar to XGBoost, CatBoost has also support for dealing with missing data values, which simplifies the data preprocessing. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<catboost.core.CatBoostClassifier at 0x1bf39c82450>"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import catboost\n",
    "from catboost import CatBoostClassifier\n",
    "\n",
    "catboost_classifier = CatBoostClassifier(verbose=False)\n",
    "catboost_classifier.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "XGBoost Ensemble accuracy is 89.2744 %\n"
     ]
    }
   ],
   "source": [
    "# evaluate\n",
    "catboost_preds = catboost_classifier.predict(X_test)\n",
    "catboost_acc = accuracy_score(y_test, catboost_preds)\n",
    "print('XGBoost Ensemble accuracy is {0:7.4f} %'.format(catboost_acc*100))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The decision on which variant of Gradient Boosting to select depends on the specific characteristics of a dataset, the available computational resources, the type of features in the dataset, and other factors. All three methods are high-performing, fast, and robust, and they are continuously developed and improved."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 14.4.3 AdaBoost Ensemble <a id=\"14.4.3-adaboost-ensemble\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**AdaBoost Ensemble** is another class of  Boosting Ensembles, however, instead of minimizing the gradient of a loss function as in Gradient Boosting models, it assigns weights to the training data instances based on the difficulty to be classified. The main idea of AdaBoost is to focus the individual models on the training instances that are difficult to classify. AdaBoost is short of *adaptive boosting*. \n",
    "\n",
    "AdaBoost begins by training a base model (which by default is a Decision Tree) on the full training dataset. After that, higher weights are assigned to the data instances that were misclassified by the previous model. Therefore, the second model will put more attention on correctly predicting the class of the misclassified data instances. By training many models sequentially, the algorithm will try harder to predict the difficult data instances. \n",
    "\n",
    "The aggregated prediction by all single models is done by averaging the predictions and using weights for each model based on their accuracy on the training set. \n",
    "\n",
    "The main parameters in the AdaBoost Classifier are the type of base estimators and the number of estimators. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-8 {color: black;}#sk-container-id-8 pre{padding: 0;}#sk-container-id-8 div.sk-toggleable {background-color: white;}#sk-container-id-8 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-8 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-8 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-8 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-8 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-8 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-8 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-8 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-8 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-8 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-8 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-8 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-8 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-8 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-8 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-8 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-8 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-8 div.sk-item {position: relative;z-index: 1;}#sk-container-id-8 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-8 div.sk-item::before, #sk-container-id-8 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-8 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-8 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-8 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-8 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-8 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-8 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-8 div.sk-label-container {text-align: center;}#sk-container-id-8 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-8 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-8\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>AdaBoostClassifier(base_estimator=DecisionTreeClassifier(class_weight=&#x27;balanced&#x27;,\n",
       "                                                         max_depth=3),\n",
       "                   learning_rate=0.5, n_estimators=300)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-16\" type=\"checkbox\" ><label for=\"sk-estimator-id-16\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">AdaBoostClassifier</label><div class=\"sk-toggleable__content\"><pre>AdaBoostClassifier(base_estimator=DecisionTreeClassifier(class_weight=&#x27;balanced&#x27;,\n",
       "                                                         max_depth=3),\n",
       "                   learning_rate=0.5, n_estimators=300)</pre></div></div></div><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-17\" type=\"checkbox\" ><label for=\"sk-estimator-id-17\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">base_estimator: DecisionTreeClassifier</label><div class=\"sk-toggleable__content\"><pre>DecisionTreeClassifier(class_weight=&#x27;balanced&#x27;, max_depth=3)</pre></div></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-18\" type=\"checkbox\" ><label for=\"sk-estimator-id-18\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">DecisionTreeClassifier</label><div class=\"sk-toggleable__content\"><pre>DecisionTreeClassifier(class_weight=&#x27;balanced&#x27;, max_depth=3)</pre></div></div></div></div></div></div></div></div></div></div>"
      ],
      "text/plain": [
       "AdaBoostClassifier(base_estimator=DecisionTreeClassifier(class_weight='balanced',\n",
       "                                                         max_depth=3),\n",
       "                   learning_rate=0.5, n_estimators=300)"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.ensemble import AdaBoostClassifier\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "\n",
    "adaboost_classifier = AdaBoostClassifier(\n",
    "    base_estimator=DecisionTreeClassifier(max_depth=3, class_weight='balanced'), \n",
    "    n_estimators=300,\n",
    "    learning_rate=0.5\n",
    ")\n",
    "\n",
    "adaboost_classifier.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "AdaBoost Ensemble accuracy is 89.9100 %\n"
     ]
    }
   ],
   "source": [
    "# evaluate\n",
    "adaboost_preds = adaboost_classifier.predict(X_test)\n",
    "adaboost_acc = accuracy_score(y_test, adaboost_preds)\n",
    "print('AdaBoost Ensemble accuracy is {0:7.4f} %'.format(adaboost_acc*100))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 14.5 Stacking Ensemble <a id=\"14.5-stacking-ensemble\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Stacking Ensemble** is also referred to as *stacked generalization*, where instead of using hard or soft voting for aggregating the results from multiple models, another model (referred to as a *blender* or *meta-learner*) is trained to perform the aggregation. \n",
    "\n",
    "In the example below, we train two base estimators: SVM classifier and Random Forest classifier. Afterward, we use Logistic Regression model as a final estimator, which is trained by using the predictions by the two base estimators as inputs. The final estimator aggregates the predictions of the individual models and outputs the final predictions. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-9 {color: black;}#sk-container-id-9 pre{padding: 0;}#sk-container-id-9 div.sk-toggleable {background-color: white;}#sk-container-id-9 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-9 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-9 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-9 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-9 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-9 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-9 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-9 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-9 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-9 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-9 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-9 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-9 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-9 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-9 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-9 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-9 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-9 div.sk-item {position: relative;z-index: 1;}#sk-container-id-9 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-9 div.sk-item::before, #sk-container-id-9 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-9 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-9 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-9 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-9 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-9 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-9 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-9 div.sk-label-container {text-align: center;}#sk-container-id-9 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-9 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-9\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>StackingClassifier(estimators=[(&#x27;rand&#x27;,\n",
       "                                RandomForestClassifier(random_state=42)),\n",
       "                               (&#x27;svc&#x27;, SVC(random_state=42))],\n",
       "                   final_estimator=LogisticRegression())</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-19\" type=\"checkbox\" ><label for=\"sk-estimator-id-19\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">StackingClassifier</label><div class=\"sk-toggleable__content\"><pre>StackingClassifier(estimators=[(&#x27;rand&#x27;,\n",
       "                                RandomForestClassifier(random_state=42)),\n",
       "                               (&#x27;svc&#x27;, SVC(random_state=42))],\n",
       "                   final_estimator=LogisticRegression())</pre></div></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><label>rand</label></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-20\" type=\"checkbox\" ><label for=\"sk-estimator-id-20\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">RandomForestClassifier</label><div class=\"sk-toggleable__content\"><pre>RandomForestClassifier(random_state=42)</pre></div></div></div></div></div></div><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><label>svc</label></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-21\" type=\"checkbox\" ><label for=\"sk-estimator-id-21\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">SVC</label><div class=\"sk-toggleable__content\"><pre>SVC(random_state=42)</pre></div></div></div></div></div></div></div></div><div class=\"sk-item\"><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><label>final_estimator</label></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-22\" type=\"checkbox\" ><label for=\"sk-estimator-id-22\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">LogisticRegression</label><div class=\"sk-toggleable__content\"><pre>LogisticRegression()</pre></div></div></div></div></div></div></div></div></div></div></div></div>"
      ],
      "text/plain": [
       "StackingClassifier(estimators=[('rand',\n",
       "                                RandomForestClassifier(random_state=42)),\n",
       "                               ('svc', SVC(random_state=42))],\n",
       "                   final_estimator=LogisticRegression())"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.ensemble import StackingClassifier\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "\n",
    "base_estimators = [\n",
    "    ('rand', RandomForestClassifier(random_state=42)),\n",
    "    ('svc', SVC(random_state=42))]\n",
    "\n",
    "final_estimator = LogisticRegression()\n",
    "\n",
    "stack_classifier = StackingClassifier(estimators = base_estimators, \n",
    "                               final_estimator = final_estimator)\n",
    "\n",
    "stack_classifier.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stacking Ensemble accuracy is 90.2189 %\n"
     ]
    }
   ],
   "source": [
    "# evaluate\n",
    "stack_preds = stack_classifier.predict(X_test)\n",
    "stack_acc = accuracy_score(y_test, stack_preds)\n",
    "print('Stacking Ensemble accuracy is {0:7.4f} %'.format(stack_acc*100))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In conclusion, Ensemble Methods are among the best-performing Machine Learning models. Whereas they may not be competitive with Neural Networks for datasets containing images, text, audio, or other data formats, for datasets with tabular data Ensemble Methods may outperform Neural Network-based approaches. In addition, Ensemble Methods can achieve good performance even without hyperparameter finetuning, in comparison to Neural Networks that typically require significant hyperparameter finetuning. As well as, Ensemble Methods can outperform Neural Networks for tasks with smaller dataset sizes, are robust to outliers, and can handle imbalanced datasets."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## References <a id=\"references\"/>\n",
    "\n",
    "1. Complete Machine Learning Package, Jean de Dieu Nyandwi, available at: [https://github.com/Nyandwi/machine_learning_complete](https://github.com/Nyandwi/machine_learning_complete).\n",
    "2. Hands-on Machine Learning with Scikit-learn, Keras & TensorFlow, Aurelien Geron, available at: [https://github.com/ageron/handson-ml/blob/master/07_ensemble_learning_and_random_forests.ipynb](https://github.com/ageron/handson-ml/blob/master/07_ensemble_learning_and_random_forests.ipynb). \n",
    "3. Python Machine Learning (2nd Ed.) Code Repository, Sebastian Raschka, available at: [https://github.com/rasbt/python-machine-learning-book-2nd-edition](https://github.com/rasbt/python-machine-learning-book-2nd-edition). \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[BACK TO TOP](#top)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}