Mostrar código
path <- 'https://raw.githubusercontent.com/ramIA-lab/MLforEducation/refs/heads/main/material/trees_ensambleMethods/idealista18_BCN_conRenta.csv'
BCN <- read.csv2(path)XGBoost
1 Descripción del problema
En este documento se estudia el algoritmo XGBoost aplicado a un problema de clasificación supervisada multiclase. El objetivo será predecir el nivel de renta del entorno de una vivienda de Barcelona (RENTA) a partir de características del inmueble, del edificio y de su localización.
La base de datos procede de anuncios de vivienda de Idealista y se ha enriquecido con información de renta media por hogar/persona a nivel de sección censal. A partir de la renta media por hogar se construye una variable categórica de tres niveles:
- Baja: renta media por hogar inferior a 30.000 €.
- Media: renta media por hogar entre 30.000 € y 50.000 €.
- Alta: renta media por hogar superior a 50.000 €.
Por tanto, el problema se puede formular como:
\[Y = f(X_1, X_2, \ldots, X_p) + \varepsilon\]
donde:
- \(Y\) es la variable objetivo
RENTA. - \(X_1, X_2, \ldots, X_p\) son las variables explicativas.
- \(f\) es una función no lineal aprendida mediante una suma secuencial de árboles.
XGBoost no construye un único árbol, ni muchos árboles independientes como Random Forest. XGBoost construye árboles de forma secuencial: cada nuevo árbol intenta corregir los errores cometidos por el conjunto de árboles anterior.
2 Introducción teórica a XGBoost
2.1 De árboles individuales a boosting
Un árbol de decisión aprende reglas del tipo:
Si
CDIS = Sarrià-Sant GervasiyDISTANCE_TO_CITY_CENTER < 3, entonces la renta probablemente esAlta.
Un único árbol suele ser fácil de interpretar, pero puede ser inestable. Random Forest reduce esa inestabilidad entrenando muchos árboles en paralelo y agregando sus votos. XGBoost usa otra lógica: entrena árboles pequeños de forma secuencial, donde cada árbol nuevo se concentra en los errores que todavía quedan por corregir.
La predicción final se expresa como una suma de árboles:
\[\hat{y}_i = \sum_{m=1}^{M} \eta f_m(x_i)\]
donde:
- \(M\) es el número de árboles.
- \(f_m\) es el árbol añadido en la iteración \(m\).
- \(\eta\) es la tasa de aprendizaje o
learning_rate. - \(x_i\) representa las variables explicativas del individuo \(i\).
La idea básica es:
- Se empieza con una predicción inicial.
- Se calcula el error del modelo.
- Se entrena un nuevo árbol para reducir ese error.
- Se añade el árbol al modelo con un peso pequeño.
- Se repite el proceso muchas veces.
2.2 Diferencia entre Random Forest y XGBoost
| Aspecto | Random Forest | XGBoost |
|---|---|---|
| Construcción de árboles | Paralela | Secuencial |
| Objetivo de cada árbol | Votar de forma independiente | Corregir errores anteriores |
| Tipo de ensamble | Bagging | Boosting |
| Control del sobreajuste | Promedio de muchos árboles | Regularización, shrinkage, profundidad, submuestreo |
| Interpretabilidad | Importancia de variables y árboles individuales | Importancia, árboles individuales y SHAP |
| Riesgo principal | Puede ser menos preciso si la señal es compleja | Puede sobreajustar si se ajusta mal |
2.3 Función objetivo de XGBoost
XGBoost optimiza una función objetivo formada por dos partes:
\[Obj = \sum_{i=1}^{n} L(y_i, \hat{y}_i) + \sum_{m=1}^{M} \Omega(f_m)\]
La primera parte mide el error de predicción:
\[ \sum_{i=1}^{n} L(y_i, \hat{y}_i) \]
La segunda parte penaliza la complejidad de los árboles:
\[ \sum_{m=1}^{M} \Omega(f_m) \]
Esto es muy importante porque XGBoost no solo intenta ajustar bien los datos, sino que también intenta evitar árboles excesivamente complejos.
Una forma habitual de escribir la penalización de complejidad es:
\[ \Omega(f) = \gamma T + \frac{1}{2}\lambda \sum_{j=1}^{T} w_j^2 \]
donde:
- \(T\) es el número de hojas del árbol.
- \(w_j\) es el peso asignado a la hoja \(j\).
- \(\gamma\) penaliza crear nuevas hojas.
- \(\lambda\) penaliza pesos demasiado grandes.
2.4 Parámetros principales
Los parámetros más importantes que se usarán durante el documento son:
nrounds/n_estimators: número de árboles.eta/learning_rate: peso de cada árbol nuevo.max_depth: profundidad máxima de cada árbol.subsample: proporción de observaciones usadas en cada iteración.colsample_bytree: proporción de variables usadas por árbol.lambda/reg_lambda: regularización L2.alpha/reg_alpha: regularización L1.objective: función de pérdida que se quiere optimizar.eval_metric: métrica usada para monitorizar el entrenamiento.
2.5 Ventajas principales
XGBoost es uno de los algoritmos más utilizados en problemas tabulares porque:
- captura no linealidades;
- captura interacciones entre variables;
- suele dar muy buen rendimiento predictivo;
- permite regularizar el modelo;
- permite hacer validación temprana mediante
early_stopping; - ofrece medidas de importancia de variables;
- permite extraer árboles concretos del modelo;
- se integra muy bien con SHAP para interpretación local y global.
2.6 Limitaciones
También tiene limitaciones:
- requiere más ajuste de hiperparámetros que un árbol simple;
- puede sobreajustar si se usan demasiados árboles o árboles muy profundos;
- necesita codificar variables categóricas si se usa una matriz numérica clásica;
- es menos interpretable que un árbol individual;
- su interpretación mediante importancia de variables puede depender de la métrica usada:
gain,cover,frequency, etc.
3 Carga de datos
El siguiente bloque carga la misma base de datos usada en los ejemplos de árboles y Random Forest.
Mostrar código
paquetes_necesarios <- c(
"dplyr", "tidyr", "ggplot2", "caret", "xgboost", "pdp"
)
paquetes_faltantes <- paquetes_necesarios[
!vapply(paquetes_necesarios, requireNamespace, logical(1), quietly = TRUE)
]
if (length(paquetes_faltantes) > 0) {
install.packages(paquetes_faltantes, repos = "https://cloud.r-project.org")
}
invisible(lapply(paquetes_necesarios, library, character.only = TRUE))Mostrar código
import importlib.util
import subprocess
import sys
paquetes_python = {
"pandas": "pandas",
"numpy": "numpy",
"sklearn": "scikit-learn",
"xgboost": "xgboost",
"matplotlib": "matplotlib",
"shap": "shap"
}
for modulo, paquete in paquetes_python.items():
if importlib.util.find_spec(modulo) is None:
subprocess.check_call([sys.executable, "-m", "pip", "install", paquete])4 Preprocesamiento de los datos
El preprocesamiento tiene cuatro objetivos:
- eliminar identificadores, geometrías y columnas que no se usarán como predictores;
- transformar variables binarias de 0/1 a etiquetas interpretables;
- crear la variable objetivo
RENTA; - preparar una matriz numérica, porque XGBoost trabaja de forma natural con matrices numéricas.
Mostrar código
columnas_a_eliminar <- c(
"X", "PRICE", "LONGITUDE", "LATITUDE", "geometry", "CONSTRUCTIONYEAR",
"ASSETID", "PERIOD", "CUSEC", "CSEC", "CMUN", "CPRO", "CCA", "CUDIS",
"CLAU2", "NPRO", "NCA", "CNUT0", "CNUT1", "CNUT2", "CNUT3", "NMUN",
"Shape_Leng", "Shape_Area", "CUMUN", "CADASTRALQUALITYID"
)
BCN <- BCN %>%
select(-any_of(columnas_a_eliminar)) %>%
mutate(
across(
.cols = matches("^(HAS|IS)"),
.fns = ~ case_when(
. == 0 ~ "No",
. == 1 ~ "Si",
TRUE ~ as.character(.)
),
.names = "{.col}"
),
AMENITYID = case_when(
AMENITYID == 1 ~ "SinMuebleSinCocina",
AMENITYID == 2 ~ "CocinaSinMuebles",
AMENITYID == 3 ~ "CocinaMuebles",
TRUE ~ "noInfo"
),
FLATLOCATIONID = case_when(
FLATLOCATIONID == 1 ~ "exterior",
FLATLOCATIONID == 2 ~ "interior",
TRUE ~ "noInfo"
),
BUILTTYPEID_1 = case_when(
BUILTTYPEID_1 == 0 ~ "noObraNueva",
BUILTTYPEID_1 == 1 ~ "obraNueva",
TRUE ~ "noInfo"
),
BUILTTYPEID_2 = case_when(
BUILTTYPEID_2 == 0 ~ "noRestaurar",
BUILTTYPEID_2 == 1 ~ "Restaurar",
TRUE ~ "noInfo"
),
BUILTTYPEID_3 = case_when(
BUILTTYPEID_3 == 0 ~ "noSegundaMano",
BUILTTYPEID_3 == 1 ~ "SegundaMano",
TRUE ~ "noInfo"
),
FLOORCLEAN = replace_na(FLOORCLEAN, 0),
CDIS = case_when(
CDIS == 1 ~ "Ciutat-Vella",
CDIS == 2 ~ "Eixample",
CDIS == 3 ~ "Sants-Montjuic",
CDIS == 4 ~ "Les Corts",
CDIS == 5 ~ "Sarrià-Sant Gervasi",
CDIS == 6 ~ "Gràcia",
CDIS == 7 ~ "Horta-Guinardó",
CDIS == 8 ~ "Nou Barris",
CDIS == 9 ~ "Sant Andreu",
CDIS == 10 ~ "Sant Martí",
TRUE ~ "noInfo"
),
RENTA = case_when(
Renta.media.por.hogar < 30000 ~ "Baja",
Renta.media.por.hogar >= 30000 & Renta.media.por.hogar <= 50000 ~ "Media",
Renta.media.por.hogar > 50000 ~ "Alta",
TRUE ~ NA_character_
)
) %>%
select(-any_of(c("Renta.media.por.hogar", "Renta.media.por.persona"))) %>%
mutate(RENTA = factor(RENTA, levels = c("Baja", "Media", "Alta"))) %>%
na.omit()5 Análisis descriptivo
Antes de entrenar un modelo de boosting conviene revisar la variable objetivo y algunas variables explicativas. En clasificación multiclase es especialmente importante mirar si las clases están equilibradas.
Mostrar código
dim(BCN)[1] 23334 36Mostrar código
str(BCN)'data.frame': 23334 obs. of 36 variables:
$ UNITPRICE : num 5232 4108 4056 6585 4200 ...
$ CONSTRUCTEDAREA : int 56 74 72 65 70 33 82 98 86 94 ...
$ ROOMNUMBER : int 1 2 3 1 3 1 2 3 3 3 ...
$ BATHNUMBER : int 1 1 1 2 1 1 2 1 1 1 ...
$ HASTERRACE : chr "No" "Si" "No" "Si" ...
$ HASLIFT : chr "Si" "Si" "Si" "No" ...
$ HASAIRCONDITIONING : chr "No" "No" "No" "Si" ...
$ AMENITYID : chr "CocinaSinMuebles" "CocinaMuebles" "CocinaMuebles" "CocinaMuebles" ...
$ HASPARKINGSPACE : chr "No" "No" "No" "No" ...
$ ISPARKINGSPACEINCLUDEDINPRICE: chr "No" "No" "No" "No" ...
$ PARKINGSPACEPRICE : num 1 1 1 1 1 1 1 1 1 1 ...
$ HASNORTHORIENTATION : chr "No" "No" "No" "No" ...
$ HASSOUTHORIENTATION : chr "No" "No" "No" "No" ...
$ HASEASTORIENTATION : chr "No" "No" "No" "No" ...
$ HASWESTORIENTATION : chr "No" "No" "No" "Si" ...
$ HASBOXROOM : chr "No" "No" "No" "No" ...
$ HASWARDROBE : chr "No" "No" "No" "No" ...
$ HASSWIMMINGPOOL : chr "No" "No" "No" "No" ...
$ HASDOORMAN : chr "No" "No" "No" "No" ...
$ HASGARDEN : chr "No" "No" "No" "No" ...
$ ISDUPLEX : chr "No" "No" "No" "No" ...
$ ISSTUDIO : chr "No" "No" "No" "No" ...
$ ISINTOPFLOOR : chr "No" "No" "No" "No" ...
$ FLOORCLEAN : int 0 4 3 2 0 0 0 6 4 4 ...
$ FLATLOCATIONID : chr "exterior" "noInfo" "exterior" "exterior" ...
$ CADCONSTRUCTIONYEAR : int 2018 1959 1959 1936 1950 1900 2018 1936 1993 1993 ...
$ CADMAXBUILDINGFLOOR : int 5 5 5 6 3 6 5 11 5 5 ...
$ CADDWELLINGCOUNT : int 11 11 11 14 3 7 11 45 44 44 ...
$ BUILTTYPEID_1 : chr "noObraNueva" "noObraNueva" "noObraNueva" "noObraNueva" ...
$ BUILTTYPEID_2 : chr "noRestaurar" "noRestaurar" "noRestaurar" "noRestaurar" ...
$ BUILTTYPEID_3 : chr "SegundaMano" "SegundaMano" "SegundaMano" "SegundaMano" ...
$ DISTANCE_TO_CITY_CENTER : num 1.7 1.75 1.75 1.68 1.66 ...
$ DISTANCE_TO_METRO : num 0.291 0.336 0.332 0.3 0.252 ...
$ DISTANCE_TO_DIAGONAL : num 2.31 2.38 2.44 2.5 2.41 ...
$ CDIS : chr "Ciutat-Vella" "Ciutat-Vella" "Ciutat-Vella" "Ciutat-Vella" ...
$ RENTA : Factor w/ 3 levels "Baja","Media",..: 1 1 1 1 1 1 1 1 1 1 ...Mostrar código
table(BCN$RENTA)
Baja Media Alta
7749 13177 2408 Mostrar código
prop.table(table(BCN$RENTA))
Baja Media Alta
0.3320905 0.5647124 0.1031971 Mostrar código
ggplot(BCN, aes(x = RENTA, fill = RENTA)) +
geom_bar(show.legend = FALSE) +
labs(
title = "Distribución de la variable objetivo",
x = "Nivel de renta",
y = "Número de observaciones"
) +
theme_minimal()
Mostrar código
ggplot(BCN, aes(x = RENTA, y = DISTANCE_TO_CITY_CENTER, fill = RENTA)) +
geom_boxplot(show.legend = FALSE, alpha = 0.8) +
labs(
title = "Distancia al centro según nivel de renta",
x = "Nivel de renta",
y = "Distancia al centro de la ciudad"
) +
theme_minimal()
6 Separación entrenamiento/test
Separamos los datos en entrenamiento y test de forma estratificada.
Mostrar código
library(dplyr)
set.seed(1994)
index <- caret::createDataPartition(BCN$RENTA, p = 0.8, list = FALSE)
rtrain <- BCN %>% dplyr::slice(index)
rtest <- BCN %>% dplyr::slice(-index)
prop.table(table(rtrain$RENTA))
Baja Media Alta
0.3321013 0.5646794 0.1032192 Mostrar código
prop.table(table(rtest$RENTA))
Baja Media Alta
0.3320472 0.5648446 0.1031083 Mostrar código
import pandas as pd
import numpy as np
pyBCN = pyBCN.dropna().copy()
y = pyBCN["RENTA"].astype(str)
X = pyBCN.drop(columns=["RENTA"])
X = pd.get_dummies(X, drop_first=False)
print(X.shape)(23334, 68)Mostrar código
print(y.value_counts(normalize=True))RENTA
Media 0.564712
Baja 0.332091
Alta 0.103197
Name: proportion, dtype: float64Mostrar código
from sklearn.model_selection import train_test_split
pyX_train, pyX_test, pyy_train, pyy_test = train_test_split(
X, y,
test_size=0.2,
random_state=1994,
stratify=y
)
print(pyX_train.shape, pyX_test.shape)(18667, 68) (4667, 68)7 Preparación específica para XGBoost
XGBoost necesita que la variable objetivo esté codificada como números. En clasificación multiclase, si hay tres clases, las etiquetas deben estar codificadas como 0, 1 y 2.
En R usaremos model.matrix() para convertir variables categóricas a variables dummy. En Python usaremos pd.get_dummies() y LabelEncoder.
Mostrar código
# dummyVars se ajusta SOLO con train y después se aplica a test.
# Así evitamos que train y test tengan columnas dummy diferentes.
dummy_model_r <- caret::dummyVars(RENTA ~ ., data = rtrain, fullRank = FALSE)
x_train <- predict(dummy_model_r, newdata = rtrain)
x_test <- predict(dummy_model_r, newdata = rtest)
x_train <- as.matrix(x_train)
x_test <- as.matrix(x_test)
y_train <- as.numeric(rtrain$RENTA) - 1
y_test <- as.numeric(rtest$RENTA) - 1
num_class <- length(levels(rtrain$RENTA))
clases <- levels(rtrain$RENTA)
dtrain <- xgb.DMatrix(data = x_train, label = y_train)
dtest <- xgb.DMatrix(data = x_test, label = y_test)
num_class[1] 3Mostrar código
head(y_train)[1] 0 0 0 0 0 0Mostrar código
from sklearn.preprocessing import LabelEncoder
le = LabelEncoder()
pyy_train_enc = le.fit_transform(pyy_train)
pyy_test_enc = le.transform(pyy_test)
print(dict(zip(le.classes_, range(len(le.classes_))))){'Alta': 0, 'Baja': 1, 'Media': 2}8 Entrenamiento de un primer modelo XGBoost
8.1 Modelo básico
Empezamos con un modelo razonable, sin una búsqueda exhaustiva de hiperparámetros. El objetivo de este primer modelo es entender el flujo completo: entrenar, predecir, evaluar e interpretar.
Mostrar código
set.seed(1994)
params_basicos <- list(
objective = "multi:softprob",
eval_metric = "mlogloss",
num_class = num_class,
eta = 0.05,
max_depth = 4,
subsample = 0.8,
colsample_bytree = 0.8,
lambda = 1,
alpha = 0
)
xgb_basico <- xgb.train(
params = params_basicos,
data = dtrain,
nrounds = 200,
watchlist = list(train = dtrain, test = dtest),
verbose = 0
)
xgb_basico##### xgb.Booster
raw: 904.9 Kb
call:
xgb.train(params = params_basicos, data = dtrain, nrounds = 200,
watchlist = list(train = dtrain, test = dtest), verbose = 0)
params (as set within xgb.train):
objective = "multi:softprob", eval_metric = "mlogloss", num_class = "3", eta = "0.05", max_depth = "4", subsample = "0.8", colsample_bytree = "0.8", lambda = "1", alpha = "0", validate_parameters = "TRUE"
xgb.attributes:
niter
callbacks:
cb.evaluation.log()
# of features: 68
niter: 200
nfeatures : 68
evaluation_log:
iter train_mlogloss test_mlogloss
<num> <num> <num>
1 1.0554322 1.0565829
2 1.0146820 1.0170784
--- --- ---
199 0.2526740 0.2932939
200 0.2521768 0.2927738Mostrar código
from xgboost import XGBClassifier
xgb_basico_py = XGBClassifier(
objective="multi:softprob",
num_class=len(le.classes_),
n_estimators=200,
learning_rate=0.1,
max_depth=4,
subsample=0.8,
colsample_bytree=0.8,
reg_lambda=1,
reg_alpha=0,
eval_metric="mlogloss",
random_state=1994,
n_jobs=-1
)
xgb_basico_py.fit(pyX_train, pyy_train_enc)XGBClassifier(base_score=None, booster=None, callbacks=None,
colsample_bylevel=None, colsample_bynode=None,
colsample_bytree=0.8, device=None, early_stopping_rounds=None,
enable_categorical=False, eval_metric='mlogloss',
feature_types=None, feature_weights=None, gamma=None,
grow_policy=None, importance_type=None,
interaction_constraints=None, learning_rate=0.1, max_bin=None,
max_cat_threshold=None, max_cat_to_onehot=None,
max_delta_step=None, max_depth=4, max_leaves=None,
min_child_weight=None, missing=nan, monotone_constraints=None,
multi_strategy=None, n_estimators=200, n_jobs=-1, num_class=3, ...)In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
Parameters
| objective | 'multi:softprob' | |
| base_score | None | |
| booster | None | |
| callbacks | None | |
| colsample_bylevel | None | |
| colsample_bynode | None | |
| colsample_bytree | 0.8 | |
| device | None | |
| early_stopping_rounds | None | |
| enable_categorical | False | |
| eval_metric | 'mlogloss' | |
| feature_types | None | |
| feature_weights | None | |
| gamma | None | |
| grow_policy | None | |
| importance_type | None | |
| interaction_constraints | None | |
| learning_rate | 0.1 | |
| max_bin | None | |
| max_cat_threshold | None | |
| max_cat_to_onehot | None | |
| max_delta_step | None | |
| max_depth | 4 | |
| max_leaves | None | |
| min_child_weight | None | |
| missing | nan | |
| monotone_constraints | None | |
| multi_strategy | None | |
| n_estimators | 200 | |
| n_jobs | -1 | |
| num_parallel_tree | None | |
| random_state | 1994 | |
| reg_alpha | 0 | |
| reg_lambda | 1 | |
| sampling_method | None | |
| scale_pos_weight | None | |
| subsample | 0.8 | |
| tree_method | None | |
| validate_parameters | None | |
| verbosity | None | |
| num_class | 3 |
9 Evaluación del modelo
En clasificación multiclase no basta con mirar la accuracy. Conviene analizar:
- matriz de confusión;
- accuracy global;
- sensibilidad/recall por clase;
- precisión por clase;
- F1-score por clase;
- log-loss multiclase;
- probabilidades predichas.
Mostrar código
pred_prob_r <- predict(xgb_basico, dtest)
pred_prob_r <- matrix(pred_prob_r, ncol = num_class, byrow = TRUE)
colnames(pred_prob_r) <- clases
pred_class_r <- max.col(pred_prob_r) - 1
pred_factor_r <- factor(clases[pred_class_r + 1], levels = clases)
real_factor_r <- factor(clases[y_test + 1], levels = clases)
caret::confusionMatrix(pred_factor_r, real_factor_r)Confusion Matrix and Statistics
Reference
Prediction Baja Media Alta
Baja 1317 125 3
Media 231 2451 128
Alta 1 59 350
Overall Statistics
Accuracy : 0.8827
95% CI : (0.8732, 0.8918)
No Information Rate : 0.5648
P-Value [Acc > NIR] : < 2.2e-16
Kappa : 0.786
Mcnemar's Test P-Value : 1.555e-12
Statistics by Class:
Class: Baja Class: Media Class: Alta
Sensitivity 0.8502 0.9302 0.72765
Specificity 0.9589 0.8232 0.98566
Pos Pred Value 0.9114 0.8722 0.85366
Neg Pred Value 0.9280 0.9008 0.96921
Prevalence 0.3320 0.5648 0.10311
Detection Rate 0.2823 0.5254 0.07503
Detection Prevalence 0.3098 0.6024 0.08789
Balanced Accuracy 0.9046 0.8767 0.85666Mostrar código
cm_r <- table(Real = real_factor_r, Predicho = pred_factor_r)
cm_r_df <- as.data.frame(cm_r)
ggplot(cm_r_df, aes(x = Predicho, y = Real, fill = Freq)) +
geom_tile() +
geom_text(aes(label = Freq), color = "white", size = 5) +
labs(
title = "Matriz de confusión - XGBoost en R",
x = "Clase predicha",
y = "Clase real"
) +
theme_minimal()
Mostrar código
from sklearn.metrics import confusion_matrix, classification_report, accuracy_score, log_loss
py_prob = xgb_basico_py.predict_proba(pyX_test)
py_pred_enc = np.argmax(py_prob, axis=1)
py_pred = le.inverse_transform(py_pred_enc)
print("Accuracy:", round(accuracy_score(pyy_test, py_pred), 4))Accuracy: 0.9087Mostrar código
print("Log-loss:", round(log_loss(pyy_test_enc, py_prob), 4))Log-loss: 0.2375Mostrar código
print(classification_report(pyy_test, py_pred)) precision recall f1-score support
Alta 0.89 0.82 0.85 482
Baja 0.93 0.88 0.90 1550
Media 0.90 0.94 0.92 2635
accuracy 0.91 4667
macro avg 0.91 0.88 0.89 4667
weighted avg 0.91 0.91 0.91 4667Mostrar código
import matplotlib.pyplot as plt
from sklearn.metrics import ConfusionMatrixDisplay
cm = confusion_matrix(pyy_test, py_pred, labels=le.classes_)
disp = ConfusionMatrixDisplay(confusion_matrix=cm, display_labels=le.classes_)
disp.plot(values_format="d")<sklearn.metrics._plot.confusion_matrix.ConfusionMatrixDisplay object at 0x0000014F2CE6F690>Mostrar código
plt.title("Matriz de confusión - XGBoost en Python")
plt.show()
10 Early stopping
Una de las ventajas prácticas de XGBoost es que permite usar early stopping. La idea es entrenar muchos árboles como máximo, pero detener el entrenamiento cuando el rendimiento en validación deja de mejorar.
Esto evita elegir manualmente el número exacto de árboles. Si el error de validación mejora hasta la ronda 87 y después empieza a empeorar, el modelo se queda con la mejor ronda encontrada.
Mostrar código
set.seed(1994)
xgb_es <- xgb.train(
params = params_basicos,
data = dtrain,
nrounds = 1000,
watchlist = list(train = dtrain, test = dtest),
early_stopping_rounds = 30,
verbose = 1
)[1] train-mlogloss:1.055432 test-mlogloss:1.056583
Multiple eval metrics are present. Will use test_mlogloss for early stopping.
Will train until test_mlogloss hasn't improved in 30 rounds.
[2] train-mlogloss:1.014682 test-mlogloss:1.017078
[3] train-mlogloss:0.976743 test-mlogloss:0.979810
[4] train-mlogloss:0.941990 test-mlogloss:0.946038
[5] train-mlogloss:0.910160 test-mlogloss:0.915022
[6] train-mlogloss:0.880563 test-mlogloss:0.886217
[7] train-mlogloss:0.851602 test-mlogloss:0.858091
[8] train-mlogloss:0.826451 test-mlogloss:0.833678
[9] train-mlogloss:0.801549 test-mlogloss:0.809306
[10] train-mlogloss:0.778893 test-mlogloss:0.787154
[11] train-mlogloss:0.758216 test-mlogloss:0.766987
[12] train-mlogloss:0.738237 test-mlogloss:0.747533
[13] train-mlogloss:0.719280 test-mlogloss:0.729176
[14] train-mlogloss:0.700783 test-mlogloss:0.711242
[15] train-mlogloss:0.684016 test-mlogloss:0.694966
[16] train-mlogloss:0.667793 test-mlogloss:0.679297
[17] train-mlogloss:0.652174 test-mlogloss:0.664246
[18] train-mlogloss:0.638070 test-mlogloss:0.650649
[19] train-mlogloss:0.624724 test-mlogloss:0.637806
[20] train-mlogloss:0.612131 test-mlogloss:0.625809
[21] train-mlogloss:0.599642 test-mlogloss:0.613706
[22] train-mlogloss:0.587706 test-mlogloss:0.602126
[23] train-mlogloss:0.576582 test-mlogloss:0.591380
[24] train-mlogloss:0.565705 test-mlogloss:0.580827
[25] train-mlogloss:0.555103 test-mlogloss:0.570428
[26] train-mlogloss:0.545979 test-mlogloss:0.561623
[27] train-mlogloss:0.536914 test-mlogloss:0.552748
[28] train-mlogloss:0.528362 test-mlogloss:0.544587
[29] train-mlogloss:0.520354 test-mlogloss:0.536926
[30] train-mlogloss:0.511999 test-mlogloss:0.528860
[31] train-mlogloss:0.504823 test-mlogloss:0.521885
[32] train-mlogloss:0.498054 test-mlogloss:0.515364
[33] train-mlogloss:0.491438 test-mlogloss:0.509036
[34] train-mlogloss:0.485009 test-mlogloss:0.502812
[35] train-mlogloss:0.478898 test-mlogloss:0.496985
[36] train-mlogloss:0.472883 test-mlogloss:0.491256
[37] train-mlogloss:0.466265 test-mlogloss:0.484946
[38] train-mlogloss:0.460602 test-mlogloss:0.479392
[39] train-mlogloss:0.454390 test-mlogloss:0.473518
[40] train-mlogloss:0.449079 test-mlogloss:0.468584
[41] train-mlogloss:0.444201 test-mlogloss:0.463963
[42] train-mlogloss:0.439196 test-mlogloss:0.459215
[43] train-mlogloss:0.434800 test-mlogloss:0.455021
[44] train-mlogloss:0.430548 test-mlogloss:0.450844
[45] train-mlogloss:0.426257 test-mlogloss:0.446745
[46] train-mlogloss:0.422009 test-mlogloss:0.442702
[47] train-mlogloss:0.418010 test-mlogloss:0.438919
[48] train-mlogloss:0.414049 test-mlogloss:0.435235
[49] train-mlogloss:0.410225 test-mlogloss:0.431632
[50] train-mlogloss:0.406541 test-mlogloss:0.428199
[51] train-mlogloss:0.402308 test-mlogloss:0.424312
[52] train-mlogloss:0.399123 test-mlogloss:0.421540
[53] train-mlogloss:0.395928 test-mlogloss:0.418636
[54] train-mlogloss:0.392638 test-mlogloss:0.415478
[55] train-mlogloss:0.389833 test-mlogloss:0.412910
[56] train-mlogloss:0.386864 test-mlogloss:0.410210
[57] train-mlogloss:0.384083 test-mlogloss:0.407631
[58] train-mlogloss:0.381170 test-mlogloss:0.405017
[59] train-mlogloss:0.378747 test-mlogloss:0.402750
[60] train-mlogloss:0.376255 test-mlogloss:0.400474
[61] train-mlogloss:0.373826 test-mlogloss:0.398313
[62] train-mlogloss:0.371628 test-mlogloss:0.396259
[63] train-mlogloss:0.369303 test-mlogloss:0.393928
[64] train-mlogloss:0.367092 test-mlogloss:0.391896
[65] train-mlogloss:0.364959 test-mlogloss:0.389937
[66] train-mlogloss:0.362846 test-mlogloss:0.387893
[67] train-mlogloss:0.361056 test-mlogloss:0.386274
[68] train-mlogloss:0.359242 test-mlogloss:0.384786
[69] train-mlogloss:0.357097 test-mlogloss:0.382721
[70] train-mlogloss:0.355144 test-mlogloss:0.380892
[71] train-mlogloss:0.353339 test-mlogloss:0.379276
[72] train-mlogloss:0.351658 test-mlogloss:0.377731
[73] train-mlogloss:0.349922 test-mlogloss:0.376126
[74] train-mlogloss:0.348337 test-mlogloss:0.374613
[75] train-mlogloss:0.346314 test-mlogloss:0.372823
[76] train-mlogloss:0.344277 test-mlogloss:0.370859
[77] train-mlogloss:0.342816 test-mlogloss:0.369497
[78] train-mlogloss:0.341391 test-mlogloss:0.368264
[79] train-mlogloss:0.339782 test-mlogloss:0.366890
[80] train-mlogloss:0.338481 test-mlogloss:0.365696
[81] train-mlogloss:0.336672 test-mlogloss:0.364067
[82] train-mlogloss:0.335133 test-mlogloss:0.362841
[83] train-mlogloss:0.333604 test-mlogloss:0.361472
[84] train-mlogloss:0.331815 test-mlogloss:0.359775
[85] train-mlogloss:0.330051 test-mlogloss:0.358136
[86] train-mlogloss:0.328444 test-mlogloss:0.356662
[87] train-mlogloss:0.327173 test-mlogloss:0.355458
[88] train-mlogloss:0.325598 test-mlogloss:0.354035
[89] train-mlogloss:0.324481 test-mlogloss:0.353085
[90] train-mlogloss:0.323213 test-mlogloss:0.351973
[91] train-mlogloss:0.321913 test-mlogloss:0.350834
[92] train-mlogloss:0.320492 test-mlogloss:0.349593
[93] train-mlogloss:0.319070 test-mlogloss:0.348480
[94] train-mlogloss:0.318047 test-mlogloss:0.347629
[95] train-mlogloss:0.317068 test-mlogloss:0.346749
[96] train-mlogloss:0.315749 test-mlogloss:0.345592
[97] train-mlogloss:0.314423 test-mlogloss:0.344379
[98] train-mlogloss:0.313311 test-mlogloss:0.343572
[99] train-mlogloss:0.312266 test-mlogloss:0.342706
[100] train-mlogloss:0.310882 test-mlogloss:0.341418
[101] train-mlogloss:0.309869 test-mlogloss:0.340577
[102] train-mlogloss:0.308812 test-mlogloss:0.339731
[103] train-mlogloss:0.307804 test-mlogloss:0.338832
[104] train-mlogloss:0.307025 test-mlogloss:0.338153
[105] train-mlogloss:0.305983 test-mlogloss:0.337385
[106] train-mlogloss:0.304959 test-mlogloss:0.336533
[107] train-mlogloss:0.304210 test-mlogloss:0.335913
[108] train-mlogloss:0.303429 test-mlogloss:0.335301
[109] train-mlogloss:0.302190 test-mlogloss:0.334172
[110] train-mlogloss:0.301063 test-mlogloss:0.333185
[111] train-mlogloss:0.300331 test-mlogloss:0.332557
[112] train-mlogloss:0.299474 test-mlogloss:0.331864
[113] train-mlogloss:0.298818 test-mlogloss:0.331302
[114] train-mlogloss:0.298029 test-mlogloss:0.330728
[115] train-mlogloss:0.296897 test-mlogloss:0.329688
[116] train-mlogloss:0.296344 test-mlogloss:0.329147
[117] train-mlogloss:0.295593 test-mlogloss:0.328570
[118] train-mlogloss:0.294869 test-mlogloss:0.328011
[119] train-mlogloss:0.294356 test-mlogloss:0.327570
[120] train-mlogloss:0.293622 test-mlogloss:0.326962
[121] train-mlogloss:0.292888 test-mlogloss:0.326382
[122] train-mlogloss:0.292316 test-mlogloss:0.325951
[123] train-mlogloss:0.291338 test-mlogloss:0.325141
[124] train-mlogloss:0.290625 test-mlogloss:0.324566
[125] train-mlogloss:0.289857 test-mlogloss:0.323869
[126] train-mlogloss:0.289370 test-mlogloss:0.323429
[127] train-mlogloss:0.288617 test-mlogloss:0.322810
[128] train-mlogloss:0.288011 test-mlogloss:0.322260
[129] train-mlogloss:0.287327 test-mlogloss:0.321757
[130] train-mlogloss:0.286544 test-mlogloss:0.321174
[131] train-mlogloss:0.285840 test-mlogloss:0.320561
[132] train-mlogloss:0.284993 test-mlogloss:0.319765
[133] train-mlogloss:0.284230 test-mlogloss:0.319226
[134] train-mlogloss:0.283531 test-mlogloss:0.318513
[135] train-mlogloss:0.282946 test-mlogloss:0.318015
[136] train-mlogloss:0.282502 test-mlogloss:0.317667
[137] train-mlogloss:0.282130 test-mlogloss:0.317351
[138] train-mlogloss:0.281317 test-mlogloss:0.316815
[139] train-mlogloss:0.280708 test-mlogloss:0.316353
[140] train-mlogloss:0.280214 test-mlogloss:0.315953
[141] train-mlogloss:0.279603 test-mlogloss:0.315364
[142] train-mlogloss:0.278910 test-mlogloss:0.314773
[143] train-mlogloss:0.278408 test-mlogloss:0.314405
[144] train-mlogloss:0.277952 test-mlogloss:0.313976
[145] train-mlogloss:0.277531 test-mlogloss:0.313637
[146] train-mlogloss:0.276947 test-mlogloss:0.313182
[147] train-mlogloss:0.276560 test-mlogloss:0.312885
[148] train-mlogloss:0.275961 test-mlogloss:0.312439
[149] train-mlogloss:0.275263 test-mlogloss:0.311846
[150] train-mlogloss:0.274700 test-mlogloss:0.311303
[151] train-mlogloss:0.274415 test-mlogloss:0.311132
[152] train-mlogloss:0.274069 test-mlogloss:0.310857
[153] train-mlogloss:0.273661 test-mlogloss:0.310547
[154] train-mlogloss:0.273125 test-mlogloss:0.310158
[155] train-mlogloss:0.272784 test-mlogloss:0.309924
[156] train-mlogloss:0.272147 test-mlogloss:0.309294
[157] train-mlogloss:0.271492 test-mlogloss:0.308670
[158] train-mlogloss:0.270949 test-mlogloss:0.308179
[159] train-mlogloss:0.270359 test-mlogloss:0.307789
[160] train-mlogloss:0.269979 test-mlogloss:0.307397
[161] train-mlogloss:0.269399 test-mlogloss:0.306954
[162] train-mlogloss:0.269124 test-mlogloss:0.306735
[163] train-mlogloss:0.268747 test-mlogloss:0.306451
[164] train-mlogloss:0.268308 test-mlogloss:0.306161
[165] train-mlogloss:0.267944 test-mlogloss:0.305869
[166] train-mlogloss:0.267206 test-mlogloss:0.305265
[167] train-mlogloss:0.266808 test-mlogloss:0.304879
[168] train-mlogloss:0.266243 test-mlogloss:0.304426
[169] train-mlogloss:0.265905 test-mlogloss:0.304132
[170] train-mlogloss:0.265644 test-mlogloss:0.303847
[171] train-mlogloss:0.265354 test-mlogloss:0.303600
[172] train-mlogloss:0.264921 test-mlogloss:0.303185
[173] train-mlogloss:0.264592 test-mlogloss:0.302928
[174] train-mlogloss:0.263959 test-mlogloss:0.302409
[175] train-mlogloss:0.263453 test-mlogloss:0.302153
[176] train-mlogloss:0.262986 test-mlogloss:0.301716
[177] train-mlogloss:0.262587 test-mlogloss:0.301421
[178] train-mlogloss:0.261990 test-mlogloss:0.300871
[179] train-mlogloss:0.261517 test-mlogloss:0.300483
[180] train-mlogloss:0.261175 test-mlogloss:0.300204
[181] train-mlogloss:0.260765 test-mlogloss:0.299884
[182] train-mlogloss:0.260301 test-mlogloss:0.299587
[183] train-mlogloss:0.259646 test-mlogloss:0.299198
[184] train-mlogloss:0.259285 test-mlogloss:0.298907
[185] train-mlogloss:0.259008 test-mlogloss:0.298692
[186] train-mlogloss:0.258419 test-mlogloss:0.298203
[187] train-mlogloss:0.257921 test-mlogloss:0.297834
[188] train-mlogloss:0.257431 test-mlogloss:0.297319
[189] train-mlogloss:0.256690 test-mlogloss:0.296617
[190] train-mlogloss:0.256407 test-mlogloss:0.296398
[191] train-mlogloss:0.256054 test-mlogloss:0.296120
[192] train-mlogloss:0.255658 test-mlogloss:0.295759
[193] train-mlogloss:0.255337 test-mlogloss:0.295447
[194] train-mlogloss:0.254707 test-mlogloss:0.294965
[195] train-mlogloss:0.254232 test-mlogloss:0.294509
[196] train-mlogloss:0.254028 test-mlogloss:0.294345
[197] train-mlogloss:0.253566 test-mlogloss:0.293986
[198] train-mlogloss:0.253055 test-mlogloss:0.293587
[199] train-mlogloss:0.252674 test-mlogloss:0.293294
[200] train-mlogloss:0.252177 test-mlogloss:0.292774
[201] train-mlogloss:0.251711 test-mlogloss:0.292430
[202] train-mlogloss:0.251286 test-mlogloss:0.292091
[203] train-mlogloss:0.250955 test-mlogloss:0.291883
[204] train-mlogloss:0.250711 test-mlogloss:0.291745
[205] train-mlogloss:0.250429 test-mlogloss:0.291483
[206] train-mlogloss:0.249645 test-mlogloss:0.290861
[207] train-mlogloss:0.249420 test-mlogloss:0.290683
[208] train-mlogloss:0.248854 test-mlogloss:0.290262
[209] train-mlogloss:0.248619 test-mlogloss:0.290104
[210] train-mlogloss:0.248021 test-mlogloss:0.289583
[211] train-mlogloss:0.247417 test-mlogloss:0.289226
[212] train-mlogloss:0.247145 test-mlogloss:0.288947
[213] train-mlogloss:0.246569 test-mlogloss:0.288533
[214] train-mlogloss:0.246288 test-mlogloss:0.288372
[215] train-mlogloss:0.245938 test-mlogloss:0.288161
[216] train-mlogloss:0.245568 test-mlogloss:0.287920
[217] train-mlogloss:0.245403 test-mlogloss:0.287774
[218] train-mlogloss:0.245075 test-mlogloss:0.287523
[219] train-mlogloss:0.244675 test-mlogloss:0.287246
[220] train-mlogloss:0.244337 test-mlogloss:0.287080
[221] train-mlogloss:0.244035 test-mlogloss:0.286906
[222] train-mlogloss:0.243551 test-mlogloss:0.286532
[223] train-mlogloss:0.242956 test-mlogloss:0.286105
[224] train-mlogloss:0.242575 test-mlogloss:0.285829
[225] train-mlogloss:0.242114 test-mlogloss:0.285451
[226] train-mlogloss:0.241800 test-mlogloss:0.285190
[227] train-mlogloss:0.241190 test-mlogloss:0.284725
[228] train-mlogloss:0.240802 test-mlogloss:0.284446
[229] train-mlogloss:0.240461 test-mlogloss:0.284068
[230] train-mlogloss:0.240149 test-mlogloss:0.283856
[231] train-mlogloss:0.239774 test-mlogloss:0.283529
[232] train-mlogloss:0.239393 test-mlogloss:0.283241
[233] train-mlogloss:0.238984 test-mlogloss:0.282968
[234] train-mlogloss:0.238647 test-mlogloss:0.282761
[235] train-mlogloss:0.238219 test-mlogloss:0.282528
[236] train-mlogloss:0.237651 test-mlogloss:0.282058
[237] train-mlogloss:0.237263 test-mlogloss:0.281790
[238] train-mlogloss:0.236829 test-mlogloss:0.281428
[239] train-mlogloss:0.236459 test-mlogloss:0.281096
[240] train-mlogloss:0.236106 test-mlogloss:0.280802
[241] train-mlogloss:0.235509 test-mlogloss:0.280239
[242] train-mlogloss:0.235109 test-mlogloss:0.279961
[243] train-mlogloss:0.234726 test-mlogloss:0.279655
[244] train-mlogloss:0.234436 test-mlogloss:0.279476
[245] train-mlogloss:0.234107 test-mlogloss:0.279301
[246] train-mlogloss:0.233686 test-mlogloss:0.279034
[247] train-mlogloss:0.233234 test-mlogloss:0.278701
[248] train-mlogloss:0.233026 test-mlogloss:0.278591
[249] train-mlogloss:0.232647 test-mlogloss:0.278287
[250] train-mlogloss:0.232298 test-mlogloss:0.278016
[251] train-mlogloss:0.231915 test-mlogloss:0.277695
[252] train-mlogloss:0.231730 test-mlogloss:0.277640
[253] train-mlogloss:0.231349 test-mlogloss:0.277350
[254] train-mlogloss:0.230971 test-mlogloss:0.277055
[255] train-mlogloss:0.230452 test-mlogloss:0.276658
[256] train-mlogloss:0.230193 test-mlogloss:0.276533
[257] train-mlogloss:0.229981 test-mlogloss:0.276420
[258] train-mlogloss:0.229505 test-mlogloss:0.275945
[259] train-mlogloss:0.229301 test-mlogloss:0.275835
[260] train-mlogloss:0.229058 test-mlogloss:0.275636
[261] train-mlogloss:0.228557 test-mlogloss:0.275205
[262] train-mlogloss:0.228295 test-mlogloss:0.275063
[263] train-mlogloss:0.227972 test-mlogloss:0.274832
[264] train-mlogloss:0.227550 test-mlogloss:0.274458
[265] train-mlogloss:0.227122 test-mlogloss:0.274016
[266] train-mlogloss:0.226566 test-mlogloss:0.273462
[267] train-mlogloss:0.226123 test-mlogloss:0.273111
[268] train-mlogloss:0.225748 test-mlogloss:0.272897
[269] train-mlogloss:0.225354 test-mlogloss:0.272629
[270] train-mlogloss:0.224928 test-mlogloss:0.272330
[271] train-mlogloss:0.224621 test-mlogloss:0.272083
[272] train-mlogloss:0.224376 test-mlogloss:0.271871
[273] train-mlogloss:0.223935 test-mlogloss:0.271520
[274] train-mlogloss:0.223443 test-mlogloss:0.271175
[275] train-mlogloss:0.222824 test-mlogloss:0.270639
[276] train-mlogloss:0.222402 test-mlogloss:0.270334
[277] train-mlogloss:0.221991 test-mlogloss:0.270072
[278] train-mlogloss:0.221660 test-mlogloss:0.269874
[279] train-mlogloss:0.221361 test-mlogloss:0.269718
[280] train-mlogloss:0.220914 test-mlogloss:0.269273
[281] train-mlogloss:0.220647 test-mlogloss:0.269168
[282] train-mlogloss:0.219973 test-mlogloss:0.268693
[283] train-mlogloss:0.219652 test-mlogloss:0.268540
[284] train-mlogloss:0.219358 test-mlogloss:0.268302
[285] train-mlogloss:0.219169 test-mlogloss:0.268265
[286] train-mlogloss:0.218848 test-mlogloss:0.267986
[287] train-mlogloss:0.218446 test-mlogloss:0.267789
[288] train-mlogloss:0.218141 test-mlogloss:0.267608
[289] train-mlogloss:0.217853 test-mlogloss:0.267440
[290] train-mlogloss:0.217488 test-mlogloss:0.267160
[291] train-mlogloss:0.217282 test-mlogloss:0.267065
[292] train-mlogloss:0.216990 test-mlogloss:0.266864
[293] train-mlogloss:0.216756 test-mlogloss:0.266687
[294] train-mlogloss:0.216440 test-mlogloss:0.266476
[295] train-mlogloss:0.216024 test-mlogloss:0.266196
[296] train-mlogloss:0.215758 test-mlogloss:0.266051
[297] train-mlogloss:0.215401 test-mlogloss:0.265833
[298] train-mlogloss:0.215194 test-mlogloss:0.265637
[299] train-mlogloss:0.214842 test-mlogloss:0.265442
[300] train-mlogloss:0.214591 test-mlogloss:0.265348
[301] train-mlogloss:0.214367 test-mlogloss:0.265214
[302] train-mlogloss:0.214063 test-mlogloss:0.265028
[303] train-mlogloss:0.213821 test-mlogloss:0.264845
[304] train-mlogloss:0.213594 test-mlogloss:0.264713
[305] train-mlogloss:0.213187 test-mlogloss:0.264382
[306] train-mlogloss:0.213027 test-mlogloss:0.264275
[307] train-mlogloss:0.212691 test-mlogloss:0.264029
[308] train-mlogloss:0.212273 test-mlogloss:0.263763
[309] train-mlogloss:0.211846 test-mlogloss:0.263415
[310] train-mlogloss:0.211668 test-mlogloss:0.263337
[311] train-mlogloss:0.211390 test-mlogloss:0.263181
[312] train-mlogloss:0.211120 test-mlogloss:0.262992
[313] train-mlogloss:0.210748 test-mlogloss:0.262814
[314] train-mlogloss:0.210310 test-mlogloss:0.262496
[315] train-mlogloss:0.210192 test-mlogloss:0.262438
[316] train-mlogloss:0.209995 test-mlogloss:0.262360
[317] train-mlogloss:0.209693 test-mlogloss:0.262170
[318] train-mlogloss:0.209443 test-mlogloss:0.262071
[319] train-mlogloss:0.209254 test-mlogloss:0.261976
[320] train-mlogloss:0.208816 test-mlogloss:0.261644
[321] train-mlogloss:0.208503 test-mlogloss:0.261483
[322] train-mlogloss:0.208398 test-mlogloss:0.261424
[323] train-mlogloss:0.208084 test-mlogloss:0.261251
[324] train-mlogloss:0.207717 test-mlogloss:0.260940
[325] train-mlogloss:0.207424 test-mlogloss:0.260796
[326] train-mlogloss:0.207201 test-mlogloss:0.260678
[327] train-mlogloss:0.207012 test-mlogloss:0.260527
[328] train-mlogloss:0.206755 test-mlogloss:0.260298
[329] train-mlogloss:0.206482 test-mlogloss:0.260118
[330] train-mlogloss:0.206252 test-mlogloss:0.259973
[331] train-mlogloss:0.205915 test-mlogloss:0.259743
[332] train-mlogloss:0.205515 test-mlogloss:0.259391
[333] train-mlogloss:0.204953 test-mlogloss:0.259019
[334] train-mlogloss:0.204717 test-mlogloss:0.258800
[335] train-mlogloss:0.204445 test-mlogloss:0.258677
[336] train-mlogloss:0.204297 test-mlogloss:0.258630
[337] train-mlogloss:0.204075 test-mlogloss:0.258490
[338] train-mlogloss:0.203855 test-mlogloss:0.258398
[339] train-mlogloss:0.203656 test-mlogloss:0.258254
[340] train-mlogloss:0.203424 test-mlogloss:0.258075
[341] train-mlogloss:0.203288 test-mlogloss:0.258012
[342] train-mlogloss:0.203014 test-mlogloss:0.257862
[343] train-mlogloss:0.202738 test-mlogloss:0.257735
[344] train-mlogloss:0.202351 test-mlogloss:0.257492
[345] train-mlogloss:0.202037 test-mlogloss:0.257295
[346] train-mlogloss:0.201946 test-mlogloss:0.257261
[347] train-mlogloss:0.201623 test-mlogloss:0.256981
[348] train-mlogloss:0.201423 test-mlogloss:0.256905
[349] train-mlogloss:0.201217 test-mlogloss:0.256725
[350] train-mlogloss:0.201094 test-mlogloss:0.256631
[351] train-mlogloss:0.200898 test-mlogloss:0.256539
[352] train-mlogloss:0.200664 test-mlogloss:0.256407
[353] train-mlogloss:0.200474 test-mlogloss:0.256296
[354] train-mlogloss:0.200282 test-mlogloss:0.256157
[355] train-mlogloss:0.199908 test-mlogloss:0.255891
[356] train-mlogloss:0.199578 test-mlogloss:0.255609
[357] train-mlogloss:0.199158 test-mlogloss:0.255304
[358] train-mlogloss:0.198944 test-mlogloss:0.255164
[359] train-mlogloss:0.198788 test-mlogloss:0.255071
[360] train-mlogloss:0.198416 test-mlogloss:0.254790
[361] train-mlogloss:0.198170 test-mlogloss:0.254610
[362] train-mlogloss:0.197856 test-mlogloss:0.254390
[363] train-mlogloss:0.197707 test-mlogloss:0.254283
[364] train-mlogloss:0.197473 test-mlogloss:0.254118
[365] train-mlogloss:0.197324 test-mlogloss:0.254087
[366] train-mlogloss:0.197058 test-mlogloss:0.253918
[367] train-mlogloss:0.196910 test-mlogloss:0.253803
[368] train-mlogloss:0.196816 test-mlogloss:0.253746
[369] train-mlogloss:0.196603 test-mlogloss:0.253606
[370] train-mlogloss:0.196293 test-mlogloss:0.253436
[371] train-mlogloss:0.196038 test-mlogloss:0.253248
[372] train-mlogloss:0.195868 test-mlogloss:0.253149
[373] train-mlogloss:0.195731 test-mlogloss:0.253071
[374] train-mlogloss:0.195584 test-mlogloss:0.252957
[375] train-mlogloss:0.195323 test-mlogloss:0.252685
[376] train-mlogloss:0.195155 test-mlogloss:0.252605
[377] train-mlogloss:0.195065 test-mlogloss:0.252531
[378] train-mlogloss:0.194899 test-mlogloss:0.252457
[379] train-mlogloss:0.194789 test-mlogloss:0.252366
[380] train-mlogloss:0.194558 test-mlogloss:0.252252
[381] train-mlogloss:0.194272 test-mlogloss:0.252012
[382] train-mlogloss:0.194029 test-mlogloss:0.251867
[383] train-mlogloss:0.193814 test-mlogloss:0.251806
[384] train-mlogloss:0.193558 test-mlogloss:0.251657
[385] train-mlogloss:0.193436 test-mlogloss:0.251589
[386] train-mlogloss:0.193291 test-mlogloss:0.251529
[387] train-mlogloss:0.192978 test-mlogloss:0.251200
[388] train-mlogloss:0.192768 test-mlogloss:0.251115
[389] train-mlogloss:0.192660 test-mlogloss:0.251052
[390] train-mlogloss:0.192407 test-mlogloss:0.250812
[391] train-mlogloss:0.192235 test-mlogloss:0.250752
[392] train-mlogloss:0.192013 test-mlogloss:0.250561
[393] train-mlogloss:0.191801 test-mlogloss:0.250453
[394] train-mlogloss:0.191605 test-mlogloss:0.250369
[395] train-mlogloss:0.191413 test-mlogloss:0.250288
[396] train-mlogloss:0.191241 test-mlogloss:0.250218
[397] train-mlogloss:0.191107 test-mlogloss:0.250138
[398] train-mlogloss:0.190757 test-mlogloss:0.249779
[399] train-mlogloss:0.190555 test-mlogloss:0.249628
[400] train-mlogloss:0.190341 test-mlogloss:0.249487
[401] train-mlogloss:0.190061 test-mlogloss:0.249230
[402] train-mlogloss:0.189856 test-mlogloss:0.249156
[403] train-mlogloss:0.189641 test-mlogloss:0.249034
[404] train-mlogloss:0.189458 test-mlogloss:0.248952
[405] train-mlogloss:0.189376 test-mlogloss:0.248893
[406] train-mlogloss:0.189227 test-mlogloss:0.248788
[407] train-mlogloss:0.189025 test-mlogloss:0.248659
[408] train-mlogloss:0.188874 test-mlogloss:0.248568
[409] train-mlogloss:0.188732 test-mlogloss:0.248501
[410] train-mlogloss:0.188481 test-mlogloss:0.248311
[411] train-mlogloss:0.188118 test-mlogloss:0.248046
[412] train-mlogloss:0.187642 test-mlogloss:0.247618
[413] train-mlogloss:0.187440 test-mlogloss:0.247525
[414] train-mlogloss:0.187337 test-mlogloss:0.247450
[415] train-mlogloss:0.187079 test-mlogloss:0.247247
[416] train-mlogloss:0.186667 test-mlogloss:0.246883
[417] train-mlogloss:0.186452 test-mlogloss:0.246734
[418] train-mlogloss:0.186226 test-mlogloss:0.246473
[419] train-mlogloss:0.185876 test-mlogloss:0.246226
[420] train-mlogloss:0.185620 test-mlogloss:0.246082
[421] train-mlogloss:0.185356 test-mlogloss:0.245937
[422] train-mlogloss:0.185134 test-mlogloss:0.245765
[423] train-mlogloss:0.184843 test-mlogloss:0.245665
[424] train-mlogloss:0.184619 test-mlogloss:0.245560
[425] train-mlogloss:0.184329 test-mlogloss:0.245390
[426] train-mlogloss:0.183990 test-mlogloss:0.245086
[427] train-mlogloss:0.183833 test-mlogloss:0.244956
[428] train-mlogloss:0.183678 test-mlogloss:0.244882
[429] train-mlogloss:0.183527 test-mlogloss:0.244769
[430] train-mlogloss:0.183337 test-mlogloss:0.244706
[431] train-mlogloss:0.183051 test-mlogloss:0.244497
[432] train-mlogloss:0.182907 test-mlogloss:0.244395
[433] train-mlogloss:0.182592 test-mlogloss:0.244157
[434] train-mlogloss:0.182410 test-mlogloss:0.244024
[435] train-mlogloss:0.182277 test-mlogloss:0.243981
[436] train-mlogloss:0.182130 test-mlogloss:0.243940
[437] train-mlogloss:0.181971 test-mlogloss:0.243882
[438] train-mlogloss:0.181810 test-mlogloss:0.243811
[439] train-mlogloss:0.181613 test-mlogloss:0.243780
[440] train-mlogloss:0.181304 test-mlogloss:0.243592
[441] train-mlogloss:0.181096 test-mlogloss:0.243478
[442] train-mlogloss:0.180901 test-mlogloss:0.243373
[443] train-mlogloss:0.180729 test-mlogloss:0.243304
[444] train-mlogloss:0.180491 test-mlogloss:0.243160
[445] train-mlogloss:0.180156 test-mlogloss:0.242871
[446] train-mlogloss:0.179979 test-mlogloss:0.242786
[447] train-mlogloss:0.179865 test-mlogloss:0.242742
[448] train-mlogloss:0.179704 test-mlogloss:0.242663
[449] train-mlogloss:0.179500 test-mlogloss:0.242486
[450] train-mlogloss:0.179267 test-mlogloss:0.242369
[451] train-mlogloss:0.178940 test-mlogloss:0.242069
[452] train-mlogloss:0.178846 test-mlogloss:0.242012
[453] train-mlogloss:0.178686 test-mlogloss:0.241910
[454] train-mlogloss:0.178408 test-mlogloss:0.241705
[455] train-mlogloss:0.178327 test-mlogloss:0.241682
[456] train-mlogloss:0.178051 test-mlogloss:0.241516
[457] train-mlogloss:0.177772 test-mlogloss:0.241316
[458] train-mlogloss:0.177469 test-mlogloss:0.241106
[459] train-mlogloss:0.177234 test-mlogloss:0.240890
[460] train-mlogloss:0.177034 test-mlogloss:0.240816
[461] train-mlogloss:0.176844 test-mlogloss:0.240737
[462] train-mlogloss:0.176646 test-mlogloss:0.240684
[463] train-mlogloss:0.176289 test-mlogloss:0.240435
[464] train-mlogloss:0.176056 test-mlogloss:0.240264
[465] train-mlogloss:0.175848 test-mlogloss:0.240192
[466] train-mlogloss:0.175566 test-mlogloss:0.239964
[467] train-mlogloss:0.175405 test-mlogloss:0.239874
[468] train-mlogloss:0.175235 test-mlogloss:0.239784
[469] train-mlogloss:0.174873 test-mlogloss:0.239552
[470] train-mlogloss:0.174757 test-mlogloss:0.239462
[471] train-mlogloss:0.174606 test-mlogloss:0.239404
[472] train-mlogloss:0.174340 test-mlogloss:0.239245
[473] train-mlogloss:0.174132 test-mlogloss:0.239093
[474] train-mlogloss:0.173994 test-mlogloss:0.239037
[475] train-mlogloss:0.173813 test-mlogloss:0.238944
[476] train-mlogloss:0.173564 test-mlogloss:0.238691
[477] train-mlogloss:0.173340 test-mlogloss:0.238532
[478] train-mlogloss:0.173023 test-mlogloss:0.238392
[479] train-mlogloss:0.172780 test-mlogloss:0.238279
[480] train-mlogloss:0.172618 test-mlogloss:0.238178
[481] train-mlogloss:0.172438 test-mlogloss:0.238050
[482] train-mlogloss:0.172181 test-mlogloss:0.237885
[483] train-mlogloss:0.172039 test-mlogloss:0.237867
[484] train-mlogloss:0.171761 test-mlogloss:0.237683
[485] train-mlogloss:0.171508 test-mlogloss:0.237582
[486] train-mlogloss:0.171202 test-mlogloss:0.237429
[487] train-mlogloss:0.170918 test-mlogloss:0.237263
[488] train-mlogloss:0.170751 test-mlogloss:0.237219
[489] train-mlogloss:0.170592 test-mlogloss:0.237165
[490] train-mlogloss:0.170425 test-mlogloss:0.237041
[491] train-mlogloss:0.170264 test-mlogloss:0.236886
[492] train-mlogloss:0.170124 test-mlogloss:0.236852
[493] train-mlogloss:0.169978 test-mlogloss:0.236781
[494] train-mlogloss:0.169861 test-mlogloss:0.236713
[495] train-mlogloss:0.169666 test-mlogloss:0.236612
[496] train-mlogloss:0.169493 test-mlogloss:0.236475
[497] train-mlogloss:0.169283 test-mlogloss:0.236306
[498] train-mlogloss:0.169093 test-mlogloss:0.236171
[499] train-mlogloss:0.168937 test-mlogloss:0.236049
[500] train-mlogloss:0.168849 test-mlogloss:0.236028
[501] train-mlogloss:0.168623 test-mlogloss:0.235906
[502] train-mlogloss:0.168489 test-mlogloss:0.235833
[503] train-mlogloss:0.168415 test-mlogloss:0.235801
[504] train-mlogloss:0.168353 test-mlogloss:0.235781
[505] train-mlogloss:0.168256 test-mlogloss:0.235746
[506] train-mlogloss:0.168059 test-mlogloss:0.235635
[507] train-mlogloss:0.167737 test-mlogloss:0.235408
[508] train-mlogloss:0.167543 test-mlogloss:0.235364
[509] train-mlogloss:0.167263 test-mlogloss:0.235183
[510] train-mlogloss:0.167133 test-mlogloss:0.235115
[511] train-mlogloss:0.166990 test-mlogloss:0.234971
[512] train-mlogloss:0.166805 test-mlogloss:0.234845
[513] train-mlogloss:0.166603 test-mlogloss:0.234674
[514] train-mlogloss:0.166427 test-mlogloss:0.234539
[515] train-mlogloss:0.166259 test-mlogloss:0.234459
[516] train-mlogloss:0.166053 test-mlogloss:0.234401
[517] train-mlogloss:0.165925 test-mlogloss:0.234333
[518] train-mlogloss:0.165752 test-mlogloss:0.234263
[519] train-mlogloss:0.165610 test-mlogloss:0.234178
[520] train-mlogloss:0.165320 test-mlogloss:0.233996
[521] train-mlogloss:0.165162 test-mlogloss:0.233926
[522] train-mlogloss:0.164910 test-mlogloss:0.233781
[523] train-mlogloss:0.164708 test-mlogloss:0.233674
[524] train-mlogloss:0.164548 test-mlogloss:0.233646
[525] train-mlogloss:0.164368 test-mlogloss:0.233510
[526] train-mlogloss:0.164265 test-mlogloss:0.233483
[527] train-mlogloss:0.164144 test-mlogloss:0.233453
[528] train-mlogloss:0.163878 test-mlogloss:0.233243
[529] train-mlogloss:0.163768 test-mlogloss:0.233193
[530] train-mlogloss:0.163509 test-mlogloss:0.233037
[531] train-mlogloss:0.163240 test-mlogloss:0.232797
[532] train-mlogloss:0.163089 test-mlogloss:0.232804
[533] train-mlogloss:0.162913 test-mlogloss:0.232735
[534] train-mlogloss:0.162695 test-mlogloss:0.232649
[535] train-mlogloss:0.162488 test-mlogloss:0.232487
[536] train-mlogloss:0.162314 test-mlogloss:0.232433
[537] train-mlogloss:0.162172 test-mlogloss:0.232367
[538] train-mlogloss:0.161986 test-mlogloss:0.232256
[539] train-mlogloss:0.161746 test-mlogloss:0.232121
[540] train-mlogloss:0.161566 test-mlogloss:0.231993
[541] train-mlogloss:0.161486 test-mlogloss:0.232000
[542] train-mlogloss:0.161352 test-mlogloss:0.231956
[543] train-mlogloss:0.161159 test-mlogloss:0.231914
[544] train-mlogloss:0.161004 test-mlogloss:0.231819
[545] train-mlogloss:0.160822 test-mlogloss:0.231714
[546] train-mlogloss:0.160686 test-mlogloss:0.231629
[547] train-mlogloss:0.160540 test-mlogloss:0.231591
[548] train-mlogloss:0.160310 test-mlogloss:0.231390
[549] train-mlogloss:0.160179 test-mlogloss:0.231332
[550] train-mlogloss:0.160049 test-mlogloss:0.231322
[551] train-mlogloss:0.159884 test-mlogloss:0.231181
[552] train-mlogloss:0.159761 test-mlogloss:0.231120
[553] train-mlogloss:0.159587 test-mlogloss:0.231041
[554] train-mlogloss:0.159417 test-mlogloss:0.230957
[555] train-mlogloss:0.159276 test-mlogloss:0.230848
[556] train-mlogloss:0.159136 test-mlogloss:0.230805
[557] train-mlogloss:0.159018 test-mlogloss:0.230736
[558] train-mlogloss:0.158759 test-mlogloss:0.230546
[559] train-mlogloss:0.158630 test-mlogloss:0.230493
[560] train-mlogloss:0.158486 test-mlogloss:0.230399
[561] train-mlogloss:0.158287 test-mlogloss:0.230262
[562] train-mlogloss:0.158104 test-mlogloss:0.230135
[563] train-mlogloss:0.157841 test-mlogloss:0.230014
[564] train-mlogloss:0.157569 test-mlogloss:0.229782
[565] train-mlogloss:0.157455 test-mlogloss:0.229723
[566] train-mlogloss:0.157327 test-mlogloss:0.229687
[567] train-mlogloss:0.157192 test-mlogloss:0.229693
[568] train-mlogloss:0.157068 test-mlogloss:0.229685
[569] train-mlogloss:0.156848 test-mlogloss:0.229518
[570] train-mlogloss:0.156696 test-mlogloss:0.229429
[571] train-mlogloss:0.156553 test-mlogloss:0.229325
[572] train-mlogloss:0.156338 test-mlogloss:0.229217
[573] train-mlogloss:0.156206 test-mlogloss:0.229092
[574] train-mlogloss:0.156070 test-mlogloss:0.228984
[575] train-mlogloss:0.155945 test-mlogloss:0.228992
[576] train-mlogloss:0.155852 test-mlogloss:0.228971
[577] train-mlogloss:0.155687 test-mlogloss:0.228874
[578] train-mlogloss:0.155496 test-mlogloss:0.228789
[579] train-mlogloss:0.155261 test-mlogloss:0.228554
[580] train-mlogloss:0.155050 test-mlogloss:0.228358
[581] train-mlogloss:0.154865 test-mlogloss:0.228203
[582] train-mlogloss:0.154733 test-mlogloss:0.228129
[583] train-mlogloss:0.154586 test-mlogloss:0.228075
[584] train-mlogloss:0.154368 test-mlogloss:0.227998
[585] train-mlogloss:0.154206 test-mlogloss:0.227931
[586] train-mlogloss:0.154086 test-mlogloss:0.227889
[587] train-mlogloss:0.153950 test-mlogloss:0.227861
[588] train-mlogloss:0.153815 test-mlogloss:0.227798
[589] train-mlogloss:0.153593 test-mlogloss:0.227643
[590] train-mlogloss:0.153381 test-mlogloss:0.227553
[591] train-mlogloss:0.153183 test-mlogloss:0.227459
[592] train-mlogloss:0.153036 test-mlogloss:0.227388
[593] train-mlogloss:0.152857 test-mlogloss:0.227336
[594] train-mlogloss:0.152718 test-mlogloss:0.227306
[595] train-mlogloss:0.152484 test-mlogloss:0.227135
[596] train-mlogloss:0.152256 test-mlogloss:0.226974
[597] train-mlogloss:0.152081 test-mlogloss:0.226890
[598] train-mlogloss:0.152020 test-mlogloss:0.226851
[599] train-mlogloss:0.151821 test-mlogloss:0.226730
[600] train-mlogloss:0.151563 test-mlogloss:0.226620
[601] train-mlogloss:0.151313 test-mlogloss:0.226410
[602] train-mlogloss:0.151155 test-mlogloss:0.226399
[603] train-mlogloss:0.151012 test-mlogloss:0.226367
[604] train-mlogloss:0.150922 test-mlogloss:0.226345
[605] train-mlogloss:0.150835 test-mlogloss:0.226329
[606] train-mlogloss:0.150713 test-mlogloss:0.226300
[607] train-mlogloss:0.150535 test-mlogloss:0.226210
[608] train-mlogloss:0.150348 test-mlogloss:0.226130
[609] train-mlogloss:0.150185 test-mlogloss:0.226034
[610] train-mlogloss:0.150025 test-mlogloss:0.225925
[611] train-mlogloss:0.149847 test-mlogloss:0.225760
[612] train-mlogloss:0.149714 test-mlogloss:0.225669
[613] train-mlogloss:0.149451 test-mlogloss:0.225465
[614] train-mlogloss:0.149269 test-mlogloss:0.225381
[615] train-mlogloss:0.149129 test-mlogloss:0.225363
[616] train-mlogloss:0.148921 test-mlogloss:0.225328
[617] train-mlogloss:0.148732 test-mlogloss:0.225244
[618] train-mlogloss:0.148616 test-mlogloss:0.225174
[619] train-mlogloss:0.148573 test-mlogloss:0.225146
[620] train-mlogloss:0.148465 test-mlogloss:0.225118
[621] train-mlogloss:0.148318 test-mlogloss:0.225027
[622] train-mlogloss:0.148122 test-mlogloss:0.224874
[623] train-mlogloss:0.147964 test-mlogloss:0.224818
[624] train-mlogloss:0.147771 test-mlogloss:0.224650
[625] train-mlogloss:0.147665 test-mlogloss:0.224599
[626] train-mlogloss:0.147563 test-mlogloss:0.224499
[627] train-mlogloss:0.147379 test-mlogloss:0.224376
[628] train-mlogloss:0.147204 test-mlogloss:0.224230
[629] train-mlogloss:0.147034 test-mlogloss:0.224134
[630] train-mlogloss:0.146911 test-mlogloss:0.224033
[631] train-mlogloss:0.146734 test-mlogloss:0.223926
[632] train-mlogloss:0.146648 test-mlogloss:0.223911
[633] train-mlogloss:0.146512 test-mlogloss:0.223883
[634] train-mlogloss:0.146459 test-mlogloss:0.223846
[635] train-mlogloss:0.146167 test-mlogloss:0.223586
[636] train-mlogloss:0.146032 test-mlogloss:0.223600
[637] train-mlogloss:0.145979 test-mlogloss:0.223588
[638] train-mlogloss:0.145843 test-mlogloss:0.223471
[639] train-mlogloss:0.145724 test-mlogloss:0.223421
[640] train-mlogloss:0.145538 test-mlogloss:0.223348
[641] train-mlogloss:0.145321 test-mlogloss:0.223197
[642] train-mlogloss:0.145104 test-mlogloss:0.223099
[643] train-mlogloss:0.144931 test-mlogloss:0.222999
[644] train-mlogloss:0.144803 test-mlogloss:0.222977
[645] train-mlogloss:0.144604 test-mlogloss:0.222867
[646] train-mlogloss:0.144522 test-mlogloss:0.222877
[647] train-mlogloss:0.144345 test-mlogloss:0.222744
[648] train-mlogloss:0.144197 test-mlogloss:0.222659
[649] train-mlogloss:0.144066 test-mlogloss:0.222586
[650] train-mlogloss:0.143952 test-mlogloss:0.222457
[651] train-mlogloss:0.143843 test-mlogloss:0.222430
[652] train-mlogloss:0.143686 test-mlogloss:0.222368
[653] train-mlogloss:0.143536 test-mlogloss:0.222324
[654] train-mlogloss:0.143374 test-mlogloss:0.222194
[655] train-mlogloss:0.143144 test-mlogloss:0.222169
[656] train-mlogloss:0.143005 test-mlogloss:0.222072
[657] train-mlogloss:0.142780 test-mlogloss:0.221936
[658] train-mlogloss:0.142632 test-mlogloss:0.221876
[659] train-mlogloss:0.142453 test-mlogloss:0.221756
[660] train-mlogloss:0.142273 test-mlogloss:0.221636
[661] train-mlogloss:0.142182 test-mlogloss:0.221640
[662] train-mlogloss:0.142001 test-mlogloss:0.221610
[663] train-mlogloss:0.141902 test-mlogloss:0.221564
[664] train-mlogloss:0.141746 test-mlogloss:0.221493
[665] train-mlogloss:0.141632 test-mlogloss:0.221399
[666] train-mlogloss:0.141546 test-mlogloss:0.221323
[667] train-mlogloss:0.141458 test-mlogloss:0.221283
[668] train-mlogloss:0.141305 test-mlogloss:0.221226
[669] train-mlogloss:0.141159 test-mlogloss:0.221189
[670] train-mlogloss:0.140981 test-mlogloss:0.221096
[671] train-mlogloss:0.140815 test-mlogloss:0.221056
[672] train-mlogloss:0.140697 test-mlogloss:0.221022
[673] train-mlogloss:0.140437 test-mlogloss:0.220780
[674] train-mlogloss:0.140306 test-mlogloss:0.220656
[675] train-mlogloss:0.140194 test-mlogloss:0.220612
[676] train-mlogloss:0.140062 test-mlogloss:0.220518
[677] train-mlogloss:0.139941 test-mlogloss:0.220437
[678] train-mlogloss:0.139868 test-mlogloss:0.220381
[679] train-mlogloss:0.139714 test-mlogloss:0.220256
[680] train-mlogloss:0.139625 test-mlogloss:0.220228
[681] train-mlogloss:0.139510 test-mlogloss:0.220188
[682] train-mlogloss:0.139424 test-mlogloss:0.220193
[683] train-mlogloss:0.139329 test-mlogloss:0.220167
[684] train-mlogloss:0.139181 test-mlogloss:0.220109
[685] train-mlogloss:0.139011 test-mlogloss:0.219996
[686] train-mlogloss:0.138824 test-mlogloss:0.219921
[687] train-mlogloss:0.138703 test-mlogloss:0.219895
[688] train-mlogloss:0.138535 test-mlogloss:0.219867
[689] train-mlogloss:0.138415 test-mlogloss:0.219777
[690] train-mlogloss:0.138261 test-mlogloss:0.219666
[691] train-mlogloss:0.138086 test-mlogloss:0.219529
[692] train-mlogloss:0.137912 test-mlogloss:0.219448
[693] train-mlogloss:0.137739 test-mlogloss:0.219333
[694] train-mlogloss:0.137606 test-mlogloss:0.219265
[695] train-mlogloss:0.137496 test-mlogloss:0.219187
[696] train-mlogloss:0.137339 test-mlogloss:0.219080
[697] train-mlogloss:0.137216 test-mlogloss:0.218992
[698] train-mlogloss:0.137063 test-mlogloss:0.218900
[699] train-mlogloss:0.136917 test-mlogloss:0.218810
[700] train-mlogloss:0.136825 test-mlogloss:0.218783
[701] train-mlogloss:0.136651 test-mlogloss:0.218562
[702] train-mlogloss:0.136490 test-mlogloss:0.218449
[703] train-mlogloss:0.136297 test-mlogloss:0.218261
[704] train-mlogloss:0.136156 test-mlogloss:0.218229
[705] train-mlogloss:0.136053 test-mlogloss:0.218218
[706] train-mlogloss:0.135937 test-mlogloss:0.218178
[707] train-mlogloss:0.135822 test-mlogloss:0.218093
[708] train-mlogloss:0.135609 test-mlogloss:0.217912
[709] train-mlogloss:0.135503 test-mlogloss:0.217831
[710] train-mlogloss:0.135386 test-mlogloss:0.217774
[711] train-mlogloss:0.135242 test-mlogloss:0.217705
[712] train-mlogloss:0.135090 test-mlogloss:0.217612
[713] train-mlogloss:0.135016 test-mlogloss:0.217558
[714] train-mlogloss:0.134927 test-mlogloss:0.217521
[715] train-mlogloss:0.134812 test-mlogloss:0.217437
[716] train-mlogloss:0.134688 test-mlogloss:0.217330
[717] train-mlogloss:0.134603 test-mlogloss:0.217263
[718] train-mlogloss:0.134505 test-mlogloss:0.217259
[719] train-mlogloss:0.134380 test-mlogloss:0.217222
[720] train-mlogloss:0.134243 test-mlogloss:0.217133
[721] train-mlogloss:0.134131 test-mlogloss:0.217109
[722] train-mlogloss:0.133973 test-mlogloss:0.216973
[723] train-mlogloss:0.133852 test-mlogloss:0.216907
[724] train-mlogloss:0.133647 test-mlogloss:0.216812
[725] train-mlogloss:0.133522 test-mlogloss:0.216713
[726] train-mlogloss:0.133380 test-mlogloss:0.216583
[727] train-mlogloss:0.133282 test-mlogloss:0.216541
[728] train-mlogloss:0.133151 test-mlogloss:0.216481
[729] train-mlogloss:0.133052 test-mlogloss:0.216450
[730] train-mlogloss:0.132928 test-mlogloss:0.216377
[731] train-mlogloss:0.132789 test-mlogloss:0.216257
[732] train-mlogloss:0.132710 test-mlogloss:0.216194
[733] train-mlogloss:0.132608 test-mlogloss:0.216125
[734] train-mlogloss:0.132450 test-mlogloss:0.216033
[735] train-mlogloss:0.132365 test-mlogloss:0.216043
[736] train-mlogloss:0.132262 test-mlogloss:0.216001
[737] train-mlogloss:0.132175 test-mlogloss:0.215990
[738] train-mlogloss:0.131989 test-mlogloss:0.215833
[739] train-mlogloss:0.131803 test-mlogloss:0.215782
[740] train-mlogloss:0.131730 test-mlogloss:0.215743
[741] train-mlogloss:0.131586 test-mlogloss:0.215629
[742] train-mlogloss:0.131397 test-mlogloss:0.215492
[743] train-mlogloss:0.131200 test-mlogloss:0.215339
[744] train-mlogloss:0.131019 test-mlogloss:0.215197
[745] train-mlogloss:0.130831 test-mlogloss:0.215102
[746] train-mlogloss:0.130715 test-mlogloss:0.215043
[747] train-mlogloss:0.130594 test-mlogloss:0.215072
[748] train-mlogloss:0.130444 test-mlogloss:0.214980
[749] train-mlogloss:0.130306 test-mlogloss:0.214943
[750] train-mlogloss:0.130218 test-mlogloss:0.214932
[751] train-mlogloss:0.130099 test-mlogloss:0.214900
[752] train-mlogloss:0.129940 test-mlogloss:0.214815
[753] train-mlogloss:0.129796 test-mlogloss:0.214690
[754] train-mlogloss:0.129642 test-mlogloss:0.214580
[755] train-mlogloss:0.129579 test-mlogloss:0.214524
[756] train-mlogloss:0.129500 test-mlogloss:0.214507
[757] train-mlogloss:0.129392 test-mlogloss:0.214439
[758] train-mlogloss:0.129271 test-mlogloss:0.214397
[759] train-mlogloss:0.129137 test-mlogloss:0.214288
[760] train-mlogloss:0.129058 test-mlogloss:0.214273
[761] train-mlogloss:0.128986 test-mlogloss:0.214271
[762] train-mlogloss:0.128835 test-mlogloss:0.214119
[763] train-mlogloss:0.128740 test-mlogloss:0.214073
[764] train-mlogloss:0.128637 test-mlogloss:0.214075
[765] train-mlogloss:0.128471 test-mlogloss:0.213974
[766] train-mlogloss:0.128330 test-mlogloss:0.213890
[767] train-mlogloss:0.128187 test-mlogloss:0.213842
[768] train-mlogloss:0.128012 test-mlogloss:0.213722
[769] train-mlogloss:0.127940 test-mlogloss:0.213685
[770] train-mlogloss:0.127848 test-mlogloss:0.213651
[771] train-mlogloss:0.127664 test-mlogloss:0.213481
[772] train-mlogloss:0.127557 test-mlogloss:0.213439
[773] train-mlogloss:0.127464 test-mlogloss:0.213381
[774] train-mlogloss:0.127361 test-mlogloss:0.213313
[775] train-mlogloss:0.127217 test-mlogloss:0.213173
[776] train-mlogloss:0.127077 test-mlogloss:0.213118
[777] train-mlogloss:0.126997 test-mlogloss:0.213096
[778] train-mlogloss:0.126872 test-mlogloss:0.213028
[779] train-mlogloss:0.126759 test-mlogloss:0.213069
[780] train-mlogloss:0.126638 test-mlogloss:0.212960
[781] train-mlogloss:0.126524 test-mlogloss:0.212910
[782] train-mlogloss:0.126414 test-mlogloss:0.212858
[783] train-mlogloss:0.126290 test-mlogloss:0.212807
[784] train-mlogloss:0.126164 test-mlogloss:0.212707
[785] train-mlogloss:0.126005 test-mlogloss:0.212573
[786] train-mlogloss:0.125941 test-mlogloss:0.212544
[787] train-mlogloss:0.125863 test-mlogloss:0.212518
[788] train-mlogloss:0.125733 test-mlogloss:0.212465
[789] train-mlogloss:0.125634 test-mlogloss:0.212418
[790] train-mlogloss:0.125532 test-mlogloss:0.212370
[791] train-mlogloss:0.125431 test-mlogloss:0.212329
[792] train-mlogloss:0.125363 test-mlogloss:0.212307
[793] train-mlogloss:0.125279 test-mlogloss:0.212282
[794] train-mlogloss:0.125095 test-mlogloss:0.212196
[795] train-mlogloss:0.124965 test-mlogloss:0.212169
[796] train-mlogloss:0.124817 test-mlogloss:0.212061
[797] train-mlogloss:0.124714 test-mlogloss:0.212012
[798] train-mlogloss:0.124616 test-mlogloss:0.211931
[799] train-mlogloss:0.124479 test-mlogloss:0.211867
[800] train-mlogloss:0.124376 test-mlogloss:0.211810
[801] train-mlogloss:0.124258 test-mlogloss:0.211800
[802] train-mlogloss:0.124126 test-mlogloss:0.211767
[803] train-mlogloss:0.123968 test-mlogloss:0.211643
[804] train-mlogloss:0.123862 test-mlogloss:0.211565
[805] train-mlogloss:0.123729 test-mlogloss:0.211471
[806] train-mlogloss:0.123604 test-mlogloss:0.211438
[807] train-mlogloss:0.123506 test-mlogloss:0.211442
[808] train-mlogloss:0.123370 test-mlogloss:0.211380
[809] train-mlogloss:0.123230 test-mlogloss:0.211241
[810] train-mlogloss:0.123094 test-mlogloss:0.211116
[811] train-mlogloss:0.122894 test-mlogloss:0.210970
[812] train-mlogloss:0.122818 test-mlogloss:0.210946
[813] train-mlogloss:0.122605 test-mlogloss:0.210793
[814] train-mlogloss:0.122503 test-mlogloss:0.210777
[815] train-mlogloss:0.122334 test-mlogloss:0.210607
[816] train-mlogloss:0.122158 test-mlogloss:0.210484
[817] train-mlogloss:0.122053 test-mlogloss:0.210433
[818] train-mlogloss:0.121961 test-mlogloss:0.210369
[819] train-mlogloss:0.121772 test-mlogloss:0.210246
[820] train-mlogloss:0.121600 test-mlogloss:0.210164
[821] train-mlogloss:0.121467 test-mlogloss:0.210099
[822] train-mlogloss:0.121321 test-mlogloss:0.210056
[823] train-mlogloss:0.121251 test-mlogloss:0.210043
[824] train-mlogloss:0.121161 test-mlogloss:0.209967
[825] train-mlogloss:0.121065 test-mlogloss:0.209902
[826] train-mlogloss:0.120979 test-mlogloss:0.209870
[827] train-mlogloss:0.120909 test-mlogloss:0.209854
[828] train-mlogloss:0.120816 test-mlogloss:0.209792
[829] train-mlogloss:0.120715 test-mlogloss:0.209694
[830] train-mlogloss:0.120605 test-mlogloss:0.209621
[831] train-mlogloss:0.120467 test-mlogloss:0.209539
[832] train-mlogloss:0.120404 test-mlogloss:0.209535
[833] train-mlogloss:0.120216 test-mlogloss:0.209415
[834] train-mlogloss:0.120088 test-mlogloss:0.209373
[835] train-mlogloss:0.120008 test-mlogloss:0.209308
[836] train-mlogloss:0.119893 test-mlogloss:0.209273
[837] train-mlogloss:0.119815 test-mlogloss:0.209225
[838] train-mlogloss:0.119711 test-mlogloss:0.209179
[839] train-mlogloss:0.119588 test-mlogloss:0.209140
[840] train-mlogloss:0.119456 test-mlogloss:0.209080
[841] train-mlogloss:0.119331 test-mlogloss:0.209077
[842] train-mlogloss:0.119230 test-mlogloss:0.209087
[843] train-mlogloss:0.119126 test-mlogloss:0.209031
[844] train-mlogloss:0.119018 test-mlogloss:0.208961
[845] train-mlogloss:0.118867 test-mlogloss:0.208879
[846] train-mlogloss:0.118756 test-mlogloss:0.208876
[847] train-mlogloss:0.118659 test-mlogloss:0.208840
[848] train-mlogloss:0.118497 test-mlogloss:0.208721
[849] train-mlogloss:0.118340 test-mlogloss:0.208645
[850] train-mlogloss:0.118260 test-mlogloss:0.208638
[851] train-mlogloss:0.118170 test-mlogloss:0.208632
[852] train-mlogloss:0.118076 test-mlogloss:0.208591
[853] train-mlogloss:0.117838 test-mlogloss:0.208390
[854] train-mlogloss:0.117731 test-mlogloss:0.208388
[855] train-mlogloss:0.117610 test-mlogloss:0.208304
[856] train-mlogloss:0.117488 test-mlogloss:0.208246
[857] train-mlogloss:0.117413 test-mlogloss:0.208245
[858] train-mlogloss:0.117302 test-mlogloss:0.208223
[859] train-mlogloss:0.117188 test-mlogloss:0.208165
[860] train-mlogloss:0.117070 test-mlogloss:0.208117
[861] train-mlogloss:0.116981 test-mlogloss:0.208100
[862] train-mlogloss:0.116846 test-mlogloss:0.208026
[863] train-mlogloss:0.116760 test-mlogloss:0.208018
[864] train-mlogloss:0.116686 test-mlogloss:0.208041
[865] train-mlogloss:0.116496 test-mlogloss:0.207901
[866] train-mlogloss:0.116390 test-mlogloss:0.207873
[867] train-mlogloss:0.116337 test-mlogloss:0.207812
[868] train-mlogloss:0.116216 test-mlogloss:0.207773
[869] train-mlogloss:0.116052 test-mlogloss:0.207654
[870] train-mlogloss:0.115963 test-mlogloss:0.207629
[871] train-mlogloss:0.115860 test-mlogloss:0.207634
[872] train-mlogloss:0.115743 test-mlogloss:0.207546
[873] train-mlogloss:0.115664 test-mlogloss:0.207539
[874] train-mlogloss:0.115576 test-mlogloss:0.207522
[875] train-mlogloss:0.115480 test-mlogloss:0.207465
[876] train-mlogloss:0.115416 test-mlogloss:0.207460
[877] train-mlogloss:0.115314 test-mlogloss:0.207445
[878] train-mlogloss:0.115203 test-mlogloss:0.207434
[879] train-mlogloss:0.115040 test-mlogloss:0.207351
[880] train-mlogloss:0.114913 test-mlogloss:0.207251
[881] train-mlogloss:0.114839 test-mlogloss:0.207250
[882] train-mlogloss:0.114772 test-mlogloss:0.207252
[883] train-mlogloss:0.114705 test-mlogloss:0.207233
[884] train-mlogloss:0.114518 test-mlogloss:0.207114
[885] train-mlogloss:0.114380 test-mlogloss:0.206973
[886] train-mlogloss:0.114307 test-mlogloss:0.206951
[887] train-mlogloss:0.114235 test-mlogloss:0.206931
[888] train-mlogloss:0.114130 test-mlogloss:0.206951
[889] train-mlogloss:0.114052 test-mlogloss:0.206859
[890] train-mlogloss:0.113949 test-mlogloss:0.206800
[891] train-mlogloss:0.113845 test-mlogloss:0.206750
[892] train-mlogloss:0.113761 test-mlogloss:0.206703
[893] train-mlogloss:0.113672 test-mlogloss:0.206623
[894] train-mlogloss:0.113580 test-mlogloss:0.206584
[895] train-mlogloss:0.113493 test-mlogloss:0.206606
[896] train-mlogloss:0.113378 test-mlogloss:0.206549
[897] train-mlogloss:0.113289 test-mlogloss:0.206493
[898] train-mlogloss:0.113151 test-mlogloss:0.206401
[899] train-mlogloss:0.113049 test-mlogloss:0.206381
[900] train-mlogloss:0.112917 test-mlogloss:0.206232
[901] train-mlogloss:0.112833 test-mlogloss:0.206198
[902] train-mlogloss:0.112729 test-mlogloss:0.206138
[903] train-mlogloss:0.112634 test-mlogloss:0.206104
[904] train-mlogloss:0.112490 test-mlogloss:0.205976
[905] train-mlogloss:0.112357 test-mlogloss:0.205918
[906] train-mlogloss:0.112231 test-mlogloss:0.205838
[907] train-mlogloss:0.112083 test-mlogloss:0.205721
[908] train-mlogloss:0.111970 test-mlogloss:0.205629
[909] train-mlogloss:0.111769 test-mlogloss:0.205450
[910] train-mlogloss:0.111654 test-mlogloss:0.205403
[911] train-mlogloss:0.111597 test-mlogloss:0.205365
[912] train-mlogloss:0.111477 test-mlogloss:0.205354
[913] train-mlogloss:0.111384 test-mlogloss:0.205300
[914] train-mlogloss:0.111282 test-mlogloss:0.205224
[915] train-mlogloss:0.111192 test-mlogloss:0.205281
[916] train-mlogloss:0.111097 test-mlogloss:0.205239
[917] train-mlogloss:0.110966 test-mlogloss:0.205209
[918] train-mlogloss:0.110847 test-mlogloss:0.205137
[919] train-mlogloss:0.110760 test-mlogloss:0.205117
[920] train-mlogloss:0.110679 test-mlogloss:0.205101
[921] train-mlogloss:0.110596 test-mlogloss:0.205086
[922] train-mlogloss:0.110496 test-mlogloss:0.205049
[923] train-mlogloss:0.110389 test-mlogloss:0.205047
[924] train-mlogloss:0.110318 test-mlogloss:0.205025
[925] train-mlogloss:0.110211 test-mlogloss:0.204939
[926] train-mlogloss:0.110125 test-mlogloss:0.204938
[927] train-mlogloss:0.110060 test-mlogloss:0.204919
[928] train-mlogloss:0.109935 test-mlogloss:0.204853
[929] train-mlogloss:0.109851 test-mlogloss:0.204866
[930] train-mlogloss:0.109771 test-mlogloss:0.204826
[931] train-mlogloss:0.109663 test-mlogloss:0.204766
[932] train-mlogloss:0.109545 test-mlogloss:0.204691
[933] train-mlogloss:0.109473 test-mlogloss:0.204687
[934] train-mlogloss:0.109378 test-mlogloss:0.204605
[935] train-mlogloss:0.109276 test-mlogloss:0.204524
[936] train-mlogloss:0.109124 test-mlogloss:0.204430
[937] train-mlogloss:0.108978 test-mlogloss:0.204339
[938] train-mlogloss:0.108860 test-mlogloss:0.204313
[939] train-mlogloss:0.108697 test-mlogloss:0.204190
[940] train-mlogloss:0.108608 test-mlogloss:0.204149
[941] train-mlogloss:0.108511 test-mlogloss:0.204118
[942] train-mlogloss:0.108423 test-mlogloss:0.204119
[943] train-mlogloss:0.108235 test-mlogloss:0.203980
[944] train-mlogloss:0.108097 test-mlogloss:0.203857
[945] train-mlogloss:0.108027 test-mlogloss:0.203856
[946] train-mlogloss:0.107930 test-mlogloss:0.203784
[947] train-mlogloss:0.107863 test-mlogloss:0.203787
[948] train-mlogloss:0.107793 test-mlogloss:0.203790
[949] train-mlogloss:0.107736 test-mlogloss:0.203772
[950] train-mlogloss:0.107603 test-mlogloss:0.203659
[951] train-mlogloss:0.107487 test-mlogloss:0.203573
[952] train-mlogloss:0.107432 test-mlogloss:0.203545
[953] train-mlogloss:0.107369 test-mlogloss:0.203509
[954] train-mlogloss:0.107231 test-mlogloss:0.203488
[955] train-mlogloss:0.107138 test-mlogloss:0.203488
[956] train-mlogloss:0.107065 test-mlogloss:0.203461
[957] train-mlogloss:0.106961 test-mlogloss:0.203441
[958] train-mlogloss:0.106898 test-mlogloss:0.203405
[959] train-mlogloss:0.106803 test-mlogloss:0.203359
[960] train-mlogloss:0.106696 test-mlogloss:0.203332
[961] train-mlogloss:0.106599 test-mlogloss:0.203314
[962] train-mlogloss:0.106489 test-mlogloss:0.203265
[963] train-mlogloss:0.106402 test-mlogloss:0.203265
[964] train-mlogloss:0.106280 test-mlogloss:0.203188
[965] train-mlogloss:0.106200 test-mlogloss:0.203152
[966] train-mlogloss:0.106082 test-mlogloss:0.203076
[967] train-mlogloss:0.105972 test-mlogloss:0.203025
[968] train-mlogloss:0.105893 test-mlogloss:0.202986
[969] train-mlogloss:0.105815 test-mlogloss:0.202959
[970] train-mlogloss:0.105734 test-mlogloss:0.202962
[971] train-mlogloss:0.105674 test-mlogloss:0.202907
[972] train-mlogloss:0.105575 test-mlogloss:0.202839
[973] train-mlogloss:0.105500 test-mlogloss:0.202804
[974] train-mlogloss:0.105405 test-mlogloss:0.202752
[975] train-mlogloss:0.105268 test-mlogloss:0.202724
[976] train-mlogloss:0.105144 test-mlogloss:0.202657
[977] train-mlogloss:0.105065 test-mlogloss:0.202645
[978] train-mlogloss:0.104974 test-mlogloss:0.202614
[979] train-mlogloss:0.104915 test-mlogloss:0.202569
[980] train-mlogloss:0.104802 test-mlogloss:0.202517
[981] train-mlogloss:0.104710 test-mlogloss:0.202495
[982] train-mlogloss:0.104596 test-mlogloss:0.202486
[983] train-mlogloss:0.104538 test-mlogloss:0.202484
[984] train-mlogloss:0.104500 test-mlogloss:0.202501
[985] train-mlogloss:0.104427 test-mlogloss:0.202483
[986] train-mlogloss:0.104341 test-mlogloss:0.202412
[987] train-mlogloss:0.104156 test-mlogloss:0.202302
[988] train-mlogloss:0.104087 test-mlogloss:0.202311
[989] train-mlogloss:0.104004 test-mlogloss:0.202236
[990] train-mlogloss:0.103910 test-mlogloss:0.202167
[991] train-mlogloss:0.103829 test-mlogloss:0.202139
[992] train-mlogloss:0.103699 test-mlogloss:0.202038
[993] train-mlogloss:0.103561 test-mlogloss:0.201939
[994] train-mlogloss:0.103470 test-mlogloss:0.201929
[995] train-mlogloss:0.103417 test-mlogloss:0.201891
[996] train-mlogloss:0.103286 test-mlogloss:0.201758
[997] train-mlogloss:0.103169 test-mlogloss:0.201640
[998] train-mlogloss:0.103091 test-mlogloss:0.201617
[999] train-mlogloss:0.103021 test-mlogloss:0.201585
[1000] train-mlogloss:0.102944 test-mlogloss:0.201534 Mostrar código
best_iteration <- xgb_es$best_iteration
best_score <- xgb_es$best_score
cat("Mejor iteración:", best_iteration, "\n")Mejor iteración: 1000 Mostrar código
cat("Mejor score:", best_score, "\n")Mejor score: 0.2015338 Mostrar código
eval_log <- xgb_es$evaluation_log
ggplot(eval_log, aes(x = iter)) +
geom_line(aes(y = train_mlogloss, color = "Train")) +
geom_line(aes(y = test_mlogloss, color = "Test")) +
labs(
title = "Curva de aprendizaje con early stopping",
x = "Número de árboles",
y = "Multiclass log-loss",
color = "Conjunto"
) +
theme_minimal()
Mostrar código
pyX_tr, pyX_val, pyy_tr, pyy_val = train_test_split(
pyX_train,
pyy_train_enc,
test_size=0.2,
random_state=1994,
stratify=pyy_train_enc
)
xgb_es_py = XGBClassifier(
objective="multi:softprob",
num_class=len(le.classes_),
n_estimators=1000,
learning_rate=0.1,
max_depth=4,
subsample=0.8,
colsample_bytree=0.8,
reg_lambda=1,
reg_alpha=0,
eval_metric="mlogloss",
random_state=1994,
n_jobs=-1,
early_stopping_rounds=30
)
try:
xgb_es_py.fit(
pyX_tr,
pyy_tr,
eval_set=[(pyX_tr, pyy_tr), (pyX_val, pyy_val)],
verbose=False
)
except TypeError:
# Compatibilidad con versiones antiguas de xgboost.
xgb_es_py.set_params(early_stopping_rounds=None)
xgb_es_py.fit(
pyX_tr,
pyy_tr,
eval_set=[(pyX_tr, pyy_tr), (pyX_val, pyy_val)],
early_stopping_rounds=30,
verbose=False
)XGBClassifier(base_score=None, booster=None, callbacks=None,
colsample_bylevel=None, colsample_bynode=None,
colsample_bytree=0.8, device=None, early_stopping_rounds=30,
enable_categorical=False, eval_metric='mlogloss',
feature_types=None, feature_weights=None, gamma=None,
grow_policy=None, importance_type=None,
interaction_constraints=None, learning_rate=0.1, max_bin=None,
max_cat_threshold=None, max_cat_to_onehot=None,
max_delta_step=None, max_depth=4, max_leaves=None,
min_child_weight=None, missing=nan, monotone_constraints=None,
multi_strategy=None, n_estimators=1000, n_jobs=-1, num_class=3, ...)In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
Parameters
| objective | 'multi:softprob' | |
| base_score | None | |
| booster | None | |
| callbacks | None | |
| colsample_bylevel | None | |
| colsample_bynode | None | |
| colsample_bytree | 0.8 | |
| device | None | |
| early_stopping_rounds | 30 | |
| enable_categorical | False | |
| eval_metric | 'mlogloss' | |
| feature_types | None | |
| feature_weights | None | |
| gamma | None | |
| grow_policy | None | |
| importance_type | None | |
| interaction_constraints | None | |
| learning_rate | 0.1 | |
| max_bin | None | |
| max_cat_threshold | None | |
| max_cat_to_onehot | None | |
| max_delta_step | None | |
| max_depth | 4 | |
| max_leaves | None | |
| min_child_weight | None | |
| missing | nan | |
| monotone_constraints | None | |
| multi_strategy | None | |
| n_estimators | 1000 | |
| n_jobs | -1 | |
| num_parallel_tree | None | |
| random_state | 1994 | |
| reg_alpha | 0 | |
| reg_lambda | 1 | |
| sampling_method | None | |
| scale_pos_weight | None | |
| subsample | 0.8 | |
| tree_method | None | |
| validate_parameters | None | |
| verbosity | None | |
| num_class | 3 |
Mostrar código
print("Mejor iteración:", getattr(xgb_es_py, "best_iteration", None))Mejor iteración: 931Mostrar código
print("Mejor score:", getattr(xgb_es_py, "best_score", None))Mejor score: 0.17749021158189618Mostrar código
results = xgb_es_py.evals_result()
plt.figure(figsize=(8, 5))
plt.plot(results["validation_0"]["mlogloss"], label="Train")
plt.plot(results["validation_1"]["mlogloss"], label="Validación")
plt.xlabel("Número de árboles")
plt.ylabel("Multiclass log-loss")
plt.title("Curva de aprendizaje con early stopping")
plt.legend()
plt.show()
11 Ajuste de hiperparámetros
XGBoost tiene muchos hiperparámetros. No conviene ajustarlos todos a la vez. Una estrategia razonable es empezar con los más influyentes:
max_depth: controla la complejidad de cada árbol.eta/learning_rate: controla cuánto aporta cada árbol.subsample: introduce aleatoriedad en observaciones.colsample_bytree: introduce aleatoriedad en variables.min_child_weight: evita particiones con muy poca información.gamma: exige una mejora mínima para crear una nueva partición.
Mostrar código
library(caret)
library(ggplot2)
head(modelLookup("xgbTree"), 4)Mostrar código
# se determina la semilla aleatoria
set.seed(101)
control <- trainControl(
method = "cv",
number = 10,
classProbs = TRUE,
summaryFunction = defaultSummary,
savePredictions = "final",
verboseIter = TRUE,
allowParallel = FALSE
)
grid_xgb <- expand.grid(
nrounds = c(100, 200),
max_depth = c(3, 4),
eta = c(0.05, 0.1),
gamma = 0,
colsample_bytree = 0.8,
min_child_weight = 1,
subsample = 0.8
)
# se entrena el modelo
xgb_caret <- train(
RENTA ~ .,
data = rtrain,
method = "xgbTree",
metric = "Accuracy",
trControl = control,
tuneGrid = grid_xgb
)+ Fold01: eta=0.05, max_depth=3, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
[17:35:22] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[17:35:22] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
- Fold01: eta=0.05, max_depth=3, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
+ Fold01: eta=0.05, max_depth=4, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
[17:35:24] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[17:35:24] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
- Fold01: eta=0.05, max_depth=4, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
+ Fold01: eta=0.10, max_depth=3, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
[17:35:26] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[17:35:26] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
- Fold01: eta=0.10, max_depth=3, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
+ Fold01: eta=0.10, max_depth=4, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
[17:35:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[17:35:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
- Fold01: eta=0.10, max_depth=4, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
+ Fold02: eta=0.05, max_depth=3, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
[17:35:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[17:35:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
- Fold02: eta=0.05, max_depth=3, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
+ Fold02: eta=0.05, max_depth=4, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
[17:35:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[17:35:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
- Fold02: eta=0.05, max_depth=4, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
+ Fold02: eta=0.10, max_depth=3, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
[17:35:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[17:35:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
- Fold02: eta=0.10, max_depth=3, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
+ Fold02: eta=0.10, max_depth=4, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
[17:35:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[17:35:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
- Fold02: eta=0.10, max_depth=4, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
+ Fold03: eta=0.05, max_depth=3, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
[17:35:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[17:35:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
- Fold03: eta=0.05, max_depth=3, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
+ Fold03: eta=0.05, max_depth=4, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
[17:35:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[17:35:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
- Fold03: eta=0.05, max_depth=4, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
+ Fold03: eta=0.10, max_depth=3, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
[17:35:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[17:35:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
- Fold03: eta=0.10, max_depth=3, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
+ Fold03: eta=0.10, max_depth=4, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
[17:35:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[17:35:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
- Fold03: eta=0.10, max_depth=4, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
+ Fold04: eta=0.05, max_depth=3, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
[17:35:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[17:35:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
- Fold04: eta=0.05, max_depth=3, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
+ Fold04: eta=0.05, max_depth=4, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
[17:35:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[17:35:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
- Fold04: eta=0.05, max_depth=4, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
+ Fold04: eta=0.10, max_depth=3, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
[17:35:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[17:35:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
- Fold04: eta=0.10, max_depth=3, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
+ Fold04: eta=0.10, max_depth=4, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
[17:35:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[17:35:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
- Fold04: eta=0.10, max_depth=4, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
+ Fold05: eta=0.05, max_depth=3, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
[17:35:52] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[17:35:52] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
- Fold05: eta=0.05, max_depth=3, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
+ Fold05: eta=0.05, max_depth=4, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
[17:35:54] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[17:35:54] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
- Fold05: eta=0.05, max_depth=4, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
+ Fold05: eta=0.10, max_depth=3, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
[17:35:56] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[17:35:56] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
- Fold05: eta=0.10, max_depth=3, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
+ Fold05: eta=0.10, max_depth=4, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
[17:35:58] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[17:35:58] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
- Fold05: eta=0.10, max_depth=4, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
+ Fold06: eta=0.05, max_depth=3, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
[17:36:00] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[17:36:00] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
- Fold06: eta=0.05, max_depth=3, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
+ Fold06: eta=0.05, max_depth=4, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
[17:36:02] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[17:36:02] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
- Fold06: eta=0.05, max_depth=4, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
+ Fold06: eta=0.10, max_depth=3, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
[17:36:04] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[17:36:04] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
- Fold06: eta=0.10, max_depth=3, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
+ Fold06: eta=0.10, max_depth=4, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
[17:36:06] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[17:36:06] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
- Fold06: eta=0.10, max_depth=4, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
+ Fold07: eta=0.05, max_depth=3, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
[17:36:07] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[17:36:07] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
- Fold07: eta=0.05, max_depth=3, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
+ Fold07: eta=0.05, max_depth=4, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
[17:36:10] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[17:36:10] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
- Fold07: eta=0.05, max_depth=4, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
+ Fold07: eta=0.10, max_depth=3, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
[17:36:11] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[17:36:11] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
- Fold07: eta=0.10, max_depth=3, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
+ Fold07: eta=0.10, max_depth=4, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
[17:36:13] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[17:36:13] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
- Fold07: eta=0.10, max_depth=4, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
+ Fold08: eta=0.05, max_depth=3, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
[17:36:15] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[17:36:15] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
- Fold08: eta=0.05, max_depth=3, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
+ Fold08: eta=0.05, max_depth=4, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
[17:36:17] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[17:36:17] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
- Fold08: eta=0.05, max_depth=4, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
+ Fold08: eta=0.10, max_depth=3, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
[17:36:19] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[17:36:19] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
- Fold08: eta=0.10, max_depth=3, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
+ Fold08: eta=0.10, max_depth=4, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
[17:36:21] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[17:36:21] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
- Fold08: eta=0.10, max_depth=4, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
+ Fold09: eta=0.05, max_depth=3, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
[17:36:23] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[17:36:23] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
- Fold09: eta=0.05, max_depth=3, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
+ Fold09: eta=0.05, max_depth=4, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
[17:36:25] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[17:36:25] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
- Fold09: eta=0.05, max_depth=4, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
+ Fold09: eta=0.10, max_depth=3, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
[17:36:26] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[17:36:26] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
- Fold09: eta=0.10, max_depth=3, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
+ Fold09: eta=0.10, max_depth=4, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
[17:36:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[17:36:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
- Fold09: eta=0.10, max_depth=4, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
+ Fold10: eta=0.05, max_depth=3, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
[17:36:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[17:36:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
- Fold10: eta=0.05, max_depth=3, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
+ Fold10: eta=0.05, max_depth=4, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
[17:36:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[17:36:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
- Fold10: eta=0.05, max_depth=4, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
+ Fold10: eta=0.10, max_depth=3, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
[17:36:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[17:36:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
- Fold10: eta=0.10, max_depth=3, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
+ Fold10: eta=0.10, max_depth=4, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
[17:36:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
[17:36:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
- Fold10: eta=0.10, max_depth=4, gamma=0, colsample_bytree=0.8, min_child_weight=1, subsample=0.8, nrounds=200
Aggregating results
Selecting tuning parameters
Fitting nrounds = 200, max_depth = 4, eta = 0.1, gamma = 0, colsample_bytree = 0.8, min_child_weight = 1, subsample = 0.8 on full training setMostrar código
xgb_careteXtreme Gradient Boosting
18669 samples
35 predictor
3 classes: 'Baja', 'Media', 'Alta'
No pre-processing
Resampling: Cross-Validated (10 fold)
Summary of sample sizes: 16802, 16803, 16801, 16801, 16803, 16802, ...
Resampling results across tuning parameters:
eta max_depth nrounds Accuracy Kappa
0.05 3 100 0.8636785 0.7499218
0.05 3 200 0.8755696 0.7730645
0.05 4 100 0.8781942 0.7775113
0.05 4 200 0.8917459 0.8031123
0.10 3 100 0.8762661 0.7744850
0.10 3 200 0.8900854 0.8003064
0.10 4 100 0.8900857 0.8001302
0.10 4 200 0.9069582 0.8314550
Tuning parameter 'gamma' was held constant at a value of 0
Tuning
Tuning parameter 'min_child_weight' was held constant at a value of 1
Tuning parameter 'subsample' was held constant at a value of 0.8
Accuracy was used to select the optimal model using the largest value.
The final values used for the model were nrounds = 200, max_depth = 4, eta
= 0.1, gamma = 0, colsample_bytree = 0.8, min_child_weight = 1 and subsample
= 0.8.Mostrar código
plot(xgb_caret)
Mostrar código
from sklearn.model_selection import GridSearchCV, StratifiedKFold
param_grid = {
"n_estimators": [100, 200],
"max_depth": [3, 5],
"learning_rate": [0.05, 0.1],
"subsample": [0.8],
"colsample_bytree": [0.8],
"min_child_weight": [1, 5]
}
base_xgb = XGBClassifier(
objective="multi:softprob",
num_class=len(le.classes_),
eval_metric="mlogloss",
random_state=1994,
n_jobs=-1
)
cv = StratifiedKFold(n_splits=3, shuffle=True, random_state=1994)
grid_search = GridSearchCV(
estimator=base_xgb,
param_grid=param_grid,
scoring="accuracy",
cv=cv,
n_jobs=1,
verbose=0
)
grid_search.fit(pyX_train, pyy_train_enc)GridSearchCV(cv=StratifiedKFold(n_splits=3, random_state=1994, shuffle=True),
estimator=XGBClassifier(base_score=None, booster=None,
callbacks=None, colsample_bylevel=None,
colsample_bynode=None,
colsample_bytree=None, device=None,
early_stopping_rounds=None,
enable_categorical=False,
eval_metric='mlogloss', feature_types=None,
feature_weights=None, gamma=None,
gr...
max_delta_step=None, max_depth=None,
max_leaves=None, min_child_weight=None,
missing=nan, monotone_constraints=None,
multi_strategy=None, n_estimators=None,
n_jobs=-1, num_class=3, ...),
n_jobs=1,
param_grid={'colsample_bytree': [0.8],
'learning_rate': [0.05, 0.1], 'max_depth': [3, 5],
'min_child_weight': [1, 5], 'n_estimators': [100, 200],
'subsample': [0.8]},
scoring='accuracy')In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
Parameters
| estimator | XGBClassifier..._class=3, ...) | |
| param_grid | {'colsample_bytree': [0.8], 'learning_rate': [0.05, 0.1], 'max_depth': [3, 5], 'min_child_weight': [1, 5], ...} | |
| scoring | 'accuracy' | |
| n_jobs | 1 | |
| refit | True | |
| cv | StratifiedKFo... shuffle=True) | |
| verbose | 0 | |
| pre_dispatch | '2*n_jobs' | |
| error_score | nan | |
| return_train_score | False |
XGBClassifier(base_score=None, booster=None, callbacks=None,
colsample_bylevel=None, colsample_bynode=None,
colsample_bytree=0.8, device=None, early_stopping_rounds=None,
enable_categorical=False, eval_metric='mlogloss',
feature_types=None, feature_weights=None, gamma=None,
grow_policy=None, importance_type=None,
interaction_constraints=None, learning_rate=0.1, max_bin=None,
max_cat_threshold=None, max_cat_to_onehot=None,
max_delta_step=None, max_depth=5, max_leaves=None,
min_child_weight=1, missing=nan, monotone_constraints=None,
multi_strategy=None, n_estimators=200, n_jobs=-1, num_class=3, ...)Parameters
| objective | 'multi:softprob' | |
| base_score | None | |
| booster | None | |
| callbacks | None | |
| colsample_bylevel | None | |
| colsample_bynode | None | |
| colsample_bytree | 0.8 | |
| device | None | |
| early_stopping_rounds | None | |
| enable_categorical | False | |
| eval_metric | 'mlogloss' | |
| feature_types | None | |
| feature_weights | None | |
| gamma | None | |
| grow_policy | None | |
| importance_type | None | |
| interaction_constraints | None | |
| learning_rate | 0.1 | |
| max_bin | None | |
| max_cat_threshold | None | |
| max_cat_to_onehot | None | |
| max_delta_step | None | |
| max_depth | 5 | |
| max_leaves | None | |
| min_child_weight | 1 | |
| missing | nan | |
| monotone_constraints | None | |
| multi_strategy | None | |
| n_estimators | 200 | |
| n_jobs | -1 | |
| num_parallel_tree | None | |
| random_state | 1994 | |
| reg_alpha | None | |
| reg_lambda | None | |
| sampling_method | None | |
| scale_pos_weight | None | |
| subsample | 0.8 | |
| tree_method | None | |
| validate_parameters | None | |
| verbosity | None | |
| num_class | 3 |
Mostrar código
print(grid_search.best_params_){'colsample_bytree': 0.8, 'learning_rate': 0.1, 'max_depth': 5, 'min_child_weight': 1, 'n_estimators': 200, 'subsample': 0.8}Mostrar código
print(round(grid_search.best_score_, 4))0.91812 Importancia de variables
XGBoost ofrece varias formas de medir la importancia de variables. Las más habituales son:
- Gain: mejora media de la función objetivo al usar una variable en una partición. Suele ser la más informativa.
- Cover: cantidad media de observaciones afectadas por las particiones donde aparece una variable.
- Frequency: número de veces que una variable aparece en los árboles.
Es importante no confundir importancia con causalidad. Una variable puede ser importante porque está correlacionada con otras, porque segmenta bien los datos o porque captura diferencias espaciales/sociales, pero eso no significa necesariamente que cause directamente la renta.
Mostrar código
importance_matrix <- xgb.importance(
feature_names = colnames(x_train),
model = xgb_es
)
head(importance_matrix, 20)Mostrar código
xgb.plot.importance(
importance_matrix = importance_matrix[1:20, ],
measure = "Gain",
rel_to_first = TRUE,
xlab = "Importancia relativa"
)
Mostrar código
importance_matrix %>%
select(Feature, Gain, Cover, Frequency) %>%
head(25)Mostrar código
importances = xgb_es_py.feature_importances_
imp_df = pd.DataFrame({
"variable": pyX_train.columns,
"importance": importances
}).sort_values("importance", ascending=False)
imp_df.head(20) variable importance
58 CDIS_Ciutat-Vella 0.279222
67 CDIS_Sarrià-Sant Gervasi 0.109396
59 CDIS_Eixample 0.081883
62 CDIS_Les Corts 0.078071
64 CDIS_Sant Andreu 0.030104
60 CDIS_Gràcia 0.026238
66 CDIS_Sants-Montjuic 0.025621
11 DISTANCE_TO_DIAGONAL 0.022157
15 HASLIFT_Si 0.018368
53 BUILTTYPEID_1_obraNueva 0.016740
65 CDIS_Sant Martí 0.015366
9 DISTANCE_TO_CITY_CENTER 0.015085
52 BUILTTYPEID_1_noObraNueva 0.011991
1 CONSTRUCTEDAREA 0.011572
63 CDIS_Nou Barris 0.010807
21 HASPARKINGSPACE_No 0.009931
61 CDIS_Horta-Guinardó 0.009152
14 HASLIFT_No 0.008878
0 UNITPRICE 0.008607
42 HASGARDEN_Si 0.007215Mostrar código
top_imp = imp_df.head(20).sort_values("importance")
plt.figure(figsize=(8, 7))
plt.barh(top_imp["variable"], top_imp["importance"])
plt.xlabel("Importancia")
plt.title("Top 20 variables importantes - XGBoost")
plt.tight_layout()
plt.show()
Mostrar código
booster = xgb_es_py.get_booster()
importance_gain = booster.get_score(importance_type="gain")
importance_cover = booster.get_score(importance_type="cover")
importance_weight = booster.get_score(importance_type="weight")
importance_types = pd.DataFrame({
"variable": list(set(list(importance_gain.keys()) + list(importance_cover.keys()) + list(importance_weight.keys()))),
})
importance_types["gain"] = importance_types["variable"].map(importance_gain).fillna(0)
importance_types["cover"] = importance_types["variable"].map(importance_cover).fillna(0)
importance_types["frequency"] = importance_types["variable"].map(importance_weight).fillna(0)
importance_types.sort_values("gain", ascending=False).head(25) variable gain cover frequency
3 CDIS_Ciutat-Vella 62.091244 1270.649292 196.0
43 CDIS_Sarrià-Sant Gervasi 24.326723 714.562012 385.0
38 CDIS_Eixample 18.208601 885.470947 285.0
44 CDIS_Les Corts 17.360847 802.645020 96.0
22 CDIS_Sant Andreu 6.694213 1024.601318 187.0
67 CDIS_Gràcia 5.834681 659.574341 329.0
34 CDIS_Sants-Montjuic 5.697492 493.080597 228.0
52 DISTANCE_TO_DIAGONAL 4.927001 319.901276 5512.0
1 HASLIFT_Si 4.084602 345.453644 43.0
23 BUILTTYPEID_1_obraNueva 3.722549 682.505737 7.0
28 CDIS_Sant Martí 3.417037 472.629456 347.0
57 DISTANCE_TO_CITY_CENTER 3.354484 368.880981 5945.0
40 BUILTTYPEID_1_noObraNueva 2.666445 728.496582 28.0
48 CONSTRUCTEDAREA 2.573203 323.895416 2727.0
35 CDIS_Nou Barris 2.403119 637.277588 208.0
55 HASPARKINGSPACE_No 2.208451 314.926422 112.0
58 CDIS_Horta-Guinardó 2.035254 399.593323 301.0
50 HASLIFT_No 1.974228 251.977203 177.0
8 UNITPRICE 1.914016 299.040985 3229.0
25 HASGARDEN_Si 1.604504 474.314331 8.0
60 CADMAXBUILDINGFLOOR 1.472515 262.913116 1441.0
46 BATHNUMBER 1.449864 200.157806 284.0
54 HASDOORMAN_Si 1.437194 232.270035 29.0
2 CADDWELLINGCOUNT 1.403906 343.029053 2714.0
65 HASGARDEN_No 1.395046 396.002228 56.012.1 Interpretación de la matriz de importancia
La matriz de importancia debe leerse de forma comparativa:
- Una variable con alto
Gainha producido particiones muy útiles para reducir el error. - Una variable con alto
Frequencyaparece muchas veces, pero no necesariamente es la más decisiva. - Una variable con alto
Coverafecta a muchas observaciones.
Ejemplo de lectura:
Si
CDIS_Sarrià-Sant Gervasiaparece con altoGain, significa que pertenecer a ese distrito ayuda mucho al modelo a separar clases de renta. SiDISTANCE_TO_CITY_CENTERtambién aparece arriba, significa que la localización espacial tiene un papel fuerte en la predicción.
13 Extracción y visualización de uno de los árboles
Aunque XGBoost es un modelo de muchos árboles, se puede extraer un árbol individual. Esto sirve para entender una pequeña parte del ensamble, no todo el modelo completo.
Un árbol individual puede responder preguntas como:
- ¿Qué variable usa en la primera partición?
- ¿Qué umbrales aparecen?
- ¿Qué hojas finales genera?
- ¿Qué peso asigna a cada hoja?
Mostrar código
arbol_1 <- xgb.model.dt.tree(
feature_names = colnames(x_train),
model = xgb_es,
trees = 0
)
head(arbol_1, 20)Mostrar código
if (requireNamespace("DiagrammeR", quietly = TRUE)) {
xgb.plot.tree(
feature_names = colnames(x_train),
model = xgb_es,
trees = 0
)
} else {
message("Para visualizar el árbol instala DiagrammeR: install.packages('DiagrammeR')")
}Mostrar código
# Estructura del primer árbol en formato texto
first_tree = booster.get_dump(with_stats=True)[0]
print(first_tree[:3000])0:[CDIS_Sarrià-Sant Gervasi] yes=1,no=2,gain=2025.23767,cover=2207.95703
1:[CDIS_Les Corts] yes=3,no=4,gain=325.062866,cover=2039.89392
3:[CONSTRUCTEDAREA<150] yes=7,no=8,missing=8,gain=146.500946,cover=1962.89575
7:[UNITPRICE<5090.90918] yes=15,no=16,missing=16,gain=28.7746277,cover=1822.96655
15:leaf=-0.0467793159,cover=1531.81763
16:leaf=-0.0124339573,cover=291.148987
8:[UNITPRICE<4842.10547] yes=17,no=18,missing=18,gain=86.5151062,cover=139.929199
17:leaf=0.00847392622,cover=93.2861328
18:leaf=0.174201354,cover=46.6430664
4:[DISTANCE_TO_DIAGONAL<0.771498144] yes=9,no=10,missing=10,gain=143.103668,cover=76.9980774
9:[DISTANCE_TO_CITY_CENTER<3.31402397] yes=19,no=20,missing=20,gain=43.5758667,cover=46.8281593
19:leaf=0.175616786,cover=23.8768082
20:leaf=0.38078782,cover=22.9513512
10:[DISTANCE_TO_CITY_CENTER<4.17750406] yes=21,no=22,missing=22,gain=18.5333767,cover=30.1699219
21:leaf=-0.0374330617,cover=23.8768082
22:leaf=0.143853143,cover=6.29311228
2:[CONSTRUCTEDAREA<158] yes=5,no=6,missing=6,gain=77.3605957,cover=168.063126
5:[DISTANCE_TO_DIAGONAL<0.560042083] yes=11,no=12,missing=12,gain=42.3699951,cover=101.430161
11:leaf=0.414218992,cover=18.6942463
12:[DISTANCE_TO_CITY_CENTER<2.68860197] yes=23,no=24,missing=24,gain=45.3856201,cover=82.7359161
23:leaf=0.0464185216,cover=11.2905836
24:leaf=0.266637951,cover=71.4453354
6:[DISTANCE_TO_METRO<0.0480681546] yes=13,no=14,missing=14,gain=3.00524902,cover=66.6329575
13:[BATHNUMBER<3] yes=25,no=26,missing=26,gain=3.47111893,cover=2.59128165
25:leaf=0.018044129,cover=1.11054921
26:leaf=0.248895884,cover=1.48073232
14:leaf=0.427871138,cover=64.0416718
13.1 Cómo interpretar el árbol extraído
Un nodo interno tiene la forma:
\[\text{variable} < \text{umbral}\]
Si la condición se cumple, la observación va hacia una rama. Si no se cumple, va hacia la otra. Al final se llega a una hoja con un valor. En boosting, ese valor no es directamente una clase, sino una contribución parcial a la predicción final.
En clasificación multiclase, XGBoost suele construir árboles asociados a las clases. Por eso, un árbol aislado solo explica una parte concreta de la predicción. La predicción final combina muchos árboles.
14 Dependencia parcial
La dependencia parcial muestra cómo cambia la predicción media del modelo cuando una variable cambia y el resto de variables se mantiene según su distribución observada.
Esto ayuda a responder preguntas como:
¿Qué ocurre con la probabilidad de renta alta cuando aumenta la distancia al centro?
Mostrar código
# Función de predicción robusta para PDP.
# El paquete pdp puede pasar newdata sin la variable objetivo.
# Como dummy_model_r fue entrenado con una fórmula RENTA ~ ., aquí añadimos
# una columna RENTA ficticia si no existe para evitar el error:
# "Variable(s) 'RENTA' are not in newdata".
pred_fun_alta <- function(object, newdata) {
newdata <- as.data.frame(newdata)
if (!"RENTA" %in% names(newdata)) {
newdata$RENTA <- factor(clases[1], levels = clases)
} else {
newdata$RENTA <- factor(newdata$RENTA, levels = clases)
}
mat <- predict(dummy_model_r, newdata = newdata)
mat <- as.matrix(mat)
# Asegurar que las columnas tienen exactamente el mismo orden que en entrenamiento.
columnas_faltantes <- setdiff(colnames(x_train), colnames(mat))
if (length(columnas_faltantes) > 0) {
faltantes <- matrix(
0,
nrow = nrow(mat),
ncol = length(columnas_faltantes),
dimnames = list(NULL, columnas_faltantes)
)
mat <- cbind(mat, faltantes)
}
mat <- mat[, colnames(x_train), drop = FALSE]
pred <- predict(object, xgb.DMatrix(mat))
pred <- matrix(pred, ncol = num_class, byrow = TRUE)
pred[, which(clases == "Alta")]
}
pdp_alta <- pdp::partial(
object = xgb_es,
pred.var = "DISTANCE_TO_CITY_CENTER",
train = rtrain,
pred.fun = pred_fun_alta,
grid.resolution = 20,
progress = FALSE
)
plot(pdp_alta) +
labs(
title = "Dependencia parcial de DISTANCE_TO_CITY_CENTER",
y = "Probabilidad media predicha de RENTA Alta"
) +
theme_minimal()
NULLMostrar código
from sklearn.inspection import PartialDependenceDisplay
feature_name = "DISTANCE_TO_CITY_CENTER"
feature_index = list(pyX_train.columns).index(feature_name)
PartialDependenceDisplay.from_estimator(
xgb_es_py,
pyX_test,
[feature_index],
target=list(le.classes_).index("Alta")
)<sklearn.inspection._plot.partial_dependence.PartialDependenceDisplay object at 0x0000014F2CFC5F90>Mostrar código
plt.title("Dependencia parcial de DISTANCE_TO_CITY_CENTER para RENTA Alta")
plt.show()
15 Interpretación mediante SHAP values
15.1 Qué son los SHAP values
Los SHAP values son una herramienta de interpretabilidad basada en teoría de juegos. La idea es repartir la predicción de un modelo entre las variables explicativas.
Para una observación concreta, se puede escribir:
\[\text{predicción} = \text{valor base} + \sum_{j=1}^{p} \phi_j\]
donde:
valor basees la predicción media del modelo;- \(\phi_j\) es la contribución de la variable \(j\);
- una contribución positiva empuja la predicción hacia arriba;
- una contribución negativa empuja la predicción hacia abajo.
En clasificación multiclase, SHAP puede calcular contribuciones para cada clase. Por ejemplo, una variable puede aumentar la probabilidad de Alta y reducir la de Baja.
15.2 Por qué SHAP es especialmente útil en XGBoost
XGBoost suele tener un rendimiento alto, pero es difícil interpretar directamente cientos de árboles. SHAP permite obtener:
- interpretación global del modelo;
- ranking de variables relevantes;
- dirección del efecto de cada variable;
- explicación individual de una predicción;
- detección de relaciones no lineales.
Mostrar código
# Para simplificar, se calculan SHAP values sobre una muestra.
set.seed(1994)
idx_shap <- sample(seq_len(nrow(x_test)), size = min(1000, nrow(x_test)))
X_shap_r <- x_test[idx_shap, ]
shap_values_r <- predict(
xgb_es,
X_shap_r,
predcontrib = TRUE
)
# En multiclase, XGBoost devuelve contribuciones por clase.
dim(shap_values_r)NULLMostrar código
# SHAPforxgboost trabaja cómodamente en problemas binarios/regresión.
# En multiclase, según la versión de xgboost, predict(..., predcontrib = TRUE)
# puede devolver:
# 1) una matriz: observaciones x [clases * (variables + BIAS)]
# 2) un array 3D: observaciones x (variables + BIAS) x clases
# Este bloque soporta ambos formatos y evita errores de dimensiones.
feature_names_r <- colnames(x_train)
num_features <- length(feature_names_r) + 1 # +1 por BIAS
# Usamos la clase "Alta" si existe. Si no existe, usamos la última clase.
clase_alta_id <- match("Alta", clases)
if (is.na(clase_alta_id)) {
clase_alta_id <- length(clases)
message("No existe la clase 'Alta'. Se usa la clase: ", clases[clase_alta_id])
}
# Aseguramos formato numérico.
if (is.list(shap_values_r)) {
# Algunas versiones devuelven lista por clase.
shap_alta <- as.matrix(shap_values_r[[clase_alta_id]])
} else if (length(dim(shap_values_r)) == 3) {
# Formato: observaciones x variables_con_bias x clases.
shap_alta <- as.matrix(shap_values_r[, , clase_alta_id])
} else {
# Formato matriz o vector.
shap_values_mat <- as.matrix(shap_values_r)
# Si solo hay una observación, as.matrix() puede dejar una sola columna.
# Reparamos el caso vectorial cuando la longitud encaja con el número esperado.
if (ncol(shap_values_mat) == 1 && length(shap_values_r) %% num_features == 0) {
shap_values_mat <- matrix(
as.numeric(shap_values_r),
ncol = num_features * num_class,
byrow = TRUE
)
}
if (ncol(shap_values_mat) == num_features) {
# Caso binario/regresión: ya viene un único bloque.
shap_alta <- shap_values_mat
} else {
# Caso multiclase: extraemos el bloque de la clase seleccionada.
cols_alta <- ((clase_alta_id - 1) * num_features + 1):(clase_alta_id * num_features)
cols_alta <- cols_alta[cols_alta <= ncol(shap_values_mat)]
shap_alta <- shap_values_mat[, cols_alta, drop = FALSE]
}
}
# Normalizamos nombres de columnas.
if (ncol(shap_alta) >= num_features) {
shap_alta <- shap_alta[, seq_len(num_features), drop = FALSE]
colnames(shap_alta) <- c(feature_names_r, "BIAS")
} else if (ncol(shap_alta) == length(feature_names_r)) {
colnames(shap_alta) <- feature_names_r
} else {
stop("El objeto SHAP no tiene el número esperado de columnas. Revisa dim(shap_values_r).")
}
shap_feature_cols <- intersect(feature_names_r, colnames(shap_alta))
shap_long_alta <- shap_alta[, shap_feature_cols, drop = FALSE] %>%
as.data.frame() %>%
mutate(id = row_number()) %>%
pivot_longer(-id, names_to = "variable", values_to = "shap")
shap_importance_alta <- shap_long_alta %>%
group_by(variable) %>%
summarise(mean_abs_shap = mean(abs(shap), na.rm = TRUE), .groups = "drop") %>%
arrange(desc(mean_abs_shap))
head(shap_importance_alta, 20)Mostrar código
shap_importance_alta %>%
slice_head(n = 20) %>%
mutate(variable = reorder(variable, mean_abs_shap)) %>%
ggplot(aes(x = mean_abs_shap, y = variable)) +
geom_col() +
labs(
title = "Importancia SHAP media absoluta para la clase Alta",
x = "Media de |SHAP value|",
y = "Variable"
) +
theme_minimal()
Mostrar código
top_vars_shap <- shap_importance_alta %>%
slice_head(n = 10) %>%
pull(variable)
shap_long_alta %>%
filter(variable %in% top_vars_shap) %>%
ggplot(aes(x = shap, y = reorder(variable, abs(shap), FUN = median))) +
geom_boxplot() +
labs(
title = "Distribución de SHAP values para la clase Alta",
x = "SHAP value",
y = "Variable"
) +
theme_minimal()
Mostrar código
import shap
# Muestra para evitar cálculos pesados
sample_shap = pyX_test.sample(n=min(1000, len(pyX_test)), random_state=1994)
explainer = shap.TreeExplainer(xgb_es_py)
shap_values = explainer.shap_values(sample_shap)
# Según la versión de shap/xgboost, puede devolver lista por clase o array 3D.
type(shap_values)<class 'numpy.ndarray'>Mostrar código
class_alta_index = list(le.classes_).index("Alta")
if isinstance(shap_values, list):
shap_alta_py = shap_values[class_alta_index]
else:
# Formato frecuente: observaciones x variables x clases
shap_alta_py = shap_values[:, :, class_alta_index]
shap.summary_plot(shap_alta_py, sample_shap, show=False)
plt.title("SHAP summary plot para la clase Alta")
plt.show()
Mostrar código
mean_abs_shap = np.abs(shap_alta_py).mean(axis=0)
shap_importance_py = pd.DataFrame({
"variable": sample_shap.columns,
"mean_abs_shap": mean_abs_shap
}).sort_values("mean_abs_shap", ascending=False)
shap_importance_py.head(20) variable mean_abs_shap
11 DISTANCE_TO_DIAGONAL 1.510816
67 CDIS_Sarrià-Sant Gervasi 0.817281
9 DISTANCE_TO_CITY_CENTER 0.753988
1 CONSTRUCTEDAREA 0.547577
0 UNITPRICE 0.453606
6 CADCONSTRUCTIONYEAR 0.410290
10 DISTANCE_TO_METRO 0.348730
58 CDIS_Ciutat-Vella 0.322575
66 CDIS_Sants-Montjuic 0.287443
8 CADDWELLINGCOUNT 0.199292
5 FLOORCLEAN 0.178454
60 CDIS_Gràcia 0.160507
7 CADMAXBUILDINGFLOOR 0.155491
61 CDIS_Horta-Guinardó 0.150152
2 ROOMNUMBER 0.114717
62 CDIS_Les Corts 0.108962
65 CDIS_Sant Martí 0.093565
16 HASAIRCONDITIONING_No 0.083400
14 HASLIFT_No 0.068099
3 BATHNUMBER 0.057134Mostrar código
top_shap = shap_importance_py.head(20).sort_values("mean_abs_shap")
plt.figure(figsize=(8, 7))
plt.barh(top_shap["variable"], top_shap["mean_abs_shap"])
plt.xlabel("Media de |SHAP value|")
plt.title("Importancia SHAP para la clase Alta")
plt.tight_layout()
plt.show()
Mostrar código
# Explicación individual de una observación concreta.
obs_id = 0
expected_value = explainer.expected_value
if isinstance(expected_value, list):
base_value = expected_value[class_alta_index]
elif isinstance(expected_value, np.ndarray):
base_value = expected_value[class_alta_index] if expected_value.ndim == 1 else expected_value[0, class_alta_index]
else:
base_value = expected_value
exp = shap.Explanation(
values=shap_alta_py[obs_id],
base_values=base_value,
data=sample_shap.iloc[obs_id].values,
feature_names=sample_shap.columns
)
shap.plots.waterfall(exp, max_display=15, show=False)
plt.title("Explicación local SHAP de una vivienda para RENTA Alta")
plt.show()
15.3 Cómo interpretar los gráficos SHAP
El gráfico de barras SHAP muestra las variables que más contribuyen, en promedio, a las predicciones del modelo. A diferencia de la importancia interna de XGBoost, SHAP mide contribuciones sobre predicciones.
El summary_plot combina dos ideas:
- eje vertical: variables ordenadas por importancia;
- eje horizontal: contribución SHAP;
- color: valor de la variable.
Lectura típica:
- valores SHAP positivos para la clase
Altaempujan la predicción haciaRENTA = Alta; - valores SHAP negativos para la clase
Altaalejan la predicción deRENTA = Alta; - si los puntos rojos de una variable aparecen a la derecha, valores altos de esa variable aumentan la probabilidad de renta alta;
- si aparecen a la izquierda, valores altos reducen la probabilidad de renta alta.
16 Comparación con un árbol individual
Para entender por qué XGBoost puede mejorar a un árbol simple, es útil recordar:
- un árbol individual aprende una única estructura de decisión;
- XGBoost aprende muchas estructuras pequeñas;
- cada árbol añade una corrección;
- el modelo final puede representar relaciones más complejas.
Sin embargo, esta mejora predictiva se paga con menor interpretabilidad directa. Por eso se combinan tres niveles de interpretación:
- Importancia de variables: visión global rápida.
- Extracción de árboles: visión estructural parcial.
- SHAP values: explicación global y local de las predicciones.
17 Predicción sobre nuevas observaciones
Una vez entrenado el modelo, podemos generar predicciones sobre nuevas viviendas. En un caso real, esta parte se conectaría con un formulario, una API o una aplicación web.
Mostrar código
# IMPORTANTE:
# dummy_model_r fue creado con la fórmula RENTA ~ .
# Por eso predict.dummyVars() exige que newdata tenga también la columna RENTA,
# aunque después esa columna NO se usa como predictor.
nuevo_registro <- rtest[1, , drop = FALSE]
# Función segura para transformar un único registro con dummyVars.
# Evita el error:
# "contrasts can be applied only to factors with 2 or more levels".
# La idea es añadir temporalmente una fila de referencia del entrenamiento para
# que model.matrix() vea todos los niveles categóricos necesarios.
predecir_dummies_seguro <- function(dummy_model, newdata, referencia_train, columnas_entrenamiento) {
columnas_necesarias <- colnames(referencia_train)
# Si falta alguna columna, se rellena con el valor de referencia del train.
faltan <- setdiff(columnas_necesarias, colnames(newdata))
if (length(faltan) > 0) {
for (col in faltan) {
newdata[[col]] <- referencia_train[[col]][1]
}
}
newdata <- newdata[, columnas_necesarias, drop = FALSE]
# Mantener exactamente los niveles/clases del entrenamiento.
for (col in columnas_necesarias) {
if (is.factor(referencia_train[[col]])) {
newdata[[col]] <- factor(as.character(newdata[[col]]), levels = levels(referencia_train[[col]]))
} else if (inherits(referencia_train[[col]], "Date")) {
newdata[[col]] <- as.Date(newdata[[col]])
} else if (is.numeric(referencia_train[[col]])) {
newdata[[col]] <- as.numeric(newdata[[col]])
} else if (is.integer(referencia_train[[col]])) {
newdata[[col]] <- as.integer(newdata[[col]])
} else {
newdata[[col]] <- as.character(newdata[[col]])
}
}
# Usamos TODO el train como referencia, no solo una fila.
# Motivo: si una variable categórica tiene el mismo nivel en la fila de referencia
# y en el nuevo registro, model.matrix() sigue viendo un único nivel y lanza:
# "contrasts can be applied only to factors with 2 or more levels".
# Al añadir todo el entrenamiento, cada factor conserva sus niveles reales.
n_nuevo <- nrow(newdata)
datos_tmp <- rbind(referencia_train, newdata)
mat_tmp <- predict(dummy_model, newdata = datos_tmp)
mat_tmp <- as.matrix(mat_tmp)
# Nos quedamos solo con las últimas filas, que corresponden al nuevo registro.
mat <- tail(mat_tmp, n_nuevo)
mat <- as.matrix(mat)
# Alinear columnas con x_train.
columnas_faltantes <- setdiff(columnas_entrenamiento, colnames(mat))
if (length(columnas_faltantes) > 0) {
faltantes_mat <- matrix(0, nrow = nrow(mat), ncol = length(columnas_faltantes))
colnames(faltantes_mat) <- columnas_faltantes
mat <- cbind(mat, faltantes_mat)
}
columnas_extra <- setdiff(colnames(mat), columnas_entrenamiento)
if (length(columnas_extra) > 0) {
mat <- mat[, setdiff(colnames(mat), columnas_extra), drop = FALSE]
}
mat <- mat[, columnas_entrenamiento, drop = FALSE]
return(mat)
}
nuevo_mat <- predecir_dummies_seguro(
dummy_model = dummy_model_r,
newdata = nuevo_registro,
referencia_train = rtrain,
columnas_entrenamiento = colnames(x_train)
)
nuevo_prob <- predict(xgb_es, xgboost::xgb.DMatrix(nuevo_mat))
nuevo_prob <- matrix(nuevo_prob, ncol = num_class, byrow = TRUE)
colnames(nuevo_prob) <- clases
nuevo_prob Baja Media Alta
[1,] 0.980497 0.01932397 0.0001790576Mostrar código
clases[which.max(nuevo_prob[1, ])][1] "Baja"Mostrar código
nuevo_registro_py = pyX_test.iloc[[0]]
prob_nuevo_py = xgb_es_py.predict_proba(nuevo_registro_py)
pred_nuevo_py = le.inverse_transform(np.argmax(prob_nuevo_py, axis=1))
pd.DataFrame(prob_nuevo_py, columns=le.classes_) Alta Baja Media
0 0.007243 0.000922 0.991835Mostrar código
print("Clase predicha:", pred_nuevo_py[0])Clase predicha: Media18 Buenas prácticas al usar XGBoost
18.1 Evitar sobreajuste
Para reducir sobreajuste:
- usar
early_stopping_rounds; - reducir
max_depth; - reducir
learning_ratey aumentarn_estimators; - usar
subsample < 1; - usar
colsample_bytree < 1; - aumentar
lambdaoalpha; - usar validación cruzada.
18.2 Evaluar con métricas adecuadas
Si las clases están desbalanceadas, la accuracy puede ser engañosa. En ese caso conviene mirar:
- macro F1;
- balanced accuracy;
- recall por clase;
- matriz de confusión;
- log-loss si importan las probabilidades.
18.3 Interpretar con cautela
Las importancias y SHAP values ayudan a entender el modelo, pero no demuestran causalidad. Si una variable como distrito aparece como muy relevante, puede estar actuando como proxy de renta, localización, calidad urbana, oferta inmobiliaria u otras variables no observadas.
19 Resumen final
En este documento se ha construido un flujo completo de XGBoost en R y Python:
- carga y limpieza de datos;
- creación de la variable objetivo
RENTA; - codificación de variables categóricas;
- partición train/test;
- entrenamiento de XGBoost;
- evaluación mediante matriz de confusión y métricas de clasificación;
- uso de early stopping;
- ajuste básico de hiperparámetros;
- matriz de importancia de variables;
- extracción de un árbol individual;
- dependencia parcial;
- interpretación mediante SHAP values;
- predicción sobre nuevas observaciones.
La conclusión principal es que XGBoost es un modelo muy potente para datos tabulares, especialmente cuando existen relaciones no lineales e interacciones entre variables. No obstante, debe acompañarse siempre de validación, control del sobreajuste e interpretación rigurosa.
20 Bibliografía y recursos recomendados
- Chen, T., & Guestrin, C. (2016). XGBoost: A Scalable Tree Boosting System. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.
- Friedman, J. H. (2001). Greedy Function Approximation: A Gradient Boosting Machine. Annals of Statistics.
- Lundberg, S. M., & Lee, S.-I. (2017). A Unified Approach to Interpreting Model Predictions. Advances in Neural Information Processing Systems.
- Documentación oficial de
xgboostpara R y Python. - Documentación de
shappara interpretación de modelos de árboles.
Aquesta web està creada por Dante Conti y Sergi Ramírez, (c) 2026