Annotation - Single-cell best practices

Steps

yml

Install conda:
- Before creating the environment, ensure that conda is installed on your system.
Save the yml content:
- Copy the content from the yml tab into a file named environment.yml.
Create the environment:
- Open a terminal or command prompt.
- Run the following command:
```
conda env create -f environment.yml
```
Activate the environment:
- After the environment is created, activate it using:
```
conda activate <environment_name>
```
- Replace <environment_name> with the name specified in the environment.yml file. In the yml file it will look like this:
```
name: <environment_name>
```
Verify the installation:
- Check that the environment was created successfully by running:
```
conda env list
```

annotation.yml

name: annotation

channels:
  - conda-forge
  - bioconda
dependencies:
  - conda-forge::python=3.14.7
  - conda-forge::ipykernel=6.29.5
  - conda-forge::scanpy=1.12.3
  - conda-forge::leidenalg=0.12.0
  - conda-forge::pynndescent=0.5.13
  - conda-forge::scvi-tools=1.5.0.post1
  - pip
  - pip:
      - celltypist==1.6.3
      - lamindb==2.10.0
      - cellmapper==0.2.6

This book uses lamindb to store, share, and load datasets and notebooks using the theislab/sc-best-practices instance. We acknowledge free hosting from Lamin Labs.

Install lamindb
- Install the lamindb Python package:
```
pip install lamindb
```
Optionally create a lamin account
- Sign up and log in following the instructions
Verify your setup
- Run the lamin connect command:
```
import lamindb as ln

ln.Artifact.connect("theislab/sc-best-practices").df()
```
You should now see up to 100 of the stored datasets.
Accessing datasets (Artifacts)
- Search for the datasets on the Artifacts page
- Load an Artifact and the corresponding object:
```
import lamindb as ln

af = ln.Artifact.connect("theislab/sc-best-practices").get(
    key="key_of_dataset", is_latest=True
)
obj = af.load()
```
The object is now accessible in memory and is ready for analysis. Adapt the lamindb.Artifact.connect("theislab/sc-best-practices").get("SOMEIDXXXX") suffix to get respective versions.
Accessing notebooks (Transforms)
- Search for the notebook on the Transforms page
- Load the notebook:
```
lamin load <notebook url>
```
which will download the notebook to the current working directory. Analogously to Artifacts, you can adapt the suffix ID to get older versions.

Motivation¶

To understand your data better and make use of existing knowledge, it is important to figure out the “cellular identity” of each of the cells in your data. The process of labeling groups of cells in your data based on known (or sometimes unknown) cellular phenotypes is called “cell annotation”. Whereas there are many ways to annotate your cells (e.g. based on batch, disease, sex and more), in this notebook we will focus on the annotation of “cell types”.

A cell type is a cellular phenotype that is robust across datasets, identifiable by specific marker genes or proteins, and tied to a distinct biological function. A classic example is the plasma B cell — a white blood cell that secretes antibodies and can be identified by characteristic markers.

However, like with any categorization the size of categories and the borders drawn between them are partly subjective and can change over time, e.g. because new technologies allow for a higher resolution view of cells, or because specific “sub-phenotypes” that were not considered biologically meaningful are found to have important biological implications (see e.g. Kadur Lakshminarasimha Murthy et al., 2022). Cell types are therefore often further classified into “subtypes” or “cell states” (e.g. activated versus resting) and some researchers use the term “cell identity” to avoid this sometimes arbitrary distinction. For a more detailed discussion of this topic, we recommend the review by Wagner et al. Wagner et al., 2016 and the recently published review by Zeng Zeng, 2022.

Similarly, multiple cell types can be part of a single continuum, where one cell type might transition or differentiate into another. For example, in hematopoiesis cells differentiate from a stem cell into a specific immune cell type. Although hard borders between early and late stages of this differentiation are often drawn, the state of these cells can more accurately be described by the differentiation coordinate between the less and more differentiated cellular phenotypes.

There are multiple ways to annotate cells. We will give an overview of the most widely used approaches below. As we are working with transcriptomic data, each of these methods is ultimately based on the expression of specific genes or gene sets, or general transcriptomic similarity between cells.

Environment setup¶

import shutil
import sys
from pathlib import Path

import celltypist
import lamindb as ln
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import pandas.core.indexes.base as pandas_indexes_base
import scanpy as sc
import scvi
import seaborn as sns
from cellmapper import CellMapper
from celltypist import models
from scipy.sparse import csr_matrix

ln.connect("theislab/sc-best-practices")

ln.track("BnpvfJLWjHuE")

→ connected lamindb: theislab/sc-best-practices

/home/lheumos/miniforge3/envs/annotation/lib/python3.14/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm

→ loaded Transform('BnpvfJLWjHuE000B', key='annotation.ipynb'), started new Run('197nMX5slReRukck') at 2026-09-29 13:05:48 UTC

→ notebook imports: cellmapper==0.2.6 celltypist==1.6.3 lamindb-core==2.10.0 matplotlib==3.11.2 numba==0.67.0 numpy==2.5.3 pandas==3.0.6 scanpy==1.12.3 scipy==1.18.1 scvi-tools==1.5.0.post1 seaborn==0.13.2

# filter out some deprecation and performance warnings that do not affect our code

import warnings

# celltypist switches off all warnings when it is imported, so we restore Python's defaults
warnings.resetwarnings()
for category in (
    DeprecationWarning,
    PendingDeprecationWarning,
    ImportWarning,
    ResourceWarning,
):
    warnings.simplefilter("ignore", category=category)

from numba.core.errors import NumbaDeprecationWarning

warnings.simplefilter("ignore", category=NumbaDeprecationWarning)
warnings.filterwarnings("ignore", category=pd.errors.PerformanceWarning)

We will continue working with the scRNA-seq dataset that we earlier preprocessed and will now annotate it.

Set figure parameters:

sc.set_figure_params(figsize=(5, 5))

Load data¶

Let’s read in the toy dataset we will use for this tutorial. It includes a single sample (“site4-donor8”) of the data also used in other parts of the book. Moreover, cells that didn’t pass QC and predicted doublets have already been removed.

af = ln.Artifact.get(key="cellular_structure/s4d8_clustered.h5ad", is_latest=True)
adata = af.load(is_run_input=False)

Manual annotation¶

The classical or oldest way to perform cell type annotation is based on a single or small set of marker genes known to be associated with a particular cell type. This approach dates back to “pre-scRNA-seq times”, when single-cell data was low-dimensional (e.g., FACS data with gene panels consisting of no more than 30-40 genes). However, when no unique markers exist for a specific cell type, this approach can quickly become challenging and less objective, with combinations of markers or expression thresholds necessary for proper annotation. A robust set of marker genes and prior knowledge or annotation experience can help here, but the approach comes with the risk of unclear and subjective decision-making.

For manual annotation, the data is usually clustered before annotation, so that we can annotate groups of cells instead of making a per-cell call. This is not only less laborious but also more robust to noise: a single cell might not have a count for a specific marker even if it was expressed in that cell, simply due to the inherent sparsity of single-cell data. Clustering enables the detection of cells highly similar in overall gene expression and can therefore account for drop-outs at the single-cell level.

Finally, there are two angles from which to approach the marker-gene-based annotation. One option is to work from a table of marker genes for all the cell types you expect in your data and check in which clusters those genes are expressed. The other option is to check which genes are highly expressed in the clusters you defined and then check if they are associated with known cell types or states. If necessary, one can move back and forth between those approaches. We will show examples of both below.

From markers to cluster annotation¶

First, we list a set of markers for cell types in the bone marrow that is based on literature: previous papers that study specific cell types and subtypes and report marker genes for those cell types. Note that markers at the protein level (e.g., used for FACS) sometimes do not work as well in transcriptomic data; hence, using markers from RNA-based papers is often more likely to work. Moreover, sometimes markers in one dataset do not turn out to work as well in other datasets. Ideally, a marker set is therefore validated across multiple datasets. It is often useful to work together with experts: as a bioinformatician, try to team up with a biologist who has more extensive knowledge of the tissue, the biology, the expected cell types and markers, etc.

# keys: cell types or populations, values: lists of marker genes
marker_genes = {
    "CD14+ Mono": ["FCN1", "CD14"],
    "CD16+ Mono": ["TCF7L2", "FCGR3A", "LYN"],
    "ID2-hi myeloid prog": [
        "CD14",
        "ID2",
        "VCAN",
        "S100A9",
        "CLEC12A",
        "KLF4",
        "PLAUR",
    ],
    "cDC1": ["CLEC9A", "CADM1"],
    "cDC2": [
        "CST3",
        "COTL1",
        "LYZ",
        "DMXL2",
        "CLEC10A",
        "FCER1A",
    ],  # Note: DMXL2 should be negative
    "Normoblast": ["SLC4A1", "SLC25A37", "HBB", "HBA2", "HBA1", "TFRC"],
    "Erythroblast": ["MKI67", "HBA1", "HBB"],
    "Proerythroblast": [
        "CDK6",
        "SYNGR1",
        "HBM",
        "GYPA",
    ],  # Note HBM and GYPA are negative markers
    "NK": ["GNLY", "NKG7", "CD247", "GRIK4", "FCER1G", "TYROBP", "KLRG1", "FCGR3A"],
    "ILC": ["ID2", "PLCG2", "GNLY", "SYNE1"],
    "Lymph prog": [
        "VPREB1",
        "MME",
        "EBF1",
        "SSBP2",
        "BACH2",
        "CD79B",
        "IGHM",
        "PAX5",
        "PRKCE",
        "DNTT",
        "IGLL1",
    ],
    "Naive CD20+ B": ["MS4A1", "IL4R", "IGHD", "FCRL1", "IGHM"],
    "B1 B": [
        "MS4A1",
        "SSPN",
        "ITGB1",
        "EPHA4",
        "COL4A4",
        "PRDM1",
        "IRF4",
        "CD38",
        "XBP1",
        "PAX5",
        "BCL11A",
        "BLK",
        "IGHD",
        "IGHM",
        "ZNF215",
    ],  # Note IGHD and IGHM are negative markers
    "Transitional B": ["MME", "CD38", "CD24", "ACSM3", "MSI2"],
    "Plasma cells": ["MZB1", "HSP90B1", "FNDC3B", "PRDM1", "IGKC", "JCHAIN"],
    "Plasmablast": ["XBP1", "RF4", "PRDM1", "PAX5"],  # Note PAX5 is a negative marker
    "CD4+ T activated": ["CD4", "IL7R", "TRBC2", "ITGB1"],
    "CD4+ T naive": ["CD4", "IL7R", "TRBC2", "CCR7"],
    "CD8+ T": ["CD8A", "CD8B", "GZMK", "GZMA", "CCL5", "GZMB", "GZMH", "GZMA"],
    "T activation": ["CD69", "CD38"],  # CD69 much better marker!
    "T naive": ["LEF1", "CCR7", "TCF7"],
    "pDC": ["GZMB", "IL3RA", "COBLL1", "TCF4"],
    "G/M prog": ["MPO", "BCL2", "KCNQ5", "CSF3R"],
    "HSC": ["NRIP1", "MECOM", "PROM1", "NKAIN2", "CD34"],
    "MK/E prog": [
        "ZNF385D",
        "ITGA2B",
        "RYR3",
        "PLCB1",
    ],  # Note PLCB1 is a negative marker
}

Negative markers

A negative marker is a gene not expressed in a specific cell type, used to exclude or distinguish it, whereas a positive marker is actively expressed to identify the cell type.

Subset to only the markers that were detected in our data. We will loop through all cell types and keep only the genes that we find in our adata object as markers for that cell type. This will prevent errors once we start plotting.

marker_genes_in_data = {}
for ct, markers in marker_genes.items():
    markers_found = []
    for marker in markers:
        if marker in adata.var.index:
            markers_found.append(marker)
    marker_genes_in_data[ct] = markers_found

To see where these markers are expressed, we can work with a 2-dimensional visualization of the data, such as a UMAP. We calculate such an embedding here based on the scran-normalized count data, using only the highly deviant genes. Note that we first perform a PCA on the normalized counts to reduce the dimensionality of the data before we generate the UMAP.

Our raw counts are stored in .layers['counts'], and we set adata.X to the scran-normalized, log-transformed counts.

adata.X = adata.layers["scran_normalization"]

We furthermore set our adata.var.highly_variable to the highly deviant genes. Scanpy uses this var column in downstream calculations, such as the PCA below.

adata.var["highly_variable"] = adata.var["highly_deviant"]

Now perform PCA. We use the highly deviant genes (set as “highly variable” above) to reduce noise and strengthen signal in our data and set number of components to the default n=50. 50 is on the high side for data of a single sample, but it will ensure that we don’t ignore important variation in our data.

sc.tl.pca(adata, n_comps=50, mask_var="highly_variable")

Calculate the neighbor graph based on the PCs:

sc.pp.neighbors(adata)

And use that neighbor graph to calculate a 2-dimensional UMAP embedding of the data:

sc.tl.umap(adata)

Now show expression of the markers using the calculated UMAP. We’ll limit ourselves to B/plasma cell subtypes for this example. Note from the marker dictionary above that there are three negative markers in our list: IGHD and IGHM for B1 B, and PAX5 for plasmablasts, meaning that this cell type is expected not to or to lowly express those markers.

Let’s list the B cell subtypes we want to show the markers for:

B_plasma_cts = [
    "Naive CD20+ B",
    "B1 B",
    "Transitional B",
    "Plasma cells",
    "Plasmablast",
]

And now plot one UMAP per marker for each of the B cell subtypes. Note that we can only plot the markers that are present in our data.

for ct in B_plasma_cts:
    print(f"{ct.upper()}:")  # print cell subtype name
    sc.pl.umap(
        adata,
        color=marker_genes_in_data[ct],
        vmin=0,
        vmax="p99",  # set vmax to the 99th percentile of the gene count instead of the maximum, to prevent outliers from making expression in other cells invisible. Note that this can cause problems for extremely lowly expressed genes.
        sort_order=False,  # do not plot highest expression on top, to not get a biased view of the mean expression among cells
        frameon=False,
        cmap="Reds",  # or choose another color map e.g. from here: https://matplotlib.org/stable/tutorials/colors/colormaps.html
    )
    print("\n\n\n")  # print white space for legibility

NAIVE CD20+ B:





B1 B:





TRANSITIONAL B:





PLASMA CELLS:





PLASMABLAST:

Even markers for a single cell type are often expressed in different subsets of the data, i.e. individual markers are often not uniquely expressed in a single cell type. Rather, it is the intersection of those subsets that will tell you where your cell type of interest is.

Of note is that markers are often sparsely expressed, i.e., it is often only a subset of cells of a cell type in which a marker was detected. This is due to the nature of scRNA-seq data: we only sequence a small subset of the total amount of RNA molecules in the cell, and due to this subsampling, we will sometimes not sample transcripts from specific genes in a cell even if they were expressed in that cell. Therefore, we do not annotate single cells based on a minimum expression threshold of, e.g., a set of markers. Instead, we first subdivide the data into groups of similar cells (i.e., “partition” the data) by clustering, thereby accounting for “missing transcripts” of single genes and rather grouping based on overall transcriptomic similarity. We can then annotate those clusters based on their overall marker expression patterns.

Let us cluster our data now. We will use the Leiden algorithm Traag et al., 2019 as discussed in the Clustering chapter to define a grouping of our data into similar subsets of cells:

sc.tl.leiden(adata, resolution=1, key_added="leiden_1", flavor="igraph", n_iterations=2)

sc.pl.umap(adata, color="leiden_1")

In case you want the partitioning to be finer, you can change the resolution to a higher level by changing the resolution parameter of the clustering:

sc.tl.leiden(adata, resolution=2, key_added="leiden_2", flavor="igraph", n_iterations=2)

You can also add the cluster numbers to the UMAP.

sc.pl.umap(adata, color="leiden_2", legend_loc="on data")

This clustering is much finer and, in some cases, will help you annotate the data with more detail. You can play around with the resolution parameter to find the setting that best captures the marker expression patterns you observe. In our case, we will stick to the original resolution:

sc.pl.umap(adata, color="leiden_1", legend_loc="on data")

Scrolling back up, you will see that cluster 11 consistently expresses Naive CD20+ B cell markers, while cluster 12 consistently expresses Transitional B cell markers.

We can also visualize this using a dotplot:

B_plasma_markers = {
    ct: [m for m in ct_markers if m in adata.var.index]
    for ct, ct_markers in marker_genes.items()
    if ct in B_plasma_cts
}

sc.pl.dotplot(
    adata,
    groupby="leiden_1",
    var_names=B_plasma_markers,
    standard_scale="var",  # standard scale: normalize each gene to range from 0 to 1
)

Using a combination of visual inspection of the UMAPs and the dotplot above we can now start annotating the clusters:

cl_annotation = {
    "11": "Naive CD20+ B",
    "12": "Transitional B",
}

You might notice that the annotation of B1 B cells is difficult, with none of the clusters expressing all the B1 B markers and several clusters expressing some of the markers. We often see that markers that work for one dataset do not work as well for others. This can be due to differences in sequencing depth, but also due to other sources of variation between datasets or samples.

Let’s visualize our annotations so far:

adata.obs["manual_celltype_annotation"] = adata.obs.leiden_1.map(cl_annotation)

sc.pl.umap(adata, color=["manual_celltype_annotation"])

... storing 'manual_celltype_annotation' as categorical

From cluster differentially expressed genes to cluster annotation¶

Conversely, we can calculate marker genes per cluster and then look up whether we can link those marker genes to any known biology, such as cell types and/or states. For marker gene calculation of clusters, simple methods such as the Wilcoxon rank-sum test are thought to perform best Pullin & McCarthy, 2022. Importantly, as the definition of the clusters is based on the same data as used for these statistical tests, the p-values of these tests will be inflated as also described here Zhang et al., 2019.

Let’s calculate the differentially expressed genes for every cluster, compared to the rest of the cells in our adata:

sc.tl.rank_genes_groups(
    adata, groupby="leiden_1", method="wilcoxon", key_added="dea_leiden_1"
)

We can visualize expression of the top differentially expressed genes per cluster with a standard scanpy dotplot:

sc.tl.dendrogram(
    adata,
    groupby="leiden_1",
)

sc.pl.rank_genes_groups_dotplot(
    adata, groupby="leiden_1", standard_scale="var", n_genes=5, key="dea_leiden_1"
)

As you can see above, a lot of the differentially expressed genes are highly expressed in multiple clusters. We can filter the differentially expressed genes to select for more cluster-specific differentially expressed genes:

sc.tl.filter_rank_genes_groups(
    adata,
    min_in_group_fraction=0.2,
    max_out_group_fraction=0.2,
    key="dea_leiden_1",
    key_added="dea_leiden_1_filtered",
)

Visualize the filtered genes :

sc.pl.rank_genes_groups_dotplot(
    adata,
    groupby="leiden_1",
    standard_scale="var",
    n_genes=5,
    key="dea_leiden_1_filtered",
)

Let’s take a look at cluster 1, which seems to have a set of relatively unique markers including MYOM2, KLRD1 and KLRF1. Some googling tells us that e.g. KLRD1 is a marker for NK cells Isola et al., 2021. In the UMAP we can see that these genes are expressed throughout cluster 1:

sc.pl.umap(
    adata,
    color=["MYOM2", "KLRD1", "KLRF1", "leiden_1"],
    vmax="p99",
    legend_loc="on data",
    frameon=False,
    cmap="Reds",
)

However, marker-based annotation can be sensitive to the cluster resolution you choose, the robustness and uniqueness of the marker sets you have, and your knowledge of the cell types to be expected in your data. Therefore, cluster annotations are not guaranteed to be definitive and should be interpreted with caution.

For this reason, the field is partly trying to move away from manual cluster annotation and rather moving towards automated annotation algorithms instead. The rest of this tutorial will focus on those options.

Before we move on, store the final bit of annotation information in our adata:

cl_annotation["1"] = "NK cells (?)"

adata.obs["manual_celltype_annotation"] = adata.obs.leiden_1.map(cl_annotation)

Automated annotation¶

General remarks¶

The remainder of the discussed methods will be methods for automated, rather than manual annotation of your data. Automated approaches are based on different principles, sometimes requiring pre-defined sets of markers, other times trained on pre-existing full scRNA-seq datasets. As discussed below, the resulting annotations can be of varying quality. It is therefore important to regard these methods as a starting point rather than an end-point of the annotation process. See also several reviews Pasquini et al., 2021, Abdelaal et al., 2019 for a more elaborate discussion of automated annotation methods.

The quality of automatically generated annotations can vary substantially. More specifically, the quality of the annotations depends on:

The type of classifier chosen: Previous benchmark studies have shown that different types of classifiers often perform comparably, with neural network-based methods generally not outperforming general-purpose models such as support vector machines or linear regression modelsAbdelaal et al., 2019, Pasquini et al., 2021, Huang & Zhang, 2021.
The quality of the data that the classifier was trained on. If the training data was not well annotated or annotated at low resolution, the classifier will do the same. Similarly, if the training data and/or its annotation was noisy, the classifier might not perform well.
The similarity of your own data to the data that the classifier was trained on. For example, if the classifier was trained on a drop-seq single cell dataset and your data is 10X single nucleus rather than single cell drop-seq, this might worsen the quality of the annotation. Classifiers trained on cross-dataset atlases including a diversity of datasets might give more robust and better quality annotations than classifiers trained on a single dataset. An example is the CellTypist (an automated annotation method that will be discussed more extensively below) classifier trained on the Human Lung Cell Atlas Sikkema et al., 2023 which includes 14 different lung datasets. This model is likely to perform better on new lung data than a model that was trained on a single lung dataset.

The aforementioned points highlight possible disadvantages of using classifiers, depending on the training data and model type. Nonetheless, there are several important advantages of using pre-trained classifiers to annotate your data. First, it is a fast and easy way to annotate your data. The annotation does not require the downloading nor preprocessing of the training data and sometimes merely involves the upload of your data to an online webpage. Second, these methods don’t rely on a partitioning of your data into clusters, as the manual annotation does. Third, pre-trained classifiers enable you to directly leverage the knowledge and information from previous studies, such as a high quality annotation. And finally, using such classifiers can help with harmonizing cell-type definitions across a field, thereby clearing the path towards a field-wide consensus on these definitions.

Finally, as these classifiers are often less transparent than e.g. manual marker-based annotation, a good uncertainty measure quantifying annotation uncertainty will improve the quality and usability of the method. We will discuss this more extensively further down.

Marker gene-based classifiers¶

One class of automated cell type annotation methods relies on a predefined set of marker genes. Cells are classified into cell types based on their expression levels of these marker genes. Examples of such methods are Garnett Pliner et al., 2019 and CellAssign Zhang et al., 2019. The more robust and generalizable the set of marker genes these models are based on, the better the model will perform. However, like with other models they are likely to be affected by batch effect-related differences between the data the model was trained on and the data that needs to be labeled. One of the advantages of these methods compared to models based on larger gene sets (see below) is that they are more transparent: we know on the basis of which genes the classification is done.
We will not show an example of marker-based classifiers in this notebook, but encourage you to explore these yourself if you are interested.

Classifiers based on a wider set of genes¶

It is worth noting that the methods discussed so far use only a small subset of the genes detected in the data: often a set of only 1 to ~10 marker genes per cell type is used. An alternative approach is to use a classifier that takes as input a larger set of genes (several thousands or more), thereby making more use of the breadth of scRNA-seq data. Such classifiers are trained on previously annotated datasets or atlases. Examples of these are CellTypist Conde et al., 2022 (see also https://www.celltypist.org, where data can be uploaded to a portal to get automated cell annotations) and Clustifyr Fu et al., 2020.

Let’s try out CellTypist on our data. Based on the CellTypist tutorial (https://www.celltypist.org/tutorials) we know we need to prepare our data so that counts are normalized to 10,000 counts per cell, then log1p-transformed:

adata_celltypist = adata.copy()  # make a copy of our adata
adata_celltypist.X = adata.layers["counts"].copy()  # set adata.X to raw counts
sc.pp.normalize_total(
    adata_celltypist, target_sum=10**4
)  # normalize to 10,000 counts per cell
sc.pp.log1p(adata_celltypist)  # log-transform
# make .X dense instead of sparse, for compatibility with celltypist:
adata_celltypist.X = adata_celltypist.X.toarray()

We’ll now download the celltypist models for immune cells:

models.download_models(
    force_update=True, model=["Immune_All_Low.pkl", "Immune_All_High.pkl"]
)

📜 Retrieving model list from server https://celltypist.cog.sanger.ac.uk/models/models.json

📚 Total models in list: 61

📂 Storing models in /home/lheumos/.celltypist/data/models

💾 Total models to download: 2

💾 Downloading model [1/2]: Immune_All_Low.pkl

💾 Downloading model [2/2]: Immune_All_High.pkl

Let’s try out both the Immune_All_Low and Immune_All_High models (these annotate immune cell types finer annotation level (low) and coarser (high)):

model_low = models.Model.load(model="Immune_All_Low.pkl")
model_high = models.Model.load(model="Immune_All_High.pkl")

/home/lheumos/miniforge3/envs/annotation/lib/python3.14/site-packages/sklearn/base.py:525: InconsistentVersionWarning: Trying to unpickle estimator LogisticRegression from version 0.24.1 when using version 1.9.1. This might lead to breaking code or invalid results. Use at your own risk. For more info please refer to:
https://scikit-learn.org/stable/model_persistence.html#security-maintainability-limitations
  warnings.warn(
/home/lheumos/miniforge3/envs/annotation/lib/python3.14/site-packages/sklearn/base.py:525: InconsistentVersionWarning: Trying to unpickle estimator StandardScaler from version 0.24.1 when using version 1.9.1. This might lead to breaking code or invalid results. Use at your own risk. For more info please refer to:
https://scikit-learn.org/stable/model_persistence.html#security-maintainability-limitations
  warnings.warn(

For each of these, we can see which cell types it includes to see if bone marrow cell types are included:

model_high.cell_types

array(['B cells', 'B-cell lineage', 'Cycling cells', 'DC', 'DC precursor',
       'Double-negative thymocytes', 'Double-positive thymocytes', 'ETP',
       'Early MK', 'Endothelial cells', 'Epithelial cells',
       'Erythrocytes', 'Erythroid', 'Fibroblasts', 'Granulocytes',
       'HSC/MPP', 'ILC', 'ILC precursor', 'MNP', 'Macrophages',
       'Mast cells', 'Megakaryocyte precursor',
       'Megakaryocytes/platelets', 'Mono-mac', 'Monocyte precursor',
       'Monocytes', 'Myelocytes', 'Plasma cells', 'Promyelocytes',
       'T cells', 'pDC', 'pDC precursor'], dtype=object)

model_low.cell_types

array(['Age-associated B cells', 'Alveolar macrophages', 'B cells',
       'CD16+ NK cells', 'CD16- NK cells', 'CD8a/a', 'CD8a/b(entry)',
       'CMP', 'CRTAM+ gamma-delta T cells', 'Classical monocytes',
       'Cycling B cells', 'Cycling DCs', 'Cycling NK cells',
       'Cycling T cells', 'Cycling gamma-delta T cells',
       'Cycling monocytes', 'DC', 'DC precursor', 'DC1', 'DC2', 'DC3',
       'Double-negative thymocytes', 'Double-positive thymocytes', 'ELP',
       'ETP', 'Early MK', 'Early erythroid', 'Early lymphoid/T lymphoid',
       'Endothelial cells', 'Epithelial cells', 'Erythrocytes',
       'Erythrophagocytic macrophages', 'Fibroblasts',
       'Follicular B cells', 'Follicular helper T cells', 'GMP',
       'Germinal center B cells', 'Granulocytes', 'HSC/MPP',
       'Hofbauer cells', 'ILC', 'ILC precursor', 'ILC1', 'ILC2', 'ILC3',
       'Intermediate macrophages', 'Intestinal macrophages',
       'Kidney-resident macrophages', 'Kupffer cells',
       'Large pre-B cells', 'Late erythroid', 'MAIT cells', 'MEMP', 'MNP',
       'Macrophages', 'Mast cells', 'Megakaryocyte precursor',
       'Megakaryocyte-erythroid-mast cell progenitor',
       'Megakaryocytes/platelets', 'Memory B cells',
       'Memory CD4+ cytotoxic T cells', 'Mid erythroid', 'Migratory DCs',
       'Mono-mac', 'Monocyte precursor', 'Monocytes', 'Myelocytes',
       'NK cells', 'NKT cells', 'Naive B cells',
       'Neutrophil-myeloid progenitor', 'Neutrophils',
       'Non-classical monocytes', 'Plasma cells', 'Plasmablasts',
       'Pre-pro-B cells', 'Pro-B cells',
       'Proliferative germinal center B cells', 'Promyelocytes',
       'Regulatory T cells', 'Small pre-B cells', 'T(agonist)',
       'Tcm/Naive cytotoxic T cells', 'Tcm/Naive helper T cells',
       'Tem/Effector helper T cells', 'Tem/Effector helper T cells PD1+',
       'Tem/Temra cytotoxic T cells', 'Tem/Trm cytotoxic T cells',
       'Transitional B cells', 'Transitional DC', 'Transitional NK',
       'Treg(diff)', 'Trm cytotoxic T cells', 'Type 1 helper T cells',
       'Type 17 helper T cells', 'gamma-delta T cells', 'pDC',
       'pDC precursor'], dtype=object)

Looks like the models include many different immune cell type progenitors!

Now let’s run the models. First the coarse one:

predictions_high = celltypist.annotate(
    adata_celltypist, model=model_high, majority_voting=True
)

🔬 Input data has 13668 cells and 20362 genes

🔗 Matching reference genes in the model

🧬 5321 features used for prediction

⚖️ Scaling input data

🖋️ Predicting labels

✅ Prediction done!

👀 Detected a neighborhood graph in the input object, will run over-clustering on the basis of it

⛓️ Over-clustering input data with resolution set to 10

/home/lheumos/miniforge3/envs/annotation/lib/python3.14/site-packages/celltypist/classifier.py:457: FutureWarning: The `igraph` implementation of leiden clustering is *orders of magnitude faster*. Set the flavor argument to (and install if needed) 'igraph' to use it.
In the future, the default backend for leiden will be igraph instead of leidenalg. To achieve the future defaults please pass: `flavor='igraph'` and `n_iterations=2`. `directed` must also be `False` to work with igraph’s implementation.
  sc.tl.leiden(self.adata, resolution=resolution, key_added='over_clustering')

🗳️ Majority voting the predictions

✅ Majority voting done!

Transform the predictions to adata to get the full output...

predictions_high_adata = predictions_high.to_adata()

...and copy the results to our original AnnData object:

adata.obs["celltypist_cell_label_coarse"] = predictions_high_adata.obs.loc[
    adata.obs.index, "majority_voting"
]
adata.obs["celltypist_conf_score_coarse"] = predictions_high_adata.obs.loc[
    adata.obs.index, "conf_score"
]

Now the same for the finer annotations:

predictions_low = celltypist.annotate(
    adata_celltypist, model=model_low, majority_voting=True
)

🔬 Input data has 13668 cells and 20362 genes

🔗 Matching reference genes in the model

🧬 5321 features used for prediction

⚖️ Scaling input data

🖋️ Predicting labels

✅ Prediction done!

👀 Detected a neighborhood graph in the input object, will run over-clustering on the basis of it

⛓️ Over-clustering input data with resolution set to 10

/home/lheumos/miniforge3/envs/annotation/lib/python3.14/site-packages/celltypist/classifier.py:457: FutureWarning: The `igraph` implementation of leiden clustering is *orders of magnitude faster*. Set the flavor argument to (and install if needed) 'igraph' to use it.
In the future, the default backend for leiden will be igraph instead of leidenalg. To achieve the future defaults please pass: `flavor='igraph'` and `n_iterations=2`. `directed` must also be `False` to work with igraph’s implementation.
  sc.tl.leiden(self.adata, resolution=resolution, key_added='over_clustering')

🗳️ Majority voting the predictions

✅ Majority voting done!

predictions_low_adata = predictions_low.to_adata()

adata.obs["celltypist_cell_label_fine"] = predictions_low_adata.obs.loc[
    adata.obs.index, "majority_voting"
]
adata.obs["celltypist_conf_score_fine"] = predictions_low_adata.obs.loc[
    adata.obs.index, "conf_score"
]

Now plot:

sc.pl.umap(
    adata,
    color=["celltypist_cell_label_coarse", "celltypist_conf_score_coarse"],
    frameon=False,
    sort_order=False,
    wspace=1,
)

... storing 'manual_celltype_annotation' as categorical

sc.pl.umap(
    adata,
    color=["celltypist_cell_label_fine", "celltypist_conf_score_fine"],
    frameon=False,
    sort_order=False,
    wspace=1,
)

One way of getting a feeling for the quality of these annotations is by looking if the observed cell type similarities correspond to our expectations:

sc.tl.dendrogram(
    adata,
    groupby="celltypist_cell_label_fine",
)

sc.pl.dendrogram(adata, groupby="celltypist_cell_label_fine")

<Axes: >

This dendrogram shows us that the cells are well clustered with each other (e.g., B cells largely clustering together). It is very important to check the automated annotation manually before simply taking them!

Cross-checking automated annotations with manual annotations

Dendrograms often reveal unexpected patterns, such as a specific B cell subtype failing to cluster with other B cells. This discrepancy often suggests inaccurate automated labeling.

To diagnose these cases:

Validate the clusters using alternative annotation sources or known marker genes.
Check confidence scores of the automated annotations. Low scores in “misplaced” clusters are a strong signal that manual intervention is required.

You can also take a look at the breakdown of a specific cluster to investigate:

pd.crosstab(adata.obs.leiden_1, adata.obs.celltypist_cell_label_fine).loc[
    "8", :
].sort_values(ascending=False)

Automated annotations may only partially correspond to manual labels and can even be inconsistent between their own coarse and fine classifications.

This underlines that automated annotation algorithms should be used with caution and should be regarded as a starting point for annotating your data, rather than as a final annotation. Ultimately, expression of known marker genes is still the most accepted support for a cell type annotation.

Large language models and foundation models¶

Large language models (LLMs) can propose cell type labels directly from the marker genes of each cluster without requiring a reference dataset Hou & Ji, 2024. Benchmarks found that large LLMs agree well with manual annotations of major cell types Hou & Ji, 2024, Crowley et al., 2025, but agreement is lower for subtypes Hou & Ji, 2024, differs substantially between models Crowley et al., 2025 and depends on how the input marker genes were selected Xiao et al., 2025. As the training data of these models is undisclosed and they can hallucinate, LLM-based labels are best used as a quick first pass that must be checked against marker gene expression and reference annotations. Since LLMs rely on previously described knowledge, labels for novel or context-specific cell states should be treated with particular caution. Single-cell foundation models are also proposed for annotation, but independent evaluations found that their zero-shot embeddings separate cell types less consistently than highly variable genes, scVI or Harmony Kedzierska et al., 2025 and that a simple logistic regression performs comparably to a fine-tuned foundation model Boiarsky et al., 2024. We therefore recommend the reference-based and marker-based approaches discussed in this chapter and regard LLMs and foundation models as, at most, an additional first pass.

Annotation by mapping to a reference¶

A final way to annotate your data is based on mapping your data to an existing, annotated single-cell reference and then performing label transfer using the resulting joint embedding. This reference can for example be a single sample that you annotated manually before, after which you would like to transfer those annotations to the rest of your dataset. Alternatively, it can be a published and ideally well-curated existing reference. In this context we refer to the “new data”, i.e. the data to be mapped and annotated, as the “query”.

There are multiple existing methods that perform such “query-to-reference mapping”, including scArches Lotfollahi et al., 2022, Symphony Kang et al., 2021, and Azimuth (Seurat) Hao et al., 2021. All of these methods enable you to map a new dataset to an existing reference without needing to reintegrate the data from the reference and without needing access to the full reference data.

As query-to-reference mapping involves embedding new data into an existing low-dimensional representation of the reference data, the dimensions and axes of that low-dimensional representation are largely pre-defined before learning from the query. Therefore, learning and incorporating unseen variation that might be present in the query (both new biological variation, e.g. unseen cell types or states and new technical variation, i.e. unseen batch effects that need to be removed) can be a challenge for these models. As a result, integration of the query data with the reference might not always be optimal and batch effects might not be fully removed from the joint query-reference embedding. However, as cell type label transfer does not necessarily require perfect integration but merely close proximity of identical cell types in the embedding, even an imperfect mapping can still be extremely helpful in annotating your data.

We will be using cellmapper Lange, 2025, specifically with scArches as the kernel method for reference-mapping-based label transfer to the new data ()“query”). This takes as its basis an existing (variational autoencoder-based) model that embeds the reference data in a low-dimensional, batch-corrected space. It then slightly extends that model to enable the mapping of an unseen dataset into the same “latent space” (i.e. the low-dimensional embedding). This model extension also enables the learning and removal of batch effects present in the mapped dataset.

Let’s start by preparing our data for the mapping to a reference. scArches, the method that enables us to adapt an existing reference model to new data, requires raw, non-normalized counts. We will therefore keep our counts layer and remove all other layers from our adata to map. We will set our .X to those raw counts as well.

adata_to_map = adata.copy()
for layer in list(adata_to_map.layers.keys()):
    if layer != "counts":
        del adata_to_map.layers[layer]
adata_to_map.X = adata_to_map.layers["counts"]

Moreover, it is important that we use the same input features (i.e. genes) as were used for training our reference model and that we put those features in the same order. The reference model’s feature information is stored together with the model. Let’s load the feature table.

af = ln.Artifact.connect("theislab/sc-best-practices").get(
    key="cellular_structure/annotation_reference_features.csv", is_latest=True
)
reference_model_features = af.load(index_col=0)

The table has both gene names and gene IDs. As gene IDs are usually less subject to change over genome annotation versions, we will use those to subset our data. We will therefore set our row names for both our adata and the reference model features to gene_ids. Importantly, we have to make sure to also store the gene names for later use: these are much easier to understand than the gene IDs.

adata_to_map.var["gene_names"] = adata_to_map.var.index
adata_to_map.var.set_index("gene_ids", inplace=True)

reference_model_features["gene_names"] = reference_model_features.index
reference_model_features.set_index("gene_ids", inplace=True)

Now, let’s check if we have all the genes we need in our query data:

print("Total number of genes needed for mapping:", reference_model_features.shape[0])

Total number of genes needed for mapping: 4000

print(
    "Number of genes found in query dataset:",
    adata_to_map.var.index.isin(reference_model_features.index).sum(),
)

Number of genes found in query dataset: 3553

We are missing some genes. These genes were detected in fewer than 20 cells and therefore removed during quality control, so we add them back with counts of 0. Let’s create an AnnData object for those missing genes with only zero values (including our raw counts layer, which will be used for the mapping). We will concatenate that to our own AnnData objects afterwards.

missing_genes = [
    gene_id
    for gene_id in reference_model_features.index
    if gene_id not in adata_to_map.var.index
]

missing_gene_adata = sc.AnnData(
    X=csr_matrix(np.zeros(shape=(adata.n_obs, len(missing_genes))), dtype="float32"),
    obs=adata.obs.iloc[:, :1],
    var=reference_model_features.loc[missing_genes, :],
)
missing_gene_adata.layers["counts"] = missing_gene_adata.X

Concatenate our original adata to the missing genes adata. To make sure we can do this concatenation without errors, we’ll remove the PCA matrix from varm.

if "PCs" in adata_to_map.varm.keys():
    del adata_to_map.varm["PCs"]

adata_to_map_augmented = sc.concat(
    [adata_to_map, missing_gene_adata],
    axis=1,
    join="outer",
    index_unique=None,
    merge="unique",
)

Now subset to the genes used in the model and order correctly:

adata_to_map_augmented = adata_to_map_augmented[
    :, reference_model_features.index
].copy()

Check if our adata gene names correspond exactly to the required gene order:

bool((adata_to_map_augmented.var.index == reference_model_features.index).all())

True

We can now set the gene indices back to gene names for easy interpretation:

adata_to_map_augmented.var["gene_ids"] = adata_to_map_augmented.var.index
adata_to_map_augmented.var.set_index("gene_names", inplace=True)

Finally, this reference model used adata.obs['batch'] as its batch variable, so we set it to the name of our sample:

adata_to_map_augmented.obs["batch"] = "s4d8"

Now let’s talk about our reference model. The better our reference model, the better our label transfer will perform. Using well-annotated reference that integrates many different datasets and that matches your data well (same organ, same single-cell technology etc.) is ideal: such models are trained on a variety of datasets and batches and are therefore expected to be more robust to batch effects. However, such references do not exist yet for all tissues. For this tutorial we will use a reference model trained on the bone marrow samples that we have been using throughout the book, excluding the sample that we will be mapping. The reference model is an scvi-model (used for data integration) that generates a low-dimensional, integrated embedding of the input data, see also the scvi-publication Lopez et al., 2018. Note that this is a toy model generated for this tutorial and it should not be used in other contexts.

We will start by patching things up for compatibility, because this model was built on previous version of pandas.

sys.modules["pandas.core.indexes.numeric"] = pandas_indexes_base
pandas_indexes_base.Int64Index = pd.Index
pandas_indexes_base.Float64Index = pd.Index

Now, let’s load the model and pass it the adata which we want to map.

af = ln.Artifact.connect("theislab/sc-best-practices").get(
    key="cellular_structure/annotation_reference_model.pt", is_latest=True
)
annotation_ref_model_path = af.cache()

model_dir = Path("./reference_model")
model_dir.mkdir(parents=True, exist_ok=True)

shutil.copy(annotation_ref_model_path, model_dir / "model.pt")

scvi.settings.seed = 0
query_model = scvi.model.SCVI.load_query_data(
    adata=adata_to_map_augmented,
    reference_model=str(model_dir),
    freeze_dropout=True,
)

INFO: Seed set to 0

Seed set to 0

INFO: Trainer will use only 1 of 2 GPUs because it is running inside an interactive / notebook environment. You may try to set `Trainer(devices=2)` but please note that multi-GPU inside interactive / notebook environments is considered experimental and unstable. Your mileage may vary.

Trainer will use only 1 of 2 GPUs because it is running inside an interactive / notebook environment. You may try to set `Trainer(devices=2)` but please note that multi-GPU inside interactive / notebook environments is considered experimental and unstable. Your mileage may vary.

INFO     File reference_model/model.pt already downloaded

We will now update this reference model so that we can embed our own data (the “query”) in the same latent space as the reference. This requires training on our query data:

query_model.train(max_epochs=500, plan_kwargs={"weight_decay": 0.0})

INFO: Trainer will use only 1 of 2 GPUs because it is running inside an interactive / notebook environment. You may try to set `Trainer(devices=2)` but please note that multi-GPU inside interactive / notebook environments is considered experimental and unstable. Your mileage may vary.

Trainer will use only 1 of 2 GPUs because it is running inside an interactive / notebook environment. You may try to set `Trainer(devices=2)` but please note that multi-GPU inside interactive / notebook environments is considered experimental and unstable. Your mileage may vary.

INFO: GPU available: True (cuda), used: True

GPU available: True (cuda), used: True

INFO: TPU available: False, using: 0 TPU cores

TPU available: False, using: 0 TPU cores

INFO: 💡 Tip: For seamless cloud logging and experiment tracking, try installing [litlogger](https://pypi.org/project/litlogger/) to enable LitLogger, which logs metrics and artifacts automatically to the Lightning Experiments platform.

💡 Tip: For seamless cloud logging and experiment tracking, try installing [litlogger](https://pypi.org/project/litlogger/) to enable LitLogger, which logs metrics and artifacts automatically to the Lightning Experiments platform.

INFO: You are using a CUDA device ('NVIDIA RTX PRO 6000 Blackwell Workstation Edition') that has Tensor Cores. To properly utilize them, you should set `torch.set_float32_matmul_precision('medium' | 'high')` which will trade-off precision for performance. For more details, read https://pytorch.org/docs/stable/generated/torch.set_float32_matmul_precision.html#torch.set_float32_matmul_precision

You are using a CUDA device ('NVIDIA RTX PRO 6000 Blackwell Workstation Edition') that has Tensor Cores. To properly utilize them, you should set `torch.set_float32_matmul_precision('medium' | 'high')` which will trade-off precision for performance. For more details, read https://pytorch.org/docs/stable/generated/torch.set_float32_matmul_precision.html#torch.set_float32_matmul_precision

INFO: LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0,1]

LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0,1]

/home/lheumos/miniforge3/envs/annotation/lib/python3.14/site-packages/lightning/pytorch/utilities/_pytree.py:21: `isinstance(treespec, LeafSpec)` is deprecated, use `isinstance(treespec, TreeSpec) and treespec.is_leaf()` instead.
/home/lheumos/miniforge3/envs/annotation/lib/python3.14/site-packages/lightning/pytorch/trainer/connectors/data_connector.py:434: The 'train_dataloader' does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` to `num_workers=63` in the `DataLoader` to improve performance.

/home/lheumos/miniforge3/envs/annotation/lib/python3.14/site-packages/lightning/pytorch/loops/fit_loop.py:538: Found 42 module(s) in eval mode at the start of training. This may lead to unexpected behavior during training. If this is intentional, you can ignore this warning.

Training:   0%|          | 0/500 [00:00<?, ?it/s]

Epoch 1/500:   0%|          | 0/500 [00:00<?, ?it/s]

Epoch 1/500:   0%|          | 1/500 [00:00<06:20,  1.31it/s]

Epoch 1/500:   0%|          | 1/500 [00:00<06:20,  1.31it/s, v_num=1, train_loss=913]

Epoch 2/500:   0%|          | 1/500 [00:00<06:20,  1.31it/s, v_num=1, train_loss=913]

Epoch 2/500:   0%|          | 2/500 [00:01<05:00,  1.66it/s, v_num=1, train_loss=913]

Epoch 2/500:   0%|          | 2/500 [00:01<05:00,  1.66it/s, v_num=1, train_loss=902]

Epoch 3/500:   0%|          | 2/500 [00:01<05:00,  1.66it/s, v_num=1, train_loss=902]

Epoch 3/500:   1%|          | 3/500 [00:01<04:27,  1.86it/s, v_num=1, train_loss=902]

Epoch 3/500:   1%|          | 3/500 [00:01<04:27,  1.86it/s, v_num=1, train_loss=896]

Epoch 4/500:   1%|          | 3/500 [00:01<04:27,  1.86it/s, v_num=1, train_loss=896]

Epoch 4/500:   1%|          | 4/500 [00:02<04:11,  1.97it/s, v_num=1, train_loss=896]

Epoch 4/500:   1%|          | 4/500 [00:02<04:11,  1.97it/s, v_num=1, train_loss=892]

Epoch 5/500:   1%|          | 4/500 [00:02<04:11,  1.97it/s, v_num=1, train_loss=892]

Epoch 5/500:   1%|          | 5/500 [00:02<04:01,  2.05it/s, v_num=1, train_loss=892]

Epoch 5/500:   1%|          | 5/500 [00:02<04:01,  2.05it/s, v_num=1, train_loss=889]

Epoch 6/500:   1%|          | 5/500 [00:02<04:01,  2.05it/s, v_num=1, train_loss=889]

Epoch 6/500:   1%|          | 6/500 [00:03<03:56,  2.09it/s, v_num=1, train_loss=889]

Epoch 6/500:   1%|          | 6/500 [00:03<03:56,  2.09it/s, v_num=1, train_loss=888]

Epoch 7/500:   1%|          | 6/500 [00:03<03:56,  2.09it/s, v_num=1, train_loss=888]

Epoch 7/500:   1%|▏         | 7/500 [00:03<03:54,  2.10it/s, v_num=1, train_loss=888]

Epoch 7/500:   1%|▏         | 7/500 [00:03<03:54,  2.10it/s, v_num=1, train_loss=886]

Epoch 8/500:   1%|▏         | 7/500 [00:03<03:54,  2.10it/s, v_num=1, train_loss=886]

Epoch 8/500:   2%|▏         | 8/500 [00:04<03:50,  2.14it/s, v_num=1, train_loss=886]

Epoch 8/500:   2%|▏         | 8/500 [00:04<03:50,  2.14it/s, v_num=1, train_loss=885]

Epoch 9/500:   2%|▏         | 8/500 [00:04<03:50,  2.14it/s, v_num=1, train_loss=885]

Epoch 9/500:   2%|▏         | 9/500 [00:04<03:45,  2.18it/s, v_num=1, train_loss=885]

Epoch 9/500:   2%|▏         | 9/500 [00:04<03:45,  2.18it/s, v_num=1, train_loss=885]

Epoch 10/500:   2%|▏         | 9/500 [00:04<03:45,  2.18it/s, v_num=1, train_loss=885]

Epoch 10/500:   2%|▏         | 10/500 [00:04<03:41,  2.21it/s, v_num=1, train_loss=885]

Epoch 10/500:   2%|▏         | 10/500 [00:04<03:41,  2.21it/s, v_num=1, train_loss=884]

Epoch 11/500:   2%|▏         | 10/500 [00:04<03:41,  2.21it/s, v_num=1, train_loss=884]

Epoch 11/500:   2%|▏         | 11/500 [00:05<03:35,  2.27it/s, v_num=1, train_loss=884]

Epoch 11/500:   2%|▏         | 11/500 [00:05<03:35,  2.27it/s, v_num=1, train_loss=884]

Epoch 12/500:   2%|▏         | 11/500 [00:05<03:35,  2.27it/s, v_num=1, train_loss=884]

Epoch 12/500:   2%|▏         | 12/500 [00:05<03:28,  2.34it/s, v_num=1, train_loss=884]

Epoch 12/500:   2%|▏         | 12/500 [00:05<03:28,  2.34it/s, v_num=1, train_loss=883]

Epoch 13/500:   2%|▏         | 12/500 [00:05<03:28,  2.34it/s, v_num=1, train_loss=883]

Epoch 13/500:   3%|▎         | 13/500 [00:06<03:24,  2.38it/s, v_num=1, train_loss=883]

Epoch 13/500:   3%|▎         | 13/500 [00:06<03:24,  2.38it/s, v_num=1, train_loss=883]

Epoch 14/500:   3%|▎         | 13/500 [00:06<03:24,  2.38it/s, v_num=1, train_loss=883]

Epoch 14/500:   3%|▎         | 14/500 [00:06<03:22,  2.40it/s, v_num=1, train_loss=883]

Epoch 14/500:   3%|▎         | 14/500 [00:06<03:22,  2.40it/s, v_num=1, train_loss=883]

Epoch 15/500:   3%|▎         | 14/500 [00:06<03:22,  2.40it/s, v_num=1, train_loss=883]

Epoch 15/500:   3%|▎         | 15/500 [00:06<03:22,  2.39it/s, v_num=1, train_loss=883]

Epoch 15/500:   3%|▎         | 15/500 [00:06<03:22,  2.39it/s, v_num=1, train_loss=882]

Epoch 16/500:   3%|▎         | 15/500 [00:06<03:22,  2.39it/s, v_num=1, train_loss=882]

Epoch 16/500:   3%|▎         | 16/500 [00:07<03:24,  2.37it/s, v_num=1, train_loss=882]

Epoch 16/500:   3%|▎         | 16/500 [00:07<03:24,  2.37it/s, v_num=1, train_loss=882]

Epoch 17/500:   3%|▎         | 16/500 [00:07<03:24,  2.37it/s, v_num=1, train_loss=882]

Epoch 17/500:   3%|▎         | 17/500 [00:07<03:28,  2.32it/s, v_num=1, train_loss=882]

Epoch 17/500:   3%|▎         | 17/500 [00:07<03:28,  2.32it/s, v_num=1, train_loss=882]

Epoch 18/500:   3%|▎         | 17/500 [00:07<03:28,  2.32it/s, v_num=1, train_loss=882]

Epoch 18/500:   4%|▎         | 18/500 [00:08<03:28,  2.31it/s, v_num=1, train_loss=882]

Epoch 18/500:   4%|▎         | 18/500 [00:08<03:28,  2.31it/s, v_num=1, train_loss=882]

Epoch 19/500:   4%|▎         | 18/500 [00:08<03:28,  2.31it/s, v_num=1, train_loss=882]

Epoch 19/500:   4%|▍         | 19/500 [00:08<03:31,  2.28it/s, v_num=1, train_loss=882]

Epoch 19/500:   4%|▍         | 19/500 [00:08<03:31,  2.28it/s, v_num=1, train_loss=882]

Epoch 20/500:   4%|▍         | 19/500 [00:08<03:31,  2.28it/s, v_num=1, train_loss=882]

Epoch 20/500:   4%|▍         | 20/500 [00:09<03:31,  2.27it/s, v_num=1, train_loss=882]

Epoch 20/500:   4%|▍         | 20/500 [00:09<03:31,  2.27it/s, v_num=1, train_loss=882]

Epoch 21/500:   4%|▍         | 20/500 [00:09<03:31,  2.27it/s, v_num=1, train_loss=882]

Epoch 21/500:   4%|▍         | 21/500 [00:09<03:30,  2.27it/s, v_num=1, train_loss=882]

Epoch 21/500:   4%|▍         | 21/500 [00:09<03:30,  2.27it/s, v_num=1, train_loss=882]

Epoch 22/500:   4%|▍         | 21/500 [00:09<03:30,  2.27it/s, v_num=1, train_loss=882]

Epoch 22/500:   4%|▍         | 22/500 [00:10<03:29,  2.29it/s, v_num=1, train_loss=882]

Epoch 22/500:   4%|▍         | 22/500 [00:10<03:29,  2.29it/s, v_num=1, train_loss=882]

Epoch 23/500:   4%|▍         | 22/500 [00:10<03:29,  2.29it/s, v_num=1, train_loss=882]

Epoch 23/500:   5%|▍         | 23/500 [00:10<03:23,  2.35it/s, v_num=1, train_loss=882]

Epoch 23/500:   5%|▍         | 23/500 [00:10<03:23,  2.35it/s, v_num=1, train_loss=882]

Epoch 24/500:   5%|▍         | 23/500 [00:10<03:23,  2.35it/s, v_num=1, train_loss=882]

Epoch 24/500:   5%|▍         | 24/500 [00:10<03:23,  2.34it/s, v_num=1, train_loss=882]

Epoch 24/500:   5%|▍         | 24/500 [00:10<03:23,  2.34it/s, v_num=1, train_loss=882]

Epoch 25/500:   5%|▍         | 24/500 [00:10<03:23,  2.34it/s, v_num=1, train_loss=882]

Epoch 25/500:   5%|▌         | 25/500 [00:11<03:24,  2.32it/s, v_num=1, train_loss=882]

Epoch 25/500:   5%|▌         | 25/500 [00:11<03:24,  2.32it/s, v_num=1, train_loss=881]

Epoch 26/500:   5%|▌         | 25/500 [00:11<03:24,  2.32it/s, v_num=1, train_loss=881]

Epoch 26/500:   5%|▌         | 26/500 [00:11<03:23,  2.33it/s, v_num=1, train_loss=881]

Epoch 26/500:   5%|▌         | 26/500 [00:11<03:23,  2.33it/s, v_num=1, train_loss=881]

Epoch 27/500:   5%|▌         | 26/500 [00:11<03:23,  2.33it/s, v_num=1, train_loss=881]

Epoch 27/500:   5%|▌         | 27/500 [00:12<03:26,  2.30it/s, v_num=1, train_loss=881]

Epoch 27/500:   5%|▌         | 27/500 [00:12<03:26,  2.30it/s, v_num=1, train_loss=881]

Epoch 28/500:   5%|▌         | 27/500 [00:12<03:26,  2.30it/s, v_num=1, train_loss=881]

Epoch 28/500:   6%|▌         | 28/500 [00:12<03:23,  2.32it/s, v_num=1, train_loss=881]

Epoch 28/500:   6%|▌         | 28/500 [00:12<03:23,  2.32it/s, v_num=1, train_loss=881]

Epoch 29/500:   6%|▌         | 28/500 [00:12<03:23,  2.32it/s, v_num=1, train_loss=881]

Epoch 29/500:   6%|▌         | 29/500 [00:13<03:24,  2.31it/s, v_num=1, train_loss=881]

Epoch 29/500:   6%|▌         | 29/500 [00:13<03:24,  2.31it/s, v_num=1, train_loss=881]

Epoch 30/500:   6%|▌         | 29/500 [00:13<03:24,  2.31it/s, v_num=1, train_loss=881]

Epoch 30/500:   6%|▌         | 30/500 [00:13<03:24,  2.30it/s, v_num=1, train_loss=881]

Epoch 30/500:   6%|▌         | 30/500 [00:13<03:24,  2.30it/s, v_num=1, train_loss=881]

Epoch 31/500:   6%|▌         | 30/500 [00:13<03:24,  2.30it/s, v_num=1, train_loss=881]

Epoch 31/500:   6%|▌         | 31/500 [00:13<03:25,  2.28it/s, v_num=1, train_loss=881]

Epoch 31/500:   6%|▌         | 31/500 [00:13<03:25,  2.28it/s, v_num=1, train_loss=881]

Epoch 32/500:   6%|▌         | 31/500 [00:13<03:25,  2.28it/s, v_num=1, train_loss=881]

Epoch 32/500:   6%|▋         | 32/500 [00:14<03:28,  2.24it/s, v_num=1, train_loss=881]

Epoch 32/500:   6%|▋         | 32/500 [00:14<03:28,  2.24it/s, v_num=1, train_loss=881]

Epoch 33/500:   6%|▋         | 32/500 [00:14<03:28,  2.24it/s, v_num=1, train_loss=881]

Epoch 33/500:   7%|▋         | 33/500 [00:14<03:25,  2.27it/s, v_num=1, train_loss=881]

Epoch 33/500:   7%|▋         | 33/500 [00:14<03:25,  2.27it/s, v_num=1, train_loss=881]

Epoch 34/500:   7%|▋         | 33/500 [00:14<03:25,  2.27it/s, v_num=1, train_loss=881]

Epoch 34/500:   7%|▋         | 34/500 [00:15<03:21,  2.31it/s, v_num=1, train_loss=881]

Epoch 34/500:   7%|▋         | 34/500 [00:15<03:21,  2.31it/s, v_num=1, train_loss=881]

Epoch 35/500:   7%|▋         | 34/500 [00:15<03:21,  2.31it/s, v_num=1, train_loss=881]

Epoch 35/500:   7%|▋         | 35/500 [00:15<03:17,  2.36it/s, v_num=1, train_loss=881]

Epoch 35/500:   7%|▋         | 35/500 [00:15<03:17,  2.36it/s, v_num=1, train_loss=881]

Epoch 36/500:   7%|▋         | 35/500 [00:15<03:17,  2.36it/s, v_num=1, train_loss=881]

Epoch 36/500:   7%|▋         | 36/500 [00:16<03:22,  2.30it/s, v_num=1, train_loss=881]

Epoch 36/500:   7%|▋         | 36/500 [00:16<03:22,  2.30it/s, v_num=1, train_loss=881]

Epoch 37/500:   7%|▋         | 36/500 [00:16<03:22,  2.30it/s, v_num=1, train_loss=881]

Epoch 37/500:   7%|▋         | 37/500 [00:16<03:21,  2.30it/s, v_num=1, train_loss=881]

Epoch 37/500:   7%|▋         | 37/500 [00:16<03:21,  2.30it/s, v_num=1, train_loss=881]

Epoch 38/500:   7%|▋         | 37/500 [00:16<03:21,  2.30it/s, v_num=1, train_loss=881]

Epoch 38/500:   8%|▊         | 38/500 [00:16<03:24,  2.26it/s, v_num=1, train_loss=881]

Epoch 38/500:   8%|▊         | 38/500 [00:16<03:24,  2.26it/s, v_num=1, train_loss=881]

Epoch 39/500:   8%|▊         | 38/500 [00:16<03:24,  2.26it/s, v_num=1, train_loss=881]

Epoch 39/500:   8%|▊         | 39/500 [00:17<03:24,  2.25it/s, v_num=1, train_loss=881]

Epoch 39/500:   8%|▊         | 39/500 [00:17<03:24,  2.25it/s, v_num=1, train_loss=881]

Epoch 40/500:   8%|▊         | 39/500 [00:17<03:24,  2.25it/s, v_num=1, train_loss=881]

Epoch 40/500:   8%|▊         | 40/500 [00:17<03:26,  2.23it/s, v_num=1, train_loss=881]

Epoch 40/500:   8%|▊         | 40/500 [00:17<03:26,  2.23it/s, v_num=1, train_loss=881]

Epoch 41/500:   8%|▊         | 40/500 [00:17<03:26,  2.23it/s, v_num=1, train_loss=881]

Epoch 41/500:   8%|▊         | 41/500 [00:18<03:25,  2.24it/s, v_num=1, train_loss=881]

Epoch 41/500:   8%|▊         | 41/500 [00:18<03:25,  2.24it/s, v_num=1, train_loss=881]

Epoch 42/500:   8%|▊         | 41/500 [00:18<03:25,  2.24it/s, v_num=1, train_loss=881]

Epoch 42/500:   8%|▊         | 42/500 [00:18<03:24,  2.24it/s, v_num=1, train_loss=881]

Epoch 42/500:   8%|▊         | 42/500 [00:18<03:24,  2.24it/s, v_num=1, train_loss=881]

Epoch 43/500:   8%|▊         | 42/500 [00:18<03:24,  2.24it/s, v_num=1, train_loss=881]

Epoch 43/500:   9%|▊         | 43/500 [00:19<03:20,  2.28it/s, v_num=1, train_loss=881]

Epoch 43/500:   9%|▊         | 43/500 [00:19<03:20,  2.28it/s, v_num=1, train_loss=882]

Epoch 44/500:   9%|▊         | 43/500 [00:19<03:20,  2.28it/s, v_num=1, train_loss=882]

Epoch 44/500:   9%|▉         | 44/500 [00:19<03:14,  2.34it/s, v_num=1, train_loss=882]

Epoch 44/500:   9%|▉         | 44/500 [00:19<03:14,  2.34it/s, v_num=1, train_loss=882]

Epoch 45/500:   9%|▉         | 44/500 [00:19<03:14,  2.34it/s, v_num=1, train_loss=882]

Epoch 45/500:   9%|▉         | 45/500 [00:19<03:11,  2.38it/s, v_num=1, train_loss=882]

Epoch 45/500:   9%|▉         | 45/500 [00:19<03:11,  2.38it/s, v_num=1, train_loss=882]

Epoch 46/500:   9%|▉         | 45/500 [00:19<03:11,  2.38it/s, v_num=1, train_loss=882]

Epoch 46/500:   9%|▉         | 46/500 [00:20<03:08,  2.41it/s, v_num=1, train_loss=882]

Epoch 46/500:   9%|▉         | 46/500 [00:20<03:08,  2.41it/s, v_num=1, train_loss=882]

Epoch 47/500:   9%|▉         | 46/500 [00:20<03:08,  2.41it/s, v_num=1, train_loss=882]

Epoch 47/500:   9%|▉         | 47/500 [00:20<03:13,  2.34it/s, v_num=1, train_loss=882]

Epoch 47/500:   9%|▉         | 47/500 [00:20<03:13,  2.34it/s, v_num=1, train_loss=882]

Epoch 48/500:   9%|▉         | 47/500 [00:20<03:13,  2.34it/s, v_num=1, train_loss=882]

Epoch 48/500:  10%|▉         | 48/500 [00:21<03:18,  2.27it/s, v_num=1, train_loss=882]

Epoch 48/500:  10%|▉         | 48/500 [00:21<03:18,  2.27it/s, v_num=1, train_loss=882]

Epoch 49/500:  10%|▉         | 48/500 [00:21<03:18,  2.27it/s, v_num=1, train_loss=882]

Epoch 49/500:  10%|▉         | 49/500 [00:21<03:20,  2.25it/s, v_num=1, train_loss=882]

Epoch 49/500:  10%|▉         | 49/500 [00:21<03:20,  2.25it/s, v_num=1, train_loss=882]

Epoch 50/500:  10%|▉         | 49/500 [00:21<03:20,  2.25it/s, v_num=1, train_loss=882]

Epoch 50/500:  10%|█         | 50/500 [00:22<03:21,  2.24it/s, v_num=1, train_loss=882]

Epoch 50/500:  10%|█         | 50/500 [00:22<03:21,  2.24it/s, v_num=1, train_loss=882]

Epoch 51/500:  10%|█         | 50/500 [00:22<03:21,  2.24it/s, v_num=1, train_loss=882]

Epoch 51/500:  10%|█         | 51/500 [00:22<03:22,  2.22it/s, v_num=1, train_loss=882]

Epoch 51/500:  10%|█         | 51/500 [00:22<03:22,  2.22it/s, v_num=1, train_loss=882]

Epoch 52/500:  10%|█         | 51/500 [00:22<03:22,  2.22it/s, v_num=1, train_loss=882]

Epoch 52/500:  10%|█         | 52/500 [00:23<03:21,  2.22it/s, v_num=1, train_loss=882]

Epoch 52/500:  10%|█         | 52/500 [00:23<03:21,  2.22it/s, v_num=1, train_loss=882]

Epoch 53/500:  10%|█         | 52/500 [00:23<03:21,  2.22it/s, v_num=1, train_loss=882]

Epoch 53/500:  11%|█         | 53/500 [00:23<03:20,  2.23it/s, v_num=1, train_loss=882]

Epoch 53/500:  11%|█         | 53/500 [00:23<03:20,  2.23it/s, v_num=1, train_loss=882]

Epoch 54/500:  11%|█         | 53/500 [00:23<03:20,  2.23it/s, v_num=1, train_loss=882]

Epoch 54/500:  11%|█         | 54/500 [00:24<03:18,  2.24it/s, v_num=1, train_loss=882]

Epoch 54/500:  11%|█         | 54/500 [00:24<03:18,  2.24it/s, v_num=1, train_loss=882]

Epoch 55/500:  11%|█         | 54/500 [00:24<03:18,  2.24it/s, v_num=1, train_loss=882]

Epoch 55/500:  11%|█         | 55/500 [00:24<03:18,  2.24it/s, v_num=1, train_loss=882]

Epoch 55/500:  11%|█         | 55/500 [00:24<03:18,  2.24it/s, v_num=1, train_loss=882]

Epoch 56/500:  11%|█         | 55/500 [00:24<03:18,  2.24it/s, v_num=1, train_loss=882]

Epoch 56/500:  11%|█         | 56/500 [00:24<03:17,  2.25it/s, v_num=1, train_loss=882]

Epoch 56/500:  11%|█         | 56/500 [00:24<03:17,  2.25it/s, v_num=1, train_loss=882]

Epoch 57/500:  11%|█         | 56/500 [00:24<03:17,  2.25it/s, v_num=1, train_loss=882]

Epoch 57/500:  11%|█▏        | 57/500 [00:25<03:15,  2.27it/s, v_num=1, train_loss=882]

Epoch 57/500:  11%|█▏        | 57/500 [00:25<03:15,  2.27it/s, v_num=1, train_loss=882]

Epoch 58/500:  11%|█▏        | 57/500 [00:25<03:15,  2.27it/s, v_num=1, train_loss=882]

Epoch 58/500:  12%|█▏        | 58/500 [00:25<03:13,  2.29it/s, v_num=1, train_loss=882]

Epoch 58/500:  12%|█▏        | 58/500 [00:25<03:13,  2.29it/s, v_num=1, train_loss=882]

Epoch 59/500:  12%|█▏        | 58/500 [00:25<03:13,  2.29it/s, v_num=1, train_loss=882]

Epoch 59/500:  12%|█▏        | 59/500 [00:26<03:12,  2.29it/s, v_num=1, train_loss=882]

Epoch 59/500:  12%|█▏        | 59/500 [00:26<03:12,  2.29it/s, v_num=1, train_loss=882]

Epoch 60/500:  12%|█▏        | 59/500 [00:26<03:12,  2.29it/s, v_num=1, train_loss=882]

Epoch 60/500:  12%|█▏        | 60/500 [00:26<03:10,  2.31it/s, v_num=1, train_loss=882]

Epoch 60/500:  12%|█▏        | 60/500 [00:26<03:10,  2.31it/s, v_num=1, train_loss=882]

Epoch 61/500:  12%|█▏        | 60/500 [00:26<03:10,  2.31it/s, v_num=1, train_loss=882]

Epoch 61/500:  12%|█▏        | 61/500 [00:27<03:10,  2.31it/s, v_num=1, train_loss=882]

Epoch 61/500:  12%|█▏        | 61/500 [00:27<03:10,  2.31it/s, v_num=1, train_loss=882]

Epoch 62/500:  12%|█▏        | 61/500 [00:27<03:10,  2.31it/s, v_num=1, train_loss=882]

Epoch 62/500:  12%|█▏        | 62/500 [00:27<03:11,  2.29it/s, v_num=1, train_loss=882]

Epoch 62/500:  12%|█▏        | 62/500 [00:27<03:11,  2.29it/s, v_num=1, train_loss=882]

Epoch 63/500:  12%|█▏        | 62/500 [00:27<03:11,  2.29it/s, v_num=1, train_loss=882]

Epoch 63/500:  13%|█▎        | 63/500 [00:27<03:14,  2.24it/s, v_num=1, train_loss=882]

Epoch 63/500:  13%|█▎        | 63/500 [00:27<03:14,  2.24it/s, v_num=1, train_loss=882]

Epoch 64/500:  13%|█▎        | 63/500 [00:27<03:14,  2.24it/s, v_num=1, train_loss=882]

Epoch 64/500:  13%|█▎        | 64/500 [00:28<03:17,  2.20it/s, v_num=1, train_loss=882]

Epoch 64/500:  13%|█▎        | 64/500 [00:28<03:17,  2.20it/s, v_num=1, train_loss=882]

Epoch 65/500:  13%|█▎        | 64/500 [00:28<03:17,  2.20it/s, v_num=1, train_loss=882]

Epoch 65/500:  13%|█▎        | 65/500 [00:28<03:16,  2.21it/s, v_num=1, train_loss=882]

Epoch 65/500:  13%|█▎        | 65/500 [00:28<03:16,  2.21it/s, v_num=1, train_loss=882]

Epoch 66/500:  13%|█▎        | 65/500 [00:28<03:16,  2.21it/s, v_num=1, train_loss=882]

Epoch 66/500:  13%|█▎        | 66/500 [00:29<03:12,  2.26it/s, v_num=1, train_loss=882]

Epoch 66/500:  13%|█▎        | 66/500 [00:29<03:12,  2.26it/s, v_num=1, train_loss=882]

Epoch 67/500:  13%|█▎        | 66/500 [00:29<03:12,  2.26it/s, v_num=1, train_loss=882]

Epoch 67/500:  13%|█▎        | 67/500 [00:29<03:08,  2.30it/s, v_num=1, train_loss=882]

Epoch 67/500:  13%|█▎        | 67/500 [00:29<03:08,  2.30it/s, v_num=1, train_loss=882]

Epoch 68/500:  13%|█▎        | 67/500 [00:29<03:08,  2.30it/s, v_num=1, train_loss=882]

Epoch 68/500:  14%|█▎        | 68/500 [00:30<03:06,  2.32it/s, v_num=1, train_loss=882]

Epoch 68/500:  14%|█▎        | 68/500 [00:30<03:06,  2.32it/s, v_num=1, train_loss=882]

Epoch 69/500:  14%|█▎        | 68/500 [00:30<03:06,  2.32it/s, v_num=1, train_loss=882]

Epoch 69/500:  14%|█▍        | 69/500 [00:30<03:04,  2.33it/s, v_num=1, train_loss=882]

Epoch 69/500:  14%|█▍        | 69/500 [00:30<03:04,  2.33it/s, v_num=1, train_loss=882]

Epoch 70/500:  14%|█▍        | 69/500 [00:30<03:04,  2.33it/s, v_num=1, train_loss=882]

Epoch 70/500:  14%|█▍        | 70/500 [00:31<03:05,  2.32it/s, v_num=1, train_loss=882]

Epoch 70/500:  14%|█▍        | 70/500 [00:31<03:05,  2.32it/s, v_num=1, train_loss=882]

Epoch 71/500:  14%|█▍        | 70/500 [00:31<03:05,  2.32it/s, v_num=1, train_loss=882]

Epoch 71/500:  14%|█▍        | 71/500 [00:31<03:07,  2.28it/s, v_num=1, train_loss=882]

Epoch 71/500:  14%|█▍        | 71/500 [00:31<03:07,  2.28it/s, v_num=1, train_loss=882]

Epoch 72/500:  14%|█▍        | 71/500 [00:31<03:07,  2.28it/s, v_num=1, train_loss=882]

Epoch 72/500:  14%|█▍        | 72/500 [00:31<03:13,  2.21it/s, v_num=1, train_loss=882]

Epoch 72/500:  14%|█▍        | 72/500 [00:31<03:13,  2.21it/s, v_num=1, train_loss=882]

Epoch 73/500:  14%|█▍        | 72/500 [00:31<03:13,  2.21it/s, v_num=1, train_loss=882]

Epoch 73/500:  15%|█▍        | 73/500 [00:32<03:16,  2.18it/s, v_num=1, train_loss=882]

Epoch 73/500:  15%|█▍        | 73/500 [00:32<03:16,  2.18it/s, v_num=1, train_loss=882]

Epoch 74/500:  15%|█▍        | 73/500 [00:32<03:16,  2.18it/s, v_num=1, train_loss=882]

Epoch 74/500:  15%|█▍        | 74/500 [00:32<03:19,  2.14it/s, v_num=1, train_loss=882]

Epoch 74/500:  15%|█▍        | 74/500 [00:32<03:19,  2.14it/s, v_num=1, train_loss=882]

Epoch 75/500:  15%|█▍        | 74/500 [00:32<03:19,  2.14it/s, v_num=1, train_loss=882]

Epoch 75/500:  15%|█▌        | 75/500 [00:33<03:15,  2.17it/s, v_num=1, train_loss=882]

Epoch 75/500:  15%|█▌        | 75/500 [00:33<03:15,  2.17it/s, v_num=1, train_loss=882]

Epoch 76/500:  15%|█▌        | 75/500 [00:33<03:15,  2.17it/s, v_num=1, train_loss=882]

Epoch 76/500:  15%|█▌        | 76/500 [00:33<03:21,  2.10it/s, v_num=1, train_loss=882]

Epoch 76/500:  15%|█▌        | 76/500 [00:33<03:21,  2.10it/s, v_num=1, train_loss=882]

Epoch 77/500:  15%|█▌        | 76/500 [00:33<03:21,  2.10it/s, v_num=1, train_loss=882]

Epoch 77/500:  15%|█▌        | 77/500 [00:34<03:31,  2.00it/s, v_num=1, train_loss=882]

Epoch 77/500:  15%|█▌        | 77/500 [00:34<03:31,  2.00it/s, v_num=1, train_loss=882]

Epoch 78/500:  15%|█▌        | 77/500 [00:34<03:31,  2.00it/s, v_num=1, train_loss=882]

Epoch 78/500:  16%|█▌        | 78/500 [00:35<03:42,  1.90it/s, v_num=1, train_loss=882]

Epoch 78/500:  16%|█▌        | 78/500 [00:35<03:42,  1.90it/s, v_num=1, train_loss=883]

Epoch 79/500:  16%|█▌        | 78/500 [00:35<03:42,  1.90it/s, v_num=1, train_loss=883]

Epoch 79/500:  16%|█▌        | 79/500 [00:35<03:36,  1.94it/s, v_num=1, train_loss=883]

Epoch 79/500:  16%|█▌        | 79/500 [00:35<03:36,  1.94it/s, v_num=1, train_loss=883]

Epoch 80/500:  16%|█▌        | 79/500 [00:35<03:36,  1.94it/s, v_num=1, train_loss=883]

Epoch 80/500:  16%|█▌        | 80/500 [00:35<03:30,  1.99it/s, v_num=1, train_loss=883]

Epoch 80/500:  16%|█▌        | 80/500 [00:35<03:30,  1.99it/s, v_num=1, train_loss=883]

Epoch 81/500:  16%|█▌        | 80/500 [00:35<03:30,  1.99it/s, v_num=1, train_loss=883]

Epoch 81/500:  16%|█▌        | 81/500 [00:36<03:26,  2.03it/s, v_num=1, train_loss=883]

Epoch 81/500:  16%|█▌        | 81/500 [00:36<03:26,  2.03it/s, v_num=1, train_loss=883]

Epoch 82/500:  16%|█▌        | 81/500 [00:36<03:26,  2.03it/s, v_num=1, train_loss=883]

Epoch 82/500:  16%|█▋        | 82/500 [00:36<03:21,  2.07it/s, v_num=1, train_loss=883]

Epoch 82/500:  16%|█▋        | 82/500 [00:36<03:21,  2.07it/s, v_num=1, train_loss=883]

Epoch 83/500:  16%|█▋        | 82/500 [00:36<03:21,  2.07it/s, v_num=1, train_loss=883]

Epoch 83/500:  17%|█▋        | 83/500 [00:37<03:16,  2.12it/s, v_num=1, train_loss=883]

Epoch 83/500:  17%|█▋        | 83/500 [00:37<03:16,  2.12it/s, v_num=1, train_loss=883]

Epoch 84/500:  17%|█▋        | 83/500 [00:37<03:16,  2.12it/s, v_num=1, train_loss=883]

Epoch 84/500:  17%|█▋        | 84/500 [00:37<03:14,  2.14it/s, v_num=1, train_loss=883]

Epoch 84/500:  17%|█▋        | 84/500 [00:37<03:14,  2.14it/s, v_num=1, train_loss=883]

Epoch 85/500:  17%|█▋        | 84/500 [00:37<03:14,  2.14it/s, v_num=1, train_loss=883]

Epoch 85/500:  17%|█▋        | 85/500 [00:38<03:10,  2.18it/s, v_num=1, train_loss=883]

Epoch 85/500:  17%|█▋        | 85/500 [00:38<03:10,  2.18it/s, v_num=1, train_loss=883]

Epoch 86/500:  17%|█▋        | 85/500 [00:38<03:10,  2.18it/s, v_num=1, train_loss=883]

Epoch 86/500:  17%|█▋        | 86/500 [00:38<03:07,  2.20it/s, v_num=1, train_loss=883]

Epoch 86/500:  17%|█▋        | 86/500 [00:38<03:07,  2.20it/s, v_num=1, train_loss=883]

Epoch 87/500:  17%|█▋        | 86/500 [00:38<03:07,  2.20it/s, v_num=1, train_loss=883]

Epoch 87/500:  17%|█▋        | 87/500 [00:39<03:07,  2.20it/s, v_num=1, train_loss=883]

Epoch 87/500:  17%|█▋        | 87/500 [00:39<03:07,  2.20it/s, v_num=1, train_loss=883]

Epoch 88/500:  17%|█▋        | 87/500 [00:39<03:07,  2.20it/s, v_num=1, train_loss=883]

Epoch 88/500:  18%|█▊        | 88/500 [00:39<03:04,  2.23it/s, v_num=1, train_loss=883]

Epoch 88/500:  18%|█▊        | 88/500 [00:39<03:04,  2.23it/s, v_num=1, train_loss=883]

Epoch 89/500:  18%|█▊        | 88/500 [00:39<03:04,  2.23it/s, v_num=1, train_loss=883]

Epoch 89/500:  18%|█▊        | 89/500 [00:40<03:02,  2.25it/s, v_num=1, train_loss=883]

Epoch 89/500:  18%|█▊        | 89/500 [00:40<03:02,  2.25it/s, v_num=1, train_loss=883]

Epoch 90/500:  18%|█▊        | 89/500 [00:40<03:02,  2.25it/s, v_num=1, train_loss=883]

Epoch 90/500:  18%|█▊        | 90/500 [00:40<03:00,  2.27it/s, v_num=1, train_loss=883]

Epoch 90/500:  18%|█▊        | 90/500 [00:40<03:00,  2.27it/s, v_num=1, train_loss=883]

Epoch 91/500:  18%|█▊        | 90/500 [00:40<03:00,  2.27it/s, v_num=1, train_loss=883]

Epoch 91/500:  18%|█▊        | 91/500 [00:40<02:59,  2.27it/s, v_num=1, train_loss=883]

Epoch 91/500:  18%|█▊        | 91/500 [00:40<02:59,  2.27it/s, v_num=1, train_loss=883]

Epoch 92/500:  18%|█▊        | 91/500 [00:40<02:59,  2.27it/s, v_num=1, train_loss=883]

Epoch 92/500:  18%|█▊        | 92/500 [00:41<02:59,  2.28it/s, v_num=1, train_loss=883]

Epoch 92/500:  18%|█▊        | 92/500 [00:41<02:59,  2.28it/s, v_num=1, train_loss=883]

Epoch 93/500:  18%|█▊        | 92/500 [00:41<02:59,  2.28it/s, v_num=1, train_loss=883]

Epoch 93/500:  19%|█▊        | 93/500 [00:41<02:57,  2.29it/s, v_num=1, train_loss=883]

Epoch 93/500:  19%|█▊        | 93/500 [00:41<02:57,  2.29it/s, v_num=1, train_loss=883]

Epoch 94/500:  19%|█▊        | 93/500 [00:41<02:57,  2.29it/s, v_num=1, train_loss=883]

Epoch 94/500:  19%|█▉        | 94/500 [00:42<03:00,  2.25it/s, v_num=1, train_loss=883]

Epoch 94/500:  19%|█▉        | 94/500 [00:42<03:00,  2.25it/s, v_num=1, train_loss=883]

Epoch 95/500:  19%|█▉        | 94/500 [00:42<03:00,  2.25it/s, v_num=1, train_loss=883]

Epoch 95/500:  19%|█▉        | 95/500 [00:42<03:01,  2.23it/s, v_num=1, train_loss=883]

Epoch 95/500:  19%|█▉        | 95/500 [00:42<03:01,  2.23it/s, v_num=1, train_loss=883]

Epoch 96/500:  19%|█▉        | 95/500 [00:42<03:01,  2.23it/s, v_num=1, train_loss=883]

Epoch 96/500:  19%|█▉        | 96/500 [00:43<02:59,  2.25it/s, v_num=1, train_loss=883]

Epoch 96/500:  19%|█▉        | 96/500 [00:43<02:59,  2.25it/s, v_num=1, train_loss=883]

Epoch 97/500:  19%|█▉        | 96/500 [00:43<02:59,  2.25it/s, v_num=1, train_loss=883]

Epoch 97/500:  19%|█▉        | 97/500 [00:43<02:58,  2.26it/s, v_num=1, train_loss=883]

Epoch 97/500:  19%|█▉        | 97/500 [00:43<02:58,  2.26it/s, v_num=1, train_loss=883]

Epoch 98/500:  19%|█▉        | 97/500 [00:43<02:58,  2.26it/s, v_num=1, train_loss=883]

Epoch 98/500:  20%|█▉        | 98/500 [00:44<03:00,  2.22it/s, v_num=1, train_loss=883]

Epoch 98/500:  20%|█▉        | 98/500 [00:44<03:00,  2.22it/s, v_num=1, train_loss=883]

Epoch 99/500:  20%|█▉        | 98/500 [00:44<03:00,  2.22it/s, v_num=1, train_loss=883]

Epoch 99/500:  20%|█▉        | 99/500 [00:44<03:00,  2.22it/s, v_num=1, train_loss=883]

Epoch 99/500:  20%|█▉        | 99/500 [00:44<03:00,  2.22it/s, v_num=1, train_loss=883]

Epoch 100/500:  20%|█▉        | 99/500 [00:44<03:00,  2.22it/s, v_num=1, train_loss=883]

Epoch 100/500:  20%|██        | 100/500 [00:44<02:55,  2.27it/s, v_num=1, train_loss=883]

Epoch 100/500:  20%|██        | 100/500 [00:44<02:55,  2.27it/s, v_num=1, train_loss=883]

Epoch 101/500:  20%|██        | 100/500 [00:44<02:55,  2.27it/s, v_num=1, train_loss=883]

Epoch 101/500:  20%|██        | 101/500 [00:45<02:50,  2.34it/s, v_num=1, train_loss=883]

Epoch 101/500:  20%|██        | 101/500 [00:45<02:50,  2.34it/s, v_num=1, train_loss=883]

Epoch 102/500:  20%|██        | 101/500 [00:45<02:50,  2.34it/s, v_num=1, train_loss=883]

Epoch 102/500:  20%|██        | 102/500 [00:45<02:50,  2.34it/s, v_num=1, train_loss=883]

Epoch 102/500:  20%|██        | 102/500 [00:45<02:50,  2.34it/s, v_num=1, train_loss=883]

Epoch 103/500:  20%|██        | 102/500 [00:45<02:50,  2.34it/s, v_num=1, train_loss=883]

Epoch 103/500:  21%|██        | 103/500 [00:46<02:49,  2.34it/s, v_num=1, train_loss=883]

Epoch 103/500:  21%|██        | 103/500 [00:46<02:49,  2.34it/s, v_num=1, train_loss=883]

Epoch 104/500:  21%|██        | 103/500 [00:46<02:49,  2.34it/s, v_num=1, train_loss=883]

Epoch 104/500:  21%|██        | 104/500 [00:46<02:47,  2.37it/s, v_num=1, train_loss=883]

Epoch 104/500:  21%|██        | 104/500 [00:46<02:47,  2.37it/s, v_num=1, train_loss=883]

Epoch 105/500:  21%|██        | 104/500 [00:46<02:47,  2.37it/s, v_num=1, train_loss=883]

Epoch 105/500:  21%|██        | 105/500 [00:46<02:46,  2.38it/s, v_num=1, train_loss=883]

Epoch 105/500:  21%|██        | 105/500 [00:46<02:46,  2.38it/s, v_num=1, train_loss=884]

Epoch 106/500:  21%|██        | 105/500 [00:46<02:46,  2.38it/s, v_num=1, train_loss=884]

Epoch 106/500:  21%|██        | 106/500 [00:47<02:50,  2.31it/s, v_num=1, train_loss=884]

Epoch 106/500:  21%|██        | 106/500 [00:47<02:50,  2.31it/s, v_num=1, train_loss=884]

Epoch 107/500:  21%|██        | 106/500 [00:47<02:50,  2.31it/s, v_num=1, train_loss=884]

Epoch 107/500:  21%|██▏       | 107/500 [00:47<02:52,  2.27it/s, v_num=1, train_loss=884]

Epoch 107/500:  21%|██▏       | 107/500 [00:47<02:52,  2.27it/s, v_num=1, train_loss=884]

Epoch 108/500:  21%|██▏       | 107/500 [00:47<02:52,  2.27it/s, v_num=1, train_loss=884]

Epoch 108/500:  22%|██▏       | 108/500 [00:48<02:53,  2.26it/s, v_num=1, train_loss=884]

Epoch 108/500:  22%|██▏       | 108/500 [00:48<02:53,  2.26it/s, v_num=1, train_loss=884]

Epoch 109/500:  22%|██▏       | 108/500 [00:48<02:53,  2.26it/s, v_num=1, train_loss=884]

Epoch 109/500:  22%|██▏       | 109/500 [00:48<02:52,  2.27it/s, v_num=1, train_loss=884]

Epoch 109/500:  22%|██▏       | 109/500 [00:48<02:52,  2.27it/s, v_num=1, train_loss=884]

Epoch 110/500:  22%|██▏       | 109/500 [00:48<02:52,  2.27it/s, v_num=1, train_loss=884]

Epoch 110/500:  22%|██▏       | 110/500 [00:49<02:52,  2.26it/s, v_num=1, train_loss=884]

Epoch 110/500:  22%|██▏       | 110/500 [00:49<02:52,  2.26it/s, v_num=1, train_loss=884]

Epoch 111/500:  22%|██▏       | 110/500 [00:49<02:52,  2.26it/s, v_num=1, train_loss=884]

Epoch 111/500:  22%|██▏       | 111/500 [00:49<02:50,  2.28it/s, v_num=1, train_loss=884]

Epoch 111/500:  22%|██▏       | 111/500 [00:49<02:50,  2.28it/s, v_num=1, train_loss=884]

Epoch 112/500:  22%|██▏       | 111/500 [00:49<02:50,  2.28it/s, v_num=1, train_loss=884]

Epoch 112/500:  22%|██▏       | 112/500 [00:50<02:52,  2.25it/s, v_num=1, train_loss=884]

Epoch 112/500:  22%|██▏       | 112/500 [00:50<02:52,  2.25it/s, v_num=1, train_loss=884]

Epoch 113/500:  22%|██▏       | 112/500 [00:50<02:52,  2.25it/s, v_num=1, train_loss=884]

Epoch 113/500:  23%|██▎       | 113/500 [00:50<02:55,  2.21it/s, v_num=1, train_loss=884]

Epoch 113/500:  23%|██▎       | 113/500 [00:50<02:55,  2.21it/s, v_num=1, train_loss=884]

Epoch 114/500:  23%|██▎       | 113/500 [00:50<02:55,  2.21it/s, v_num=1, train_loss=884]

Epoch 114/500:  23%|██▎       | 114/500 [00:51<02:54,  2.21it/s, v_num=1, train_loss=884]

Epoch 114/500:  23%|██▎       | 114/500 [00:51<02:54,  2.21it/s, v_num=1, train_loss=884]

Epoch 115/500:  23%|██▎       | 114/500 [00:51<02:54,  2.21it/s, v_num=1, train_loss=884]

Epoch 115/500:  23%|██▎       | 115/500 [00:51<02:53,  2.21it/s, v_num=1, train_loss=884]

Epoch 115/500:  23%|██▎       | 115/500 [00:51<02:53,  2.21it/s, v_num=1, train_loss=884]

Epoch 116/500:  23%|██▎       | 115/500 [00:51<02:53,  2.21it/s, v_num=1, train_loss=884]

Epoch 116/500:  23%|██▎       | 116/500 [00:51<02:59,  2.13it/s, v_num=1, train_loss=884]

Epoch 116/500:  23%|██▎       | 116/500 [00:51<02:59,  2.13it/s, v_num=1, train_loss=884]

Epoch 117/500:  23%|██▎       | 116/500 [00:51<02:59,  2.13it/s, v_num=1, train_loss=884]

Epoch 117/500:  23%|██▎       | 117/500 [00:52<03:14,  1.97it/s, v_num=1, train_loss=884]

Epoch 117/500:  23%|██▎       | 117/500 [00:52<03:14,  1.97it/s, v_num=1, train_loss=884]

Epoch 118/500:  23%|██▎       | 117/500 [00:52<03:14,  1.97it/s, v_num=1, train_loss=884]

Epoch 118/500:  24%|██▎       | 118/500 [00:53<03:15,  1.95it/s, v_num=1, train_loss=884]

Epoch 118/500:  24%|██▎       | 118/500 [00:53<03:15,  1.95it/s, v_num=1, train_loss=884]

Epoch 119/500:  24%|██▎       | 118/500 [00:53<03:15,  1.95it/s, v_num=1, train_loss=884]

Epoch 119/500:  24%|██▍       | 119/500 [00:53<03:10,  2.00it/s, v_num=1, train_loss=884]

Epoch 119/500:  24%|██▍       | 119/500 [00:53<03:10,  2.00it/s, v_num=1, train_loss=884]

Epoch 120/500:  24%|██▍       | 119/500 [00:53<03:10,  2.00it/s, v_num=1, train_loss=884]

Epoch 120/500:  24%|██▍       | 120/500 [00:54<03:08,  2.01it/s, v_num=1, train_loss=884]

Epoch 120/500:  24%|██▍       | 120/500 [00:54<03:08,  2.01it/s, v_num=1, train_loss=884]

Epoch 121/500:  24%|██▍       | 120/500 [00:54<03:08,  2.01it/s, v_num=1, train_loss=884]

Epoch 121/500:  24%|██▍       | 121/500 [00:54<03:06,  2.03it/s, v_num=1, train_loss=884]

Epoch 121/500:  24%|██▍       | 121/500 [00:54<03:06,  2.03it/s, v_num=1, train_loss=884]

Epoch 122/500:  24%|██▍       | 121/500 [00:54<03:06,  2.03it/s, v_num=1, train_loss=884]

Epoch 122/500:  24%|██▍       | 122/500 [00:55<03:06,  2.03it/s, v_num=1, train_loss=884]

Epoch 122/500:  24%|██▍       | 122/500 [00:55<03:06,  2.03it/s, v_num=1, train_loss=884]

Epoch 123/500:  24%|██▍       | 122/500 [00:55<03:06,  2.03it/s, v_num=1, train_loss=884]

Epoch 123/500:  25%|██▍       | 123/500 [00:55<03:00,  2.09it/s, v_num=1, train_loss=884]

Epoch 123/500:  25%|██▍       | 123/500 [00:55<03:00,  2.09it/s, v_num=1, train_loss=884]

Epoch 124/500:  25%|██▍       | 123/500 [00:55<03:00,  2.09it/s, v_num=1, train_loss=884]

Epoch 124/500:  25%|██▍       | 124/500 [00:56<03:05,  2.02it/s, v_num=1, train_loss=884]

Epoch 124/500:  25%|██▍       | 124/500 [00:56<03:05,  2.02it/s, v_num=1, train_loss=884]

Epoch 125/500:  25%|██▍       | 124/500 [00:56<03:05,  2.02it/s, v_num=1, train_loss=884]

Epoch 125/500:  25%|██▌       | 125/500 [00:56<03:05,  2.02it/s, v_num=1, train_loss=884]

Epoch 125/500:  25%|██▌       | 125/500 [00:56<03:05,  2.02it/s, v_num=1, train_loss=884]

Epoch 126/500:  25%|██▌       | 125/500 [00:56<03:05,  2.02it/s, v_num=1, train_loss=884]

Epoch 126/500:  25%|██▌       | 126/500 [00:57<03:03,  2.04it/s, v_num=1, train_loss=884]

Epoch 126/500:  25%|██▌       | 126/500 [00:57<03:03,  2.04it/s, v_num=1, train_loss=884]

Epoch 127/500:  25%|██▌       | 126/500 [00:57<03:03,  2.04it/s, v_num=1, train_loss=884]

Epoch 127/500:  25%|██▌       | 127/500 [00:57<02:57,  2.10it/s, v_num=1, train_loss=884]

Epoch 127/500:  25%|██▌       | 127/500 [00:57<02:57,  2.10it/s, v_num=1, train_loss=884]

Epoch 128/500:  25%|██▌       | 127/500 [00:57<02:57,  2.10it/s, v_num=1, train_loss=884]

Epoch 128/500:  26%|██▌       | 128/500 [00:57<02:53,  2.14it/s, v_num=1, train_loss=884]

Epoch 128/500:  26%|██▌       | 128/500 [00:57<02:53,  2.14it/s, v_num=1, train_loss=884]

Epoch 129/500:  26%|██▌       | 128/500 [00:57<02:53,  2.14it/s, v_num=1, train_loss=884]

Epoch 129/500:  26%|██▌       | 129/500 [00:58<02:51,  2.17it/s, v_num=1, train_loss=884]

Epoch 129/500:  26%|██▌       | 129/500 [00:58<02:51,  2.17it/s, v_num=1, train_loss=885]

Epoch 130/500:  26%|██▌       | 129/500 [00:58<02:51,  2.17it/s, v_num=1, train_loss=885]

Epoch 130/500:  26%|██▌       | 130/500 [00:58<02:50,  2.18it/s, v_num=1, train_loss=885]

Epoch 130/500:  26%|██▌       | 130/500 [00:58<02:50,  2.18it/s, v_num=1, train_loss=885]

Epoch 131/500:  26%|██▌       | 130/500 [00:58<02:50,  2.18it/s, v_num=1, train_loss=885]

Epoch 131/500:  26%|██▌       | 131/500 [00:59<02:48,  2.19it/s, v_num=1, train_loss=885]

Epoch 131/500:  26%|██▌       | 131/500 [00:59<02:48,  2.19it/s, v_num=1, train_loss=885]

Epoch 132/500:  26%|██▌       | 131/500 [00:59<02:48,  2.19it/s, v_num=1, train_loss=885]

Epoch 132/500:  26%|██▋       | 132/500 [00:59<02:47,  2.20it/s, v_num=1, train_loss=885]

Epoch 132/500:  26%|██▋       | 132/500 [00:59<02:47,  2.20it/s, v_num=1, train_loss=885]

Epoch 133/500:  26%|██▋       | 132/500 [00:59<02:47,  2.20it/s, v_num=1, train_loss=885]

Epoch 133/500:  27%|██▋       | 133/500 [01:00<02:45,  2.22it/s, v_num=1, train_loss=885]

Epoch 133/500:  27%|██▋       | 133/500 [01:00<02:45,  2.22it/s, v_num=1, train_loss=885]

Epoch 134/500:  27%|██▋       | 133/500 [01:00<02:45,  2.22it/s, v_num=1, train_loss=885]

Epoch 134/500:  27%|██▋       | 134/500 [01:00<02:43,  2.24it/s, v_num=1, train_loss=885]

Epoch 134/500:  27%|██▋       | 134/500 [01:00<02:43,  2.24it/s, v_num=1, train_loss=885]

Epoch 135/500:  27%|██▋       | 134/500 [01:00<02:43,  2.24it/s, v_num=1, train_loss=885]

Epoch 135/500:  27%|██▋       | 135/500 [01:01<02:42,  2.24it/s, v_num=1, train_loss=885]

Epoch 135/500:  27%|██▋       | 135/500 [01:01<02:42,  2.24it/s, v_num=1, train_loss=885]

Epoch 136/500:  27%|██▋       | 135/500 [01:01<02:42,  2.24it/s, v_num=1, train_loss=885]

Epoch 136/500:  27%|██▋       | 136/500 [01:01<02:42,  2.23it/s, v_num=1, train_loss=885]

Epoch 136/500:  27%|██▋       | 136/500 [01:01<02:42,  2.23it/s, v_num=1, train_loss=885]

Epoch 137/500:  27%|██▋       | 136/500 [01:01<02:42,  2.23it/s, v_num=1, train_loss=885]

Epoch 137/500:  27%|██▋       | 137/500 [01:01<02:42,  2.23it/s, v_num=1, train_loss=885]

Epoch 137/500:  27%|██▋       | 137/500 [01:01<02:42,  2.23it/s, v_num=1, train_loss=885]

Epoch 138/500:  27%|██▋       | 137/500 [01:01<02:42,  2.23it/s, v_num=1, train_loss=885]

Epoch 138/500:  28%|██▊       | 138/500 [01:02<02:42,  2.22it/s, v_num=1, train_loss=885]

Epoch 138/500:  28%|██▊       | 138/500 [01:02<02:42,  2.22it/s, v_num=1, train_loss=885]

Epoch 139/500:  28%|██▊       | 138/500 [01:02<02:42,  2.22it/s, v_num=1, train_loss=885]

Epoch 139/500:  28%|██▊       | 139/500 [01:02<02:41,  2.24it/s, v_num=1, train_loss=885]

Epoch 139/500:  28%|██▊       | 139/500 [01:02<02:41,  2.24it/s, v_num=1, train_loss=885]

Epoch 140/500:  28%|██▊       | 139/500 [01:02<02:41,  2.24it/s, v_num=1, train_loss=885]

Epoch 140/500:  28%|██▊       | 140/500 [01:03<02:40,  2.24it/s, v_num=1, train_loss=885]

Epoch 140/500:  28%|██▊       | 140/500 [01:03<02:40,  2.24it/s, v_num=1, train_loss=885]

Epoch 141/500:  28%|██▊       | 140/500 [01:03<02:40,  2.24it/s, v_num=1, train_loss=885]

Epoch 141/500:  28%|██▊       | 141/500 [01:03<02:40,  2.24it/s, v_num=1, train_loss=885]

Epoch 141/500:  28%|██▊       | 141/500 [01:03<02:40,  2.24it/s, v_num=1, train_loss=885]

Epoch 142/500:  28%|██▊       | 141/500 [01:03<02:40,  2.24it/s, v_num=1, train_loss=885]

Epoch 142/500:  28%|██▊       | 142/500 [01:04<02:38,  2.25it/s, v_num=1, train_loss=885]

Epoch 142/500:  28%|██▊       | 142/500 [01:04<02:38,  2.25it/s, v_num=1, train_loss=885]

Epoch 143/500:  28%|██▊       | 142/500 [01:04<02:38,  2.25it/s, v_num=1, train_loss=885]

Epoch 143/500:  29%|██▊       | 143/500 [01:04<02:34,  2.32it/s, v_num=1, train_loss=885]

Epoch 143/500:  29%|██▊       | 143/500 [01:04<02:34,  2.32it/s, v_num=1, train_loss=885]

Epoch 144/500:  29%|██▊       | 143/500 [01:04<02:34,  2.32it/s, v_num=1, train_loss=885]

Epoch 144/500:  29%|██▉       | 144/500 [01:04<02:28,  2.39it/s, v_num=1, train_loss=885]

Epoch 144/500:  29%|██▉       | 144/500 [01:04<02:28,  2.39it/s, v_num=1, train_loss=885]

Epoch 145/500:  29%|██▉       | 144/500 [01:04<02:28,  2.39it/s, v_num=1, train_loss=885]

Epoch 145/500:  29%|██▉       | 145/500 [01:05<02:24,  2.45it/s, v_num=1, train_loss=885]

Epoch 145/500:  29%|██▉       | 145/500 [01:05<02:24,  2.45it/s, v_num=1, train_loss=885]

Epoch 146/500:  29%|██▉       | 145/500 [01:05<02:24,  2.45it/s, v_num=1, train_loss=885]

Epoch 146/500:  29%|██▉       | 146/500 [01:05<02:22,  2.48it/s, v_num=1, train_loss=885]

Epoch 146/500:  29%|██▉       | 146/500 [01:05<02:22,  2.48it/s, v_num=1, train_loss=885]

Epoch 147/500:  29%|██▉       | 146/500 [01:05<02:22,  2.48it/s, v_num=1, train_loss=885]

Epoch 147/500:  29%|██▉       | 147/500 [01:06<02:20,  2.51it/s, v_num=1, train_loss=885]

Epoch 147/500:  29%|██▉       | 147/500 [01:06<02:20,  2.51it/s, v_num=1, train_loss=885]

Epoch 148/500:  29%|██▉       | 147/500 [01:06<02:20,  2.51it/s, v_num=1, train_loss=885]

Epoch 148/500:  30%|██▉       | 148/500 [01:06<02:19,  2.53it/s, v_num=1, train_loss=885]

Epoch 148/500:  30%|██▉       | 148/500 [01:06<02:19,  2.53it/s, v_num=1, train_loss=885]

Epoch 149/500:  30%|██▉       | 148/500 [01:06<02:19,  2.53it/s, v_num=1, train_loss=885]

Epoch 149/500:  30%|██▉       | 149/500 [01:06<02:19,  2.51it/s, v_num=1, train_loss=885]

Epoch 149/500:  30%|██▉       | 149/500 [01:06<02:19,  2.51it/s, v_num=1, train_loss=885]

Epoch 150/500:  30%|██▉       | 149/500 [01:06<02:19,  2.51it/s, v_num=1, train_loss=885]

Epoch 150/500:  30%|███       | 150/500 [01:07<02:22,  2.46it/s, v_num=1, train_loss=885]

Epoch 150/500:  30%|███       | 150/500 [01:07<02:22,  2.46it/s, v_num=1, train_loss=885]

Epoch 151/500:  30%|███       | 150/500 [01:07<02:22,  2.46it/s, v_num=1, train_loss=885]

Epoch 151/500:  30%|███       | 151/500 [01:07<02:22,  2.46it/s, v_num=1, train_loss=885]

Epoch 151/500:  30%|███       | 151/500 [01:07<02:22,  2.46it/s, v_num=1, train_loss=885]

Epoch 152/500:  30%|███       | 151/500 [01:07<02:22,  2.46it/s, v_num=1, train_loss=885]

Epoch 152/500:  30%|███       | 152/500 [01:08<02:20,  2.48it/s, v_num=1, train_loss=885]

Epoch 152/500:  30%|███       | 152/500 [01:08<02:20,  2.48it/s, v_num=1, train_loss=885]

Epoch 153/500:  30%|███       | 152/500 [01:08<02:20,  2.48it/s, v_num=1, train_loss=885]

Epoch 153/500:  31%|███       | 153/500 [01:08<02:20,  2.47it/s, v_num=1, train_loss=885]

Epoch 153/500:  31%|███       | 153/500 [01:08<02:20,  2.47it/s, v_num=1, train_loss=886]

Epoch 154/500:  31%|███       | 153/500 [01:08<02:20,  2.47it/s, v_num=1, train_loss=886]

Epoch 154/500:  31%|███       | 154/500 [01:09<02:34,  2.24it/s, v_num=1, train_loss=886]

Epoch 154/500:  31%|███       | 154/500 [01:09<02:34,  2.24it/s, v_num=1, train_loss=886]

Epoch 155/500:  31%|███       | 154/500 [01:09<02:34,  2.24it/s, v_num=1, train_loss=886]

Epoch 155/500:  31%|███       | 155/500 [01:09<02:29,  2.31it/s, v_num=1, train_loss=886]

Epoch 155/500:  31%|███       | 155/500 [01:09<02:29,  2.31it/s, v_num=1, train_loss=886]

Epoch 156/500:  31%|███       | 155/500 [01:09<02:29,  2.31it/s, v_num=1, train_loss=886]

Epoch 156/500:  31%|███       | 156/500 [01:09<02:23,  2.40it/s, v_num=1, train_loss=886]

Epoch 156/500:  31%|███       | 156/500 [01:09<02:23,  2.40it/s, v_num=1, train_loss=886]

Epoch 157/500:  31%|███       | 156/500 [01:09<02:23,  2.40it/s, v_num=1, train_loss=886]

Epoch 157/500:  31%|███▏      | 157/500 [01:10<02:19,  2.46it/s, v_num=1, train_loss=886]

Epoch 157/500:  31%|███▏      | 157/500 [01:10<02:19,  2.46it/s, v_num=1, train_loss=886]

Epoch 158/500:  31%|███▏      | 157/500 [01:10<02:19,  2.46it/s, v_num=1, train_loss=886]

Epoch 158/500:  32%|███▏      | 158/500 [01:10<02:16,  2.50it/s, v_num=1, train_loss=886]

Epoch 158/500:  32%|███▏      | 158/500 [01:10<02:16,  2.50it/s, v_num=1, train_loss=886]

Epoch 159/500:  32%|███▏      | 158/500 [01:10<02:16,  2.50it/s, v_num=1, train_loss=886]

Epoch 159/500:  32%|███▏      | 159/500 [01:10<02:14,  2.53it/s, v_num=1, train_loss=886]

Epoch 159/500:  32%|███▏      | 159/500 [01:10<02:14,  2.53it/s, v_num=1, train_loss=886]

Epoch 160/500:  32%|███▏      | 159/500 [01:11<02:14,  2.53it/s, v_num=1, train_loss=886]

Epoch 160/500:  32%|███▏      | 160/500 [01:11<02:13,  2.55it/s, v_num=1, train_loss=886]

Epoch 160/500:  32%|███▏      | 160/500 [01:11<02:13,  2.55it/s, v_num=1, train_loss=886]

Epoch 161/500:  32%|███▏      | 160/500 [01:11<02:13,  2.55it/s, v_num=1, train_loss=886]

Epoch 161/500:  32%|███▏      | 161/500 [01:11<02:11,  2.57it/s, v_num=1, train_loss=886]

Epoch 161/500:  32%|███▏      | 161/500 [01:11<02:11,  2.57it/s, v_num=1, train_loss=886]

Epoch 162/500:  32%|███▏      | 161/500 [01:11<02:11,  2.57it/s, v_num=1, train_loss=886]

Epoch 162/500:  32%|███▏      | 162/500 [01:12<02:11,  2.58it/s, v_num=1, train_loss=886]

Epoch 162/500:  32%|███▏      | 162/500 [01:12<02:11,  2.58it/s, v_num=1, train_loss=886]

Epoch 163/500:  32%|███▏      | 162/500 [01:12<02:11,  2.58it/s, v_num=1, train_loss=886]

Epoch 163/500:  33%|███▎      | 163/500 [01:12<02:13,  2.53it/s, v_num=1, train_loss=886]

Epoch 163/500:  33%|███▎      | 163/500 [01:12<02:13,  2.53it/s, v_num=1, train_loss=886]

Epoch 164/500:  33%|███▎      | 163/500 [01:12<02:13,  2.53it/s, v_num=1, train_loss=886]

Epoch 164/500:  33%|███▎      | 164/500 [01:13<02:17,  2.44it/s, v_num=1, train_loss=886]

Epoch 164/500:  33%|███▎      | 164/500 [01:13<02:17,  2.44it/s, v_num=1, train_loss=886]

Epoch 165/500:  33%|███▎      | 164/500 [01:13<02:17,  2.44it/s, v_num=1, train_loss=886]

Epoch 165/500:  33%|███▎      | 165/500 [01:13<02:21,  2.36it/s, v_num=1, train_loss=886]

Epoch 165/500:  33%|███▎      | 165/500 [01:13<02:21,  2.36it/s, v_num=1, train_loss=886]

Epoch 166/500:  33%|███▎      | 165/500 [01:13<02:21,  2.36it/s, v_num=1, train_loss=886]

Epoch 166/500:  33%|███▎      | 166/500 [01:13<02:20,  2.38it/s, v_num=1, train_loss=886]

Epoch 166/500:  33%|███▎      | 166/500 [01:13<02:20,  2.38it/s, v_num=1, train_loss=886]

Epoch 167/500:  33%|███▎      | 166/500 [01:13<02:20,  2.38it/s, v_num=1, train_loss=886]

Epoch 167/500:  33%|███▎      | 167/500 [01:14<02:23,  2.32it/s, v_num=1, train_loss=886]

Epoch 167/500:  33%|███▎      | 167/500 [01:14<02:23,  2.32it/s, v_num=1, train_loss=886]

Epoch 168/500:  33%|███▎      | 167/500 [01:14<02:23,  2.32it/s, v_num=1, train_loss=886]

Epoch 168/500:  34%|███▎      | 168/500 [01:14<02:26,  2.27it/s, v_num=1, train_loss=886]

Epoch 168/500:  34%|███▎      | 168/500 [01:14<02:26,  2.27it/s, v_num=1, train_loss=886]

Epoch 169/500:  34%|███▎      | 168/500 [01:14<02:26,  2.27it/s, v_num=1, train_loss=886]

Epoch 169/500:  34%|███▍      | 169/500 [01:15<02:28,  2.22it/s, v_num=1, train_loss=886]

Epoch 169/500:  34%|███▍      | 169/500 [01:15<02:28,  2.22it/s, v_num=1, train_loss=886]

Epoch 170/500:  34%|███▍      | 169/500 [01:15<02:28,  2.22it/s, v_num=1, train_loss=886]

Epoch 170/500:  34%|███▍      | 170/500 [01:15<02:31,  2.18it/s, v_num=1, train_loss=886]

Epoch 170/500:  34%|███▍      | 170/500 [01:15<02:31,  2.18it/s, v_num=1, train_loss=886]

Epoch 171/500:  34%|███▍      | 170/500 [01:15<02:31,  2.18it/s, v_num=1, train_loss=886]

Epoch 171/500:  34%|███▍      | 171/500 [01:16<02:32,  2.16it/s, v_num=1, train_loss=886]

Epoch 171/500:  34%|███▍      | 171/500 [01:16<02:32,  2.16it/s, v_num=1, train_loss=886]

Epoch 172/500:  34%|███▍      | 171/500 [01:16<02:32,  2.16it/s, v_num=1, train_loss=886]

Epoch 172/500:  34%|███▍      | 172/500 [01:16<02:28,  2.21it/s, v_num=1, train_loss=886]

Epoch 172/500:  34%|███▍      | 172/500 [01:16<02:28,  2.21it/s, v_num=1, train_loss=886]

Epoch 173/500:  34%|███▍      | 172/500 [01:16<02:28,  2.21it/s, v_num=1, train_loss=886]

Epoch 173/500:  35%|███▍      | 173/500 [01:17<02:28,  2.21it/s, v_num=1, train_loss=886]

Epoch 173/500:  35%|███▍      | 173/500 [01:17<02:28,  2.21it/s, v_num=1, train_loss=886]

Epoch 174/500:  35%|███▍      | 173/500 [01:17<02:28,  2.21it/s, v_num=1, train_loss=886]

Epoch 174/500:  35%|███▍      | 174/500 [01:17<02:26,  2.23it/s, v_num=1, train_loss=886]

Epoch 174/500:  35%|███▍      | 174/500 [01:17<02:26,  2.23it/s, v_num=1, train_loss=886]

Epoch 175/500:  35%|███▍      | 174/500 [01:17<02:26,  2.23it/s, v_num=1, train_loss=886]

Epoch 175/500:  35%|███▌      | 175/500 [01:17<02:25,  2.23it/s, v_num=1, train_loss=886]

Epoch 175/500:  35%|███▌      | 175/500 [01:17<02:25,  2.23it/s, v_num=1, train_loss=886]

Epoch 176/500:  35%|███▌      | 175/500 [01:17<02:25,  2.23it/s, v_num=1, train_loss=886]

Epoch 176/500:  35%|███▌      | 176/500 [01:18<02:22,  2.28it/s, v_num=1, train_loss=886]

Epoch 176/500:  35%|███▌      | 176/500 [01:18<02:22,  2.28it/s, v_num=1, train_loss=886]

Epoch 177/500:  35%|███▌      | 176/500 [01:18<02:22,  2.28it/s, v_num=1, train_loss=886]

Epoch 177/500:  35%|███▌      | 177/500 [01:18<02:16,  2.37it/s, v_num=1, train_loss=886]

Epoch 177/500:  35%|███▌      | 177/500 [01:18<02:16,  2.37it/s, v_num=1, train_loss=886]

Epoch 178/500:  35%|███▌      | 177/500 [01:18<02:16,  2.37it/s, v_num=1, train_loss=886]

Epoch 178/500:  36%|███▌      | 178/500 [01:19<02:12,  2.42it/s, v_num=1, train_loss=886]

Epoch 178/500:  36%|███▌      | 178/500 [01:19<02:12,  2.42it/s, v_num=1, train_loss=886]

Epoch 179/500:  36%|███▌      | 178/500 [01:19<02:12,  2.42it/s, v_num=1, train_loss=886]

Epoch 179/500:  36%|███▌      | 179/500 [01:19<02:10,  2.46it/s, v_num=1, train_loss=886]

Epoch 179/500:  36%|███▌      | 179/500 [01:19<02:10,  2.46it/s, v_num=1, train_loss=886]

Epoch 180/500:  36%|███▌      | 179/500 [01:19<02:10,  2.46it/s, v_num=1, train_loss=886]

Epoch 180/500:  36%|███▌      | 180/500 [01:19<02:10,  2.46it/s, v_num=1, train_loss=886]

Epoch 180/500:  36%|███▌      | 180/500 [01:19<02:10,  2.46it/s, v_num=1, train_loss=887]

Epoch 181/500:  36%|███▌      | 180/500 [01:19<02:10,  2.46it/s, v_num=1, train_loss=887]

Epoch 181/500:  36%|███▌      | 181/500 [01:20<02:08,  2.48it/s, v_num=1, train_loss=887]

Epoch 181/500:  36%|███▌      | 181/500 [01:20<02:08,  2.48it/s, v_num=1, train_loss=887]

Epoch 182/500:  36%|███▌      | 181/500 [01:20<02:08,  2.48it/s, v_num=1, train_loss=887]

Epoch 182/500:  36%|███▋      | 182/500 [01:20<02:12,  2.39it/s, v_num=1, train_loss=887]

Epoch 182/500:  36%|███▋      | 182/500 [01:20<02:12,  2.39it/s, v_num=1, train_loss=887]

Epoch 183/500:  36%|███▋      | 182/500 [01:20<02:12,  2.39it/s, v_num=1, train_loss=887]

Epoch 183/500:  37%|███▋      | 183/500 [01:21<02:09,  2.44it/s, v_num=1, train_loss=887]

Epoch 183/500:  37%|███▋      | 183/500 [01:21<02:09,  2.44it/s, v_num=1, train_loss=887]

Epoch 184/500:  37%|███▋      | 183/500 [01:21<02:09,  2.44it/s, v_num=1, train_loss=887]

Epoch 184/500:  37%|███▋      | 184/500 [01:21<02:07,  2.47it/s, v_num=1, train_loss=887]

Epoch 184/500:  37%|███▋      | 184/500 [01:21<02:07,  2.47it/s, v_num=1, train_loss=887]

Epoch 185/500:  37%|███▋      | 184/500 [01:21<02:07,  2.47it/s, v_num=1, train_loss=887]

Epoch 185/500:  37%|███▋      | 185/500 [01:21<02:05,  2.51it/s, v_num=1, train_loss=887]

Epoch 185/500:  37%|███▋      | 185/500 [01:21<02:05,  2.51it/s, v_num=1, train_loss=887]

Epoch 186/500:  37%|███▋      | 185/500 [01:21<02:05,  2.51it/s, v_num=1, train_loss=887]

Epoch 186/500:  37%|███▋      | 186/500 [01:22<02:03,  2.54it/s, v_num=1, train_loss=887]

Epoch 186/500:  37%|███▋      | 186/500 [01:22<02:03,  2.54it/s, v_num=1, train_loss=887]

Epoch 187/500:  37%|███▋      | 186/500 [01:22<02:03,  2.54it/s, v_num=1, train_loss=887]

Epoch 187/500:  37%|███▋      | 187/500 [01:22<02:02,  2.56it/s, v_num=1, train_loss=887]

Epoch 187/500:  37%|███▋      | 187/500 [01:22<02:02,  2.56it/s, v_num=1, train_loss=887]

Epoch 188/500:  37%|███▋      | 187/500 [01:22<02:02,  2.56it/s, v_num=1, train_loss=887]

Epoch 188/500:  38%|███▊      | 188/500 [01:23<02:01,  2.56it/s, v_num=1, train_loss=887]

Epoch 188/500:  38%|███▊      | 188/500 [01:23<02:01,  2.56it/s, v_num=1, train_loss=887]

Epoch 189/500:  38%|███▊      | 188/500 [01:23<02:01,  2.56it/s, v_num=1, train_loss=887]

Epoch 189/500:  38%|███▊      | 189/500 [01:23<02:02,  2.55it/s, v_num=1, train_loss=887]

Epoch 189/500:  38%|███▊      | 189/500 [01:23<02:02,  2.55it/s, v_num=1, train_loss=887]

Epoch 190/500:  38%|███▊      | 189/500 [01:23<02:02,  2.55it/s, v_num=1, train_loss=887]

Epoch 190/500:  38%|███▊      | 190/500 [01:23<02:00,  2.56it/s, v_num=1, train_loss=887]

Epoch 190/500:  38%|███▊      | 190/500 [01:23<02:00,  2.56it/s, v_num=1, train_loss=887]

Epoch 191/500:  38%|███▊      | 190/500 [01:23<02:00,  2.56it/s, v_num=1, train_loss=887]

Epoch 191/500:  38%|███▊      | 191/500 [01:24<02:00,  2.57it/s, v_num=1, train_loss=887]

Epoch 191/500:  38%|███▊      | 191/500 [01:24<02:00,  2.57it/s, v_num=1, train_loss=887]

Epoch 192/500:  38%|███▊      | 191/500 [01:24<02:00,  2.57it/s, v_num=1, train_loss=887]

Epoch 192/500:  38%|███▊      | 192/500 [01:24<01:59,  2.57it/s, v_num=1, train_loss=887]

Epoch 192/500:  38%|███▊      | 192/500 [01:24<01:59,  2.57it/s, v_num=1, train_loss=887]

Epoch 193/500:  38%|███▊      | 192/500 [01:24<01:59,  2.57it/s, v_num=1, train_loss=887]

Epoch 193/500:  39%|███▊      | 193/500 [01:25<01:59,  2.58it/s, v_num=1, train_loss=887]

Epoch 193/500:  39%|███▊      | 193/500 [01:25<01:59,  2.58it/s, v_num=1, train_loss=887]

Epoch 194/500:  39%|███▊      | 193/500 [01:25<01:59,  2.58it/s, v_num=1, train_loss=887]

Epoch 194/500:  39%|███▉      | 194/500 [01:25<01:58,  2.58it/s, v_num=1, train_loss=887]

Epoch 194/500:  39%|███▉      | 194/500 [01:25<01:58,  2.58it/s, v_num=1, train_loss=887]

Epoch 195/500:  39%|███▉      | 194/500 [01:25<01:58,  2.58it/s, v_num=1, train_loss=887]

Epoch 195/500:  39%|███▉      | 195/500 [01:25<01:58,  2.57it/s, v_num=1, train_loss=887]

Epoch 195/500:  39%|███▉      | 195/500 [01:25<01:58,  2.57it/s, v_num=1, train_loss=887]

Epoch 196/500:  39%|███▉      | 195/500 [01:25<01:58,  2.57it/s, v_num=1, train_loss=887]

Epoch 196/500:  39%|███▉      | 196/500 [01:26<01:57,  2.58it/s, v_num=1, train_loss=887]

Epoch 196/500:  39%|███▉      | 196/500 [01:26<01:57,  2.58it/s, v_num=1, train_loss=887]

Epoch 197/500:  39%|███▉      | 196/500 [01:26<01:57,  2.58it/s, v_num=1, train_loss=887]

Epoch 197/500:  39%|███▉      | 197/500 [01:26<01:57,  2.59it/s, v_num=1, train_loss=887]

Epoch 197/500:  39%|███▉      | 197/500 [01:26<01:57,  2.59it/s, v_num=1, train_loss=887]

Epoch 198/500:  39%|███▉      | 197/500 [01:26<01:57,  2.59it/s, v_num=1, train_loss=887]

Epoch 198/500:  40%|███▉      | 198/500 [01:27<01:56,  2.59it/s, v_num=1, train_loss=887]

Epoch 198/500:  40%|███▉      | 198/500 [01:27<01:56,  2.59it/s, v_num=1, train_loss=887]

Epoch 199/500:  40%|███▉      | 198/500 [01:27<01:56,  2.59it/s, v_num=1, train_loss=887]

Epoch 199/500:  40%|███▉      | 199/500 [01:27<01:55,  2.60it/s, v_num=1, train_loss=887]

Epoch 199/500:  40%|███▉      | 199/500 [01:27<01:55,  2.60it/s, v_num=1, train_loss=887]

Epoch 200/500:  40%|███▉      | 199/500 [01:27<01:55,  2.60it/s, v_num=1, train_loss=887]

Epoch 200/500:  40%|████      | 200/500 [01:27<01:54,  2.61it/s, v_num=1, train_loss=887]

Epoch 200/500:  40%|████      | 200/500 [01:27<01:54,  2.61it/s, v_num=1, train_loss=887]

Epoch 201/500:  40%|████      | 200/500 [01:27<01:54,  2.61it/s, v_num=1, train_loss=887]

Epoch 201/500:  40%|████      | 201/500 [01:28<01:54,  2.61it/s, v_num=1, train_loss=887]

Epoch 201/500:  40%|████      | 201/500 [01:28<01:54,  2.61it/s, v_num=1, train_loss=887]

Epoch 202/500:  40%|████      | 201/500 [01:28<01:54,  2.61it/s, v_num=1, train_loss=887]

Epoch 202/500:  40%|████      | 202/500 [01:28<01:53,  2.62it/s, v_num=1, train_loss=887]

Epoch 202/500:  40%|████      | 202/500 [01:28<01:53,  2.62it/s, v_num=1, train_loss=887]

Epoch 203/500:  40%|████      | 202/500 [01:28<01:53,  2.62it/s, v_num=1, train_loss=887]

Epoch 203/500:  41%|████      | 203/500 [01:28<01:53,  2.62it/s, v_num=1, train_loss=887]

Epoch 203/500:  41%|████      | 203/500 [01:28<01:53,  2.62it/s, v_num=1, train_loss=887]

Epoch 204/500:  41%|████      | 203/500 [01:28<01:53,  2.62it/s, v_num=1, train_loss=887]

Epoch 204/500:  41%|████      | 204/500 [01:29<01:54,  2.58it/s, v_num=1, train_loss=887]

Epoch 204/500:  41%|████      | 204/500 [01:29<01:54,  2.58it/s, v_num=1, train_loss=887]

Epoch 205/500:  41%|████      | 204/500 [01:29<01:54,  2.58it/s, v_num=1, train_loss=887]

Epoch 205/500:  41%|████      | 205/500 [01:29<02:07,  2.32it/s, v_num=1, train_loss=887]

Epoch 205/500:  41%|████      | 205/500 [01:29<02:07,  2.32it/s, v_num=1, train_loss=888]

Epoch 206/500:  41%|████      | 205/500 [01:29<02:07,  2.32it/s, v_num=1, train_loss=888]

Epoch 206/500:  41%|████      | 206/500 [01:30<02:07,  2.30it/s, v_num=1, train_loss=888]

Epoch 206/500:  41%|████      | 206/500 [01:30<02:07,  2.30it/s, v_num=1, train_loss=887]

Epoch 207/500:  41%|████      | 206/500 [01:30<02:07,  2.30it/s, v_num=1, train_loss=887]

Epoch 207/500:  41%|████▏     | 207/500 [01:30<02:06,  2.32it/s, v_num=1, train_loss=887]

Epoch 207/500:  41%|████▏     | 207/500 [01:30<02:06,  2.32it/s, v_num=1, train_loss=888]

Epoch 208/500:  41%|████▏     | 207/500 [01:30<02:06,  2.32it/s, v_num=1, train_loss=888]

Epoch 208/500:  42%|████▏     | 208/500 [01:31<02:05,  2.33it/s, v_num=1, train_loss=888]

Epoch 208/500:  42%|████▏     | 208/500 [01:31<02:05,  2.33it/s, v_num=1, train_loss=888]

Epoch 209/500:  42%|████▏     | 208/500 [01:31<02:05,  2.33it/s, v_num=1, train_loss=888]

Epoch 209/500:  42%|████▏     | 209/500 [01:31<02:06,  2.30it/s, v_num=1, train_loss=888]

Epoch 209/500:  42%|████▏     | 209/500 [01:31<02:06,  2.30it/s, v_num=1, train_loss=888]

Epoch 210/500:  42%|████▏     | 209/500 [01:31<02:06,  2.30it/s, v_num=1, train_loss=888]

Epoch 210/500:  42%|████▏     | 210/500 [01:32<02:04,  2.33it/s, v_num=1, train_loss=888]

Epoch 210/500:  42%|████▏     | 210/500 [01:32<02:04,  2.33it/s, v_num=1, train_loss=888]

Epoch 211/500:  42%|████▏     | 210/500 [01:32<02:04,  2.33it/s, v_num=1, train_loss=888]

Epoch 211/500:  42%|████▏     | 211/500 [01:32<02:02,  2.35it/s, v_num=1, train_loss=888]

Epoch 211/500:  42%|████▏     | 211/500 [01:32<02:02,  2.35it/s, v_num=1, train_loss=888]

Epoch 212/500:  42%|████▏     | 211/500 [01:32<02:02,  2.35it/s, v_num=1, train_loss=888]

Epoch 212/500:  42%|████▏     | 212/500 [01:32<01:59,  2.42it/s, v_num=1, train_loss=888]

Epoch 212/500:  42%|████▏     | 212/500 [01:32<01:59,  2.42it/s, v_num=1, train_loss=888]

Epoch 213/500:  42%|████▏     | 212/500 [01:32<01:59,  2.42it/s, v_num=1, train_loss=888]

Epoch 213/500:  43%|████▎     | 213/500 [01:33<01:56,  2.46it/s, v_num=1, train_loss=888]

Epoch 213/500:  43%|████▎     | 213/500 [01:33<01:56,  2.46it/s, v_num=1, train_loss=888]

Epoch 214/500:  43%|████▎     | 213/500 [01:33<01:56,  2.46it/s, v_num=1, train_loss=888]

Epoch 214/500:  43%|████▎     | 214/500 [01:33<02:09,  2.21it/s, v_num=1, train_loss=888]

Epoch 214/500:  43%|████▎     | 214/500 [01:33<02:09,  2.21it/s, v_num=1, train_loss=888]

Epoch 215/500:  43%|████▎     | 214/500 [01:33<02:09,  2.21it/s, v_num=1, train_loss=888]

Epoch 215/500:  43%|████▎     | 215/500 [01:34<02:08,  2.21it/s, v_num=1, train_loss=888]

Epoch 215/500:  43%|████▎     | 215/500 [01:34<02:08,  2.21it/s, v_num=1, train_loss=888]

Epoch 216/500:  43%|████▎     | 215/500 [01:34<02:08,  2.21it/s, v_num=1, train_loss=888]

Epoch 216/500:  43%|████▎     | 216/500 [01:34<02:04,  2.29it/s, v_num=1, train_loss=888]

Epoch 216/500:  43%|████▎     | 216/500 [01:34<02:04,  2.29it/s, v_num=1, train_loss=888]

Epoch 217/500:  43%|████▎     | 216/500 [01:34<02:04,  2.29it/s, v_num=1, train_loss=888]

Epoch 217/500:  43%|████▎     | 217/500 [01:35<02:00,  2.35it/s, v_num=1, train_loss=888]

Epoch 217/500:  43%|████▎     | 217/500 [01:35<02:00,  2.35it/s, v_num=1, train_loss=888]

Epoch 218/500:  43%|████▎     | 217/500 [01:35<02:00,  2.35it/s, v_num=1, train_loss=888]

Epoch 218/500:  44%|████▎     | 218/500 [01:35<01:58,  2.39it/s, v_num=1, train_loss=888]

Epoch 218/500:  44%|████▎     | 218/500 [01:35<01:58,  2.39it/s, v_num=1, train_loss=888]

Epoch 219/500:  44%|████▎     | 218/500 [01:35<01:58,  2.39it/s, v_num=1, train_loss=888]

Epoch 219/500:  44%|████▍     | 219/500 [01:35<01:57,  2.39it/s, v_num=1, train_loss=888]

Epoch 219/500:  44%|████▍     | 219/500 [01:35<01:57,  2.39it/s, v_num=1, train_loss=888]

Epoch 220/500:  44%|████▍     | 219/500 [01:35<01:57,  2.39it/s, v_num=1, train_loss=888]

Epoch 220/500:  44%|████▍     | 220/500 [01:36<02:00,  2.33it/s, v_num=1, train_loss=888]

Epoch 220/500:  44%|████▍     | 220/500 [01:36<02:00,  2.33it/s, v_num=1, train_loss=888]

Epoch 221/500:  44%|████▍     | 220/500 [01:36<02:00,  2.33it/s, v_num=1, train_loss=888]

Epoch 221/500:  44%|████▍     | 221/500 [01:36<02:01,  2.29it/s, v_num=1, train_loss=888]

Epoch 221/500:  44%|████▍     | 221/500 [01:36<02:01,  2.29it/s, v_num=1, train_loss=888]

Epoch 222/500:  44%|████▍     | 221/500 [01:36<02:01,  2.29it/s, v_num=1, train_loss=888]

Epoch 222/500:  44%|████▍     | 222/500 [01:37<02:00,  2.30it/s, v_num=1, train_loss=888]

Epoch 222/500:  44%|████▍     | 222/500 [01:37<02:00,  2.30it/s, v_num=1, train_loss=888]

Epoch 223/500:  44%|████▍     | 222/500 [01:37<02:00,  2.30it/s, v_num=1, train_loss=888]

Epoch 223/500:  45%|████▍     | 223/500 [01:37<01:59,  2.31it/s, v_num=1, train_loss=888]

Epoch 223/500:  45%|████▍     | 223/500 [01:37<01:59,  2.31it/s, v_num=1, train_loss=888]

Epoch 224/500:  45%|████▍     | 223/500 [01:37<01:59,  2.31it/s, v_num=1, train_loss=888]

Epoch 224/500:  45%|████▍     | 224/500 [01:38<01:58,  2.32it/s, v_num=1, train_loss=888]

Epoch 224/500:  45%|████▍     | 224/500 [01:38<01:58,  2.32it/s, v_num=1, train_loss=888]

Epoch 225/500:  45%|████▍     | 224/500 [01:38<01:58,  2.32it/s, v_num=1, train_loss=888]

Epoch 225/500:  45%|████▌     | 225/500 [01:38<01:57,  2.33it/s, v_num=1, train_loss=888]

Epoch 225/500:  45%|████▌     | 225/500 [01:38<01:57,  2.33it/s, v_num=1, train_loss=888]

Epoch 226/500:  45%|████▌     | 225/500 [01:38<01:57,  2.33it/s, v_num=1, train_loss=888]

Epoch 226/500:  45%|████▌     | 226/500 [01:38<02:02,  2.24it/s, v_num=1, train_loss=888]

Epoch 226/500:  45%|████▌     | 226/500 [01:38<02:02,  2.24it/s, v_num=1, train_loss=888]

Epoch 227/500:  45%|████▌     | 226/500 [01:38<02:02,  2.24it/s, v_num=1, train_loss=888]

Epoch 227/500:  45%|████▌     | 227/500 [01:39<02:01,  2.24it/s, v_num=1, train_loss=888]

Epoch 227/500:  45%|████▌     | 227/500 [01:39<02:01,  2.24it/s, v_num=1, train_loss=888]

Epoch 228/500:  45%|████▌     | 227/500 [01:39<02:01,  2.24it/s, v_num=1, train_loss=888]

Epoch 228/500:  46%|████▌     | 228/500 [01:39<01:57,  2.32it/s, v_num=1, train_loss=888]

Epoch 228/500:  46%|████▌     | 228/500 [01:39<01:57,  2.32it/s, v_num=1, train_loss=888]

Epoch 229/500:  46%|████▌     | 228/500 [01:39<01:57,  2.32it/s, v_num=1, train_loss=888]

Epoch 229/500:  46%|████▌     | 229/500 [01:40<01:54,  2.37it/s, v_num=1, train_loss=888]

Epoch 229/500:  46%|████▌     | 229/500 [01:40<01:54,  2.37it/s, v_num=1, train_loss=888]

Epoch 230/500:  46%|████▌     | 229/500 [01:40<01:54,  2.37it/s, v_num=1, train_loss=888]

Epoch 230/500:  46%|████▌     | 230/500 [01:40<01:51,  2.42it/s, v_num=1, train_loss=888]

Epoch 230/500:  46%|████▌     | 230/500 [01:40<01:51,  2.42it/s, v_num=1, train_loss=888]

Epoch 231/500:  46%|████▌     | 230/500 [01:40<01:51,  2.42it/s, v_num=1, train_loss=888]

Epoch 231/500:  46%|████▌     | 231/500 [01:40<01:49,  2.46it/s, v_num=1, train_loss=888]

Epoch 231/500:  46%|████▌     | 231/500 [01:40<01:49,  2.46it/s, v_num=1, train_loss=889]

Epoch 232/500:  46%|████▌     | 231/500 [01:40<01:49,  2.46it/s, v_num=1, train_loss=889]

Epoch 232/500:  46%|████▋     | 232/500 [01:41<01:48,  2.46it/s, v_num=1, train_loss=889]

Epoch 232/500:  46%|████▋     | 232/500 [01:41<01:48,  2.46it/s, v_num=1, train_loss=888]

Epoch 233/500:  46%|████▋     | 232/500 [01:41<01:48,  2.46it/s, v_num=1, train_loss=888]

Epoch 233/500:  47%|████▋     | 233/500 [01:41<01:54,  2.34it/s, v_num=1, train_loss=888]

Epoch 233/500:  47%|████▋     | 233/500 [01:41<01:54,  2.34it/s, v_num=1, train_loss=888]

Epoch 234/500:  47%|████▋     | 233/500 [01:41<01:54,  2.34it/s, v_num=1, train_loss=888]

Epoch 234/500:  47%|████▋     | 234/500 [01:42<01:57,  2.26it/s, v_num=1, train_loss=888]

Epoch 234/500:  47%|████▋     | 234/500 [01:42<01:57,  2.26it/s, v_num=1, train_loss=889]

Epoch 235/500:  47%|████▋     | 234/500 [01:42<01:57,  2.26it/s, v_num=1, train_loss=889]

Epoch 235/500:  47%|████▋     | 235/500 [01:42<01:56,  2.27it/s, v_num=1, train_loss=889]

Epoch 235/500:  47%|████▋     | 235/500 [01:42<01:56,  2.27it/s, v_num=1, train_loss=889]

Epoch 236/500:  47%|████▋     | 235/500 [01:42<01:56,  2.27it/s, v_num=1, train_loss=889]

Epoch 236/500:  47%|████▋     | 236/500 [01:43<02:00,  2.20it/s, v_num=1, train_loss=889]

Epoch 236/500:  47%|████▋     | 236/500 [01:43<02:00,  2.20it/s, v_num=1, train_loss=889]

Epoch 237/500:  47%|████▋     | 236/500 [01:43<02:00,  2.20it/s, v_num=1, train_loss=889]

Epoch 237/500:  47%|████▋     | 237/500 [01:43<02:00,  2.18it/s, v_num=1, train_loss=889]

Epoch 237/500:  47%|████▋     | 237/500 [01:43<02:00,  2.18it/s, v_num=1, train_loss=889]

Epoch 238/500:  47%|████▋     | 237/500 [01:43<02:00,  2.18it/s, v_num=1, train_loss=889]

Epoch 238/500:  48%|████▊     | 238/500 [01:44<02:01,  2.16it/s, v_num=1, train_loss=889]

Epoch 238/500:  48%|████▊     | 238/500 [01:44<02:01,  2.16it/s, v_num=1, train_loss=889]

Epoch 239/500:  48%|████▊     | 238/500 [01:44<02:01,  2.16it/s, v_num=1, train_loss=889]

Epoch 239/500:  48%|████▊     | 239/500 [01:44<02:09,  2.01it/s, v_num=1, train_loss=889]

Epoch 239/500:  48%|████▊     | 239/500 [01:44<02:09,  2.01it/s, v_num=1, train_loss=889]

Epoch 240/500:  48%|████▊     | 239/500 [01:44<02:09,  2.01it/s, v_num=1, train_loss=889]

Epoch 240/500:  48%|████▊     | 240/500 [01:45<02:11,  1.98it/s, v_num=1, train_loss=889]

Epoch 240/500:  48%|████▊     | 240/500 [01:45<02:11,  1.98it/s, v_num=1, train_loss=889]

Epoch 241/500:  48%|████▊     | 240/500 [01:45<02:11,  1.98it/s, v_num=1, train_loss=889]

Epoch 241/500:  48%|████▊     | 241/500 [01:45<02:06,  2.05it/s, v_num=1, train_loss=889]

Epoch 241/500:  48%|████▊     | 241/500 [01:45<02:06,  2.05it/s, v_num=1, train_loss=889]

Epoch 242/500:  48%|████▊     | 241/500 [01:45<02:06,  2.05it/s, v_num=1, train_loss=889]

Epoch 242/500:  48%|████▊     | 242/500 [01:46<02:02,  2.11it/s, v_num=1, train_loss=889]

Epoch 242/500:  48%|████▊     | 242/500 [01:46<02:02,  2.11it/s, v_num=1, train_loss=889]

Epoch 243/500:  48%|████▊     | 242/500 [01:46<02:02,  2.11it/s, v_num=1, train_loss=889]

Epoch 243/500:  49%|████▊     | 243/500 [01:46<01:59,  2.16it/s, v_num=1, train_loss=889]

Epoch 243/500:  49%|████▊     | 243/500 [01:46<01:59,  2.16it/s, v_num=1, train_loss=889]

Epoch 244/500:  49%|████▊     | 243/500 [01:46<01:59,  2.16it/s, v_num=1, train_loss=889]

Epoch 244/500:  49%|████▉     | 244/500 [01:47<01:58,  2.16it/s, v_num=1, train_loss=889]

Epoch 244/500:  49%|████▉     | 244/500 [01:47<01:58,  2.16it/s, v_num=1, train_loss=889]

Epoch 245/500:  49%|████▉     | 244/500 [01:47<01:58,  2.16it/s, v_num=1, train_loss=889]

Epoch 245/500:  49%|████▉     | 245/500 [01:47<01:58,  2.16it/s, v_num=1, train_loss=889]

Epoch 245/500:  49%|████▉     | 245/500 [01:47<01:58,  2.16it/s, v_num=1, train_loss=889]

Epoch 246/500:  49%|████▉     | 245/500 [01:47<01:58,  2.16it/s, v_num=1, train_loss=889]

Epoch 246/500:  49%|████▉     | 246/500 [01:47<01:56,  2.17it/s, v_num=1, train_loss=889]

Epoch 246/500:  49%|████▉     | 246/500 [01:47<01:56,  2.17it/s, v_num=1, train_loss=889]

Epoch 247/500:  49%|████▉     | 246/500 [01:47<01:56,  2.17it/s, v_num=1, train_loss=889]

Epoch 247/500:  49%|████▉     | 247/500 [01:48<01:56,  2.17it/s, v_num=1, train_loss=889]

Epoch 247/500:  49%|████▉     | 247/500 [01:48<01:56,  2.17it/s, v_num=1, train_loss=889]

Epoch 248/500:  49%|████▉     | 247/500 [01:48<01:56,  2.17it/s, v_num=1, train_loss=889]

Epoch 248/500:  50%|████▉     | 248/500 [01:48<01:56,  2.16it/s, v_num=1, train_loss=889]

Epoch 248/500:  50%|████▉     | 248/500 [01:48<01:56,  2.16it/s, v_num=1, train_loss=889]

Epoch 249/500:  50%|████▉     | 248/500 [01:48<01:56,  2.16it/s, v_num=1, train_loss=889]

Epoch 249/500:  50%|████▉     | 249/500 [01:49<01:53,  2.22it/s, v_num=1, train_loss=889]

Epoch 249/500:  50%|████▉     | 249/500 [01:49<01:53,  2.22it/s, v_num=1, train_loss=889]

Epoch 250/500:  50%|████▉     | 249/500 [01:49<01:53,  2.22it/s, v_num=1, train_loss=889]

Epoch 250/500:  50%|█████     | 250/500 [01:49<01:50,  2.26it/s, v_num=1, train_loss=889]

Epoch 250/500:  50%|█████     | 250/500 [01:49<01:50,  2.26it/s, v_num=1, train_loss=889]

Epoch 251/500:  50%|█████     | 250/500 [01:49<01:50,  2.26it/s, v_num=1, train_loss=889]

Epoch 251/500:  50%|█████     | 251/500 [01:50<01:50,  2.26it/s, v_num=1, train_loss=889]

Epoch 251/500:  50%|█████     | 251/500 [01:50<01:50,  2.26it/s, v_num=1, train_loss=889]

Epoch 252/500:  50%|█████     | 251/500 [01:50<01:50,  2.26it/s, v_num=1, train_loss=889]

Epoch 252/500:  50%|█████     | 252/500 [01:50<01:50,  2.25it/s, v_num=1, train_loss=889]

Epoch 252/500:  50%|█████     | 252/500 [01:50<01:50,  2.25it/s, v_num=1, train_loss=889]

Epoch 253/500:  50%|█████     | 252/500 [01:50<01:50,  2.25it/s, v_num=1, train_loss=889]

Epoch 253/500:  51%|█████     | 253/500 [01:51<01:52,  2.19it/s, v_num=1, train_loss=889]

Epoch 253/500:  51%|█████     | 253/500 [01:51<01:52,  2.19it/s, v_num=1, train_loss=889]

Epoch 254/500:  51%|█████     | 253/500 [01:51<01:52,  2.19it/s, v_num=1, train_loss=889]

Epoch 254/500:  51%|█████     | 254/500 [01:51<01:51,  2.21it/s, v_num=1, train_loss=889]

Epoch 254/500:  51%|█████     | 254/500 [01:51<01:51,  2.21it/s, v_num=1, train_loss=889]

Epoch 255/500:  51%|█████     | 254/500 [01:51<01:51,  2.21it/s, v_num=1, train_loss=889]

Epoch 255/500:  51%|█████     | 255/500 [01:52<01:48,  2.26it/s, v_num=1, train_loss=889]

Epoch 255/500:  51%|█████     | 255/500 [01:52<01:48,  2.26it/s, v_num=1, train_loss=889]

Epoch 256/500:  51%|█████     | 255/500 [01:52<01:48,  2.26it/s, v_num=1, train_loss=889]

Epoch 256/500:  51%|█████     | 256/500 [01:52<01:44,  2.34it/s, v_num=1, train_loss=889]

Epoch 256/500:  51%|█████     | 256/500 [01:52<01:44,  2.34it/s, v_num=1, train_loss=889]

Epoch 257/500:  51%|█████     | 256/500 [01:52<01:44,  2.34it/s, v_num=1, train_loss=889]

Epoch 257/500:  51%|█████▏    | 257/500 [01:52<01:40,  2.41it/s, v_num=1, train_loss=889]

Epoch 257/500:  51%|█████▏    | 257/500 [01:52<01:40,  2.41it/s, v_num=1, train_loss=889]

Epoch 258/500:  51%|█████▏    | 257/500 [01:52<01:40,  2.41it/s, v_num=1, train_loss=889]

Epoch 258/500:  52%|█████▏    | 258/500 [01:53<01:38,  2.46it/s, v_num=1, train_loss=889]

Epoch 258/500:  52%|█████▏    | 258/500 [01:53<01:38,  2.46it/s, v_num=1, train_loss=889]

Epoch 259/500:  52%|█████▏    | 258/500 [01:53<01:38,  2.46it/s, v_num=1, train_loss=889]

Epoch 259/500:  52%|█████▏    | 259/500 [01:53<01:39,  2.43it/s, v_num=1, train_loss=889]

Epoch 259/500:  52%|█████▏    | 259/500 [01:53<01:39,  2.43it/s, v_num=1, train_loss=889]

Epoch 260/500:  52%|█████▏    | 259/500 [01:53<01:39,  2.43it/s, v_num=1, train_loss=889]

Epoch 260/500:  52%|█████▏    | 260/500 [01:54<01:42,  2.35it/s, v_num=1, train_loss=889]

Epoch 260/500:  52%|█████▏    | 260/500 [01:54<01:42,  2.35it/s, v_num=1, train_loss=890]

Epoch 261/500:  52%|█████▏    | 260/500 [01:54<01:42,  2.35it/s, v_num=1, train_loss=890]

Epoch 261/500:  52%|█████▏    | 261/500 [01:54<01:43,  2.30it/s, v_num=1, train_loss=890]

Epoch 261/500:  52%|█████▏    | 261/500 [01:54<01:43,  2.30it/s, v_num=1, train_loss=890]

Epoch 262/500:  52%|█████▏    | 261/500 [01:54<01:43,  2.30it/s, v_num=1, train_loss=890]

Epoch 262/500:  52%|█████▏    | 262/500 [01:55<01:51,  2.13it/s, v_num=1, train_loss=890]

Epoch 262/500:  52%|█████▏    | 262/500 [01:55<01:51,  2.13it/s, v_num=1, train_loss=890]

Epoch 263/500:  52%|█████▏    | 262/500 [01:55<01:51,  2.13it/s, v_num=1, train_loss=890]

Epoch 263/500:  53%|█████▎    | 263/500 [01:55<01:56,  2.03it/s, v_num=1, train_loss=890]

Epoch 263/500:  53%|█████▎    | 263/500 [01:55<01:56,  2.03it/s, v_num=1, train_loss=890]

Epoch 264/500:  53%|█████▎    | 263/500 [01:55<01:56,  2.03it/s, v_num=1, train_loss=890]

Epoch 264/500:  53%|█████▎    | 264/500 [01:56<02:00,  1.96it/s, v_num=1, train_loss=890]

Epoch 264/500:  53%|█████▎    | 264/500 [01:56<02:00,  1.96it/s, v_num=1, train_loss=890]

Epoch 265/500:  53%|█████▎    | 264/500 [01:56<02:00,  1.96it/s, v_num=1, train_loss=890]

Epoch 265/500:  53%|█████▎    | 265/500 [01:56<02:02,  1.91it/s, v_num=1, train_loss=890]

Epoch 265/500:  53%|█████▎    | 265/500 [01:56<02:02,  1.91it/s, v_num=1, train_loss=890]

Epoch 266/500:  53%|█████▎    | 265/500 [01:56<02:02,  1.91it/s, v_num=1, train_loss=890]

Epoch 266/500:  53%|█████▎    | 266/500 [01:57<01:59,  1.97it/s, v_num=1, train_loss=890]

Epoch 266/500:  53%|█████▎    | 266/500 [01:57<01:59,  1.97it/s, v_num=1, train_loss=890]

Epoch 267/500:  53%|█████▎    | 266/500 [01:57<01:59,  1.97it/s, v_num=1, train_loss=890]

Epoch 267/500:  53%|█████▎    | 267/500 [01:57<01:59,  1.94it/s, v_num=1, train_loss=890]

Epoch 267/500:  53%|█████▎    | 267/500 [01:57<01:59,  1.94it/s, v_num=1, train_loss=890]

Epoch 268/500:  53%|█████▎    | 267/500 [01:57<01:59,  1.94it/s, v_num=1, train_loss=890]

Epoch 268/500:  54%|█████▎    | 268/500 [01:58<01:54,  2.03it/s, v_num=1, train_loss=890]

Epoch 268/500:  54%|█████▎    | 268/500 [01:58<01:54,  2.03it/s, v_num=1, train_loss=890]

Epoch 269/500:  54%|█████▎    | 268/500 [01:58<01:54,  2.03it/s, v_num=1, train_loss=890]

Epoch 269/500:  54%|█████▍    | 269/500 [01:58<01:57,  1.97it/s, v_num=1, train_loss=890]

Epoch 269/500:  54%|█████▍    | 269/500 [01:58<01:57,  1.97it/s, v_num=1, train_loss=890]

Epoch 270/500:  54%|█████▍    | 269/500 [01:58<01:57,  1.97it/s, v_num=1, train_loss=890]

Epoch 270/500:  54%|█████▍    | 270/500 [01:59<01:49,  2.10it/s, v_num=1, train_loss=890]

Epoch 270/500:  54%|█████▍    | 270/500 [01:59<01:49,  2.10it/s, v_num=1, train_loss=890]

Epoch 271/500:  54%|█████▍    | 270/500 [01:59<01:49,  2.10it/s, v_num=1, train_loss=890]

Epoch 271/500:  54%|█████▍    | 271/500 [01:59<01:53,  2.03it/s, v_num=1, train_loss=890]

Epoch 271/500:  54%|█████▍    | 271/500 [01:59<01:53,  2.03it/s, v_num=1, train_loss=890]

Epoch 272/500:  54%|█████▍    | 271/500 [01:59<01:53,  2.03it/s, v_num=1, train_loss=890]

Epoch 272/500:  54%|█████▍    | 272/500 [02:00<01:45,  2.16it/s, v_num=1, train_loss=890]

Epoch 272/500:  54%|█████▍    | 272/500 [02:00<01:45,  2.16it/s, v_num=1, train_loss=890]

Epoch 273/500:  54%|█████▍    | 272/500 [02:00<01:45,  2.16it/s, v_num=1, train_loss=890]

Epoch 273/500:  55%|█████▍    | 273/500 [02:00<01:40,  2.26it/s, v_num=1, train_loss=890]

Epoch 273/500:  55%|█████▍    | 273/500 [02:00<01:40,  2.26it/s, v_num=1, train_loss=890]

Epoch 274/500:  55%|█████▍    | 273/500 [02:00<01:40,  2.26it/s, v_num=1, train_loss=890]

Epoch 274/500:  55%|█████▍    | 274/500 [02:00<01:40,  2.25it/s, v_num=1, train_loss=890]

Epoch 274/500:  55%|█████▍    | 274/500 [02:00<01:40,  2.25it/s, v_num=1, train_loss=890]

Epoch 275/500:  55%|█████▍    | 274/500 [02:00<01:40,  2.25it/s, v_num=1, train_loss=890]

Epoch 275/500:  55%|█████▌    | 275/500 [02:01<01:40,  2.25it/s, v_num=1, train_loss=890]

Epoch 275/500:  55%|█████▌    | 275/500 [02:01<01:40,  2.25it/s, v_num=1, train_loss=890]

Epoch 276/500:  55%|█████▌    | 275/500 [02:01<01:40,  2.25it/s, v_num=1, train_loss=890]

Epoch 276/500:  55%|█████▌    | 276/500 [02:01<01:39,  2.25it/s, v_num=1, train_loss=890]

Epoch 276/500:  55%|█████▌    | 276/500 [02:01<01:39,  2.25it/s, v_num=1, train_loss=890]

Epoch 277/500:  55%|█████▌    | 276/500 [02:01<01:39,  2.25it/s, v_num=1, train_loss=890]

Epoch 277/500:  55%|█████▌    | 277/500 [02:02<01:38,  2.27it/s, v_num=1, train_loss=890]

Epoch 277/500:  55%|█████▌    | 277/500 [02:02<01:38,  2.27it/s, v_num=1, train_loss=890]

Epoch 278/500:  55%|█████▌    | 277/500 [02:02<01:38,  2.27it/s, v_num=1, train_loss=890]

Epoch 278/500:  56%|█████▌    | 278/500 [02:02<01:36,  2.30it/s, v_num=1, train_loss=890]

Epoch 278/500:  56%|█████▌    | 278/500 [02:02<01:36,  2.30it/s, v_num=1, train_loss=890]

Epoch 279/500:  56%|█████▌    | 278/500 [02:02<01:36,  2.30it/s, v_num=1, train_loss=890]

Epoch 279/500:  56%|█████▌    | 279/500 [02:03<01:34,  2.34it/s, v_num=1, train_loss=890]

Epoch 279/500:  56%|█████▌    | 279/500 [02:03<01:34,  2.34it/s, v_num=1, train_loss=890]

Epoch 280/500:  56%|█████▌    | 279/500 [02:03<01:34,  2.34it/s, v_num=1, train_loss=890]

Epoch 280/500:  56%|█████▌    | 280/500 [02:03<01:33,  2.36it/s, v_num=1, train_loss=890]

Epoch 280/500:  56%|█████▌    | 280/500 [02:03<01:33,  2.36it/s, v_num=1, train_loss=890]

Epoch 281/500:  56%|█████▌    | 280/500 [02:03<01:33,  2.36it/s, v_num=1, train_loss=890]

Epoch 281/500:  56%|█████▌    | 281/500 [02:03<01:33,  2.35it/s, v_num=1, train_loss=890]

Epoch 281/500:  56%|█████▌    | 281/500 [02:03<01:33,  2.35it/s, v_num=1, train_loss=890]

Epoch 282/500:  56%|█████▌    | 281/500 [02:03<01:33,  2.35it/s, v_num=1, train_loss=890]

Epoch 282/500:  56%|█████▋    | 282/500 [02:04<01:30,  2.42it/s, v_num=1, train_loss=890]

Epoch 282/500:  56%|█████▋    | 282/500 [02:04<01:30,  2.42it/s, v_num=1, train_loss=890]

Epoch 283/500:  56%|█████▋    | 282/500 [02:04<01:30,  2.42it/s, v_num=1, train_loss=890]

Epoch 283/500:  57%|█████▋    | 283/500 [02:04<01:29,  2.42it/s, v_num=1, train_loss=890]

Epoch 283/500:  57%|█████▋    | 283/500 [02:04<01:29,  2.42it/s, v_num=1, train_loss=890]

Epoch 284/500:  57%|█████▋    | 283/500 [02:04<01:29,  2.42it/s, v_num=1, train_loss=890]

Epoch 284/500:  57%|█████▋    | 284/500 [02:05<01:27,  2.46it/s, v_num=1, train_loss=890]

Epoch 284/500:  57%|█████▋    | 284/500 [02:05<01:27,  2.46it/s, v_num=1, train_loss=890]

Epoch 285/500:  57%|█████▋    | 284/500 [02:05<01:27,  2.46it/s, v_num=1, train_loss=890]

Epoch 285/500:  57%|█████▋    | 285/500 [02:05<01:25,  2.51it/s, v_num=1, train_loss=890]

Epoch 285/500:  57%|█████▋    | 285/500 [02:05<01:25,  2.51it/s, v_num=1, train_loss=890]

Epoch 286/500:  57%|█████▋    | 285/500 [02:05<01:25,  2.51it/s, v_num=1, train_loss=890]

Epoch 286/500:  57%|█████▋    | 286/500 [02:05<01:28,  2.43it/s, v_num=1, train_loss=890]

Epoch 286/500:  57%|█████▋    | 286/500 [02:05<01:28,  2.43it/s, v_num=1, train_loss=890]

Epoch 287/500:  57%|█████▋    | 286/500 [02:05<01:28,  2.43it/s, v_num=1, train_loss=890]

Epoch 287/500:  57%|█████▋    | 287/500 [02:06<01:29,  2.38it/s, v_num=1, train_loss=890]

Epoch 287/500:  57%|█████▋    | 287/500 [02:06<01:29,  2.38it/s, v_num=1, train_loss=890]

Epoch 288/500:  57%|█████▋    | 287/500 [02:06<01:29,  2.38it/s, v_num=1, train_loss=890]

Epoch 288/500:  58%|█████▊    | 288/500 [02:06<01:30,  2.34it/s, v_num=1, train_loss=890]

Epoch 288/500:  58%|█████▊    | 288/500 [02:06<01:30,  2.34it/s, v_num=1, train_loss=890]

Epoch 289/500:  58%|█████▊    | 288/500 [02:06<01:30,  2.34it/s, v_num=1, train_loss=890]

Epoch 289/500:  58%|█████▊    | 289/500 [02:07<01:31,  2.31it/s, v_num=1, train_loss=890]

Epoch 289/500:  58%|█████▊    | 289/500 [02:07<01:31,  2.31it/s, v_num=1, train_loss=890]

Epoch 290/500:  58%|█████▊    | 289/500 [02:07<01:31,  2.31it/s, v_num=1, train_loss=890]

Epoch 290/500:  58%|█████▊    | 290/500 [02:07<01:33,  2.24it/s, v_num=1, train_loss=890]

Epoch 290/500:  58%|█████▊    | 290/500 [02:07<01:33,  2.24it/s, v_num=1, train_loss=891]

Epoch 291/500:  58%|█████▊    | 290/500 [02:07<01:33,  2.24it/s, v_num=1, train_loss=891]

Epoch 291/500:  58%|█████▊    | 291/500 [02:08<01:33,  2.23it/s, v_num=1, train_loss=891]

Epoch 291/500:  58%|█████▊    | 291/500 [02:08<01:33,  2.23it/s, v_num=1, train_loss=891]

Epoch 292/500:  58%|█████▊    | 291/500 [02:08<01:33,  2.23it/s, v_num=1, train_loss=891]

Epoch 292/500:  58%|█████▊    | 292/500 [02:08<01:32,  2.26it/s, v_num=1, train_loss=891]

Epoch 292/500:  58%|█████▊    | 292/500 [02:08<01:32,  2.26it/s, v_num=1, train_loss=891]

Epoch 293/500:  58%|█████▊    | 292/500 [02:08<01:32,  2.26it/s, v_num=1, train_loss=891]

Epoch 293/500:  59%|█████▊    | 293/500 [02:08<01:29,  2.32it/s, v_num=1, train_loss=891]

Epoch 293/500:  59%|█████▊    | 293/500 [02:08<01:29,  2.32it/s, v_num=1, train_loss=891]

Epoch 294/500:  59%|█████▊    | 293/500 [02:08<01:29,  2.32it/s, v_num=1, train_loss=891]

Epoch 294/500:  59%|█████▉    | 294/500 [02:09<01:26,  2.38it/s, v_num=1, train_loss=891]

Epoch 294/500:  59%|█████▉    | 294/500 [02:09<01:26,  2.38it/s, v_num=1, train_loss=891]

Epoch 295/500:  59%|█████▉    | 294/500 [02:09<01:26,  2.38it/s, v_num=1, train_loss=891]

Epoch 295/500:  59%|█████▉    | 295/500 [02:09<01:23,  2.44it/s, v_num=1, train_loss=891]

Epoch 295/500:  59%|█████▉    | 295/500 [02:09<01:23,  2.44it/s, v_num=1, train_loss=891]

Epoch 296/500:  59%|█████▉    | 295/500 [02:09<01:23,  2.44it/s, v_num=1, train_loss=891]

Epoch 296/500:  59%|█████▉    | 296/500 [02:10<01:21,  2.49it/s, v_num=1, train_loss=891]

Epoch 296/500:  59%|█████▉    | 296/500 [02:10<01:21,  2.49it/s, v_num=1, train_loss=891]

Epoch 297/500:  59%|█████▉    | 296/500 [02:10<01:21,  2.49it/s, v_num=1, train_loss=891]

Epoch 297/500:  59%|█████▉    | 297/500 [02:10<01:20,  2.54it/s, v_num=1, train_loss=891]

Epoch 297/500:  59%|█████▉    | 297/500 [02:10<01:20,  2.54it/s, v_num=1, train_loss=891]

Epoch 298/500:  59%|█████▉    | 297/500 [02:10<01:20,  2.54it/s, v_num=1, train_loss=891]

Epoch 298/500:  60%|█████▉    | 298/500 [02:10<01:18,  2.56it/s, v_num=1, train_loss=891]

Epoch 298/500:  60%|█████▉    | 298/500 [02:10<01:18,  2.56it/s, v_num=1, train_loss=891]

Epoch 299/500:  60%|█████▉    | 298/500 [02:10<01:18,  2.56it/s, v_num=1, train_loss=891]

Epoch 299/500:  60%|█████▉    | 299/500 [02:11<01:17,  2.58it/s, v_num=1, train_loss=891]

Epoch 299/500:  60%|█████▉    | 299/500 [02:11<01:17,  2.58it/s, v_num=1, train_loss=891]

Epoch 300/500:  60%|█████▉    | 299/500 [02:11<01:17,  2.58it/s, v_num=1, train_loss=891]

Epoch 300/500:  60%|██████    | 300/500 [02:11<01:16,  2.60it/s, v_num=1, train_loss=891]

Epoch 300/500:  60%|██████    | 300/500 [02:11<01:16,  2.60it/s, v_num=1, train_loss=891]

Epoch 301/500:  60%|██████    | 300/500 [02:11<01:16,  2.60it/s, v_num=1, train_loss=891]

Epoch 301/500:  60%|██████    | 301/500 [02:12<01:16,  2.61it/s, v_num=1, train_loss=891]

Epoch 301/500:  60%|██████    | 301/500 [02:12<01:16,  2.61it/s, v_num=1, train_loss=891]

Epoch 302/500:  60%|██████    | 301/500 [02:12<01:16,  2.61it/s, v_num=1, train_loss=891]

Epoch 302/500:  60%|██████    | 302/500 [02:12<01:17,  2.56it/s, v_num=1, train_loss=891]

Epoch 302/500:  60%|██████    | 302/500 [02:12<01:17,  2.56it/s, v_num=1, train_loss=891]

Epoch 303/500:  60%|██████    | 302/500 [02:12<01:17,  2.56it/s, v_num=1, train_loss=891]

Epoch 303/500:  61%|██████    | 303/500 [02:12<01:16,  2.56it/s, v_num=1, train_loss=891]

Epoch 303/500:  61%|██████    | 303/500 [02:12<01:16,  2.56it/s, v_num=1, train_loss=891]

Epoch 304/500:  61%|██████    | 303/500 [02:12<01:16,  2.56it/s, v_num=1, train_loss=891]

Epoch 304/500:  61%|██████    | 304/500 [02:13<01:18,  2.48it/s, v_num=1, train_loss=891]

Epoch 304/500:  61%|██████    | 304/500 [02:13<01:18,  2.48it/s, v_num=1, train_loss=891]

Epoch 305/500:  61%|██████    | 304/500 [02:13<01:18,  2.48it/s, v_num=1, train_loss=891]

Epoch 305/500:  61%|██████    | 305/500 [02:13<01:17,  2.52it/s, v_num=1, train_loss=891]

Epoch 305/500:  61%|██████    | 305/500 [02:13<01:17,  2.52it/s, v_num=1, train_loss=891]

Epoch 306/500:  61%|██████    | 305/500 [02:13<01:17,  2.52it/s, v_num=1, train_loss=891]

Epoch 306/500:  61%|██████    | 306/500 [02:14<01:17,  2.52it/s, v_num=1, train_loss=891]

Epoch 306/500:  61%|██████    | 306/500 [02:14<01:17,  2.52it/s, v_num=1, train_loss=891]

Epoch 307/500:  61%|██████    | 306/500 [02:14<01:17,  2.52it/s, v_num=1, train_loss=891]

Epoch 307/500:  61%|██████▏   | 307/500 [02:14<01:16,  2.53it/s, v_num=1, train_loss=891]

Epoch 307/500:  61%|██████▏   | 307/500 [02:14<01:16,  2.53it/s, v_num=1, train_loss=891]

Epoch 308/500:  61%|██████▏   | 307/500 [02:14<01:16,  2.53it/s, v_num=1, train_loss=891]

Epoch 308/500:  62%|██████▏   | 308/500 [02:14<01:24,  2.27it/s, v_num=1, train_loss=891]

Epoch 308/500:  62%|██████▏   | 308/500 [02:14<01:24,  2.27it/s, v_num=1, train_loss=891]

Epoch 309/500:  62%|██████▏   | 308/500 [02:14<01:24,  2.27it/s, v_num=1, train_loss=891]

Epoch 309/500:  62%|██████▏   | 309/500 [02:15<01:25,  2.22it/s, v_num=1, train_loss=891]

Epoch 309/500:  62%|██████▏   | 309/500 [02:15<01:25,  2.22it/s, v_num=1, train_loss=891]

Epoch 310/500:  62%|██████▏   | 309/500 [02:15<01:25,  2.22it/s, v_num=1, train_loss=891]

Epoch 310/500:  62%|██████▏   | 310/500 [02:15<01:22,  2.31it/s, v_num=1, train_loss=891]

Epoch 310/500:  62%|██████▏   | 310/500 [02:15<01:22,  2.31it/s, v_num=1, train_loss=891]

Epoch 311/500:  62%|██████▏   | 310/500 [02:15<01:22,  2.31it/s, v_num=1, train_loss=891]

Epoch 311/500:  62%|██████▏   | 311/500 [02:16<01:19,  2.38it/s, v_num=1, train_loss=891]

Epoch 311/500:  62%|██████▏   | 311/500 [02:16<01:19,  2.38it/s, v_num=1, train_loss=891]

Epoch 312/500:  62%|██████▏   | 311/500 [02:16<01:19,  2.38it/s, v_num=1, train_loss=891]

Epoch 312/500:  62%|██████▏   | 312/500 [02:16<01:19,  2.37it/s, v_num=1, train_loss=891]

Epoch 312/500:  62%|██████▏   | 312/500 [02:16<01:19,  2.37it/s, v_num=1, train_loss=891]

Epoch 313/500:  62%|██████▏   | 312/500 [02:16<01:19,  2.37it/s, v_num=1, train_loss=891]

Epoch 313/500:  63%|██████▎   | 313/500 [02:17<01:17,  2.42it/s, v_num=1, train_loss=891]

Epoch 313/500:  63%|██████▎   | 313/500 [02:17<01:17,  2.42it/s, v_num=1, train_loss=891]

Epoch 314/500:  63%|██████▎   | 313/500 [02:17<01:17,  2.42it/s, v_num=1, train_loss=891]

Epoch 314/500:  63%|██████▎   | 314/500 [02:17<01:16,  2.43it/s, v_num=1, train_loss=891]

Epoch 314/500:  63%|██████▎   | 314/500 [02:17<01:16,  2.43it/s, v_num=1, train_loss=891]

Epoch 315/500:  63%|██████▎   | 314/500 [02:17<01:16,  2.43it/s, v_num=1, train_loss=891]

Epoch 315/500:  63%|██████▎   | 315/500 [02:17<01:15,  2.44it/s, v_num=1, train_loss=891]

Epoch 315/500:  63%|██████▎   | 315/500 [02:17<01:15,  2.44it/s, v_num=1, train_loss=891]

Epoch 316/500:  63%|██████▎   | 315/500 [02:17<01:15,  2.44it/s, v_num=1, train_loss=891]

Epoch 316/500:  63%|██████▎   | 316/500 [02:18<01:16,  2.41it/s, v_num=1, train_loss=891]

Epoch 316/500:  63%|██████▎   | 316/500 [02:18<01:16,  2.41it/s, v_num=1, train_loss=891]

Epoch 317/500:  63%|██████▎   | 316/500 [02:18<01:16,  2.41it/s, v_num=1, train_loss=891]

Epoch 317/500:  63%|██████▎   | 317/500 [02:18<01:15,  2.41it/s, v_num=1, train_loss=891]

Epoch 317/500:  63%|██████▎   | 317/500 [02:18<01:15,  2.41it/s, v_num=1, train_loss=891]

Epoch 318/500:  63%|██████▎   | 317/500 [02:18<01:15,  2.41it/s, v_num=1, train_loss=891]

Epoch 318/500:  64%|██████▎   | 318/500 [02:19<01:14,  2.45it/s, v_num=1, train_loss=891]

Epoch 318/500:  64%|██████▎   | 318/500 [02:19<01:14,  2.45it/s, v_num=1, train_loss=891]

Epoch 319/500:  64%|██████▎   | 318/500 [02:19<01:14,  2.45it/s, v_num=1, train_loss=891]

Epoch 319/500:  64%|██████▍   | 319/500 [02:19<01:15,  2.38it/s, v_num=1, train_loss=891]

Epoch 319/500:  64%|██████▍   | 319/500 [02:19<01:15,  2.38it/s, v_num=1, train_loss=892]

Epoch 320/500:  64%|██████▍   | 319/500 [02:19<01:15,  2.38it/s, v_num=1, train_loss=892]

Epoch 320/500:  64%|██████▍   | 320/500 [02:20<01:17,  2.32it/s, v_num=1, train_loss=892]

Epoch 320/500:  64%|██████▍   | 320/500 [02:20<01:17,  2.32it/s, v_num=1, train_loss=891]

Epoch 321/500:  64%|██████▍   | 320/500 [02:20<01:17,  2.32it/s, v_num=1, train_loss=891]

Epoch 321/500:  64%|██████▍   | 321/500 [02:20<01:16,  2.35it/s, v_num=1, train_loss=891]

Epoch 321/500:  64%|██████▍   | 321/500 [02:20<01:16,  2.35it/s, v_num=1, train_loss=891]

Epoch 322/500:  64%|██████▍   | 321/500 [02:20<01:16,  2.35it/s, v_num=1, train_loss=891]

Epoch 322/500:  64%|██████▍   | 322/500 [02:20<01:14,  2.38it/s, v_num=1, train_loss=891]

Epoch 322/500:  64%|██████▍   | 322/500 [02:20<01:14,  2.38it/s, v_num=1, train_loss=892]

Epoch 323/500:  64%|██████▍   | 322/500 [02:20<01:14,  2.38it/s, v_num=1, train_loss=892]

Epoch 323/500:  65%|██████▍   | 323/500 [02:21<01:14,  2.39it/s, v_num=1, train_loss=892]

Epoch 323/500:  65%|██████▍   | 323/500 [02:21<01:14,  2.39it/s, v_num=1, train_loss=891]

Epoch 324/500:  65%|██████▍   | 323/500 [02:21<01:14,  2.39it/s, v_num=1, train_loss=891]

Epoch 324/500:  65%|██████▍   | 324/500 [02:21<01:12,  2.41it/s, v_num=1, train_loss=891]

Epoch 324/500:  65%|██████▍   | 324/500 [02:21<01:12,  2.41it/s, v_num=1, train_loss=892]

Epoch 325/500:  65%|██████▍   | 324/500 [02:21<01:12,  2.41it/s, v_num=1, train_loss=892]

Epoch 325/500:  65%|██████▌   | 325/500 [02:22<01:11,  2.44it/s, v_num=1, train_loss=892]

Epoch 325/500:  65%|██████▌   | 325/500 [02:22<01:11,  2.44it/s, v_num=1, train_loss=892]

Epoch 326/500:  65%|██████▌   | 325/500 [02:22<01:11,  2.44it/s, v_num=1, train_loss=892]

Epoch 326/500:  65%|██████▌   | 326/500 [02:22<01:12,  2.41it/s, v_num=1, train_loss=892]

Epoch 326/500:  65%|██████▌   | 326/500 [02:22<01:12,  2.41it/s, v_num=1, train_loss=892]

Epoch 327/500:  65%|██████▌   | 326/500 [02:22<01:12,  2.41it/s, v_num=1, train_loss=892]

Epoch 327/500:  65%|██████▌   | 327/500 [02:22<01:11,  2.42it/s, v_num=1, train_loss=892]

Epoch 327/500:  65%|██████▌   | 327/500 [02:22<01:11,  2.42it/s, v_num=1, train_loss=892]

Epoch 328/500:  65%|██████▌   | 327/500 [02:22<01:11,  2.42it/s, v_num=1, train_loss=892]

Epoch 328/500:  66%|██████▌   | 328/500 [02:23<01:10,  2.44it/s, v_num=1, train_loss=892]

Epoch 328/500:  66%|██████▌   | 328/500 [02:23<01:10,  2.44it/s, v_num=1, train_loss=892]

Epoch 329/500:  66%|██████▌   | 328/500 [02:23<01:10,  2.44it/s, v_num=1, train_loss=892]

Epoch 329/500:  66%|██████▌   | 329/500 [02:23<01:10,  2.44it/s, v_num=1, train_loss=892]

Epoch 329/500:  66%|██████▌   | 329/500 [02:23<01:10,  2.44it/s, v_num=1, train_loss=892]

Epoch 330/500:  66%|██████▌   | 329/500 [02:23<01:10,  2.44it/s, v_num=1, train_loss=892]

Epoch 330/500:  66%|██████▌   | 330/500 [02:24<01:10,  2.40it/s, v_num=1, train_loss=892]

Epoch 330/500:  66%|██████▌   | 330/500 [02:24<01:10,  2.40it/s, v_num=1, train_loss=892]

Epoch 331/500:  66%|██████▌   | 330/500 [02:24<01:10,  2.40it/s, v_num=1, train_loss=892]

Epoch 331/500:  66%|██████▌   | 331/500 [02:24<01:09,  2.42it/s, v_num=1, train_loss=892]

Epoch 331/500:  66%|██████▌   | 331/500 [02:24<01:09,  2.42it/s, v_num=1, train_loss=892]

Epoch 332/500:  66%|██████▌   | 331/500 [02:24<01:09,  2.42it/s, v_num=1, train_loss=892]

Epoch 332/500:  66%|██████▋   | 332/500 [02:24<01:08,  2.44it/s, v_num=1, train_loss=892]

Epoch 332/500:  66%|██████▋   | 332/500 [02:24<01:08,  2.44it/s, v_num=1, train_loss=892]

Epoch 333/500:  66%|██████▋   | 332/500 [02:24<01:08,  2.44it/s, v_num=1, train_loss=892]

Epoch 333/500:  67%|██████▋   | 333/500 [02:25<01:08,  2.45it/s, v_num=1, train_loss=892]

Epoch 333/500:  67%|██████▋   | 333/500 [02:25<01:08,  2.45it/s, v_num=1, train_loss=892]

Epoch 334/500:  67%|██████▋   | 333/500 [02:25<01:08,  2.45it/s, v_num=1, train_loss=892]

Epoch 334/500:  67%|██████▋   | 334/500 [02:25<01:07,  2.46it/s, v_num=1, train_loss=892]

Epoch 334/500:  67%|██████▋   | 334/500 [02:25<01:07,  2.46it/s, v_num=1, train_loss=892]

Epoch 335/500:  67%|██████▋   | 334/500 [02:25<01:07,  2.46it/s, v_num=1, train_loss=892]

Epoch 335/500:  67%|██████▋   | 335/500 [02:26<01:07,  2.45it/s, v_num=1, train_loss=892]

Epoch 335/500:  67%|██████▋   | 335/500 [02:26<01:07,  2.45it/s, v_num=1, train_loss=892]

Epoch 336/500:  67%|██████▋   | 335/500 [02:26<01:07,  2.45it/s, v_num=1, train_loss=892]

Epoch 336/500:  67%|██████▋   | 336/500 [02:26<01:08,  2.40it/s, v_num=1, train_loss=892]

Epoch 336/500:  67%|██████▋   | 336/500 [02:26<01:08,  2.40it/s, v_num=1, train_loss=892]

Epoch 337/500:  67%|██████▋   | 336/500 [02:26<01:08,  2.40it/s, v_num=1, train_loss=892]

Epoch 337/500:  67%|██████▋   | 337/500 [02:27<01:09,  2.34it/s, v_num=1, train_loss=892]

Epoch 337/500:  67%|██████▋   | 337/500 [02:27<01:09,  2.34it/s, v_num=1, train_loss=892]

Epoch 338/500:  67%|██████▋   | 337/500 [02:27<01:09,  2.34it/s, v_num=1, train_loss=892]

Epoch 338/500:  68%|██████▊   | 338/500 [02:27<01:09,  2.33it/s, v_num=1, train_loss=892]

Epoch 338/500:  68%|██████▊   | 338/500 [02:27<01:09,  2.33it/s, v_num=1, train_loss=892]

Epoch 339/500:  68%|██████▊   | 338/500 [02:27<01:09,  2.33it/s, v_num=1, train_loss=892]

Epoch 339/500:  68%|██████▊   | 339/500 [02:27<01:10,  2.29it/s, v_num=1, train_loss=892]

Epoch 339/500:  68%|██████▊   | 339/500 [02:27<01:10,  2.29it/s, v_num=1, train_loss=892]

Epoch 340/500:  68%|██████▊   | 339/500 [02:27<01:10,  2.29it/s, v_num=1, train_loss=892]

Epoch 340/500:  68%|██████▊   | 340/500 [02:28<01:12,  2.19it/s, v_num=1, train_loss=892]

Epoch 340/500:  68%|██████▊   | 340/500 [02:28<01:12,  2.19it/s, v_num=1, train_loss=892]

Epoch 341/500:  68%|██████▊   | 340/500 [02:28<01:12,  2.19it/s, v_num=1, train_loss=892]

Epoch 341/500:  68%|██████▊   | 341/500 [02:28<01:17,  2.06it/s, v_num=1, train_loss=892]

Epoch 341/500:  68%|██████▊   | 341/500 [02:28<01:17,  2.06it/s, v_num=1, train_loss=892]

Epoch 342/500:  68%|██████▊   | 341/500 [02:28<01:17,  2.06it/s, v_num=1, train_loss=892]

Epoch 342/500:  68%|██████▊   | 342/500 [02:29<01:17,  2.04it/s, v_num=1, train_loss=892]

Epoch 342/500:  68%|██████▊   | 342/500 [02:29<01:17,  2.04it/s, v_num=1, train_loss=892]

Epoch 343/500:  68%|██████▊   | 342/500 [02:29<01:17,  2.04it/s, v_num=1, train_loss=892]

Epoch 343/500:  69%|██████▊   | 343/500 [02:29<01:14,  2.11it/s, v_num=1, train_loss=892]

Epoch 343/500:  69%|██████▊   | 343/500 [02:29<01:14,  2.11it/s, v_num=1, train_loss=892]

Epoch 344/500:  69%|██████▊   | 343/500 [02:29<01:14,  2.11it/s, v_num=1, train_loss=892]

Epoch 344/500:  69%|██████▉   | 344/500 [02:30<01:11,  2.17it/s, v_num=1, train_loss=892]

Epoch 344/500:  69%|██████▉   | 344/500 [02:30<01:11,  2.17it/s, v_num=1, train_loss=892]

Epoch 345/500:  69%|██████▉   | 344/500 [02:30<01:11,  2.17it/s, v_num=1, train_loss=892]

Epoch 345/500:  69%|██████▉   | 345/500 [02:30<01:09,  2.22it/s, v_num=1, train_loss=892]

Epoch 345/500:  69%|██████▉   | 345/500 [02:30<01:09,  2.22it/s, v_num=1, train_loss=892]

Epoch 346/500:  69%|██████▉   | 345/500 [02:30<01:09,  2.22it/s, v_num=1, train_loss=892]

Epoch 346/500:  69%|██████▉   | 346/500 [02:31<01:08,  2.25it/s, v_num=1, train_loss=892]

Epoch 346/500:  69%|██████▉   | 346/500 [02:31<01:08,  2.25it/s, v_num=1, train_loss=892]

Epoch 347/500:  69%|██████▉   | 346/500 [02:31<01:08,  2.25it/s, v_num=1, train_loss=892]

Epoch 347/500:  69%|██████▉   | 347/500 [02:31<01:07,  2.27it/s, v_num=1, train_loss=892]

Epoch 347/500:  69%|██████▉   | 347/500 [02:31<01:07,  2.27it/s, v_num=1, train_loss=892]

Epoch 348/500:  69%|██████▉   | 347/500 [02:31<01:07,  2.27it/s, v_num=1, train_loss=892]

Epoch 348/500:  70%|██████▉   | 348/500 [02:32<01:06,  2.28it/s, v_num=1, train_loss=892]

Epoch 348/500:  70%|██████▉   | 348/500 [02:32<01:06,  2.28it/s, v_num=1, train_loss=892]

Epoch 349/500:  70%|██████▉   | 348/500 [02:32<01:06,  2.28it/s, v_num=1, train_loss=892]

Epoch 349/500:  70%|██████▉   | 349/500 [02:32<01:05,  2.30it/s, v_num=1, train_loss=892]

Epoch 349/500:  70%|██████▉   | 349/500 [02:32<01:05,  2.30it/s, v_num=1, train_loss=892]

Epoch 350/500:  70%|██████▉   | 349/500 [02:32<01:05,  2.30it/s, v_num=1, train_loss=892]

Epoch 350/500:  70%|███████   | 350/500 [02:32<01:05,  2.30it/s, v_num=1, train_loss=892]

Epoch 350/500:  70%|███████   | 350/500 [02:32<01:05,  2.30it/s, v_num=1, train_loss=892]

Epoch 351/500:  70%|███████   | 350/500 [02:32<01:05,  2.30it/s, v_num=1, train_loss=892]

Epoch 351/500:  70%|███████   | 351/500 [02:33<01:08,  2.18it/s, v_num=1, train_loss=892]

Epoch 351/500:  70%|███████   | 351/500 [02:33<01:08,  2.18it/s, v_num=1, train_loss=893]

Epoch 352/500:  70%|███████   | 351/500 [02:33<01:08,  2.18it/s, v_num=1, train_loss=893]

Epoch 352/500:  70%|███████   | 352/500 [02:33<01:08,  2.17it/s, v_num=1, train_loss=893]

Epoch 352/500:  70%|███████   | 352/500 [02:33<01:08,  2.17it/s, v_num=1, train_loss=893]

Epoch 353/500:  70%|███████   | 352/500 [02:33<01:08,  2.17it/s, v_num=1, train_loss=893]

Epoch 353/500:  71%|███████   | 353/500 [02:34<01:08,  2.16it/s, v_num=1, train_loss=893]

Epoch 353/500:  71%|███████   | 353/500 [02:34<01:08,  2.16it/s, v_num=1, train_loss=893]

Epoch 354/500:  71%|███████   | 353/500 [02:34<01:08,  2.16it/s, v_num=1, train_loss=893]

Epoch 354/500:  71%|███████   | 354/500 [02:34<01:08,  2.15it/s, v_num=1, train_loss=893]

Epoch 354/500:  71%|███████   | 354/500 [02:34<01:08,  2.15it/s, v_num=1, train_loss=893]

Epoch 355/500:  71%|███████   | 354/500 [02:34<01:08,  2.15it/s, v_num=1, train_loss=893]

Epoch 355/500:  71%|███████   | 355/500 [02:35<01:05,  2.20it/s, v_num=1, train_loss=893]

Epoch 355/500:  71%|███████   | 355/500 [02:35<01:05,  2.20it/s, v_num=1, train_loss=893]

Epoch 356/500:  71%|███████   | 355/500 [02:35<01:05,  2.20it/s, v_num=1, train_loss=893]

Epoch 356/500:  71%|███████   | 356/500 [02:35<01:05,  2.21it/s, v_num=1, train_loss=893]

Epoch 356/500:  71%|███████   | 356/500 [02:35<01:05,  2.21it/s, v_num=1, train_loss=893]

Epoch 357/500:  71%|███████   | 356/500 [02:35<01:05,  2.21it/s, v_num=1, train_loss=893]

Epoch 357/500:  71%|███████▏  | 357/500 [02:36<01:06,  2.16it/s, v_num=1, train_loss=893]

Epoch 357/500:  71%|███████▏  | 357/500 [02:36<01:06,  2.16it/s, v_num=1, train_loss=893]

Epoch 358/500:  71%|███████▏  | 357/500 [02:36<01:06,  2.16it/s, v_num=1, train_loss=893]

Epoch 358/500:  72%|███████▏  | 358/500 [02:36<01:06,  2.15it/s, v_num=1, train_loss=893]

Epoch 358/500:  72%|███████▏  | 358/500 [02:36<01:06,  2.15it/s, v_num=1, train_loss=893]

Epoch 359/500:  72%|███████▏  | 358/500 [02:36<01:06,  2.15it/s, v_num=1, train_loss=893]

Epoch 359/500:  72%|███████▏  | 359/500 [02:37<01:09,  2.02it/s, v_num=1, train_loss=893]

Epoch 359/500:  72%|███████▏  | 359/500 [02:37<01:09,  2.02it/s, v_num=1, train_loss=893]

Epoch 360/500:  72%|███████▏  | 359/500 [02:37<01:09,  2.02it/s, v_num=1, train_loss=893]

Epoch 360/500:  72%|███████▏  | 360/500 [02:37<01:09,  2.02it/s, v_num=1, train_loss=893]

Epoch 360/500:  72%|███████▏  | 360/500 [02:37<01:09,  2.02it/s, v_num=1, train_loss=893]

Epoch 361/500:  72%|███████▏  | 360/500 [02:37<01:09,  2.02it/s, v_num=1, train_loss=893]

Epoch 361/500:  72%|███████▏  | 361/500 [02:38<01:13,  1.90it/s, v_num=1, train_loss=893]

Epoch 361/500:  72%|███████▏  | 361/500 [02:38<01:13,  1.90it/s, v_num=1, train_loss=893]

Epoch 362/500:  72%|███████▏  | 361/500 [02:38<01:13,  1.90it/s, v_num=1, train_loss=893]

Epoch 362/500:  72%|███████▏  | 362/500 [02:38<01:14,  1.86it/s, v_num=1, train_loss=893]

Epoch 362/500:  72%|███████▏  | 362/500 [02:38<01:14,  1.86it/s, v_num=1, train_loss=893]

Epoch 363/500:  72%|███████▏  | 362/500 [02:38<01:14,  1.86it/s, v_num=1, train_loss=893]

Epoch 363/500:  73%|███████▎  | 363/500 [02:39<01:11,  1.91it/s, v_num=1, train_loss=893]

Epoch 363/500:  73%|███████▎  | 363/500 [02:39<01:11,  1.91it/s, v_num=1, train_loss=893]

Epoch 364/500:  73%|███████▎  | 363/500 [02:39<01:11,  1.91it/s, v_num=1, train_loss=893]

Epoch 364/500:  73%|███████▎  | 364/500 [02:39<01:11,  1.90it/s, v_num=1, train_loss=893]

Epoch 364/500:  73%|███████▎  | 364/500 [02:39<01:11,  1.90it/s, v_num=1, train_loss=893]

Epoch 365/500:  73%|███████▎  | 364/500 [02:39<01:11,  1.90it/s, v_num=1, train_loss=893]

Epoch 365/500:  73%|███████▎  | 365/500 [02:40<01:11,  1.89it/s, v_num=1, train_loss=893]

Epoch 365/500:  73%|███████▎  | 365/500 [02:40<01:11,  1.89it/s, v_num=1, train_loss=893]

Epoch 366/500:  73%|███████▎  | 365/500 [02:40<01:11,  1.89it/s, v_num=1, train_loss=893]

Epoch 366/500:  73%|███████▎  | 366/500 [02:41<01:11,  1.87it/s, v_num=1, train_loss=893]

Epoch 366/500:  73%|███████▎  | 366/500 [02:41<01:11,  1.87it/s, v_num=1, train_loss=893]

Epoch 367/500:  73%|███████▎  | 366/500 [02:41<01:11,  1.87it/s, v_num=1, train_loss=893]

Epoch 367/500:  73%|███████▎  | 367/500 [02:41<01:10,  1.88it/s, v_num=1, train_loss=893]

Epoch 367/500:  73%|███████▎  | 367/500 [02:41<01:10,  1.88it/s, v_num=1, train_loss=893]

Epoch 368/500:  73%|███████▎  | 367/500 [02:41<01:10,  1.88it/s, v_num=1, train_loss=893]

Epoch 368/500:  74%|███████▎  | 368/500 [02:42<01:10,  1.89it/s, v_num=1, train_loss=893]

Epoch 368/500:  74%|███████▎  | 368/500 [02:42<01:10,  1.89it/s, v_num=1, train_loss=893]

Epoch 369/500:  74%|███████▎  | 368/500 [02:42<01:10,  1.89it/s, v_num=1, train_loss=893]

Epoch 369/500:  74%|███████▍  | 369/500 [02:42<01:10,  1.87it/s, v_num=1, train_loss=893]

Epoch 369/500:  74%|███████▍  | 369/500 [02:42<01:10,  1.87it/s, v_num=1, train_loss=893]

Epoch 370/500:  74%|███████▍  | 369/500 [02:42<01:10,  1.87it/s, v_num=1, train_loss=893]

Epoch 370/500:  74%|███████▍  | 370/500 [02:43<01:10,  1.84it/s, v_num=1, train_loss=893]

Epoch 370/500:  74%|███████▍  | 370/500 [02:43<01:10,  1.84it/s, v_num=1, train_loss=893]

Epoch 371/500:  74%|███████▍  | 370/500 [02:43<01:10,  1.84it/s, v_num=1, train_loss=893]

Epoch 371/500:  74%|███████▍  | 371/500 [02:43<01:09,  1.84it/s, v_num=1, train_loss=893]

Epoch 371/500:  74%|███████▍  | 371/500 [02:43<01:09,  1.84it/s, v_num=1, train_loss=893]

Epoch 372/500:  74%|███████▍  | 371/500 [02:43<01:09,  1.84it/s, v_num=1, train_loss=893]

Epoch 372/500:  74%|███████▍  | 372/500 [02:44<01:12,  1.76it/s, v_num=1, train_loss=893]

Epoch 372/500:  74%|███████▍  | 372/500 [02:44<01:12,  1.76it/s, v_num=1, train_loss=893]

Epoch 373/500:  74%|███████▍  | 372/500 [02:44<01:12,  1.76it/s, v_num=1, train_loss=893]

Epoch 373/500:  75%|███████▍  | 373/500 [02:44<01:09,  1.81it/s, v_num=1, train_loss=893]

Epoch 373/500:  75%|███████▍  | 373/500 [02:44<01:09,  1.81it/s, v_num=1, train_loss=893]

Epoch 374/500:  75%|███████▍  | 373/500 [02:44<01:09,  1.81it/s, v_num=1, train_loss=893]

Epoch 374/500:  75%|███████▍  | 374/500 [02:45<01:06,  1.89it/s, v_num=1, train_loss=893]

Epoch 374/500:  75%|███████▍  | 374/500 [02:45<01:06,  1.89it/s, v_num=1, train_loss=893]

Epoch 375/500:  75%|███████▍  | 374/500 [02:45<01:06,  1.89it/s, v_num=1, train_loss=893]

Epoch 375/500:  75%|███████▌  | 375/500 [02:45<01:04,  1.94it/s, v_num=1, train_loss=893]

Epoch 375/500:  75%|███████▌  | 375/500 [02:45<01:04,  1.94it/s, v_num=1, train_loss=893]

Epoch 376/500:  75%|███████▌  | 375/500 [02:45<01:04,  1.94it/s, v_num=1, train_loss=893]

Epoch 376/500:  75%|███████▌  | 376/500 [02:46<01:02,  2.00it/s, v_num=1, train_loss=893]

Epoch 376/500:  75%|███████▌  | 376/500 [02:46<01:02,  2.00it/s, v_num=1, train_loss=893]

Epoch 377/500:  75%|███████▌  | 376/500 [02:46<01:02,  2.00it/s, v_num=1, train_loss=893]

Epoch 377/500:  75%|███████▌  | 377/500 [02:46<00:58,  2.09it/s, v_num=1, train_loss=893]

Epoch 377/500:  75%|███████▌  | 377/500 [02:46<00:58,  2.09it/s, v_num=1, train_loss=893]

Epoch 378/500:  75%|███████▌  | 377/500 [02:46<00:58,  2.09it/s, v_num=1, train_loss=893]

Epoch 378/500:  76%|███████▌  | 378/500 [02:47<00:57,  2.13it/s, v_num=1, train_loss=893]

Epoch 378/500:  76%|███████▌  | 378/500 [02:47<00:57,  2.13it/s, v_num=1, train_loss=893]

Epoch 379/500:  76%|███████▌  | 378/500 [02:47<00:57,  2.13it/s, v_num=1, train_loss=893]

Epoch 379/500:  76%|███████▌  | 379/500 [02:47<00:55,  2.16it/s, v_num=1, train_loss=893]

Epoch 379/500:  76%|███████▌  | 379/500 [02:47<00:55,  2.16it/s, v_num=1, train_loss=893]

Epoch 380/500:  76%|███████▌  | 379/500 [02:47<00:55,  2.16it/s, v_num=1, train_loss=893]

Epoch 380/500:  76%|███████▌  | 380/500 [02:48<00:58,  2.05it/s, v_num=1, train_loss=893]

Epoch 380/500:  76%|███████▌  | 380/500 [02:48<00:58,  2.05it/s, v_num=1, train_loss=893]

Epoch 381/500:  76%|███████▌  | 380/500 [02:48<00:58,  2.05it/s, v_num=1, train_loss=893]

Epoch 381/500:  76%|███████▌  | 381/500 [02:48<01:01,  1.95it/s, v_num=1, train_loss=893]

Epoch 381/500:  76%|███████▌  | 381/500 [02:48<01:01,  1.95it/s, v_num=1, train_loss=893]

Epoch 382/500:  76%|███████▌  | 381/500 [02:48<01:01,  1.95it/s, v_num=1, train_loss=893]

Epoch 382/500:  76%|███████▋  | 382/500 [02:49<01:02,  1.90it/s, v_num=1, train_loss=893]

Epoch 382/500:  76%|███████▋  | 382/500 [02:49<01:02,  1.90it/s, v_num=1, train_loss=893]

Epoch 383/500:  76%|███████▋  | 382/500 [02:49<01:02,  1.90it/s, v_num=1, train_loss=893]

Epoch 383/500:  77%|███████▋  | 383/500 [02:49<01:03,  1.85it/s, v_num=1, train_loss=893]

Epoch 383/500:  77%|███████▋  | 383/500 [02:49<01:03,  1.85it/s, v_num=1, train_loss=893]

Epoch 384/500:  77%|███████▋  | 383/500 [02:49<01:03,  1.85it/s, v_num=1, train_loss=893]

Epoch 384/500:  77%|███████▋  | 384/500 [02:50<01:01,  1.88it/s, v_num=1, train_loss=893]

Epoch 384/500:  77%|███████▋  | 384/500 [02:50<01:01,  1.88it/s, v_num=1, train_loss=893]

Epoch 385/500:  77%|███████▋  | 384/500 [02:50<01:01,  1.88it/s, v_num=1, train_loss=893]

Epoch 385/500:  77%|███████▋  | 385/500 [02:50<01:01,  1.87it/s, v_num=1, train_loss=893]

Epoch 385/500:  77%|███████▋  | 385/500 [02:50<01:01,  1.87it/s, v_num=1, train_loss=893]

Epoch 386/500:  77%|███████▋  | 385/500 [02:50<01:01,  1.87it/s, v_num=1, train_loss=893]

Epoch 386/500:  77%|███████▋  | 386/500 [02:51<01:02,  1.81it/s, v_num=1, train_loss=893]

Epoch 386/500:  77%|███████▋  | 386/500 [02:51<01:02,  1.81it/s, v_num=1, train_loss=894]

Epoch 387/500:  77%|███████▋  | 386/500 [02:51<01:02,  1.81it/s, v_num=1, train_loss=894]

Epoch 387/500:  77%|███████▋  | 387/500 [02:52<01:00,  1.87it/s, v_num=1, train_loss=894]

Epoch 387/500:  77%|███████▋  | 387/500 [02:52<01:00,  1.87it/s, v_num=1, train_loss=893]

Epoch 388/500:  77%|███████▋  | 387/500 [02:52<01:00,  1.87it/s, v_num=1, train_loss=893]

Epoch 388/500:  78%|███████▊  | 388/500 [02:52<00:56,  1.99it/s, v_num=1, train_loss=893]

Epoch 388/500:  78%|███████▊  | 388/500 [02:52<00:56,  1.99it/s, v_num=1, train_loss=894]

Epoch 389/500:  78%|███████▊  | 388/500 [02:52<00:56,  1.99it/s, v_num=1, train_loss=894]

Epoch 389/500:  78%|███████▊  | 389/500 [02:52<00:52,  2.10it/s, v_num=1, train_loss=894]

Epoch 389/500:  78%|███████▊  | 389/500 [02:52<00:52,  2.10it/s, v_num=1, train_loss=894]

Epoch 390/500:  78%|███████▊  | 389/500 [02:52<00:52,  2.10it/s, v_num=1, train_loss=894]

Epoch 390/500:  78%|███████▊  | 390/500 [02:53<00:51,  2.12it/s, v_num=1, train_loss=894]

Epoch 390/500:  78%|███████▊  | 390/500 [02:53<00:51,  2.12it/s, v_num=1, train_loss=894]

Epoch 391/500:  78%|███████▊  | 390/500 [02:53<00:51,  2.12it/s, v_num=1, train_loss=894]

Epoch 391/500:  78%|███████▊  | 391/500 [02:53<00:52,  2.07it/s, v_num=1, train_loss=894]

Epoch 391/500:  78%|███████▊  | 391/500 [02:53<00:52,  2.07it/s, v_num=1, train_loss=894]

Epoch 392/500:  78%|███████▊  | 391/500 [02:53<00:52,  2.07it/s, v_num=1, train_loss=894]

Epoch 392/500:  78%|███████▊  | 392/500 [02:54<00:51,  2.08it/s, v_num=1, train_loss=894]

Epoch 392/500:  78%|███████▊  | 392/500 [02:54<00:51,  2.08it/s, v_num=1, train_loss=894]

Epoch 393/500:  78%|███████▊  | 392/500 [02:54<00:51,  2.08it/s, v_num=1, train_loss=894]

Epoch 393/500:  79%|███████▊  | 393/500 [02:54<00:48,  2.21it/s, v_num=1, train_loss=894]

Epoch 393/500:  79%|███████▊  | 393/500 [02:54<00:48,  2.21it/s, v_num=1, train_loss=894]

Epoch 394/500:  79%|███████▊  | 393/500 [02:54<00:48,  2.21it/s, v_num=1, train_loss=894]

Epoch 394/500:  79%|███████▉  | 394/500 [02:55<00:46,  2.30it/s, v_num=1, train_loss=894]

Epoch 394/500:  79%|███████▉  | 394/500 [02:55<00:46,  2.30it/s, v_num=1, train_loss=894]

Epoch 395/500:  79%|███████▉  | 394/500 [02:55<00:46,  2.30it/s, v_num=1, train_loss=894]

Epoch 395/500:  79%|███████▉  | 395/500 [02:55<00:44,  2.37it/s, v_num=1, train_loss=894]

Epoch 395/500:  79%|███████▉  | 395/500 [02:55<00:44,  2.37it/s, v_num=1, train_loss=894]

Epoch 396/500:  79%|███████▉  | 395/500 [02:55<00:44,  2.37it/s, v_num=1, train_loss=894]

Epoch 396/500:  79%|███████▉  | 396/500 [02:55<00:44,  2.34it/s, v_num=1, train_loss=894]

Epoch 396/500:  79%|███████▉  | 396/500 [02:55<00:44,  2.34it/s, v_num=1, train_loss=894]

Epoch 397/500:  79%|███████▉  | 396/500 [02:55<00:44,  2.34it/s, v_num=1, train_loss=894]

Epoch 397/500:  79%|███████▉  | 397/500 [02:56<00:45,  2.24it/s, v_num=1, train_loss=894]

Epoch 397/500:  79%|███████▉  | 397/500 [02:56<00:45,  2.24it/s, v_num=1, train_loss=894]

Epoch 398/500:  79%|███████▉  | 397/500 [02:56<00:45,  2.24it/s, v_num=1, train_loss=894]

Epoch 398/500:  80%|███████▉  | 398/500 [02:56<00:46,  2.18it/s, v_num=1, train_loss=894]

Epoch 398/500:  80%|███████▉  | 398/500 [02:56<00:46,  2.18it/s, v_num=1, train_loss=894]

Epoch 399/500:  80%|███████▉  | 398/500 [02:56<00:46,  2.18it/s, v_num=1, train_loss=894]

Epoch 399/500:  80%|███████▉  | 399/500 [02:57<00:47,  2.15it/s, v_num=1, train_loss=894]

Epoch 399/500:  80%|███████▉  | 399/500 [02:57<00:47,  2.15it/s, v_num=1, train_loss=894]

Epoch 400/500:  80%|███████▉  | 399/500 [02:57<00:47,  2.15it/s, v_num=1, train_loss=894]

Epoch 400/500:  80%|████████  | 400/500 [02:57<00:47,  2.12it/s, v_num=1, train_loss=894]

Epoch 400/500:  80%|████████  | 400/500 [02:57<00:47,  2.12it/s, v_num=1, train_loss=894]

Epoch 401/500:  80%|████████  | 400/500 [02:57<00:47,  2.12it/s, v_num=1, train_loss=894]

Epoch 401/500:  80%|████████  | 401/500 [02:58<00:48,  2.04it/s, v_num=1, train_loss=894]

Epoch 401/500:  80%|████████  | 401/500 [02:58<00:48,  2.04it/s, v_num=1, train_loss=894]

Epoch 402/500:  80%|████████  | 401/500 [02:58<00:48,  2.04it/s, v_num=1, train_loss=894]

Epoch 402/500:  80%|████████  | 402/500 [02:59<00:54,  1.81it/s, v_num=1, train_loss=894]

Epoch 402/500:  80%|████████  | 402/500 [02:59<00:54,  1.81it/s, v_num=1, train_loss=894]

Epoch 403/500:  80%|████████  | 402/500 [02:59<00:54,  1.81it/s, v_num=1, train_loss=894]

Epoch 403/500:  81%|████████  | 403/500 [02:59<00:53,  1.81it/s, v_num=1, train_loss=894]

Epoch 403/500:  81%|████████  | 403/500 [02:59<00:53,  1.81it/s, v_num=1, train_loss=894]

Epoch 404/500:  81%|████████  | 403/500 [02:59<00:53,  1.81it/s, v_num=1, train_loss=894]

Epoch 404/500:  81%|████████  | 404/500 [03:00<00:54,  1.78it/s, v_num=1, train_loss=894]

Epoch 404/500:  81%|████████  | 404/500 [03:00<00:54,  1.78it/s, v_num=1, train_loss=894]

Epoch 405/500:  81%|████████  | 404/500 [03:00<00:54,  1.78it/s, v_num=1, train_loss=894]

Epoch 405/500:  81%|████████  | 405/500 [03:00<00:51,  1.86it/s, v_num=1, train_loss=894]

Epoch 405/500:  81%|████████  | 405/500 [03:00<00:51,  1.86it/s, v_num=1, train_loss=894]

Epoch 406/500:  81%|████████  | 405/500 [03:00<00:51,  1.86it/s, v_num=1, train_loss=894]

Epoch 406/500:  81%|████████  | 406/500 [03:01<00:49,  1.91it/s, v_num=1, train_loss=894]

Epoch 406/500:  81%|████████  | 406/500 [03:01<00:49,  1.91it/s, v_num=1, train_loss=894]

Epoch 407/500:  81%|████████  | 406/500 [03:01<00:49,  1.91it/s, v_num=1, train_loss=894]

Epoch 407/500:  81%|████████▏ | 407/500 [03:01<00:47,  1.95it/s, v_num=1, train_loss=894]

Epoch 407/500:  81%|████████▏ | 407/500 [03:01<00:47,  1.95it/s, v_num=1, train_loss=894]

Epoch 408/500:  81%|████████▏ | 407/500 [03:01<00:47,  1.95it/s, v_num=1, train_loss=894]

Epoch 408/500:  82%|████████▏ | 408/500 [03:02<00:46,  1.96it/s, v_num=1, train_loss=894]

Epoch 408/500:  82%|████████▏ | 408/500 [03:02<00:46,  1.96it/s, v_num=1, train_loss=894]

Epoch 409/500:  82%|████████▏ | 408/500 [03:02<00:46,  1.96it/s, v_num=1, train_loss=894]

Epoch 409/500:  82%|████████▏ | 409/500 [03:02<00:45,  1.98it/s, v_num=1, train_loss=894]

Epoch 409/500:  82%|████████▏ | 409/500 [03:02<00:45,  1.98it/s, v_num=1, train_loss=894]

Epoch 410/500:  82%|████████▏ | 409/500 [03:02<00:45,  1.98it/s, v_num=1, train_loss=894]

Epoch 410/500:  82%|████████▏ | 410/500 [03:03<00:44,  2.02it/s, v_num=1, train_loss=894]

Epoch 410/500:  82%|████████▏ | 410/500 [03:03<00:44,  2.02it/s, v_num=1, train_loss=894]

Epoch 411/500:  82%|████████▏ | 410/500 [03:03<00:44,  2.02it/s, v_num=1, train_loss=894]

Epoch 411/500:  82%|████████▏ | 411/500 [03:03<00:43,  2.05it/s, v_num=1, train_loss=894]

Epoch 411/500:  82%|████████▏ | 411/500 [03:03<00:43,  2.05it/s, v_num=1, train_loss=894]

Epoch 412/500:  82%|████████▏ | 411/500 [03:03<00:43,  2.05it/s, v_num=1, train_loss=894]

Epoch 412/500:  82%|████████▏ | 412/500 [03:04<00:43,  2.04it/s, v_num=1, train_loss=894]

Epoch 412/500:  82%|████████▏ | 412/500 [03:04<00:43,  2.04it/s, v_num=1, train_loss=894]

Epoch 413/500:  82%|████████▏ | 412/500 [03:04<00:43,  2.04it/s, v_num=1, train_loss=894]

Epoch 413/500:  83%|████████▎ | 413/500 [03:04<00:42,  2.07it/s, v_num=1, train_loss=894]

Epoch 413/500:  83%|████████▎ | 413/500 [03:04<00:42,  2.07it/s, v_num=1, train_loss=894]

Epoch 414/500:  83%|████████▎ | 413/500 [03:04<00:42,  2.07it/s, v_num=1, train_loss=894]

Epoch 414/500:  83%|████████▎ | 414/500 [03:05<00:44,  1.94it/s, v_num=1, train_loss=894]

Epoch 414/500:  83%|████████▎ | 414/500 [03:05<00:44,  1.94it/s, v_num=1, train_loss=894]

Epoch 415/500:  83%|████████▎ | 414/500 [03:05<00:44,  1.94it/s, v_num=1, train_loss=894]

Epoch 415/500:  83%|████████▎ | 415/500 [03:05<00:44,  1.93it/s, v_num=1, train_loss=894]

Epoch 415/500:  83%|████████▎ | 415/500 [03:05<00:44,  1.93it/s, v_num=1, train_loss=894]

Epoch 416/500:  83%|████████▎ | 415/500 [03:05<00:44,  1.93it/s, v_num=1, train_loss=894]

Epoch 416/500:  83%|████████▎ | 416/500 [03:06<00:41,  2.02it/s, v_num=1, train_loss=894]

Epoch 416/500:  83%|████████▎ | 416/500 [03:06<00:41,  2.02it/s, v_num=1, train_loss=894]

Epoch 417/500:  83%|████████▎ | 416/500 [03:06<00:41,  2.02it/s, v_num=1, train_loss=894]

Epoch 417/500:  83%|████████▎ | 417/500 [03:06<00:39,  2.12it/s, v_num=1, train_loss=894]

Epoch 417/500:  83%|████████▎ | 417/500 [03:06<00:39,  2.12it/s, v_num=1, train_loss=894]

Epoch 418/500:  83%|████████▎ | 417/500 [03:06<00:39,  2.12it/s, v_num=1, train_loss=894]

Epoch 418/500:  84%|████████▎ | 418/500 [03:07<00:38,  2.14it/s, v_num=1, train_loss=894]

Epoch 418/500:  84%|████████▎ | 418/500 [03:07<00:38,  2.14it/s, v_num=1, train_loss=894]

Epoch 419/500:  84%|████████▎ | 418/500 [03:07<00:38,  2.14it/s, v_num=1, train_loss=894]

Epoch 419/500:  84%|████████▍ | 419/500 [03:07<00:36,  2.21it/s, v_num=1, train_loss=894]

Epoch 419/500:  84%|████████▍ | 419/500 [03:07<00:36,  2.21it/s, v_num=1, train_loss=894]

Epoch 420/500:  84%|████████▍ | 419/500 [03:07<00:36,  2.21it/s, v_num=1, train_loss=894]

Epoch 420/500:  84%|████████▍ | 420/500 [03:07<00:35,  2.26it/s, v_num=1, train_loss=894]

Epoch 420/500:  84%|████████▍ | 420/500 [03:07<00:35,  2.26it/s, v_num=1, train_loss=894]

Epoch 421/500:  84%|████████▍ | 420/500 [03:07<00:35,  2.26it/s, v_num=1, train_loss=894]

Epoch 421/500:  84%|████████▍ | 421/500 [03:08<00:36,  2.19it/s, v_num=1, train_loss=894]

Epoch 421/500:  84%|████████▍ | 421/500 [03:08<00:36,  2.19it/s, v_num=1, train_loss=894]

Epoch 422/500:  84%|████████▍ | 421/500 [03:08<00:36,  2.19it/s, v_num=1, train_loss=894]

Epoch 422/500:  84%|████████▍ | 422/500 [03:08<00:36,  2.11it/s, v_num=1, train_loss=894]

Epoch 422/500:  84%|████████▍ | 422/500 [03:08<00:36,  2.11it/s, v_num=1, train_loss=894]

Epoch 423/500:  84%|████████▍ | 422/500 [03:08<00:36,  2.11it/s, v_num=1, train_loss=894]

Epoch 423/500:  85%|████████▍ | 423/500 [03:09<00:35,  2.15it/s, v_num=1, train_loss=894]

Epoch 423/500:  85%|████████▍ | 423/500 [03:09<00:35,  2.15it/s, v_num=1, train_loss=894]

Epoch 424/500:  85%|████████▍ | 423/500 [03:09<00:35,  2.15it/s, v_num=1, train_loss=894]

Epoch 424/500:  85%|████████▍ | 424/500 [03:09<00:34,  2.22it/s, v_num=1, train_loss=894]

Epoch 424/500:  85%|████████▍ | 424/500 [03:09<00:34,  2.22it/s, v_num=1, train_loss=894]

Epoch 425/500:  85%|████████▍ | 424/500 [03:09<00:34,  2.22it/s, v_num=1, train_loss=894]

Epoch 425/500:  85%|████████▌ | 425/500 [03:10<00:33,  2.26it/s, v_num=1, train_loss=894]

Epoch 425/500:  85%|████████▌ | 425/500 [03:10<00:33,  2.26it/s, v_num=1, train_loss=894]

Epoch 426/500:  85%|████████▌ | 425/500 [03:10<00:33,  2.26it/s, v_num=1, train_loss=894]

Epoch 426/500:  85%|████████▌ | 426/500 [03:10<00:32,  2.31it/s, v_num=1, train_loss=894]

Epoch 426/500:  85%|████████▌ | 426/500 [03:10<00:32,  2.31it/s, v_num=1, train_loss=894]

Epoch 427/500:  85%|████████▌ | 426/500 [03:10<00:32,  2.31it/s, v_num=1, train_loss=894]

Epoch 427/500:  85%|████████▌ | 427/500 [03:10<00:31,  2.32it/s, v_num=1, train_loss=894]

Epoch 427/500:  85%|████████▌ | 427/500 [03:10<00:31,  2.32it/s, v_num=1, train_loss=894]

Epoch 428/500:  85%|████████▌ | 427/500 [03:10<00:31,  2.32it/s, v_num=1, train_loss=894]

Epoch 428/500:  86%|████████▌ | 428/500 [03:11<00:32,  2.24it/s, v_num=1, train_loss=894]

Epoch 428/500:  86%|████████▌ | 428/500 [03:11<00:32,  2.24it/s, v_num=1, train_loss=894]

Epoch 429/500:  86%|████████▌ | 428/500 [03:11<00:32,  2.24it/s, v_num=1, train_loss=894]

Epoch 429/500:  86%|████████▌ | 429/500 [03:11<00:32,  2.16it/s, v_num=1, train_loss=894]

Epoch 429/500:  86%|████████▌ | 429/500 [03:11<00:32,  2.16it/s, v_num=1, train_loss=894]

Epoch 430/500:  86%|████████▌ | 429/500 [03:11<00:32,  2.16it/s, v_num=1, train_loss=894]

Epoch 430/500:  86%|████████▌ | 430/500 [03:12<00:33,  2.08it/s, v_num=1, train_loss=894]

Epoch 430/500:  86%|████████▌ | 430/500 [03:12<00:33,  2.08it/s, v_num=1, train_loss=894]

Epoch 431/500:  86%|████████▌ | 430/500 [03:12<00:33,  2.08it/s, v_num=1, train_loss=894]

Epoch 431/500:  86%|████████▌ | 431/500 [03:13<00:34,  2.02it/s, v_num=1, train_loss=894]

Epoch 431/500:  86%|████████▌ | 431/500 [03:13<00:34,  2.02it/s, v_num=1, train_loss=894]

Epoch 432/500:  86%|████████▌ | 431/500 [03:13<00:34,  2.02it/s, v_num=1, train_loss=894]

Epoch 432/500:  86%|████████▋ | 432/500 [03:13<00:33,  2.01it/s, v_num=1, train_loss=894]

Epoch 432/500:  86%|████████▋ | 432/500 [03:13<00:33,  2.01it/s, v_num=1, train_loss=894]

Epoch 433/500:  86%|████████▋ | 432/500 [03:13<00:33,  2.01it/s, v_num=1, train_loss=894]

Epoch 433/500:  87%|████████▋ | 433/500 [03:14<00:33,  1.98it/s, v_num=1, train_loss=894]

Epoch 433/500:  87%|████████▋ | 433/500 [03:14<00:33,  1.98it/s, v_num=1, train_loss=894]

Epoch 434/500:  87%|████████▋ | 433/500 [03:14<00:33,  1.98it/s, v_num=1, train_loss=894]

Epoch 434/500:  87%|████████▋ | 434/500 [03:14<00:35,  1.87it/s, v_num=1, train_loss=894]

Epoch 434/500:  87%|████████▋ | 434/500 [03:14<00:35,  1.87it/s, v_num=1, train_loss=894]

Epoch 435/500:  87%|████████▋ | 434/500 [03:14<00:35,  1.87it/s, v_num=1, train_loss=894]

Epoch 435/500:  87%|████████▋ | 435/500 [03:15<00:35,  1.82it/s, v_num=1, train_loss=894]

Epoch 435/500:  87%|████████▋ | 435/500 [03:15<00:35,  1.82it/s, v_num=1, train_loss=894]

Epoch 436/500:  87%|████████▋ | 435/500 [03:15<00:35,  1.82it/s, v_num=1, train_loss=894]

Epoch 436/500:  87%|████████▋ | 436/500 [03:15<00:34,  1.85it/s, v_num=1, train_loss=894]

Epoch 436/500:  87%|████████▋ | 436/500 [03:15<00:34,  1.85it/s, v_num=1, train_loss=894]

Epoch 437/500:  87%|████████▋ | 436/500 [03:15<00:34,  1.85it/s, v_num=1, train_loss=894]

Epoch 437/500:  87%|████████▋ | 437/500 [03:16<00:33,  1.87it/s, v_num=1, train_loss=894]

Epoch 437/500:  87%|████████▋ | 437/500 [03:16<00:33,  1.87it/s, v_num=1, train_loss=894]

Epoch 438/500:  87%|████████▋ | 437/500 [03:16<00:33,  1.87it/s, v_num=1, train_loss=894]

Epoch 438/500:  88%|████████▊ | 438/500 [03:16<00:32,  1.88it/s, v_num=1, train_loss=894]

Epoch 438/500:  88%|████████▊ | 438/500 [03:16<00:32,  1.88it/s, v_num=1, train_loss=894]

Epoch 439/500:  88%|████████▊ | 438/500 [03:16<00:32,  1.88it/s, v_num=1, train_loss=894]

Epoch 439/500:  88%|████████▊ | 439/500 [03:17<00:31,  1.92it/s, v_num=1, train_loss=894]

Epoch 439/500:  88%|████████▊ | 439/500 [03:17<00:31,  1.92it/s, v_num=1, train_loss=894]

Epoch 440/500:  88%|████████▊ | 439/500 [03:17<00:31,  1.92it/s, v_num=1, train_loss=894]

Epoch 440/500:  88%|████████▊ | 440/500 [03:17<00:30,  1.94it/s, v_num=1, train_loss=894]

Epoch 440/500:  88%|████████▊ | 440/500 [03:17<00:30,  1.94it/s, v_num=1, train_loss=894]

Epoch 441/500:  88%|████████▊ | 440/500 [03:17<00:30,  1.94it/s, v_num=1, train_loss=894]

Epoch 441/500:  88%|████████▊ | 441/500 [03:18<00:29,  1.97it/s, v_num=1, train_loss=894]

Epoch 441/500:  88%|████████▊ | 441/500 [03:18<00:29,  1.97it/s, v_num=1, train_loss=894]

Epoch 442/500:  88%|████████▊ | 441/500 [03:18<00:29,  1.97it/s, v_num=1, train_loss=894]

Epoch 442/500:  88%|████████▊ | 442/500 [03:18<00:29,  1.96it/s, v_num=1, train_loss=894]

Epoch 442/500:  88%|████████▊ | 442/500 [03:18<00:29,  1.96it/s, v_num=1, train_loss=894]

Epoch 443/500:  88%|████████▊ | 442/500 [03:18<00:29,  1.96it/s, v_num=1, train_loss=894]

Epoch 443/500:  89%|████████▊ | 443/500 [03:19<00:29,  1.93it/s, v_num=1, train_loss=894]

Epoch 443/500:  89%|████████▊ | 443/500 [03:19<00:29,  1.93it/s, v_num=1, train_loss=894]

Epoch 444/500:  89%|████████▊ | 443/500 [03:19<00:29,  1.93it/s, v_num=1, train_loss=894]

Epoch 444/500:  89%|████████▉ | 444/500 [03:19<00:29,  1.90it/s, v_num=1, train_loss=894]

Epoch 444/500:  89%|████████▉ | 444/500 [03:19<00:29,  1.90it/s, v_num=1, train_loss=894]

Epoch 445/500:  89%|████████▉ | 444/500 [03:19<00:29,  1.90it/s, v_num=1, train_loss=894]

Epoch 445/500:  89%|████████▉ | 445/500 [03:20<00:29,  1.89it/s, v_num=1, train_loss=894]

Epoch 445/500:  89%|████████▉ | 445/500 [03:20<00:29,  1.89it/s, v_num=1, train_loss=894]

Epoch 446/500:  89%|████████▉ | 445/500 [03:20<00:29,  1.89it/s, v_num=1, train_loss=894]

Epoch 446/500:  89%|████████▉ | 446/500 [03:20<00:28,  1.91it/s, v_num=1, train_loss=894]

Epoch 446/500:  89%|████████▉ | 446/500 [03:20<00:28,  1.91it/s, v_num=1, train_loss=894]

Epoch 447/500:  89%|████████▉ | 446/500 [03:20<00:28,  1.91it/s, v_num=1, train_loss=894]

Epoch 447/500:  89%|████████▉ | 447/500 [03:21<00:27,  1.94it/s, v_num=1, train_loss=894]

Epoch 447/500:  89%|████████▉ | 447/500 [03:21<00:27,  1.94it/s, v_num=1, train_loss=894]

Epoch 448/500:  89%|████████▉ | 447/500 [03:21<00:27,  1.94it/s, v_num=1, train_loss=894]

Epoch 448/500:  90%|████████▉ | 448/500 [03:21<00:26,  1.98it/s, v_num=1, train_loss=894]

Epoch 448/500:  90%|████████▉ | 448/500 [03:21<00:26,  1.98it/s, v_num=1, train_loss=894]

Epoch 449/500:  90%|████████▉ | 448/500 [03:21<00:26,  1.98it/s, v_num=1, train_loss=894]

Epoch 449/500:  90%|████████▉ | 449/500 [03:22<00:29,  1.75it/s, v_num=1, train_loss=894]

Epoch 449/500:  90%|████████▉ | 449/500 [03:22<00:29,  1.75it/s, v_num=1, train_loss=894]

Epoch 450/500:  90%|████████▉ | 449/500 [03:22<00:29,  1.75it/s, v_num=1, train_loss=894]

Epoch 450/500:  90%|█████████ | 450/500 [03:23<00:30,  1.66it/s, v_num=1, train_loss=894]

Epoch 450/500:  90%|█████████ | 450/500 [03:23<00:30,  1.66it/s, v_num=1, train_loss=894]

Epoch 451/500:  90%|█████████ | 450/500 [03:23<00:30,  1.66it/s, v_num=1, train_loss=894]

Epoch 451/500:  90%|█████████ | 451/500 [03:24<00:33,  1.46it/s, v_num=1, train_loss=894]

Epoch 451/500:  90%|█████████ | 451/500 [03:24<00:33,  1.46it/s, v_num=1, train_loss=894]

Epoch 452/500:  90%|█████████ | 451/500 [03:24<00:33,  1.46it/s, v_num=1, train_loss=894]

Epoch 452/500:  90%|█████████ | 452/500 [03:24<00:30,  1.59it/s, v_num=1, train_loss=894]

Epoch 452/500:  90%|█████████ | 452/500 [03:24<00:30,  1.59it/s, v_num=1, train_loss=894]

Epoch 453/500:  90%|█████████ | 452/500 [03:24<00:30,  1.59it/s, v_num=1, train_loss=894]

Epoch 453/500:  91%|█████████ | 453/500 [03:25<00:29,  1.62it/s, v_num=1, train_loss=894]

Epoch 453/500:  91%|█████████ | 453/500 [03:25<00:29,  1.62it/s, v_num=1, train_loss=894]

Epoch 454/500:  91%|█████████ | 453/500 [03:25<00:29,  1.62it/s, v_num=1, train_loss=894]

Epoch 454/500:  91%|█████████ | 454/500 [03:25<00:26,  1.72it/s, v_num=1, train_loss=894]

Epoch 454/500:  91%|█████████ | 454/500 [03:25<00:26,  1.72it/s, v_num=1, train_loss=894]

Epoch 455/500:  91%|█████████ | 454/500 [03:25<00:26,  1.72it/s, v_num=1, train_loss=894]

Epoch 455/500:  91%|█████████ | 455/500 [03:26<00:25,  1.77it/s, v_num=1, train_loss=894]

Epoch 455/500:  91%|█████████ | 455/500 [03:26<00:25,  1.77it/s, v_num=1, train_loss=894]

Epoch 456/500:  91%|█████████ | 455/500 [03:26<00:25,  1.77it/s, v_num=1, train_loss=894]

Epoch 456/500:  91%|█████████ | 456/500 [03:26<00:25,  1.74it/s, v_num=1, train_loss=894]

Epoch 456/500:  91%|█████████ | 456/500 [03:26<00:25,  1.74it/s, v_num=1, train_loss=894]

Epoch 457/500:  91%|█████████ | 456/500 [03:26<00:25,  1.74it/s, v_num=1, train_loss=894]

Epoch 457/500:  91%|█████████▏| 457/500 [03:27<00:25,  1.68it/s, v_num=1, train_loss=894]

Epoch 457/500:  91%|█████████▏| 457/500 [03:27<00:25,  1.68it/s, v_num=1, train_loss=894]

Epoch 458/500:  91%|█████████▏| 457/500 [03:27<00:25,  1.68it/s, v_num=1, train_loss=894]

Epoch 458/500:  92%|█████████▏| 458/500 [03:28<00:24,  1.70it/s, v_num=1, train_loss=894]

Epoch 458/500:  92%|█████████▏| 458/500 [03:28<00:24,  1.70it/s, v_num=1, train_loss=894]

Epoch 459/500:  92%|█████████▏| 458/500 [03:28<00:24,  1.70it/s, v_num=1, train_loss=894]

Epoch 459/500:  92%|█████████▏| 459/500 [03:28<00:22,  1.80it/s, v_num=1, train_loss=894]

Epoch 459/500:  92%|█████████▏| 459/500 [03:28<00:22,  1.80it/s, v_num=1, train_loss=894]

Epoch 460/500:  92%|█████████▏| 459/500 [03:28<00:22,  1.80it/s, v_num=1, train_loss=894]

Epoch 460/500:  92%|█████████▏| 460/500 [03:29<00:22,  1.77it/s, v_num=1, train_loss=894]

Epoch 460/500:  92%|█████████▏| 460/500 [03:29<00:22,  1.77it/s, v_num=1, train_loss=894]

Epoch 461/500:  92%|█████████▏| 460/500 [03:29<00:22,  1.77it/s, v_num=1, train_loss=894]

Epoch 461/500:  92%|█████████▏| 461/500 [03:29<00:20,  1.87it/s, v_num=1, train_loss=894]

Epoch 461/500:  92%|█████████▏| 461/500 [03:29<00:20,  1.87it/s, v_num=1, train_loss=894]

Epoch 462/500:  92%|█████████▏| 461/500 [03:29<00:20,  1.87it/s, v_num=1, train_loss=894]

Epoch 462/500:  92%|█████████▏| 462/500 [03:30<00:27,  1.38it/s, v_num=1, train_loss=894]

Epoch 462/500:  92%|█████████▏| 462/500 [03:30<00:27,  1.38it/s, v_num=1, train_loss=894]

Epoch 463/500:  92%|█████████▏| 462/500 [03:30<00:27,  1.38it/s, v_num=1, train_loss=894]

Epoch 463/500:  93%|█████████▎| 463/500 [03:32<00:34,  1.06it/s, v_num=1, train_loss=894]

Epoch 463/500:  93%|█████████▎| 463/500 [03:32<00:34,  1.06it/s, v_num=1, train_loss=894]

Epoch 464/500:  93%|█████████▎| 463/500 [03:32<00:34,  1.06it/s, v_num=1, train_loss=894]

Epoch 464/500:  93%|█████████▎| 464/500 [03:32<00:29,  1.22it/s, v_num=1, train_loss=894]

Epoch 464/500:  93%|█████████▎| 464/500 [03:32<00:29,  1.22it/s, v_num=1, train_loss=894]

Epoch 465/500:  93%|█████████▎| 464/500 [03:32<00:29,  1.22it/s, v_num=1, train_loss=894]

Epoch 465/500:  93%|█████████▎| 465/500 [03:33<00:25,  1.37it/s, v_num=1, train_loss=894]

Epoch 465/500:  93%|█████████▎| 465/500 [03:33<00:25,  1.37it/s, v_num=1, train_loss=894]

Epoch 466/500:  93%|█████████▎| 465/500 [03:33<00:25,  1.37it/s, v_num=1, train_loss=894]

Epoch 466/500:  93%|█████████▎| 466/500 [03:33<00:23,  1.47it/s, v_num=1, train_loss=894]

Epoch 466/500:  93%|█████████▎| 466/500 [03:33<00:23,  1.47it/s, v_num=1, train_loss=894]

Epoch 467/500:  93%|█████████▎| 466/500 [03:33<00:23,  1.47it/s, v_num=1, train_loss=894]

Epoch 467/500:  93%|█████████▎| 467/500 [03:34<00:20,  1.59it/s, v_num=1, train_loss=894]

Epoch 467/500:  93%|█████████▎| 467/500 [03:34<00:20,  1.59it/s, v_num=1, train_loss=894]

Epoch 468/500:  93%|█████████▎| 467/500 [03:34<00:20,  1.59it/s, v_num=1, train_loss=894]

Epoch 468/500:  94%|█████████▎| 468/500 [03:34<00:19,  1.68it/s, v_num=1, train_loss=894]

Epoch 468/500:  94%|█████████▎| 468/500 [03:34<00:19,  1.68it/s, v_num=1, train_loss=894]

Epoch 469/500:  94%|█████████▎| 468/500 [03:34<00:19,  1.68it/s, v_num=1, train_loss=894]

Epoch 469/500:  94%|█████████▍| 469/500 [03:35<00:17,  1.74it/s, v_num=1, train_loss=894]

Epoch 469/500:  94%|█████████▍| 469/500 [03:35<00:17,  1.74it/s, v_num=1, train_loss=894]

Epoch 470/500:  94%|█████████▍| 469/500 [03:35<00:17,  1.74it/s, v_num=1, train_loss=894]

Epoch 470/500:  94%|█████████▍| 470/500 [03:35<00:16,  1.79it/s, v_num=1, train_loss=894]

Epoch 470/500:  94%|█████████▍| 470/500 [03:35<00:16,  1.79it/s, v_num=1, train_loss=894]

Epoch 471/500:  94%|█████████▍| 470/500 [03:35<00:16,  1.79it/s, v_num=1, train_loss=894]

Epoch 471/500:  94%|█████████▍| 471/500 [03:36<00:15,  1.85it/s, v_num=1, train_loss=894]

Epoch 471/500:  94%|█████████▍| 471/500 [03:36<00:15,  1.85it/s, v_num=1, train_loss=894]

Epoch 472/500:  94%|█████████▍| 471/500 [03:36<00:15,  1.85it/s, v_num=1, train_loss=894]

Epoch 472/500:  94%|█████████▍| 472/500 [03:36<00:14,  1.91it/s, v_num=1, train_loss=894]

Epoch 472/500:  94%|█████████▍| 472/500 [03:36<00:14,  1.91it/s, v_num=1, train_loss=894]

Epoch 473/500:  94%|█████████▍| 472/500 [03:36<00:14,  1.91it/s, v_num=1, train_loss=894]

Epoch 473/500:  95%|█████████▍| 473/500 [03:37<00:13,  1.94it/s, v_num=1, train_loss=894]

Epoch 473/500:  95%|█████████▍| 473/500 [03:37<00:13,  1.94it/s, v_num=1, train_loss=894]

Epoch 474/500:  95%|█████████▍| 473/500 [03:37<00:13,  1.94it/s, v_num=1, train_loss=894]

Epoch 474/500:  95%|█████████▍| 474/500 [03:37<00:13,  1.92it/s, v_num=1, train_loss=894]

Epoch 474/500:  95%|█████████▍| 474/500 [03:37<00:13,  1.92it/s, v_num=1, train_loss=894]

Epoch 475/500:  95%|█████████▍| 474/500 [03:37<00:13,  1.92it/s, v_num=1, train_loss=894]

Epoch 475/500:  95%|█████████▌| 475/500 [03:38<00:12,  1.96it/s, v_num=1, train_loss=894]

Epoch 475/500:  95%|█████████▌| 475/500 [03:38<00:12,  1.96it/s, v_num=1, train_loss=894]

Epoch 476/500:  95%|█████████▌| 475/500 [03:38<00:12,  1.96it/s, v_num=1, train_loss=894]

Epoch 476/500:  95%|█████████▌| 476/500 [03:38<00:11,  2.00it/s, v_num=1, train_loss=894]

Epoch 476/500:  95%|█████████▌| 476/500 [03:38<00:11,  2.00it/s, v_num=1, train_loss=894]

Epoch 477/500:  95%|█████████▌| 476/500 [03:38<00:11,  2.00it/s, v_num=1, train_loss=894]

Epoch 477/500:  95%|█████████▌| 477/500 [03:40<00:17,  1.32it/s, v_num=1, train_loss=894]

Epoch 477/500:  95%|█████████▌| 477/500 [03:40<00:17,  1.32it/s, v_num=1, train_loss=894]

Epoch 478/500:  95%|█████████▌| 477/500 [03:40<00:17,  1.32it/s, v_num=1, train_loss=894]

Epoch 478/500:  96%|█████████▌| 478/500 [03:41<00:17,  1.23it/s, v_num=1, train_loss=894]

Epoch 478/500:  96%|█████████▌| 478/500 [03:41<00:17,  1.23it/s, v_num=1, train_loss=894]

Epoch 479/500:  96%|█████████▌| 478/500 [03:41<00:17,  1.23it/s, v_num=1, train_loss=894]

Epoch 479/500:  96%|█████████▌| 479/500 [03:41<00:15,  1.39it/s, v_num=1, train_loss=894]

Epoch 479/500:  96%|█████████▌| 479/500 [03:41<00:15,  1.39it/s, v_num=1, train_loss=894]

Epoch 480/500:  96%|█████████▌| 479/500 [03:41<00:15,  1.39it/s, v_num=1, train_loss=894]

Epoch 480/500:  96%|█████████▌| 480/500 [03:42<00:16,  1.19it/s, v_num=1, train_loss=894]

Epoch 480/500:  96%|█████████▌| 480/500 [03:42<00:16,  1.19it/s, v_num=1, train_loss=894]

Epoch 481/500:  96%|█████████▌| 480/500 [03:42<00:16,  1.19it/s, v_num=1, train_loss=894]

Epoch 481/500:  96%|█████████▌| 481/500 [03:43<00:17,  1.08it/s, v_num=1, train_loss=894]

Epoch 481/500:  96%|█████████▌| 481/500 [03:43<00:17,  1.08it/s, v_num=1, train_loss=894]

Epoch 482/500:  96%|█████████▌| 481/500 [03:43<00:17,  1.08it/s, v_num=1, train_loss=894]

Epoch 482/500:  96%|█████████▋| 482/500 [03:45<00:18,  1.05s/it, v_num=1, train_loss=894]

Epoch 482/500:  96%|█████████▋| 482/500 [03:45<00:18,  1.05s/it, v_num=1, train_loss=894]

Epoch 483/500:  96%|█████████▋| 482/500 [03:45<00:18,  1.05s/it, v_num=1, train_loss=894]

Epoch 483/500:  97%|█████████▋| 483/500 [03:46<00:17,  1.01s/it, v_num=1, train_loss=894]

Epoch 483/500:  97%|█████████▋| 483/500 [03:46<00:17,  1.01s/it, v_num=1, train_loss=894]

Epoch 484/500:  97%|█████████▋| 483/500 [03:46<00:17,  1.01s/it, v_num=1, train_loss=894]

Epoch 484/500:  97%|█████████▋| 484/500 [03:47<00:15,  1.04it/s, v_num=1, train_loss=894]

Epoch 484/500:  97%|█████████▋| 484/500 [03:47<00:15,  1.04it/s, v_num=1, train_loss=894]

Epoch 485/500:  97%|█████████▋| 484/500 [03:47<00:15,  1.04it/s, v_num=1, train_loss=894]

Epoch 485/500:  97%|█████████▋| 485/500 [03:48<00:15,  1.05s/it, v_num=1, train_loss=894]

Epoch 485/500:  97%|█████████▋| 485/500 [03:48<00:15,  1.05s/it, v_num=1, train_loss=894]

Epoch 486/500:  97%|█████████▋| 485/500 [03:48<00:15,  1.05s/it, v_num=1, train_loss=894]

Epoch 486/500:  97%|█████████▋| 486/500 [03:49<00:15,  1.10s/it, v_num=1, train_loss=894]

Epoch 486/500:  97%|█████████▋| 486/500 [03:49<00:15,  1.10s/it, v_num=1, train_loss=894]

Epoch 487/500:  97%|█████████▋| 486/500 [03:49<00:15,  1.10s/it, v_num=1, train_loss=894]

Epoch 487/500:  97%|█████████▋| 487/500 [03:50<00:15,  1.17s/it, v_num=1, train_loss=894]

Epoch 487/500:  97%|█████████▋| 487/500 [03:50<00:15,  1.17s/it, v_num=1, train_loss=894]

Epoch 488/500:  97%|█████████▋| 487/500 [03:50<00:15,  1.17s/it, v_num=1, train_loss=894]

Epoch 488/500:  98%|█████████▊| 488/500 [03:51<00:12,  1.03s/it, v_num=1, train_loss=894]

Epoch 488/500:  98%|█████████▊| 488/500 [03:51<00:12,  1.03s/it, v_num=1, train_loss=894]

Epoch 489/500:  98%|█████████▊| 488/500 [03:51<00:12,  1.03s/it, v_num=1, train_loss=894]

Epoch 489/500:  98%|█████████▊| 489/500 [03:52<00:10,  1.02it/s, v_num=1, train_loss=894]

Epoch 489/500:  98%|█████████▊| 489/500 [03:52<00:10,  1.02it/s, v_num=1, train_loss=894]

Epoch 490/500:  98%|█████████▊| 489/500 [03:52<00:10,  1.02it/s, v_num=1, train_loss=894]

Epoch 490/500:  98%|█████████▊| 490/500 [03:53<00:10,  1.05s/it, v_num=1, train_loss=894]

Epoch 490/500:  98%|█████████▊| 490/500 [03:53<00:10,  1.05s/it, v_num=1, train_loss=894]

Epoch 491/500:  98%|█████████▊| 490/500 [03:53<00:10,  1.05s/it, v_num=1, train_loss=894]

Epoch 491/500:  98%|█████████▊| 491/500 [03:54<00:09,  1.09s/it, v_num=1, train_loss=894]

Epoch 491/500:  98%|█████████▊| 491/500 [03:54<00:09,  1.09s/it, v_num=1, train_loss=893]

Epoch 492/500:  98%|█████████▊| 491/500 [03:54<00:09,  1.09s/it, v_num=1, train_loss=893]

Epoch 492/500:  98%|█████████▊| 492/500 [03:56<00:09,  1.14s/it, v_num=1, train_loss=893]

Epoch 492/500:  98%|█████████▊| 492/500 [03:56<00:09,  1.14s/it, v_num=1, train_loss=894]

Epoch 493/500:  98%|█████████▊| 492/500 [03:56<00:09,  1.14s/it, v_num=1, train_loss=894]

Epoch 493/500:  99%|█████████▊| 493/500 [03:56<00:07,  1.02s/it, v_num=1, train_loss=894]

Epoch 493/500:  99%|█████████▊| 493/500 [03:56<00:07,  1.02s/it, v_num=1, train_loss=894]

Epoch 494/500:  99%|█████████▊| 493/500 [03:56<00:07,  1.02s/it, v_num=1, train_loss=894]

Epoch 494/500:  99%|█████████▉| 494/500 [03:57<00:06,  1.01s/it, v_num=1, train_loss=894]

Epoch 494/500:  99%|█████████▉| 494/500 [03:57<00:06,  1.01s/it, v_num=1, train_loss=894]

Epoch 495/500:  99%|█████████▉| 494/500 [03:57<00:06,  1.01s/it, v_num=1, train_loss=894]

Epoch 495/500:  99%|█████████▉| 495/500 [03:58<00:05,  1.04s/it, v_num=1, train_loss=894]

Epoch 495/500:  99%|█████████▉| 495/500 [03:58<00:05,  1.04s/it, v_num=1, train_loss=894]

Epoch 496/500:  99%|█████████▉| 495/500 [03:58<00:05,  1.04s/it, v_num=1, train_loss=894]

Epoch 496/500:  99%|█████████▉| 496/500 [04:00<00:04,  1.10s/it, v_num=1, train_loss=894]

Epoch 496/500:  99%|█████████▉| 496/500 [04:00<00:04,  1.10s/it, v_num=1, train_loss=894]

Epoch 497/500:  99%|█████████▉| 496/500 [04:00<00:04,  1.10s/it, v_num=1, train_loss=894]

Epoch 497/500:  99%|█████████▉| 497/500 [04:01<00:03,  1.11s/it, v_num=1, train_loss=894]

Epoch 497/500:  99%|█████████▉| 497/500 [04:01<00:03,  1.11s/it, v_num=1, train_loss=894]

Epoch 498/500:  99%|█████████▉| 497/500 [04:01<00:03,  1.11s/it, v_num=1, train_loss=894]

Epoch 498/500: 100%|█████████▉| 498/500 [04:02<00:01,  1.00it/s, v_num=1, train_loss=894]

Epoch 498/500: 100%|█████████▉| 498/500 [04:02<00:01,  1.00it/s, v_num=1, train_loss=894]

Epoch 499/500: 100%|█████████▉| 498/500 [04:02<00:01,  1.00it/s, v_num=1, train_loss=894]

Epoch 499/500: 100%|█████████▉| 499/500 [04:02<00:00,  1.06it/s, v_num=1, train_loss=894]

Epoch 499/500: 100%|█████████▉| 499/500 [04:02<00:00,  1.06it/s, v_num=1, train_loss=894]

Epoch 500/500: 100%|█████████▉| 499/500 [04:02<00:00,  1.06it/s, v_num=1, train_loss=894]

Epoch 500/500: 100%|██████████| 500/500 [04:04<00:00,  1.03s/it, v_num=1, train_loss=894]

Epoch 500/500: 100%|██████████| 500/500 [04:04<00:00,  1.03s/it, v_num=1, train_loss=894]

INFO: `Trainer.fit` stopped: `max_epochs=500` reached.

`Trainer.fit` stopped: `max_epochs=500` reached.

Epoch 500/500: 100%|██████████| 500/500 [04:04<00:00,  2.05it/s, v_num=1, train_loss=894]

Now that we have updated the model, we can calculate the (ideally batch-corrected) latent representation of our query:

adata.obsm["X_scVI"] = query_model.get_latent_representation()

We can now use this newly calculated low-dimensional embedding as a basis for visualization and clustering. Let’s calculate the new UMAP using the scVI-based representation of the data.

sc.pp.neighbors(adata, use_rep="X_scVI")
sc.tl.umap(adata)

To see if the mapping-based UMAP makes general sense, let’s look at a few markers and if their expression is localized to specific parts of the UMAP:

sc.pl.umap(
    adata,
    color=["IGHD", "IGHM", "PRDM1"],
    vmin=0,
    vmax="p99",  # set vmax to the 99th percentile of the gene count instead of the maximum, to prevent outliers from making expression in other cells invisible. Note that this can cause problems for extremely lowly expressed genes.
    sort_order=False,  # do not plot highest expression on top, to not get a biased view of the mean expression among cells
    frameon=False,
    cmap="Reds",  # or choose another color map e.g. from here: https://matplotlib.org/stable/tutorials/colors/colormaps.html
)

Now the essential step is that we can combine the inferred latent space embedding of our query data with the existing reference embedding. Using this joint embedding, we will not only be able to e.g., visualize and cluster the two together, but we can also do label transfer from the query to the reference.

Let’s load the reference embedding: this is often made publicly available with existing atlases.

af = ln.Artifact.get(
    key="cellular_structure/annotation_reference_embedding.h5ad", is_latest=True
)
ref_emb = af.load(is_run_input=False)

We’ll store a variable specifying that these cells are from the reference.

ref_emb.obs["reference_or_query"] = "reference"

Let’s see what’s in this reference object:

ref_emb

AnnData object with n_obs × n_vars = 86332 × 10
    obs: 'donor', 'batch', 'site', 'cell_type', 'reference_or_query'
    uns: 'neighbors', 'umap'
    obsm: 'X_umap'
    obsp: 'connectivities', 'distances'
    layers: None (.X)

As you can see it has only 10 dimensions (in .X) which together represent the latent space embedding of the reference cells. Our query embedding that we calculated for our own data also has 10 dimensions. The 10 dimensions of the reference and query are the same and can be combined!

Moreover, it has cell type labels in .obs['cell_type']. We will use these labels to annotate our own data.

To perform the label transfer, we will first concatenate the reference and query data using the 10-dimensional embedding. To get there, we will create the same type of AnnData object from our query data as we have from the reference (with the embedding under .X) and concatenate the two. With that, we can jointly analyze reference and query including doing transfer from one to the other.

adata_emb = sc.AnnData(X=adata.obsm["X_scVI"], obs=adata.obs)

adata_emb.obs["reference_or_query"] = "query"

We will set the cell type labels in .obs["cell_type"] as None, so that we can see how we can annotate cell types based on the surroundings.

adata_emb.obs["cell_type"] = None

emb_ref_query = sc.concat(
    [ref_emb, adata_emb],
    axis=0,
    join="outer",
    index_unique=None,
    merge="unique",
)

Let’s visualize the joint embedding with a UMAP.

sc.pp.neighbors(emb_ref_query)
sc.tl.umap(emb_ref_query)

We can visually get a first impression of whether the reference and query integrated well based on the UMAP:

sc.pl.umap(
    emb_ref_query,
    color=["reference_or_query"],
    sort_order=False,
    frameon=False,
)

... storing 'cell_type' as categorical

... storing 'reference_or_query' as categorical

The (partial) mixing of query and reference in this UMAP is a good sign! When mapping completely fails, you will often see a full separation of query and reference in the UMAP.

Now let’s look at the cell type annotations from the reference. All cells from the query are set to NA here as they don’t have annotations yet and shown in black.

We’ll make this figure a bit bigger so that we can read the legend well:

sc.set_figure_params(figsize=(8, 8))

sc.pl.umap(
    emb_ref_query,
    color=["cell_type"],
    sort_order=False,
    frameon=False,
    legend_loc="on data",
    legend_fontsize=10,
    na_color="black",
)

As you can already tell from the UMAP, we can guess the cell type of each of our own cells (in black) by looking at which cell types from the reference surrounding it. This is exactly what a nearest-neighbor-graph-based label transfer approach does: for each query cell it checks what is the most common cell type among its neighboring reference cells. The higher the fraction of reference cells coming from a single cell type, the more confident the label transfer is.

Let’s perform the KNN-based label transfer.

First we set up the label transfer model:

cmapper = CellMapper(query=adata_emb, reference=ref_emb)
cmapper.compute_neighbors(
    use_rep="X",  # location of our joint embedding
    n_neighbors=15,
)

INFO     Initialized CellMapper with 13668 query cells and 86332 reference cells.

WARNING  Using sklearn for neighbor search with large dataset (86332 cells). Consider using approximate k-NN search
         (e.g. pynndescent) or GPU acceleration (e.g. faiss or rapids)

INFO     Using sklearn to compute 15 neighbors.

Now we perform the label transfer:

cmapper.compute_mapping_matrix(kernel_method="scarches")
cmapper.map_obs(
    key="cell_type",  # obs column name for which to transfer labels
    prediction_postfix="_pred",
    confidence_postfix="_conf",
)

INFO     Computing mapping matrix using kernel method 'scarches'.

INFO     Mapping categorical data for key 'cell_type' using direct multiplication.

INFO     Categorical data mapped and stored in query.obs['cell_type_pred'].

And store the results in our adata:

adata_emb.obs["transf_cell_type"] = adata_emb.obs["cell_type_pred"]
adata_emb.obs["transf_cell_type_unc"] = 1 - adata_emb.obs["cell_type_conf"]

Let’s transfer the results to our query adata object which also has our UMAP and gene counts, so that we can visualize all of those together.

adata.obs.loc[adata_emb.obs.index, "transf_cell_type"] = adata_emb.obs[
    "transf_cell_type"
]
adata.obs.loc[adata_emb.obs.index, "transf_cell_type_unc"] = adata_emb.obs[
    "transf_cell_type_unc"
]
adata.obs["transf_cell_type_unc"] = adata.obs["transf_cell_type_unc"].astype(
    float
)  # ensure uncertainty is float, for compatibility with downstream plotting functions

We can now visualize the transferred labels in our previously calculated UMAP of our own data:

Let’s set the figure size smaller again:

sc.set_figure_params(figsize=(5, 5))

sc.pl.umap(adata, color="transf_cell_type", frameon=False)

Based on the neighbors of each of our query cells we can not only guess the cell type these cells belong to, but also generate a measure for certainty of that label: if a cell has neighbors from several different cell types, our guess will be highly uncertain. This is relevant to assess to what extent we can “trust” the transferred labels! Let’s visualize the uncertainty scores:

sc.pl.umap(adata, color="transf_cell_type_unc", frameon=False)

Let’s check for each cell type label how high the label transfer uncertainty levels were. This gives us a first impression of which annotations are more contentious/need more manual checks.

fig, ax = plt.subplots(figsize=(8, 3))
ct_order = (
    adata.obs.groupby("transf_cell_type")
    .agg({"transf_cell_type_unc": "median"})
    .sort_values(by="transf_cell_type_unc", ascending=False)
)
sns.boxplot(
    adata.obs,
    x="transf_cell_type",
    y="transf_cell_type_unc",
    color="grey",
    ax=ax,
    order=ct_order.index,
)
ax.tick_params(rotation=90, axis="x")

You’ll notice that e.g. progenitor cells are often more difficult to distinguish than other cell types. Same for the rather unspecific category “Other T” cells in our annotations. Even for pDC, a cell type that is known to be quite transcriptionally distinct and therefore easier to recognize and label, the uncertainty of some cells is high.

To incorporate this uncertainty information in our transferred labels, we can set cells with an uncertainty score above e.g. 0.2 to “unknown”:

adata.obs["transf_cell_type_certain"] = adata.obs.transf_cell_type.tolist()
adata.obs.loc[adata.obs.transf_cell_type_unc > 0.2, "transf_cell_type_certain"] = (
    "Unknown"
)

Let’s see what our annotations look like after this filtering. Note the Unknown color in the legend and the UMAP.

sc.pl.umap(adata, color="transf_cell_type_certain", frameon=False)

... storing 'transf_cell_type_certain' as categorical

To ease legibility, we can color only the “unknown” cells. This will make it easier for us to see how many of those there are. You can do the same with any of the other cell type labels.

sc.pl.umap(adata, color="transf_cell_type_certain", groups="Unknown")

There are quite many of them! These cells will need particularly careful manual reviewing. However, the low-uncertainty annotations surrounding the “unknown cells” will already give us a first idea of what cell type we can expect each cell to belong to.

Now let’s take a look at our more certain annotations. We will check for a few cell types (chosen at random here) to what extent the reference-transferred annotation matches our known marker genes from above. In reality, this should be done systematically for all annotations!

cell_types_to_check = [
    "CD14+ Mono",
    "cDC2",
    "NK",
    "B1 B",
    "CD4+ T activated",
    "T naive",
    "MK/E prog",
]

Conveniently, for each of these cell types we have markers in our dictionary. Let’s plot marker expression for all our newly annotated cell types. You will notice that marker expression generally corresponds to the automated annotations, a good sign!

sc.pl.dotplot(
    adata,
    var_names={
        ct: marker_genes_in_data[ct] for ct in cell_types_to_check
    },  # gene names grouped by cell type in a dictionary
    groupby="transf_cell_type_certain",
    standard_scale="var",  # normalize gene scores from 0 to 1
)

As you can see, the marker groups are generally most highly expressed in the cells annotated with the matching label. This means these labels are likely (at least partially) correct!

Let’s go back one more time to our UMAP colored by uncertainty:

sc.pl.umap(
    adata, color=["transf_cell_type_unc", "transf_cell_type_certain"], frameon=False
)

The uncertainty not only helps us identify regions where the algorithm is unsure which cell type a cell belongs to (e.g., because it falls between two annotated phenotypes), but can also highlight unseen cell types or new cell states. For example, your reference might consist of healthy cells while your query could be from a diseased sample. The uncertainty score can then highlight disease-specific cell states, as they might not have neighbors from the reference that consistently come from a single cell type. Especially when your reference is based on a large dataset, the uncertainty score is useful for flagging parts of the query data that could be worth investigating. Reference-based label transfer not only helps you annotate your data but can also speed up its exploration and interpretation. However, as with any metric, these uncertainty scores are often imperfect and, in some cases, fail to highlight new cell types or states. For a more extensive discussion of uncertainty metrics, see e.g. Engelmann et al., 2022.

Like with any of the methods discussed in this notebook, the quality of the transferred annotations depends on the quality of the “training data” (in this case the reference) and its annotations, the quality of the model, and the match of your own data with the training data!

The quality of the transferred annotations should therefore always be validated with manual inspection using marker gene expression and refinement of the initial annotations might be needed.

# formatting the 'names' column as string, to prevent problems with saving to h5ad format
fields = adata.uns["dea_leiden_1_filtered"]["names"].dtype.names
for field in fields:
    adata.uns["dea_leiden_1_filtered"]["names"][field] = adata.uns[
        "dea_leiden_1_filtered"
    ]["names"][field].astype(str)


af = ln.Artifact.from_anndata(
    adata,
    key="cellular_structure/s4d8_annotated.h5ad",
    description="anndata after annotation",
).save()
af

→ creating new artifact version for key 'cellular_structure/s4d8_annotated.h5ad' in storage 's3://lamin-eu-central-1/VPwcjx3CDAa2'

... uploading 7d6LdOoS4ZSxqQPw0009.h5ad:  0.0%

... uploading 7d6LdOoS4ZSxqQPw0009.h5ad:  6.5%

... uploading 7d6LdOoS4ZSxqQPw0009.h5ad: 25.9%

... uploading 7d6LdOoS4ZSxqQPw0009.h5ad: 64.8%

... uploading 7d6LdOoS4ZSxqQPw0009.h5ad: 67.6%

... uploading 7d6LdOoS4ZSxqQPw0009.h5ad: 74.1%

... uploading 7d6LdOoS4ZSxqQPw0009.h5ad: 100.0%

• replacing the existing cache path /home/lheumos/.cache/lamindb/lamin-eu-central-1/VPwcjx3CDAa2/cellular_structure/s4d8_annotated.h5ad

Artifact(uid='7d6LdOoS4ZSxqQPw0009', key='cellular_structure/s4d8_annotated.h5ad', description='anndata after annotation', suffix='.h5ad', kind='dataset', otype='AnnData', size=809408078, hash='VV_QyngqcJDzL5YJVCLElp', n_files=None, n_observations=13668, extra_data=None, branch_id=1, created_on_id=1, space_id=1, storage_id=1, run_id=158, schema_id=None, created_by_id=2, created_at=2026-09-29 13:11:56 UTC, is_locked=False, version_tag=None, is_latest=True)

Contributors¶

We gratefully acknowledge the contributions of:

Authors¶

Lisa Sikkema
Maren Büttner
Seo H. Kim

Reviewers¶

Lukas Heumos

References¶

Kadur Lakshminarasimha Murthy, P., Sontake, V., Tata, A., Kobayashi, Y., Macadlo, L., Okuda, K., Conchola, A. S., Nakano, S., Gregory, S., Miller, L. A., Spence, J. R., Engelhardt, J. F., Boucher, R. C., Rock, J. R., Randell, S. H., & Tata, P. R. (2022). Human distal lung maps and lineage hierarchies reveal a bipotent progenitor. Nature, 604(7904), 111–119. 10.1038/s41586-022-04541-3
Wagner, A., Regev, A., & Yosef, N. (2016). Revealing the vectors of cellular identity with single-cell genomics. Nature Biotechnology, 34(11), 1145–1160. 10.1038/nbt.3711
Zeng, H. (2022). What is a cell type and how to define it? Cell, 185(15), 2739–2755. https://doi.org/10.1016/j.cell.2022.06.031
Traag, V. A., Waltman, L., & van Eck, N. J. (2019). From Louvain to Leiden: guaranteeing well-connected communities. Scientific Reports, 9(1), 5233. 10.1038/s41598-019-41695-z
Pullin, J. M., & McCarthy, D. J. (2022). A comparison of marker gene selection methods for single-cell RNA sequencing data. bioRxiv. 10.1101/2022.05.09.490241
Zhang, J. M., Kamath, G. M., & Tse, D. N. (2019). Valid Post-clustering Differential Analysis for Single-Cell RNA-Seq. Cell Systems, 9(4), 383-392.e6. https://doi.org/10.1016/j.cels.2019.07.012
Isola, I., Brasó-Maristany, F., Moreno, D. F., Mena, M.-P., Oliver-Calders, A., Paré, L., Rodríguez-Lobato, L. G., Martin-Antonio, B., Cibeira, M. T., Bladé, J., Rosiñol, L., Prat, A., Lozano, E., & Fernández De Larrea, C. (2021). Gene Expression Analysis of the Bone Marrow Microenvironment Reveals Distinct Immunotypes in Smoldering Multiple Myeloma Associated to Progression to Symptomatic Disease. Frontiers in Immunology, 12, 792609. 10.3389/fimmu.2021.792609
Pasquini, G., Rojo Arias, J. E., Schäfer, P., & Busskamp, V. (2021). Automated methods for cell type annotation on scRNA-seq data. Computational and Structural Biotechnology Journal, 19, 961–969. https://doi.org/10.1016/j.csbj.2021.01.015
Abdelaal, T., Michielsen, L., Cats, D., Hoogduin, D., Mei, H., Reinders, M. J. T., & Mahfouz, A. (2019). A comparison of automatic cell identification methods for single-cell RNA sequencing data. Genome Biology, 20(1), 194. 10.1186/s13059-019-1795-z
Huang, Y., & Zhang, P. (2021). Evaluation of machine learning approaches for cell-type identification from single-cell transcriptomics data. Briefings in Bioinformatics. 10.1093/bib/bbab035
Sikkema, L., Ramı́rez-Suástegui, C., Strobl, D. C., Gillett, T. E., Zappia, L., Madissoon, E., Markov, N. S., Zaragosi, L.-E., Ji, Y., Ansari, M., Arguel, M.-J., Apperloo, L., Banchero, M., Bécavin, C., Berg, M., Chichelnitskiy, E., i Mei-Chung, Collin, A., Gay, A. C. A., … and, F. J. T. (2023). An integrated cell atlas of the lung in health and disease. Nature Medicine. 10.1038/s41591-023-02327-2
Pliner, H. A., Shendure, J., & Trapnell, C. (2019). Supervised classification enables rapid annotation of cell atlases. Nature Methods, 16(10), 983–986. 10.1038/s41592-019-0535-3
Zhang, A. W., O’Flanagan, C., Chavez, E. A., Lim, J. L. P., Ceglia, N., McPherson, A., Wiens, M., Walters, P., Chan, T., Hewitson, B., Lai, D., Mottok, A., Sarkozy, C., Chong, L., Aoki, T., Wang, X., Weng, A. P., McAlpine, J. N., Aparicio, S., … Shah, S. P. (2019). Probabilistic cell-type assignment of single-cell RNA-seq for tumor microenvironment profiling. Nature Methods, 16(10), 1007–1015. 10.1038/s41592-019-0529-1
Conde, C. D., Xu, C., Jarvis, L. B., Rainbow, D. B., Wells, S. B., Gomes, T., Howlett, S. K., Suchanek, O., Polanski, K., King, H. W., Mamanova, L., Huang, N., Szabo, P. A., Richardson, L., Bolt, L., Fasouli, E. S., Mahbubani, K. T., Prete, M., Tuck, L., … S. A. Teichmann. (2022). Cross-tissue immune cell analysis reveals tissue-specific features in humans. Science, 376(6594), eabl5197. 10.1126/science.abl5197
Fu, R., Gillen, A. E., Sheridan, R. M., Tian, C., Daya, M., Hao, Y., Hesselberth, J. R., & Riemondy, K. A. (2020). clustifyr: an R package for automated single-cell RNA sequencing cluster classification. F1000Research, 9, 223. 10.12688/f1000research.22969.2