Deep eye learning experience

Experiment: RMB two classification, training a two classifier, can distinguish the fourth set of RMB 1 yuan and 100 yuan.

5 steps of model training, data, model, loss function, optimizer, iterative training

This article only discusses the data.

Data: data collection --- img, label

Data segmentation---train, valid, test

Data reading---DataLoader(Sampler (generate index),Dataset(According to the index to get img and label))

Data pre-processing---transforms

DataLoader: Building an iterable data loader

torch.utils.data.DataLoader(dataset,batch_size=1,shuffle=False,sampler=None,batch_sampler=None,num_workers=0,

collate_fn=None,pin_memory=None,drop_last=False,timeout=0,worker_init_fn=None,multiprocessing_context=None)

    dataset:Dataset class,Decide where and how to read data

    batch_size: Batch size

    shuffle: Whether each epoch of training samples is out of order

    num_workers: Number of processes reading data

    drop_last: When the number of samples cannot be divided by batch_size, whether to discard the last batch of data.

The relationship between three common names:

epoch: All training samples are input into the model once, which is called an epoch

iteration: A batch of samples is input into the model, which is called an iteration

batch_size: The number of samples in a batch of samples, called batch_size, which determines how many iterations an epoch has.

For example: an epoch is 120 samples, if batch_size=10, then there are 12 iterations

An epoch is 125 samples, if batch_size=10, drop_last=True, then there are 12 iterations

�� ��

Dataset: Dataset abstract class, all custom Datasets need to inherit it, and duplicate __getitem__(), getitem receives the index and returns the sample (img, label)

Data reading: which data to read: which batch_size samples should be read for each iteration, the Index output by the Sampler

Where to read data from: set hard disk data path, data_dir in Dataset

How to read data: getitem in Dataset

The data reading is shown in the figure:

Blog writing is not standardized, mainly for easy search and knowledge memory.

Next articleData preprocessing--transforms