Python tips and tricks - 9: Performing additional tasks during data augmentation in keras

For example scaling inputs, performing preprocessing operations, converting masks ​to categorical​, etc.
Code snippet from the video...
#Libraries
import segmentation_models as sm
from sklearn.preprocessing import MinMaxScaler
scaler = MinMaxScaler()
from keras.utils import to_categorical
#Some scaling operation to be applied to images
from sklearn.preprocessing import MinMaxScaler
scaler = MinMaxScaler()
from keras.utils import to_categorical
#Some preprocessing operation on images
BACKBONE = 'resnet34'
preprocess_input = sm.get_preprocessing(BACKBONE)
#Define a function to perform additional preprocessing after datagen.
#For example, scale images, convert masks to categorical, etc.
def preprocess_data(img, mask, num_class):
#Scale images
img = scaler.fit_transform(img.reshape(-1, img.shape[-1])).reshape(img.shape)
img = preprocess_input(img) #Preprocess based on the pretrained backbone...
#Convert mask to one-hot
mask = to_categorical(mask, num_class)
return (img,mask)
#Define the generator.
#We are not doing any rotation or zoom to make sure mask values are not interpolated.
#It is important to keep pixel values in mask as 0, 1, 2, 3, .....
from tensorflow.keras.preprocessing.image import ImageDataGenerator
def trainGenerator(train_img_path, train_mask_path, num_class):
img_data_gen_args = dict(horizontal_flip=True,
vertical_flip=True,
fill_mode='reflect')
image_datagen = ImageDataGenerator(**img_data_gen_args)
mask_datagen = ImageDataGenerator(**img_data_gen_args)
image_generator = image_datagen.flow_from_directory(
train_img_path,
class_mode = None,
batch_size = batch_size,
seed = seed)
mask_generator = mask_datagen.flow_from_directory(
train_mask_path,
class_mode = None,
color_mode = 'grayscale',
batch_size = batch_size,
seed = seed)
train_generator = zip(image_generator, mask_generator)
for (img, mask) in train_generator:
img, mask = preprocess_data(img, mask, num_class)
yield (img, mask)
train_img_path = "data/data_for_keras_aug/train_images/"
train_mask_path = "data/data_for_keras_aug/train_masks/"
train_img_gen = trainGenerator(train_img_path, train_mask_path, num_class=4)
#Make sure the generator is working and that images and masks are indeed lined up.
#Verify generator.... In python 3 next() is renamed as __next__()
x, y = train_img_gen.__next__()
for i in range(0,3):
image = x[i]
mask = np.argmax(y[i], axis=2)
plt.subplot(1,2,1)
plt.imshow(image)
plt.subplot(1,2,2)
plt.imshow(mask, cmap='gray')
plt.show()

[@julenbhy1185]

Thanks for this great tutorial, i'm new at machine learning and i've been struggling with this problem for weeks. I've been searching for this kind of tutorial for a long time and i haven't found anithing really helpfull until i've discovered your channel.

[@DigitalSreeni]

Great to hear!

[@medbenzou]

Hello sir. I hope you are fine .
I have an error while running the line "x_val, y_val = val_img_gen.__next__()"
ValueError: Found array with 0 sample(s) (shape=(0, 3)) while a minimum of 1 is required by MinMaxScaler.
Any reason why . Thanks

[@rimumia4773]

I am also facing the same problem. If you found the solution please help me.
Thank you

[@banbiyancheng]

Thanks for your sharing! That's exactly what I am searching for🥰

[@DigitalSreeni]

You’re welcome 😊

[@dimitheodoro]

in ImageDataGenerator there is an argument (preprocessing_function).
perhaps you could there define the custom preproccess_data function !

[@liviuene7177]

Hi, regarding the altered mask values, did you try with interpolation= 'nearest' (keras.io/api/preprocessing/image/)?

[@user-wd7tk6tw1r]

Sir I was following your video for data augmentation to perform semantic segmentation (UNet) on my dataset , but it showing an error as: could not convert string to float: '/content/drive/MyDrive/USGDataset/train/images/XZ_vol101_f_1_1. and unknown function yeild()
I am sending you my whole code, please help me :
def load_data(path):
train_x=sorted(glob(os.path.join(path,'train','images','*')))
train_y=sorted(glob(os.path.join(path,'train','mask','*')))
valid_x=sorted(glob(os.path.join(path,'valid','images','*')))
valid_y=sorted(glob(os.path.join(path,'valid','mask','*')))
return (train_x,train_y),(valid_x,valid_y)
def read_images(path):
path=path.decode()
x=cv2.imread(path,cv2.IMREAD_COLOR)
x=x/255.0

return x
def read_mask(path):
path=path.decode()
x=cv2.imread(path,cv2.IMREAD_GRAYSCALE)
x=x/255.0
x=np.expand_dims(x,axis=-1)
return x
def tf_parse(x,y):
def _parse(x,y):
x=read_images(x)
y=read_mask(y)
return x,y
x,y=tf.numpy_function(_parse,[x,y],[tf.float64,tf.float64])
x.set_shape([height,width,3])
y.set_shape([height,width, 1])
return x,y
def tf_dataset(x,y,batch=4):
dataset=tf.data.Dataset.from_tensor_slices((x,y))
dataset=dataset.map(tf_parse,num_parallel_calls=tf.data.AUTOTUNE)
dataset=dataset.batch(batch)
dataset=dataset.prefetch(tf.data.AUTOTUNE)
return dataset
(train_x,train_y),(valid_x,valid_y)=load_data(dataset_path)
from inspect import Arguments
seed=24
img_data_gen_args=dict(
rotation_range=90,
width_shift_range=0.3,
height_shift_range=0.3,
shear_range=0.5,
zoom_range=0.3,
horizontal_flip=True,
vertical_flip=True,
fill_mode='reflect'
)
mask_data_gen_args=dict(
rotation_range=90,
width_shift_range=0.3,
height_shift_range=0.3,
shear_range=0.5,
zoom_range=0.3,
horizontal_flip=True,
vertical_flip=True,
fill_mode='reflect',
preprocessing_function=lambda x:np.where(x>0,1,0).astype(x.dtype)
)
image_data_generator=ImageDataGenerator(**img_data_gen_args)
image_data_generator.fit(train_x,augment=True,seed=seed)
image_generator=image_data_generator.flow(train_image_path,seed=seed)
valid_image_generator=image_data_generator.flow(valid_image_path,seed=seed)
mask_data_generator=ImageDataGenerator(**mask_data_gen_args)
mask_data_generator.fit(train_y,augment=True,seed=seed)
mask_generator=mask_data_generator.flow(train_maske_path,seed=seed)
valid_mask_generator=mask_data_generator.flow(valid_mask_path,seed=seed)
def my_image_mask_generator(image_generator,mask_generator):
train_generator=zip(image_generator,mask_generator)
for (img,mask) in train_generator:
yeild(img,mask)
my_generator=my_image_mask_generator(image_generator,mask_generator)
validation_datagen=my_image_mask_generator(valid_image_generator,valid_mask_generator)

[@surflaweb]

Hi bro! Interesting.

[@CharlanDellon]

It's possible to apply the image generator in other data types unlike image for to make a any data generator ? How do It?

[@DRMohinderKalwa]

Hello sir. I hope you are fine . I have a question about train_test_split . If I have 100 of classes(1-100) in my data set that I want to split . In datat set i have 10 samples for each class and the database is sorted according to the classes that needs to be classified. Is it possible that while dividing train_test_split can leave one or more classes totally in the training data set . If it is possible then what is the solution.

[@07jyothir]

When using augmentation and flow from directory, the images and masks are not displayed properly. The masks are shown empty even though all the images are loaded properly. Will images with different shapes affects while loading images in batches?

[@aliqsaeed2878]

same issue happened with me, flow from directory fetches images but not the masks even though the masks exist in the folder. later on, i noticed that images or masks with GIF extension can't be fetched by the ImageDataGenerator, thus i believe you have to change the format of your masks to png or any other type

[@pedroribeirossb]

Thanks for the great tutorials! One question, It seems that when plotting train_images and mask_images, they are not being shown as the same. It seems it's picking different img numbers for train and mask. Any reason why? On yours seems to be presented the same.

[@DigitalSreeni]

Images and masks may not line up on non-windows systems, for example on colab. The best way to tackle it would be to sort the image and mask name list. If you have your image list as ‘img_list’ then add a line as following to sort..
img_list = img_list.sort()
Do the same for masks.

[@pedroribeirossb]

@@DigitalSreeni Thank you for your reply - I'm actually using Colab as you mention. It's still not that clear for me where the list should be placed (probably after link in file path?), but I'm a beginner in python. I'm also having some issues importing your Unet_simple_model file in Colab. Can you do a video or a short explanation about it? Thank you so much. You have been an excellent source of learning!.

[@07jyothir]

@@DigitalSreeni Sir, when we load the data using flow from directory after augmentation, will this happen? because the losses are decreased immediately and become negative vallues after i've done this. since we are loading the images directly as batched, how do we sort ?