Page tree
Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 2 Current »

1. Dataset

  • Spreadsheet: sidewalks.csv
OBJECTIDID_SIDEWALSTREETNAME...OBSTRUCTIOOBSTRUCT_1DEFICIENCYDEFICIEN_1...BEGIN_IMAGEND_IMAGEFILENAME
45422836-2N HAMPTON RD_R...None1None4...https://ivision.fugro.com/CityofDallas_Video/ROW/16BD0VD7B00/000000000123/000000144172.jpghttps://ivision.fugro.com/CityofDallas_Video/ROW/16BD0VD7B00/000000000123/000000172144.jpg6BD0VD7B
  • Images dir: DAll16/DAL16_85092/v12

2. Convert original dataset to training dataset

  • Run
    • $ python categorize_with_feature.py
  • Results


3. Create a basic CNN model with Keras

cnn.py
# Part 1 - Building the CNN

# Importing the Keras libraries and packages
from keras.models import Sequential
from keras.layers import Convolution2D
from keras.layers import MaxPooling2D
from keras.layers import Flatten
from keras.layers import Dense

# Initialising the CNN
classifier = Sequential()

# Step 1 - Convolution
classifier.add(Convolution2D(32, 3, 3, input_shape = (64, 64, 3), activation = 'relu'))

# Step 2 - Pooling
classifier.add(MaxPooling2D(pool_size = (2, 2)))

# Adding a second convolutional layer
classifier.add(Convolution2D(32, 3, 3, activation = 'relu'))
classifier.add(MaxPooling2D(pool_size = (2, 2)))

# Step 3 - Flattening
classifier.add(Flatten())

# Step 4 - Full connection
classifier.add(Dense(output_dim = 128, activation = 'relu'))
classifier.add(Dense(output_dim = 1, activation = 'sigmoid'))

# Compiling the CNN
classifier.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy'])


# Part 2 - Fitting the CNN to the images

from keras.preprocessing.image import ImageDataGenerator

train_datagen = ImageDataGenerator(rescale = 1./255,
                                   shear_range = 0.2,
                                   zoom_range = 0.2,
                                   horizontal_flip = True)

test_datagen = ImageDataGenerator(rescale = 1./255)

training_set = train_datagen.flow_from_directory('train',
                                                 target_size = (64, 64),
                                                 batch_size = 10,
                                                 class_mode = 'binary')

test_set = test_datagen.flow_from_directory('test',
                                            target_size = (64, 64),
                                            batch_size = 10,
                                            class_mode = 'binary')

classifier.fit_generator(training_set,
                         samples_per_epoch = 1500,
                         nb_epoch = 250,
                         validation_data = test_set,
                         nb_val_samples = 300)


4. Training in HAL cluster

  • If your job may run more than 4 hours, you should change next two things.
    • --partition=debug → --partition=solo
    • --time=04:00:00 → --time=72:00:00
cnn.sb
#!/bin/bash

#SBATCH --job-name="cnn-train-debug"
#SBATCH --output="cnn-train-debug%j.%N.out"
#SBATCH --error="cnn-train-debug%j.%N.err"
#SBATCH --partition=debug
#SBATCH --nodes=1
#SBATCH --mem-per-cpu=1500
#SBATCH --ntasks-per-node=12
#SBATCH --cores-per-socket=12
#SBATCH --gres=gpu:v100:1
#SBATCH --export=ALL
#SBATCH --time=04:00:00

module load ibm/powerai
module list

python cnn.py

5. Results

  • Training 3,000 images
  • Test 600 images
cnn-train-debug6075.hal02.out
Found 3000 images belonging to 3 classes.
Found 600 images belonging to 3 classes.
Epoch 1/250

1/150 [..............................] - ETA: 8:45 - loss: 0.7129 - acc: 0.3000
10/150 [=>............................] - ETA: 50s - loss: 0.3547 - acc: 0.2600
11/150 [=>............................] - ETA: 47s - loss: 0.3451 - acc: 0.2636
12/150 [=>............................] - ETA: 47s - loss: 0.2712 - acc: 0.2667
13/150 [=>............................] - ETA: 48s - loss: 0.1345 - acc: 0.2538
14/150 [=>............................] - ETA: 48s - loss: 0.0803 - acc: 0.2714
15/150 [==>...........................] - ETA: 49s - loss: 0.1817 - acc: 0.2667
16/150 [==>...........................] - ETA: 48s - loss: -0.0491 - acc: 0.2625
17/150 [==>...........................] - ETA: 50s - loss: -0.1718 - acc: 0.2824
18/150 [==>...........................] - ETA: 50s - loss: -0.5505 - acc: 0.2833
19/150 [==>...........................] - ETA: 50s - loss: -0.5997 - acc: 0.2737
20/150 [===>..........................] - ETA: 50s - loss: -0.6494 - acc: 0.2750
21/150 [===>..........................] - ETA: 49s - loss: -0.5426 - acc: 0.2762
22/150 [===>..........................] - ETA: 50s - loss: -0.5179 - acc: 0.2818
23/150 [===>..........................] - ETA: 49s - loss: -0.3568 - acc: 0.2957
...
142/150 [===========================>..] - ETA: 3s - loss: 0.3929 - acc: 0.3289
143/150 [===========================>..] - ETA: 2s - loss: 0.3902 - acc: 0.3308
144/150 [===========================>..] - ETA: 2s - loss: 0.3875 - acc: 0.3313
145/150 [============================>.] - ETA: 2s - loss: 0.3628 - acc: 0.3303
146/150 [============================>.] - ETA: 1s - loss: 0.3385 - acc: 0.3308
147/150 [============================>.] - ETA: 1s - loss: 0.3579 - acc: 0.3299
148/150 [============================>.] - ETA: 0s - loss: 0.3770 - acc: 0.3291
149/150 [============================>.] - ETA: 0s - loss: 0.3745 - acc: 0.3309
150/150 [==============================] - 151s 1s/step - loss: 0.3188 - acc: 0.3307 - val_loss: -6.6757e-09 - val_acc: 0.3333

...


Reference

  • No labels