Dynamic Image Cropping In Tensorflow

I'm trying to figure out how to take a crop of an image determined dynamically in Tensorflow. Below is an example of what I am trying to accomplish, however I can't seem to make i

Solution 1:

I saw this code from here.

elems = (np.array([1, 2, 3]), np.array([-1, 1, -1]))
alternate = map_fn(lambda x: x[0] * x[1], elems, dtype=tf.int64)
# alternate == [-1, 2, -3]

It is possible to use a tuple or a list to pack several elements into one so I tried this.

import tensorflow as tf
from matplotlib import pyplot as plt
import numpy as np

sess = tf.InteractiveSession()

img1 = np.random.random([400, 600, 3])
img2 = np.random.random([400, 600, 3])
img3 = np.random.random([400, 600, 3])
images = np.array([img1, img2, img3])
# images = tf.convert_to_tensor(images)  # it can be uncommented.

img1_crop = [100, 100, 100, 100]
img2_crop = [200, 150, 100, 100]
img3_crop = [150, 200, 100, 100]
crop_values = np.array([img1_crop, img2_crop, img3_crop])
# crop_values = tf.convert_to_tensor(crop_values)  # it can be uncommented.

def crop_image(img, crop):
    return tf.image.crop_to_bounding_box(img,
                                         crop[0],
                                         crop[1],
                                         crop[2],
                                         crop[3])

fn = lambda x: crop_image(x[0], x[1])
elems = (images, crop_values)

cropped_image = tf.map_fn(fn, elems=elems, dtype=tf.float64)
result = sess.run(cropped_image)

print result.shape

plt.imshow(result[0])
plt.show()

It works on my machine with tf version 0.11 and python2. Hope this can help you.

Solution 2:

Couple of things :

• You do not have a return statement in the crop_image function.
• map_fn accepts a single argument.
• I strongly advise you to separate the graph definition and the session usage.

--

# Graph def
def crop_image(img, crop):
    return tf.image.crop_to_bounding_box(img,
                                  crop[0],
                                  crop[1],
                                  crop[2],
                                  crop[3])

image_placeholder = tf.placeholder(tf.float32, [None, 400, 600, 3])
crop_placeholder = tf.placeholder(dtype=tf.int32)
cropped_image = tf.map_fn(lambda inputs: crop_image(*inputs), elems=[image_placeholder, crop_placeholder], dtype=tf.float32)


# Session
sess = tf.InteractiveSession()

img1 = np.random.random([400, 600, 3])
img2 = np.random.random([400, 600, 3])
img3 = np.random.random([400, 600, 3])

images = [img1, img2, img3]

img1_crop = [100, 100, 100, 100]
img2_crop = [200, 150, 100, 100]
img3_crop = [150, 200, 100, 100]

crop_values = [img1_crop, img2_crop, img3_crop]

sess.run(tf.global_variables_initializer())

result = sess.run(cropped_image, feed_dict={image_placeholder: images, crop_placeholder:crop_values})

plt.imshow(result[0])
plt.show()

Solution 3:

tf.map_fn(f, l) runs function f for every tensor in list l. In your case, your function expects 2 arguments, but since you supply a flat list, map_fn() sends them one by one. According to docs, map_fn() supports variable arity, so what you should do is something like this

tf.map_fn(lambda img, crop: crop_image(img, crop), 
          elems=([image_placeholder, crop_placeholder], ))

so the list you pass to map_fn contains pairs of arguments.