Assignment 1

CS5187 Vision and Image

instance search

You are given a collection of 28,493 images and 50 testing query images. You can download it from this link:

https://portland-my.sharepoint.com/:f:/g/personal/srwang3-c_my_cityu_edu_hk/EjUuFjjMbnNIov1CyoKwzuQB7NzIfjYFyiluHRer_UZ-kg?e=usgpS4.

Each image contains one or two instances (objects). Your task is to implement any two object matching methods (e.g., histogram, LBP, filter bank, AlexNet, YOLO, Fast R-CNN, Faster R-CNN, SIFT, Bag-of-words) for instance search. You can use the methods learned in this course, or you could also use the other methods in the literature. It is also encouraged to develop your own method.

Given an image with one or multiple bounding boxes as a query, retrieve the ten most similar images containing the same instance(s) from the collection of images. Show the top ten matching images with the bounding box(es) of instance(s) for queries 1-5 in the report. You need to show the results for at least one of the methods. ---(60%)

It is worth mentioning that if your implementations for both methods are correct and at least one method is based on the deep neural networks (e.g., AlexNet, VGG, ResNet, YOLO, Fast R-CNN) and can locate the instances of retrieved images, you will get all 60% of marks. If only one method works, you could only get 40% of the marks instead of 60%.

The similarity between two images can be based on any distance or similarity function, including the followings:

Euclidean distance between query and image :

Cosine similarity between query and image :

List the retrieval results (a rank list of 28,493 images in descending order of similarity) for the 50 queries in a text file: rankList.txt (see submission guideline in the next page). You can submit at most two RUNs (two rankLists), each from a different method. The mark will be allocated based on the retrieval performance of your better RUN. (20%)

Report should be brief and no more than 6 pages. The report should briefly describe and analyze your methods. (20%)

Submission

Please zip the followings and submit to Canvas:

Computer program: Python/Matlab/C++ recommended. If you want to use another program language, please send an email to our Teaching Assistant.

Report

Rank list: Two text files corresponding to two runs, showing the descending order of the images. Sample text file:

Q1: 7 12 214 350 …

Q2: 301 501 1990 2 …

Q3: 288 345 389 1290 …

Q4: 248 293 1098 2000 …

Q5: 380 287 392 478 222 …

Remark: For each text file, 50 rows only (corresponding to the 50 testing query images). Each row should list the names of the 28,493 images (named in number) in descending order of their similarities to a query.