Bing + Python Image Search

Sometimes it is necessary to type pictures on a specific topic in order to be able to choose the necessary one from the existing set, etc. Current search engines provide such an opportunity, but you need to open the browser, navigate through the pages, work with the mouse and, in general, do it. I would like to have a console utility “launched and forgotten” for a set of necessary images. We consider the Bing API, getting started in Python and linking them to search for images.

Introduction

This is my first more or less large Python program, which I began to study recently (by the way, many thanks to kossmak for his translations of articles). Examples of use at the end of the article.

TK

A console program that accepts a search line as input and the required number of images. At the output - a subdirectory in the current search results.
')

Why bing

Some time ago, it was necessary to test an asynchronous loader for ActionScript. For the load, Google was chosen, however, as a result, it turned out that Google produces no more than 64 results for requests through the API. This (at that time) was enough, but the sediment remained. After the search, it was found: Yahoo (with comments that many of the data it issued is outdated) and Bing (which on its page promises up to 1000 results). Bing was chosen, because, besides the request itself, it allows you to overlay filters on it (see below)

Bing

Development for Bing begins with the Bing Developer Center page. There you need to get APP_ID for signing each request, registration is minute. I didn’t really understand the restrictions imposed (maybe there are simply no them), so I’m posting my test APP_ID along with the examples (if I’m going to use it, I recommend that you add and put your APP_ID into the code).

Bing API

The API exists for VB / C # / C ++ / F # / JS, but in this example the final http request is used. API description for image search here
So, the minimum request for image search and JSON response is as follows:
api.search.live.net/json.aspx?appid=APP_ID&sources=image&query=SEARCH_QUERY
Request example (search for apple):
http://api.search.live.net/json.aspx?appid=4EFC2F2CA1F9547B3C048B40C33A6A4FEF1FAF3B&sources=image&query=apple

Python

Everything is simple and cross-platform. The python itself (version 2.6.x) is set from here . As a development environment, I really liked PyDev. We put Eclipse (if not yet) and from under it we put PyDev

Algorithm

I will not comment block by block, there are a lot of comments in the code, besides, it is not so big as to not put it in one block. Short:

• In the main loop, a request is sent to the Bing API and the image.offset parameter is increased until either the required number of images is typed, or the Bing API does not show that the results are over.
• Each request asks for 8 pictures (stopped at this size, 4 is too small, for 16 it is sometimes long to wait for an answer, maximum 50).
• For each picture found, the URL is extracted, and a thread is created that downloads the picture to memory and saves it to disk. Here I ran into a problem - the pictures are very often called the same. So the save function “blocks” the remaining threads, and adds to the file name "_" in front, until it turns out that there is no such file yet. Next, save and unlock.

Code

# import used libraries
import urllib , json , sys , os , threading

def load_url (url, filename, filesystem_lock):
try :
# open connection to URL
socket = urllib . urlopen(url)
# read data
data = socket . read()
# close connection
socket . close()
# on all exceptions
except :
print "error loading" , url
# if no exceptions
else :
# save loaded data
save_to_file(data, filename, filesystem_lock)

def save_to_file (data, filename, filesystem_lock):
# wait for file system and block it
filesystem_lock . acquire()
try :
# while already have file with this name
while os . path . isfile(filename):
# append '_' to the beginning of file name
filename = os . path . dirname(filename) + "/_" + os . path . basename(filename)
# open for binary writing
with open (filename, 'wb' ) as f:
# and save data
f . write(data)
f . close()
print filename
except :
print "error saving" , filename
# release file system
filesystem_lock . release()

def main ():
# Bing search URL
SERVICE_URL = "http://api.search.live.net/json.aspx"
# request parameters dictionary (will append to SERVICE_URL)
params = {}
params[ "appid" ] = "4EFC2F2CA1F9547B3C048B40C33A6A4FEF1FAF3B"
params[ "sources" ] = "image"
params[ "image.count" ] = 8
params[ "image.offset" ] = 00

# try to read command line parameters
try :
params[ "query" ] = sys . argv[ 1 ]
images_count = int (sys . argv[ 2 ])
if len (sys . argv) > 3 :
params[ "image.filters" ] = sys . argv[ 3 ]
# if have less than 2 parameters (IndexError) or
# if second parameter cannot be cast to int (ValueError)
except ( IndexError , ValueError ):
# print usage string
print "Bing image search tool"
print "Usage: bing.py search_str images_count [filters]"
# end exit
return 1

# make directory at current path
dir_name = "./" + params[ "query" ] + "/"
if not os . path . isdir(dir_name):
os . mkdir(dir_name)

# list to store loading threads
loaders = []
# file system lock object
filesystem_lock = threading . Lock()

try :

# loop for images count
while (params[ "image.offset" ] < images_count):

# combine URL string, open it and parse with JSON
response = json . load(urllib . urlopen(SERVICE_URL + "? %s " % urllib . urlencode(params)))
# extract image section
images_section = response[ "SearchResponse" ][ "Image" ]

# if current search offset greater or equal to returned total files
if "Total" not in images_section or params[ "image.offset" ] >= images_section[ "Total" ]:
# then break search loop
break

# extract image results section
results = images_section[ "Results" ]
# loop for results
for result in results:
# extract image URL
image_url = result[ "MediaUrl" ]
# create new loading thread
loader = threading . Thread(\
target = load_url,\
args = (\
image_url,\
dir_name + os . path . basename(str(image_url)),\
filesystem_lock))
# start loading thread
loader . start()
# and add it to loaders list
loaders . append(loader)
# advance search offset
params[ "image.offset" ] += 1
# break if no more images needed
if params[ "image.offset" ] >= images_count:
break ;

# on all exceptions
except :
print "error occured"
return 1

# wait for all loading threads to complete
for loader in loaders:
loader . join()

# all done
print "done"
return 0 ;

if __name__ == '__main__' :
status = main()
sys . exit(status)


Query examples

To refine the query, you can use Bing API filters , separated by a space.

• bing.py apple 1000 - find 1000 pictures for apple.
• bing.py "obama" 16 "size:large style:graphics face:face" - find 16 portraits of Obama, large in illustration style.
• bing.py "warhammer wallpaper" 16 "size:width:1280 size:height:1024" - find 16 wallpapers on the topic "warhammer", sizes 1280x1024

Creating single-exe under win32

To do this, you need py2exe, you can install it from here . Next, in the folder with the program, the setup.py file is created with the following contents (the program is in the bing.py file):

from distutils.core import setup
import py2exe , sys , os

sys . argv . append( 'py2exe' )

setup(
console = [ 'bing.py' ],
options = { 'py2exe' : { 'bundle_files' : 1 }},
zipfile = None ,
)


It is launched by the command “python setup.py”. As a result of execution, the “compiled” program appears in the ./dist folder (the w9xpopen.exe file can be erased)
Then you can shake it with UPX (from 5182Kb it’s down to 4061Kb)

What I would like to improve

• Requests in Russian
• General progress indicator for all files
• Download progress for each file
• Using time-out when trying to download an image (it seems to be minute by default)
• Normal error handling

PS

Strange habra-glitch.
<code><font color="#666666">0</font></code>
Does not display anything.
Also view links
<code>http://api.google.com</code>
Output without http: //

Pps

Compiled exe under Win32 here .