Celery Causing Out of Memory with OpenCv

Recently I posted about using openCV to grab webcam images on a pi.  The problem turned out not to be the limited memory, but how I was using celery and opencv.  Basically, there was a memory leak.  On my laptop this occurred as well, but at a much slower pace.  On the pi, the poor computer very rapidly ran out of memory.

The setup was that there was a celery task (posted below), which was set in celleryconfig to run every 10 seconds or so.

@app.task
def getwebcamimage():
    logger.debug(“capturing image attempt…”)
    c = cv2.VideoCapture(0)
    for i in range(10):
        flag, frame=c.read()
    flag, frame = c.read()
    cv2.imwrite(PATH_TO_IMG,frame)
    logger.debug(‘saved image…hopefully’)
    c.release()
    return 0

I would start celery beat with

$ celery -B -A celerytest worker –loglevel=DEBUG”

The problem has something to do with how celery/python treats objects that are created inside of tasks.  Each time, python was creating a cv2.VideoCapture object, but cv2.VideoCaptue.release() doesen’t seem to release the object- instead it releases the lock on the camera and keeps the object because it might be needed later.  I used the Top command to look at how processes were using the memory, and the memory for celery would grow out of control.  Bad.

The solution I came up with was just to run two threads- the server in one, and an image updater in the other.  This is nice because the cv2.VideoCapture object is only made once, and it can run independently of celery.  Just to make sure I had actually found the error, I tried running the same code from the celery task in a thread alongside the server, and I got the same kind of memory leak.  So it was not celery related, but jut python not giving up memory, which caused the celery task to OOM.

The new code snippets look like this:

def get_image():
    c = cv2.VideoCapture(0)  #make videocapture object
    while 1:                              #capture frame ever 5s
        sleep(5)
        print(“getting image!”)
        for i in range(10):        #let camera adjust*
            flag, frame=c.read()
         flag, frame = c.read()
        cv2.imwrite(PATH,frame)

    try:
        thread.start_new_thread(app.run,())#start server
        thread.start_new_thread(get_image,())
    except:
        print(“unable to start thread”)
    while 1:
        pass

*Note: my webcam seems to take some time to wake up between accesses, so I read a few frames before capturing the final image.

Posted in: ENG

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s