Image Recognition

What is image recognition?

A holy grail of camera trap analysis is automated image recognition, where a computer automatically inspects your images and identifies what is occurring within them.

Recognizers can be broadly categorized as one of two types.

  1. A detector detects whether something is in an image or not, assigns a coarse identifying label to it (e.g., empty, animal, person, vehicle) as well as a confidence value that very roughly indicates the likelihood that the detection is correct. The detector is incredibly useful for highlighting entities in an image that you may otherwise miss, for finding and discarding ’empty’ images (e.g., triggered due to wind effects), and removing images with people (for privacy).
  2. A classifier performs a fine-grained classification of what the detector detected. A classifier will typically produce a list of possible classifications for each detection, where each possibility is assigned a confidence value. For example, for wildlife images classifications will typically be different species of wildlife (e.g., elk, wolf, bear, dog). The classifier is useful for inspecting and bulk-tagging images species by species.

Is image recognition useful for you?

Some knowledge can help you decide on whether and when to use image recognition. The time taken to peruse the materials below is well worth the effort.

  • Read: Automated Image Recognition for Wildlife Camera Traps: Making it Work for You. Greenberg, S. (2020), Research report, University of Calgary: Prism Digital Repository, August 21, 15 pages.
    • This brief but must-read primer is oriented towards scientists (specifically wildlife ecologists) who are thinking about incorporating image recognition. It describes problems and prospects of using it for camera trap tagging, as well as how you can incorporate it effectively in the tagging workflow.
  • Watch: Image Recognition for Camera Traps: Making it Work for You [~20 minutes]. In Conference: Scaling Up Camera Trap Surveys to Inform Regional Wildlife Conservation, Columbia Mountains Institute of Applied Ecology, May 18, 2021.
    • This video serves as a companion to the previous reading. It illustrates image recognition: what it is, its fallibilities, and how it can still be incorporated for more efficient image tagging by adjusting your workflow..
  • Read: Timelapse Image Recognition Guide and view its Companion Video. Try it out on the ImageRecognitionPracticeSet
    • This tutorial guide and accompanying video walks you through using image recognition in Timelapse. You can use Timelapse on the practice image set to try out the guide’s explanations as you read it.

Timelapse and the EcoAssist image recognizer

Timelapse does not do image recognition by itself. Rather, it uses EcoAssist (which internally uses Microsoft’s widely used MegaDetector) to do the actual image recognition processing. The good news is that you can download from the Timelapse recognition menu.

Using EcoAssist is easy, as illustrated in the accompanying video. Start EcoAssistfrom the recognition menu: a dialog asks you to indicate a folder containing the images you want to recognize. EcoAssist processes those images and creates a recognition file in the same folder, which you then import back into Timelapse. You will then see bounding boxes around recognized entities in your images. You can also select subsets of images by its classification via the Select | Custom Select menu (e.g., all images with people in it) . 

The Timelapse Image Recognition Guide provides details. It explains how to download, install and run EcoAssist, and import the recognition file it creates. It also illustrates an efficient recognition workflow that lets you quickly review (and correct) recognitions for a category, and then tag most images in that category in a single operation.

Using MegaDetector by itself

If you are technically inclined and have a strong reason to do so (e.g., to tweak parameters, or perhaps to create and run your own models, or because you are an AI person), you can download and run Megadetector by itself (i.e., without EcoAssist).

There are two versions of Megadetector available (yes, this is confusing).

  • The original MegaDetector. Microsoft’s AI for Earth originally developed MegaDetector, but they then ceased development on it. Dan Morris [email] , who had worked on MegaDetector, took over its support. He helped many ecologists get going with image recognition, and truly understands what ecologists are trying to do – he is amazing. See also his release notes.
  • The reborn MegaDetector. Microsoft later created the AI for Good group. One of their projects, called Pytorch Wildlife, returned to MegaDetector development.

I have no strong opinion as to which is ‘better’. I collaborated with Dan Morris over many years. I find him instrumental in advocating for MegaDetector, in making it work with Timelapse, and in freely offering his own time to help ecologists. I have also had a few interactions with the AI for Good staff, who seem dedicated in advancing technical aspects of MegaDetector.

Both versions work well with Timelapse

Note. The MegaDetector version numbers are also confusing: while the reborn MegaDetector is version 6 and the original version 5, its best to view them as somewhat different rather than one being newer than the other.

Getting help doing image recognition

While EcoAssist is great for letting you do image recognition by yourself, there are times when you may need help (see below). This is when you should contact Dan Morris [email], as he is willing to give you a hand at no cost. He can discuss the kinds of images you have and whether image recognition would actually help you. He is also willing to process your images through his own (fast) computers, and return an image recognition file back to you. This would normally require you to either upload your images to the cloud, or send him a drive containing the images.

Here are some example cases where you may want Dan to process your images.

  1. You have built up a huge backlog of images that exceeds your capacity to process them (this is especially true for video).  In these cases, Dan can help you get out of the backlog. After those files are processed, you will likely find it reasonable to incrementally run EcoAssist for images as they are collected. 
  2. Your machine is limited. For example, EcoAssist on your laptop is great if you have 50,000 images; if you have 5 million images, that is likely a different story unless you have a fast and powerful machine. Similarly, if your computer does not have a suitable GPU (graphics processing unit) processing large numbers of images will be slow.
  3. You have difficult images: Running “vanilla MegaDetector” is not enough, and some exploration is required to make it work better. For example, you may have unusual images and entities that don’t quite match how MegaDetector recognizes things. Both you and Dan will benefit, as these exchanges are the crux of how MegaDetector improves over time, as it help incorporate such ‘difficult’ images into its tooling and model.
  4. The results don’t look right. You’ve run EcoAssist, but it seems to do really poorly. Dan can help understand why and perhaps offer some solutions.