[MUSIC] In the last video, we described feature detection or the process of identifying feature points in an image in a variety ways. However, remember that our end goal is to use match features between two different images for localization, object detection, and other perception tasks that require depth estimation to points in the environment. To do so, we need to describe features in a way that it allows for feature comparison to determine the best match between frames. We therefore assign a descriptor to every feature point in an image. In this video, you will learn what makes a good feature descriptor for computer vision applications. You'll learn how to derive these descriptors from images so we can use them in autonomous driving. Let's begin by defining what a feature descriptor is. Mathematically, we define a feature point by its coordinates u and v in the image frame. We describe a descriptor f as an n dimensional vector associated with each feature. The descriptor has the task of providing a summary of the image information in the vicinity of the feature itself, and can take on many forms. Similar to the design of feature detectors we also have some favorable characteristics required for the design of descriptors to allow for robust feature matching. As with feature detectors descriptors should be repeatable, that means that regardless of shifts in position, scale, and illumination, the same point of interest in two images should have approximately the same descriptor. This invariance in transformations is one of the most researched topics when it comes to descriptor design. And a large amount of work has been done to provide descriptors that are invariant to scale, illumination, and other variables in image formation. The second important characteristic of a feature descriptor is distinctiveness. Two nearby features should not have similar descriptors, as this will confuse our feature matching process later on. Finally, descriptors should be compact and efficient to compute. This is because we will usually require matching to be performed in real time for autonomous driving applications. A wide variety of effective descriptors have been developed for feature matching. So let's take a look at a specific case study on the design of a single feature descriptors to give you sense for descriptors work. We will describe how to compute the shift features descriptors specifically designed by David Lowe in 1999. The procedure for computing shift feature descriptors is as follows. Given a feature in the image, the shift descriptor takes a 16 by 16 window of pixels around it, we call this window the features local neighborhood. We then separate this window in to four, 4 by 4 cells such that each cell contains 16 pixels. Next we compute the edges and edge orientation of each pixel in each cell using the gradient filters we discussed in module one. For stability of the descriptor, we suppress weak edges using a predefined threshold as they are likely to vary significantly in orientation with small amounts of noise between images. Finally, we compute a 32 dimensional histogram of orientations for each cell. And concatenate the histograms for all four cells to get a final 128 dimensional histogram for the feature at hand, we call this histogram or descriptor. Some additional post processing is done as well in that it helps the 128 dimensional vector retain stable values under variable contrast, game, and other fundametric variations. SIFT is an example of a very well human engineered feature descriptor, and is used in many state-of-the-art systems. It is usually computed over multiple scales and orientations for better scale and rotation invariants. Finally, when combined with a scale invariant feature detector, such as the difference of Gaussian's detector, it results in a highly robust feature detector and descriptor pair. It is worth mentioning that there is huge literature out there for feature detectors and descriptors. The surf descriptive for example uses similar concepts to SIFT while being significantly faster to compute. Many other variants exist in the literature including the Gradient Location-Orientation Histogram or GLOH descriptor. The Binary Robust Independent Elementary Features descriptor or BRIEF, and the Oriented Fast and Rotated Brief descriptor or ORB. This is a lot of acronyms to remember, but don't worry, we don't expect you to remember all of these. But you may see them in the implementations available for use in Open Source Computer Vision Libraries. We've now completed our discussion on feature detectors and descriptors. Although most of the discussed algorithms have open source implementations, some like SIFT and SURF are patented and should not be used commercially without approval of the authors. Fortunately, the feature detector and descriptor literature up there is vast and some really good algorithms such as ORB match the performance of SIFT and SURF and are free to use even commercially. In this lesson, you learned what comprises a feature descriptor, what characteristics are favorable when designing these descriptors. And different algorithms that are available in the open source libraries to extract feature descriptors as you need them. In combination with the feature extractors we talked about in the previous video, you're now ready to take on the challenging tasks of matching features between images using their computed descriptors. You'll learn more about this in the next video. See you then. [MUSIC]
