✳flâneur — a map of the web's best reading

Creating In-Video Search - DEV Community

dev.to · 1,354 words · saved by 1 readers

In-video search is ability to search for a specific content within a video. This can include searching for particular words spoken, objects shown or description of a scene. With the current advancement in transformers the process of in-video search have become more accurate and fairly simple. Although most of the transformers doesn’t have a joint embedding space for multiple modalities but there are few models like Meta’s ImageBind that a joint embedding space between text, image, audio, depth, thermal and IMU, or OpenAI’s CLiP model have joint embedding space between text and image. We can use these models to create a relatively quick and accurate in-video service. A typical video would have the following data with them -: Frames -: A video is basically a series of the frames. Some of these frames are complete frames called i-frames and others will be partial frames, also known as p-frames and b-frames, which contain only the changes from the previous frame. Audio -: A video also migh

In-video search is ability to search for a specific content within a video. This can include searching for particular words spoken, objects shown or description of a scene. With the current advancement in transformers the process of in-video search have become more accurate and fairly simple. Although most of the transformers doesn’t have a joint embedding space for multiple modalities but there are few models like Meta’s ImageBind that a joint embedding space between text, image, audio, depth, thermal and IMU, or OpenAI’s CLiP model have joint embedding space between text and image. We can us

Explore this link on the map →

saved by

related reading