AutoOverlay AviSynth plugin

Requirements

Installation

Description

The plugin is designed for optimal overlay of one video clip on another.
The comparison is performed by testing different coordinates of the overlay, resolutions, aspect ratios and rotations of the clip in order to find the best combination of these parameters. The function of comparing the areas of two frames is the standard deviation (RMSE). The result of the function is denoted as diff (because other functions were tested too).
The mapping is broken down into several steps by scaling and defining the boundaries of the blending parameters to shorten the processing time. If the required accuracy is achieved at a certain step, the remaining steps are not performed.
After matching the frames, the plug-in allows you to adapt and overlay them one by one in different ways, having received the resulting video collected from several sources.
The plugin contains several filters that interact with each other to break the work into several stages.

Plugin connection

LoadPlugin("AvsFilterNet.dll") LoadNetPlugin("AutoOverlay.dll") 

In AviSynth+ plugins are loaded automatically.

Filters

OverlayConfig

OverlayConfig(float minOverlayArea, float minSourceArea, float aspectRatio1, float aspectRatio2,  \
float angle1, float angle2, int minSampleArea, int requiredSampleArea, float maxSampleDifference, \
int subpixel, float scaleBase, int branches, float acceptableDiff, int correction, int minX,      \
int maxX, int minY, int maxY, int minArea, int maxArea, bool fixedAspectRatio) 

The filter describes the configuration of searching for optimal blending parameters: sets the limits of the values ​​and settings of the search algorithm. The output filter gives a service clip consisting of only one frame, in which the transferred parameters are encoded, which can be read by another filter.

Options

OverlayEngine

OverlayEngine(clip source, clip overlay, string statFile, int backwardFrames, int forwardFrames, \
clip sourceMask, clip overlayMask, float maxDiff, float maxDiffIncrease, float maxDeviation, \
bool stabilize, clip configs, string downsize, string upsize, string rotate, bool editor)

The filter takes two clips at the input and finds the optimal blending parameters, that is, minimizes the RMS between the clips in the intersection area. This information is recorded in the output frame and can be used by other filters. Statistics can be written to a file for reuse, saving time and editing manually using the built-in editor.

Options

Pritsnip work

The engine finds the optimal intersection of two frames. The registration parameters are described by the following properties: the coordinates of the upper corner of the overlay image (x, y), the angle of rotation, the width and height of the image (width and height), the cropping value for subpixel precision (on either side of the image be a maximum of a pixel in the range of real numbers), DSC (diff).

The search parameters are specified using the configuration chain OverlayConfig in the configs parameter. The engine takes turns turning all the configurations until they end or an acceptable MSE value specified in OverlayConfig.AcceptableDiff is received . Testing each configuration also consists of several scaling steps. In the first step, all possible overlapping combinations are tested at the lowest possible resolution. The next steps are testing the refined parameters.
Resize images using the functions passed in the parameters downsize and upsize . The first is mainly used in the initial stages, the second one on the last stages, so it can be explicitly redefined to a more complex one, for example, BicubicResize . Signature functions must have the following signature: Resize(clip clip, int target_width, int target_height, float src_left, float src_top, float src_width, float src_height) . Additional parameters are allowed, that is, the signature corresponds to standard resize functions. It is recommended that you use the ResampleMT plug-in, which provides the same functions that work internally in multiple threads.

In addition to searching for the optimal superposition of a single frame, the engine analyzes the sequence of neighboring frames. For this, the parameters backwardFrames, forwardFrames, maxDiff, maxDiffIncrease, maxDeviation, stabilize meet .
When the engine requests the parameters of superimposing a certain frame, the following order of actions occurs:

Visual editor

Runs if OverlayEngine.editor = true.
Left preview of the frame. Below the trackbar for the number of frames and the field for entering the current frame. On the right is a table that displays frames with the same blending parameters combined into episodes. Between episodes you can switch. Under the control panel.
Overlay settings - Overlay settings for the current episode. Crop in a thousandth pixel in the range from 0 to 1000.
Button AutoOverlay frame - repeated run AutoOverlay for the current frame, the characteristics of the overlay apply to the whole episode.
Button AutoOverlay scene - repeated run AutoOverlay for all frames that make up the episode, as a result of which it can be divided into several.
Changed and unsaved episodes are highlighted in yellow in the grid. Button save - save changes. Reset - reset changes and reload data. Reload - Reload the characteristics for the current frame, extending to the entire episode.
Separate - separating the frame. Join prev - attach the frames of the previous episode. Join next - attach frames of the next episode. Join to - Attach frames to the entered ones inclusive.

Hotkeys :

OverlayRender

OverlayRender(clip engine, clip source, clip overlay, clip sourceMask, clip overlayMask,      \
bool lumaOnly, int width, int height, int gradient, int noise, bool dynamicNoise, bool mode, \
float opacity, int colorAdjust, string matrix, string upsize, string downsize, string rotate, \
bool debug)

The filter renders the result of combining two clips with certain settings.

Options

OverlayCompare

OverlayCompare(clip, clip, clip, string sourceText, string overlayText, int sourceColor, \
int overlayColor, int borderSize, float opacity, int width, int height, bool info) 

ColorRangeMask

ColorRangeMask(clip, int low, int high) 

The auxiliary filter creates a mask, including pixels in the range from low to high on the input clip.

tbd

Example of use

OM=AviSource("c:\test\OpenMatte.avs")                                     # YV12 clip 
WS=AviSource("c:\test\Widescreen.avs")                                    # YV12 clip 
config=OverlayConfig(subpixel=2) + OverlayConfig(angle1=-0.5, angle2=0.5) # rotate only if needed
OverlayEngine(OM.ConvertToY8(), WS.ConvertToY8(), configs=config) # ConvertToY8 for better perfomance
return last # uncomment for analysis pass OverlayRender(OM.ConvertToYV24(), WS.ConvertToYV24(), debug=true, noise=50, upsize="Spline64Resize") \ ConvertToYV12() # from YV24 # Prefetch(4) # uncomment after analysis pass