Skip to content

Commit 6d71057

Browse files
committed
Bump HAVC Version to 5.6.0
1 parent f6810d2 commit 6d71057

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

71 files changed

+12899
-756
lines changed

README.md

Lines changed: 23 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ The filter (_HAVC_ in short) can be considered the _Swiss Army knife_ for colori
1414

1515
## Quick Start
1616

17-
This filter is distributed with the torch package provided with the **Hybrid Windows Addons**. To use it on Desktop (Windows) it is necessary install [Hybrid](https://www.selur.de/downloads) and the related [Addons](https://drive.google.com/drive/folders/1vC_pxwxL0o8fjmg8Okn0RA5rsodTcv9G?usp=drive_link). **Hybrid** is a Qt-based frontend for a lot video filters (including this one) which can convert most input formats to common audio & video formats and containers. Hybrid represents one of the most comprehensive solutions for implementing A.I. video filters and offers the most user-friendly approach to image colorization using the HAVC filter via [VapourSynth](https://www.vapoursynth.com/). In the folder _documentation_ is available a [User Guide](https://github.com/dan64/vs-deoldify/blob/main/documentation/HAVC%20User%20Guide.pdf) that provides detailed information on how to install Hybrid and use it to colorize videos. The Guide also provides tips on how to improve the final quality of colored movies.
17+
To use the HAVC filter is necessary a GPU supporting CUDA, a NVIDIA RTX3060 is the minimum requirement to use this filter satisfactorily. The filter is distributed with the torch package provided with the **Hybrid Windows Addons**. To use it on Desktop (Windows) it is necessary install [Hybrid](https://www.selur.de/downloads) and the related [Addons](https://drive.google.com/drive/folders/1vC_pxwxL0o8fjmg8Okn0RA5rsodTcv9G?usp=drive_link). **Hybrid** is a Qt-based frontend for a lot video filters (including this one) which can convert most input formats to common audio & video formats and containers. Hybrid represents one of the most comprehensive solutions for implementing A.I. video filters and offers the most user-friendly approach to image colorization using the HAVC filter via [VapourSynth](https://www.vapoursynth.com/). In the folder _documentation_ is available a [User Guide](https://github.com/dan64/vs-deoldify/blob/main/documentation/HAVC%20User%20Guide.pdf) that provides detailed information on how to install Hybrid and use it to colorize videos. The Guide also provides tips on how to improve the final quality of colored movies.
1818

1919

2020
## Dependencies
@@ -145,7 +145,7 @@ As explained previously the stabilization is performed by averaging the past/fut
145145

146146
**DDColor Tweaks**: This filter is available only for DDColor and has been added because has been observed that the DDcolor's _inference_ is quite poor on dark/bright scenes depending on the luma value. This filter will force the luma of input image to don't be below the threshold defined by the parameter _luma_min_. Moreover this filter allows to apply a dynamic gamma correction. The gamma adjustment will be applied when the average luma is below the parameter _gamma_luma_min_. A _gamma_ value > 2.0 improves the DDColor stability on bright scenes, while a _gamma_ < 1 improves the DDColor stability on dark scenes.
147147

148-
**B&W tune**: Starting with HAVC version 5.5.0, a new post-processing filter called B&W Tune was introduced, which can automatically correct most color allocation errors. Unfortunately, forcing color stability has the side effect of producing washed-out colors with a slight pink cast (similar to skin tone). This new post-processing filter can automatically correct this problem and restore image colors to a more natural color. With B&W Tune, HAVC has evolved beyond basic colorization, it now delivers vivid, natural colors while ensuring consistent color stability throughout films.
148+
**B&W tune**: Starting with HAVC version 5.5.0, a new post-processing filter called B&W Tune was introduced, which can automatically correct most color allocation errors. The color adjustment capability has been further improved with the version 5.6.0 where the Retinex filter and LUTs have been included to improve the overall output color quality. Unfortunately, forcing color stability has the side effect of producing washed-out colors with a slight pink cast (similar to skin tone). This new post-processing filter can automatically correct this problem and restore image colors to a more natural color. With the version 5.6.0, HAVC has evolved beyond basic colorization, it now delivers vivid, natural colors while ensuring consistent color stability throughout films.
149149

150150
### Chroma Adjustment
151151

@@ -193,7 +193,7 @@ In the [HAVC User Guide](https://github.com/dan64/vs-deoldify/blob/main/document
193193

194194
### Merging the models
195195

196-
As explained previously, this filter is able to combine the results provided by DeOldify and DDColor, to perform this combination has been implemented 6 methods:
196+
As explained previously, this filter is able to combine the results provided by DeOldify and DDColor, to perform this combination has been implemented 8 methods:
197197

198198
0. _DeOldify_ only coloring model.
199199

@@ -205,9 +205,14 @@ As explained previously, this filter is able to combine the results provided by
205205

206206
4. _Luma Masked Merge_: the behaviour is similar to the method _Adaptive Luma Merge_. With this method the frames are combined using a _masked merge_. The pixels of DDColor's frame with luma < _luma_limit_ will be filled with the (de-saturated) pixels of DeOldify, while the pixels above the _white_limit_ threshold will be left untouched. All the pixels in the middle will be gradually replaced depending on the luma value. If the parameter _merge_weight_ is < 1.0, the resulting masked frames will be merged again with the non de-saturated frames of DeOldify using the _Simple Merge_.
207207

208-
5. _Adaptive Luma Merge_: given that the DDcolor performance is quite bad on dark scenes, with this method the images are combined by decreasing the weight assigned to DDcolor frames when the luma is below the _luma_threshold_. For example with: luma_threshold = 0.6 and alpha = 1, the weight assigned to DDcolor frames will start to decrease linearly when the luma < 60% till _min_weight_. For _alpha_=2, the weight begins to decrease quadratically.
208+
5. _Adaptive Luma Merge_: given that the DDcolor performance is quite bad on dark scenes, with this method the images are combined by decreasing the weight assigned to DDcolor frames when the luma is below the _luma_threshold_. For example with: luma_threshold = 0.6 and alpha = 1, the weight assigned to DDcolor frames will start to decrease linearly when the luma < 60% till _min_weight_. For _alpha_=2, the weight begins to decrease quadratically.
209+
6. _Chroma Retention Merge_: Given that the colors provided by deoldify() are more conservative and stable than the colors obtained with ddcolor(). This function try to restore the colors of gray pixels provide by deoldify() by using the colors provided by ddcolor(). The gray pixels are identified by the parameter "tht". Once are identified the gray pixels are substituted with the desaturated colors in deoldify(), the level of desaturation is identified by the parameter "sat". It is performed a "gradient" substitution, i.e. the gray pixels are gradually substituted depending on the level of gray gradient. The steepness of gradient curve is controlled by the parameter "alpha". Optionally is possible to resize the frame before the filter application to speed up the filter by setting True the parameter ?chroma_resize?.
210+
7. _ChromaBound Adaptive_: Adaptive version of Constrained-Chroma. In this version the chroma tolerance is adaptive, i.e., it is applied an approach that will allow more color variation in textured/complex regions and less in smooth areas. The texture strength is computed via Laplacian and chroma tolerance is controlled by the following parameters:
211+
- base_tol: int = 20, Base chroma tolerance (smooth areas)
212+
- max_extra: int = 24, Extra tolerance for textured areas
213+
209214

210-
The merging methods 2-5 are leveraging on the fact that usually the DeOldify _Video_ model provides frames which are more stable, this feature is exploited to stabilize also DDColor. The methods 3 and 4 are similar to _Simple Merge_, but before the merge with _DeOldify_ the _DDColor_ frame is limited in the chroma changes (method 3) or limited based on the luma (method 4). The method 5 is a _Simple Merge_ where the weight decrease with luma.
215+
The merging methods 2-7 are leveraging on the fact that usually the DeOldify _Video_ model provides frames which are more stable, this feature is exploited to stabilize also DDColor. The methods 3, 4 and 7 are similar to _Simple Merge_, but before the merge with _DeOldify_ the _DDColor_ frame is limited in the chroma changes (method 3) or limited based on the luma (method 4). The method 5 is a _Simple Merge_ where the weight decrease with luma and the method 7 is an hybrid model that combines the approach of methods 4 and 5.
211216

212217
## Comparison of Models
213218

@@ -365,27 +370,33 @@ Finally the flag **Reference frames only** can be used to export the reference f
365370

366371
As stated previously the simplest way to colorize images with the HAVC filter it to use [Hybrid](https://www.selur.de/downloads). To simplify the usage has been introduced standard Presets that automatically apply all the filter's settings. A set of parameters that are able to provide a satisfactory colorization are the following:
367372

368-
- **Speed:** medium or fast (_fast_ will increase the speed with a little decrease in color accuracy)
369-
- **Color map:** none
370-
- **Color tweaks:** violet/red
371-
- **Denoise:** light
372-
- **Stabilize:** stable or morestable
373+
- **Speed:** slower
374+
- **Color map:** red->brown
375+
- **Color tweaks:** retinex/red
376+
- **Denoise:** medium
377+
- **Stabilize:** balanced
378+
- **B&W tune:** light
379+
- **B&W mode:** CLAHE (luma)
380+
- **Interpolation:** 3
373381

374382
then enable the _Exemplar Models_ check box and set
375383
- **Method:** HAVC
376384
- **SC thresh:** 0.10
377385
- **SC SSIM thresh:** 0.0
378-
- **SC min freq:** 15 (5 if is used the _local_ mode)
386+
- **SC min freq:** 0
379387
- **normalize:** checked
380388
- **Mode:** remote
381389
- **Frames:** 0
382-
- **Preset:** medium (_slow_ will increase the color accuracy but the speed will decrease of 40%)
390+
- **Preset:** medium
383391
- **Vivid:** checked
392+
- **Ref merge:** high
384393

385394
In the following picture are shown the suggested parameters:
386395

387396
![Hybrid Preset](https://github.com/dan64/vs-deoldify/blob/main/hybrid_setup/Model_Presets.JPG)
388397

398+
The suggested settings are appropriate for a medium powered GPU (RTX4070 or above). In the [HAVC User Guide](https://github.com/dan64/vs-deoldify/blob/main/documentation/HAVC%20User%20Guide.pdf) are provided more settings depending on the available hardware.
399+
389400
## Conclusions
390401

391402
In Summary **HAVC** is able to provide often a final colorized image that is better than the image obtained from the individual models, and can be considered an improvement respect to the current Models. It is highly recommended to read the [HAVC User Guide](https://github.com/dan64/vs-deoldify/blob/main/documentation/HAVC%20User%20Guide.pdf) which provides useful tips on how to improve the colored movies.

documentation/HAVC User Guide.pdf

3.02 MB
Binary file not shown.

hybrid_setup/Model_D+D.JPG

7.21 KB
Loading

hybrid_setup/Model_D+D_filters.JPG

5.02 KB
Loading

hybrid_setup/Model_DD.JPG

1.83 KB
Loading

hybrid_setup/Model_DS.JPG

4.02 KB
Loading

hybrid_setup/Model_DV.JPG

1.63 KB
Loading

hybrid_setup/Model_Presets.JPG

6.26 KB
Loading

pyproject.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ exclude = [
1111

1212
[project]
1313
name = "vsdeoldify"
14-
version = "5.5.1"
14+
version = "5.6.0"
1515
description = "HAVC functions for VapourSynth"
1616
readme = "README.md"
1717
requires-python = ">=3.10"

0 commit comments

Comments
 (0)