Specify convention for estimate_objective

[Red-Portal]

The objectives used in variational inference are sometimes minimized or maximized, which can be confusing. For instance the KL divergence is supposed to minimized, but the ELBO is usually maximized. Allowing this to vary on an algorithm-to-algorithm basis doesn't make a lot of sense. Therefore, this PR enforces the convention that all objective values (that are only accessible by calling estimate_objective) are expected to be minimized. For instance, for ELBO maximization algorithms, estimate_objective will return the negative ELBO.

[github-actions]

AdvancedVI.jl documentation for PR #215 is available at:
https://TuringLang.github.io/AdvancedVI.jl/previews/PR215/

[github-actions]

Benchmark Results

Details

Benchmark suite	Current: a77a4ec	Previous: b14bb9f	Ratio
normal/RepGradELBO + STL/meanfield/Zygote	4111928055.5 ns	3920438191 ns	1.05
normal/RepGradELBO + STL/meanfield/ReverseDiff	1143921579 ns	1137314480 ns	1.01
normal/RepGradELBO + STL/meanfield/Mooncake	1212530059 ns	1248798164 ns	0.97
normal/RepGradELBO + STL/fullrank/Zygote	4090539011.5 ns	3888598013 ns	1.05
normal/RepGradELBO + STL/fullrank/ReverseDiff	1687724094 ns	1660774838 ns	1.02
normal/RepGradELBO + STL/fullrank/Mooncake	1264704478.5 ns	1250085352 ns	1.01
normal/RepGradELBO/meanfield/Zygote	2903273540.5 ns	2775544305.5 ns	1.05
normal/RepGradELBO/meanfield/ReverseDiff	807433943 ns	780909736 ns	1.03
normal/RepGradELBO/meanfield/Mooncake	1105499181 ns	1086150808 ns	1.02
normal/RepGradELBO/fullrank/Zygote	2929161011 ns	2770044101 ns	1.06
normal/RepGradELBO/fullrank/ReverseDiff	995244863.5 ns	973336667 ns	1.02
normal/RepGradELBO/fullrank/Mooncake	1124814062 ns	1113137202 ns	1.01
normal + bijector/RepGradELBO + STL/meanfield/Zygote	5684673652 ns	5583939500 ns	1.02
normal + bijector/RepGradELBO + STL/meanfield/ReverseDiff	2511164435 ns	2408835091 ns	1.04
normal + bijector/RepGradELBO + STL/meanfield/Mooncake	4214620793.5 ns	4158213745.5 ns	1.01
normal + bijector/RepGradELBO + STL/fullrank/Zygote	5849709284 ns	5577515624 ns	1.05
normal + bijector/RepGradELBO + STL/fullrank/ReverseDiff	3169720192.5 ns	3063136975 ns	1.03
normal + bijector/RepGradELBO + STL/fullrank/Mooncake	4343776652.5 ns	4187254056 ns	1.04
normal + bijector/RepGradELBO/meanfield/Zygote	4573742532 ns	4312418628 ns	1.06
normal + bijector/RepGradELBO/meanfield/ReverseDiff	2131237741 ns	2049108141 ns	1.04
normal + bijector/RepGradELBO/meanfield/Mooncake	4068000162.5 ns	3927457037.5 ns	1.04
normal + bijector/RepGradELBO/fullrank/Zygote	4647694027 ns	4322323571 ns	1.08
normal + bijector/RepGradELBO/fullrank/ReverseDiff	2413311298 ns	2294744589 ns	1.05
normal + bijector/RepGradELBO/fullrank/Mooncake	4258594545 ns	4054198058 ns	1.05

This comment was automatically generated by workflow using github-action-benchmark.

[mhauru]

Makes sense to me. I only have one docstring question to raise, and a couple of tiny phrasing points.

[mhauru]

LGTM