Questions wrt. gradient computation

[matthiasreisser]

I'm not 100% clear about pytorch syntax. Should the two following ways to compute the gradients df/dtheta be equivalent? Why are they not? :) I'm not entirely sure what loss.backward(backward_ones) does. Is this df/d1 ?

loss.mean().backward(retain_variables=True)
print(reg_funcs.params.grad.data)
reg_funcs.params.data.zero_()
loss.backward(backward_ones)
print(reg_funcs.params.grad.data)

[toologicbv]

this is (what I learned from pytorch documentation) necessary if the loss variable is not a scaler, which is the case in my software, it is a vector. It wouldn't work without the "backward_ones" vector.

[matthiasreisser]

Yes it's not a vector because you have a batchsize of 125 functions, so you would want to compute
df/dtheta for each f. Each of these gradients is an average over n_samples data-points.
By averaging over the 125 functions, we get a scalar and can compute the gradients without the vector of 1s. Since the 125 functions don't share parameters theta (each f has individual thetas), taking the gradient of the average (or sum) should again decompose into 125 vectors (which it does indeed).
Now if both methods were equivalent then we should expect the same values in the above print statements (up to a multiplicative scalar), right?

Maybe you could clarify what the vector of ones actually does.

[toologicbv]

your are right that "loss" is a matrix of batch_size x num_of_params_optimizee.
I think that it should not be necessary to set the parameter "retain_variables" to True. We should only call backward() once after we calculated the optimizee losses.

It's true that the statement loss.mean().backward() also generates the gradient matrix. But I think that the other version, which calculates the gradients per function/parameter based on the loss the specific function "produced", is more accurate and generates gradients that are specific to the optimizee, which should help when we feed them into the LSTM.

I tried it with "loss.mean().backward() " but the tests that I run reveal that calculating the optimizee gradients this way has a negative effect on the performance of the (LSTM) optmizer.

[matthiasreisser]

Of course the above code was just for printing the gradients. We need not compute the gradient twice.

There is just one correct way of computing the gradients, all others will be wrong (not more or less accurate?) All gradients should be specific to each optimizee. My mathematical argument is that the gradient of a sum of independent functions wrt to their independent individual parameters (jacobian) is equivalent to a matrix filled with the gradients of each function wrt. their individual parameters.

What I was pointing out is just that I don't understand what this vector of ones is doing (mathematically).
If you are convinced that your implementation is doing what it is supposed to, then that's fine :)