Reconsider equality of `ScalarValue`s for encoded types (REE/Dictionary)

[Jefffrey]

Is your feature request related to a problem or challenge?

@brancz brought up a point on how we might want to not consider the index/key types of REE/Dict scalarvalues when checking equality:

• Add ScalarValue::RunEndEncoded variant #19895 (comment)

Code reference:

datafusion/datafusion/common/src/scalar/mod.rs

Lines 727 to 739 in 8c478e9

	(Dictionary(k1, v1), Dictionary(k2, v2)) => {
	// Don't compare if the key types don't match (it is effectively a different datatype)
	if k1 == k2 { v1.partial_cmp(v2) } else { None }
	}
	(Dictionary(_, _), _) => None,
	(RunEndEncoded(rf1, vf1, v1), RunEndEncoded(rf2, vf2, v2)) => {
	// Don't compare if the run ends fields don't match (it is effectively a different datatype)
	if rf1 == rf2 && vf1 == vf2 {
	v1.partial_cmp(v2)
	} else {
	None
	}
	}

Describe the solution you'd like

Consider ignoring the index/key types for scalarvalue equality of REE/dictionary values. What knock-on effects will this have? What does it enable for us?

Describe alternatives you've considered

Don't do this if we prefer to stick to strict equality.

Additional context

No response