add nixops eval subcommand

[manveru]

This lets you pass a file to eval to nixops eval.
With nixops eval you can evaluate nix code that has access to your deployment, so you can implement tooling around nixops that depends on your actual configuration easier. This is currently not possible using just nixops show-option, since that only accesses a single attribute.

The simplest example would be to get the list of nodes:

Given this eval.nix file:

{ nodes, ... }: builtins.attrNames nodes

nixops eval --json ./eval.nix
# => [ "node1", "node2", "node3" ]

A more complex example would be generating keys that nixops depends on, using a custom generator attribute on the key:

{ nodes, defaults, ... }:

let
  inherit (defaults.nixpkgs.pkgs) lib;
  inherit (lib)
    pipe filterAttrs mapAttrsToList concatStrings flatten attrValues
    optionalString all concatMapStrings;
in pipe nodes [
  (mapAttrsToList (name: config: config.deployment.keys))
  (map attrValues)
  flatten
  (concatMapStrings ({ keyFile, generator, ... }:
    optionalString (all (x: x != null) [ keyFile generator ]) ''
      set -e

      keyFile=${toString keyFile}
      if ! [[ -f $keyFile ]]; then
        1>&2 printf "Generating $(basename $keyFile)... "
        (${generator}) > $keyFile && 1>&2 echo "done" || (1>&2 echo "failed!" && exit 1)
      fi
    ''))
]

There are many more possibilities, but this really is a lot simpler than having to figure out the correct syntax for eval-machine-info yourself.

[grahamc]

Before I review this, a few things: can you fixup the failures with black, and take a look to see if you can fix the ratchet's complaints about reduced type coverage?

And also if you could write a bit about what this is useful for, like what use cases you have for it -- etc. that'd be helpful!

[manveru]

Yeah... working on it :)

[grahamc]

I left some feedback focusing mostly on the help messages. One thing this does is it sort of implicitly makes the structure of the evaluation's results an API. I know you already treat it like an API, but NixOps doesn't really make a claim about that right now.

I am not sure if we should do that, but want to still let you do what you need to do.

My thinking is two-fold:

1. having a standard way for a NixOps network to become hydra-buildable is a great idea (#1320)
2. if your goal is to have pre-deploy steps, then maybe a plugin and hooks would be a better approach, so a nixops deploy is doing the right thing?

I'm especially curious about your keyfile code there :)

what do you think?

[grahamc]

Suggested change

	subparsers, "eval", help="eval the given file is nix code in the network expression"
	subparsers, "eval", help="evaluate a Nix expression with the NixOps network as arguments"

my proposed wording isn't quite right, but maybe something like this?

[grahamc]

Suggested change

	subparser.add_argument("code", metavar="CODE", help="code")
	subparser.add_argument("file", metavar="FILE", help="File containing a Nix expression")

[grahamc]

Suggested change

	"--json", action="store_true", help="print the option value in JSON format"
	"--json", action="store_true", help="print the resulting value as an JSON representation of the abstract syntax tree"

what does it print without --json?

[grahamc]

I don't have a specific suggestion, but we might want to crib what the nix-instantiate manual says:

recursively evaluate list elements and attributes. Normally, such sub-expressions are left unevaluated
(since the Nix expression language is lazy).
Warning
This option can cause non-termination, because lazy data structures can be infinitely large.

I wonder why someone wouldn't use --strict --json? Should --json imply --strict?

[manveru]

Yes, --json will only work with --strict if it's more than one level deep.

The use-case here was if you wanted to evaluate something but finely control the level of strictness and output Nix instead of JSON. For that you'd want lazy evaluation of functions, which is impossible if --strict is hardcoded.

Given that this is supposed to be used by other tooling (that's why I don't pass a plain string but a file instead), it shouldn't be too confusing or hard to configure it for the specific usage.

[grahamc]

Not necessarily to be fixed here, but it would be nice to think of a way for the deployment API to not require being careful about calling evaluate.