Adding Support for Asynchronous Test Syntax in Various Functions

[1eyewonder]

Suggestion

When using various functions within the library, I think it would be nice if we could add support for async/task test cases. The main two I can think of are testParam and testFixture. If there are other functions that already support this or there is a better way to do this with existing functionality, I would appreciate any insight you have.

Comments

I know I reused the same code for both examples but I just wanted a quick way to illustrate the async/task ergonomics. I realize a http client factory may not be the best use of one or both of these functions.

testParam Example

Current Support

open Expecto
open System.Threading
open Microsoft.AspNetCore.Mvc.Testing
open MyApi.Api

[<Tests>]
let allTests =

  let factory = new WebApplicationFactory<Startup>()

  testList "MyApi Integration Tests" [
    testParam (factory.CreateClient()) [
      "GET /",
      fun client () ->
        task {
          let! response = client.GetAsync("/")
          let! body = response.Content.ReadAsStringAsync()
          Expect.equal "Hello World!" body "Expected 'Hello World!'"
        }
        |> Async.AwaitTask // currently have to do this
        |> Async.RunSynchronously // currently have to do this
    ]
    |> List.ofSeq
    |> testList "Routing Tests"
  ]

[<EntryPoint>]
let main args =
  let cts = new CancellationTokenSource()
  runTestsWithCLIArgsAndCancel cts.Token [| Summary |] args allTests

Ideal Support

Add support for testParamAsync and testParamTask to allow creating the testCaseAsync and testCaseTask under the hood.

open Expecto
open System.Threading
open Microsoft.AspNetCore.Mvc.Testing
open MyApi.Api

[<Tests>]
let allTests =

  let factory = new WebApplicationFactory<Startup>()

  testList "MyApi Integration Tests" [
    testParamTask (factory.CreateClient()) [
      "GET /",
      fun client () ->
        task {
          let! response = client.GetAsync("/")
          let! body = response.Content.ReadAsStringAsync()
          Expect.equal "Hello World!" body "Expected 'Hello World!'"
        }
    ]
    |> List.ofSeq
    |> testList "Routing Tests"
  ]

[<EntryPoint>]
let main args =
  let cts = new CancellationTokenSource()
  runTestsWithCLIArgsAndCancel cts.Token [| Summary |] args allTests

testFixture Example

Current Support

While testFixtureAsync is an existing function, it only allows for asynchronous setup, I believe.

open Expecto
open System.Threading
open Microsoft.AspNetCore.Mvc.Testing
open MyApi.Api

[<Tests>]
let allTests =

  let factory = new WebApplicationFactory<Startup>()

  let withClient f () =
    use client = factory.CreateClient()
    f client

  testList "MyApi Integration Tests" [
    yield! testFixture withClient [
      "GET /",
      fun client ->
        task {
          let! response = client.GetAsync("/")
          let! body = response.Content.ReadAsStringAsync()
          Expect.equal "Hello World!" body "Expected 'Hello World!'"
        }
        |> Async.AwaitTask // currently have to do this
        |> Async.RunSynchronously // currently have to do this
    ]
  ]

[<EntryPoint>]
let main args =
  let cts = new CancellationTokenSource()
  runTestsWithCLIArgsAndCancel cts.Token [| Summary |] args allTests

Ideal Support

This one may be weird to name if it is desired since we have testFixture and testFixtureAsync. In order to not break existing functionality, does it make sense to add async/task after test to show the signature of the test and then have async/task after fixture to show the signature of the fixture?

open Expecto
open System.Threading
open Microsoft.AspNetCore.Mvc.Testing
open MyApi.Api

[<Tests>]
let allTests =

  let factory = new WebApplicationFactory<Startup>()

  let withClient f () =
    use client = factory.CreateClient()
    f client

  testList "MyApi Integration Tests" [
    yield! testTaskFixture withClient [
      "GET /",
      fun client ->
        task {
          let! response = client.GetAsync("/")
          let! body = response.Content.ReadAsStringAsync()
          Expect.equal "Hello World!" body "Expected 'Hello World!'"
        }
    ]
  ]

[<EntryPoint>]
let main args =
  let cts = new CancellationTokenSource()
  runTestsWithCLIArgsAndCancel cts.Token [| Summary |] args allTests

Notes

I am willing to try and add the necessary support if we think this would be beneficial to add to the library.

[farlee2121]

It looks like testParam indeed lacks async and task variants.
This is an inconsistency worth patching

However, it appears that testFixtureAsync and testFixtureTask behave in the preferred way you describe: they respectively take async and task types who's evaluation is managed by the library. See the documentation, testFixtureTask tests, and some testFixtureAsync tests