an eDSL for composing computation steps

[mccraigmccraig]

I wanted an eDSL for composing computation steps (to give steps names, capture relevant values, log, and resume failed computations). Here's a couple of examples of what I came up with, based on a monad over thunks:

pub fun make-test-dsl1()
  with x <- ({1}).step("x").bind
  with y <- step({10},"y").bind
  with z <- ({x + y}).step("z").bind
  ({2 * z}).step("result")

pub fun test-dsl1()
  val td = make-test-dsl1()
  td()

--->

x: 1
y: 10
z: 11
result: 22

and

fun stuff(i: int)
  step({i}, "stuff")

pub fun make-test-dsl2(i: int)
  with x <- ({i}).step("x").bind
  with y <- ({ if i>5 then [10,11] else [] }).step("y").bind
  val aa' = match y
    [a, b] -> 
      with z <- ({(x + a + b)}).step("z").bind
      ({z}).step("aa'-pair")
    _ -> ({0}).step("aa'-default")
  with aa <- aa'.bind
  with bb <- stuff(i).bind
  ({aa.show ++ "/" ++ bb.show}).step("result")

pub fun test-dsl2()
  println(make-test-dsl2(1)())
  println("--")
  println(make-test-dsl2(10)())

---> 
x: 1
y: []
aa'-default: 0
stuff: 1
result: 0/1
--
x: 10
y: [10, 11]
z: 31
aa'-pair: 31
stuff: 10
result: 31/10

what do you think ? (any criticisms, suggestions, bring 'em on)

https://github.com/mccraigmccraig/koka-noodles/blob/trunk/src/steps.kk

[mccraigmccraig]

well this turned out quite fun ... this code:

fun testdsl/stuff(i: int)
  step({i}, "stuff")

pub fun make-test-dsl2(i: int)
  with x <- ({i}).step("x").bind
  with y <- ({ if i>5 then [10,11] else [] }).step("y").bind
  val aa' = match y
    [a, b] -> 
      with z <- ({foldl([x, a, b], 0, (+))}).step("z").bind
      ({z}).step("aa'-pair", ?strde=int/strde)
    _ -> ({0}).step("aa'-default")
  with aa <- aa'.bind
  with bb <- testdsl/stuff(i).bind
  ({aa.int/show ++ "/" ++ bb.int/show}).step("result")

pub fun test-dsl2()
  val (_, log) = make-test-dsl2(10).run-nofx
  logger/print-computation-log(log)

creates and prints a structured log of the computation path:

computation: bind--x
  step: x
  bound-value: 10
  children:
  | step: y
  | bound-value: [10, 11]
  | children:
  | | step: bind--z
  | | bound-value: 31
  | | children:
  | | | step: z
  | | | bound-value: 31
  | | | outcome: run
  | | | output: 31
  | | | --
  | | | step: stuff
  | | | bound-value: 10
  | | | outcome: run
  | | | output: 31/10
  | | outcome: run
  | | output: 31/10
  | outcome: run
  | output: 31/10
  outcome: run
  output: 31/10

[TimWhiting]

I think this is great, especially the fact that it is serializable which is something you don't get with Koka's effect system. It's also surprisingly clean even without something like metaprogramming. Koka's with syntax is really nice in general.

Though it would be useful to have some syntax like:

pub fun make-test-dsl2(i: int)
  do
    x <- ({i}).step("x").bind
    y <- ({ if i>5 then [10,11] else [] }).step("y").bind
    val aa' = match y
      [a, b] -> do
        z <- ({foldl([x, a, b], 0, (+))}).step("z").bind
        ({z}).step("aa'-pair", ?strde=int/strde)
      _ -> ({0}).step("aa'-default")
    aa <- aa'.bind
    bb <- testdsl/stuff(i).bind
    ({aa.int/show ++ "/" ++ bb.int/show}).step("result")

Or even better:

pub fun make-test-dsl2(i: int)
  do/bind
    x <- ({i}).step("x")
    y <- ({ if i>5 then [10,11] else [] }).step("y")
    val aa' = match y
      [a, b] -> do/bind
        z <- ({foldl([x, a, b], 0, (+))}).step("z")
        ({z}).step("aa'-pair", ?strde=int/strde)
      _ -> ({0}).step("aa'-default")
    aa <- aa'
    bb <- testdsl/stuff(i)
    ({aa.int/show ++ "/" ++ bb.int/show}).step("result")

Then the only other improvement I would make is to swap the order of arguments:

pub fun make-test-dsl2(i: int)
  do/bind
    x <- step("x") { i }
    y <- step("y") { if i>5 then [10,11] else [] }
    val aa' = match y
      [a, b] -> do/bind
        z <- step("z") { foldl([x, a, b], 0, (+)) }
        step("aa'-pair", ?strde=int/strde) { z }
      _ -> step("aa'-default") { 0 }
    aa <- aa'
    bb <- testdsl/stuff(i)
    step("result") { aa.int/show ++ "/" ++ bb.int/show}

[mccraigmccraig]

yeah, an actual do syntax would be awesome...

although one problem would need resolving first - bind (and also pure, which isn't implemented) need an implicit parameter for the serialization, so I couldn't make them work with the struct approach to monads used in the samples

[mccraigmccraig]

As it turns out, the step argument order can already be swapped, to save some () - for some reason I did not expect that the trailing lambdas would desugar to a position before the implicit args, but they do:

pub fun make-test-dsl1()
  with x: int <- step("x"){1}.bind
  with y <- step("y"){10}.bind
  with z <- step("z"){x + y}.bind
  step("result"){2 * z}

I'm still not sure which way round I prefer though - there are fewer parens, but the lambda is the important bit and it seems to get a bit lost in the middle of the expression

It's also a bit annoying that I sometimes need explicit types on a bind variable (or elsewhere) - I started needing them after I added an implicit arg to the bind function to serialize the with fn output - I guess it introduced too much ambiguity for the inference

[mccraigmccraig]

so, it seems combining a trailing lambda, a binding with, and implicit args all together works well - so I can get most of the way to your improvements without any new do/bind syntax @TimWhiting

pub fun make-test-dsl1()
  with x: int <- sb("x") {1}
  with y <- sb("y") {10}
  with z <- sb("z") {x + y}
  s("result") {2 * z}

pub fun make-test-dsl3(i: int)
  with x: int <- sb("x") {i}
  with y <- sb("y") { if i>5 then [10,11] else [] }
  val aa' = match y
    [a, b] -> 
      with z <- sb("z") {x + a + b}
      s("aa'-pair", ?strde=int/strde) {z}
    _ -> s("aa'-default") {0}
  with aa: int <- aa'.bind
  with bb: int <- testdsl/stuff(i).bind
  s("result") {aa.show ++ "/" ++ bb.show}