Runtime & Better Contracts

[spy16]

This PR has the new structure and types as per the discussions on #25 . New contracts are nested within a sub-package sabre for the time being (Just to make it easy to move.)..

Highlights:

• runtime contains all the core interfaces/types that Sabre deals with - Runtime, Value, Invokable, Seq, Seqable, Vector, Map, Set & a common Error type.
• reader package contains the Reader. reader.New returns a reader instance that can read the primitive types defined in runtime. For map, vectors, sets,reader provides MapReader(mapFactory), VectorReader(vectorFactory), SetReader(setFactory) reader-macro implementation.. With the factory argument, makes it easy to use any map implementation.
• repl package remains the same.
• core package is in the works and will contain the function value, macro system, special form type and built-in special forms.

Opening this incomplete PR to get feedbacks early on. (Expect lot of random commits here. I will be rewriting commit history to make sense before merge)

[spy16]

Hi, I have been super busy and haven't been able to do all the planned changes yet. But, I am slowly working on this. One decision I need to take now is about the Special Forms.

I am considering to bake the special-forms handling into the Runtime itself. It will remain the responsibility of Runtime to handle the special form during Eval. This has the following advantages:

1. Special forms are special evaluation rules and it makes sense that they are defined with the Runtime.
2. Adding custom special forms becomes possible but only when extreme customisation is needed (In cases where a new Runtime would be required anyway)

For the built-in basic runtime implementation provided in the runtime package, the factory function will be runtime.New(flags int32, parent Runtime) Runtime.. (flags can control what built in features should be enabled). And the built-in special forms will be:

1. (fn name? [args] expr*) - and the multi-arity variant.
2. (def symbol expr doc?)
3. (cond test1 expr1 ...) --> No if-then-else form since cond can be used for it.
4. (do expr*)
5. (let [bindings] expr*)
6. (error expr) --> need to think about this bit more and also the handling form.

@lthibault Let me know your thoughts. And also, if i have missed any essential special form.

[lthibault]

LGTM. I agree with both points regarding the integration of special forms into Runtime.

Regarding runtime.New, any particular reason you're leaning towards int32 flags? I'm guessing some of the bits will be reserved, and others will allow for customization. If that's the case, I'm a bit nervous about running out of space - 32 options isn't a lot.

Would it make sense to use functional options, as in the repl package? I quite like the idea of writing:

// signature: func New(parent runtime.Runtime, opt ...runtime.Option) runtime.Runtime
runtime.New(nil, runtime.WithDefault, somelib.WithFoo)

I think this is a nicer API for users as well.

[spy16]

Yea it is. I didn't really go with flags. :) I mentioned it here but then decided probably not a good idea.

[lthibault]

@spy16 Just wondering: does anything else still need implementing, or can we begin the formal code-review/merge process?

[lthibault]

@spy16 can we get rid of the strings.TrimSpace in repl.WithBanner? It's interfering with my template, which specifies a newline at the end.

Right now my banner looks like this:

$ go run cmd/ww/main.go
Wetware v0.0.0, go1.14
Copyright 2020 The Wetware Project
ww » 

But I want it to look like this:

$ go run cmd/ww/main.go
Wetware v0.0.0, go1.14
Copyright 2020 The Wetware Project

ww » 

[lthibault]

@spy16 I have some spare cycles beginning next week -- is there anything I can do to help move this forward? This is becoming a bit of a blocking issue on my end, so I'm happy to pick up some of the slack. 🙂

[spy16]

Hey @lthibault, Very sorry for the delay here. I ran into couple of issues while working on this refactor.

1. Analyse turned out to be a really complicated thing to do when we use the interface for everything approach.. (For example, if a map is encountered, Analyse needs to check each key and value and expand it if necessary and re-construct a map with the new set of key-values. But since how to construct a map is allowed to vary, Analyse cannot construct a map easily)..
2. Since Eval rules are left to the Value type, it is hard to control things from the Runtime. For example, if you want to be able to support stack-traces, Runtime needs to "know" what is being invoked and maintain a stack frame for it. This will require List to be able to know how stack is implemented in the Runtime causing a cyclic relation..

I was experimenting with an approach very similar to Clojure and the Joker (#18) and i think i was close to solving both the issues. But my father had a heart attack (He is fine now) and I had to take a break from work and this.

I plan on looking at this again this weekend but unsure about timelines. Here's what Clojure does:

• Clojure deals directly with Object type. Translates to type Value interface{} for us.
• Clojure has a runtime RT (A singleton) that every other data structure talks to. This RT does not define the evaluation rules.
• Clojure has a Compiler class which defines eval(Object form) method. Algorithm of this method:

def eval(form):
  meta = get_meta(form) # may be null if not set.
  bindings = merge(meta, get_position_info(form)) # file, line, col
  form = macro_expand(form)
  expr = analyze(form)
  return expr.eval()

Notice how instead of evaluating the value directly, it turns values into Expr interface type. For example any constant value is wrapped into ConstExpr and returned which on eval() simply returns the wrapped value. Similarly, DefExpr which on eval() creates a global binding. InvokeExpr that performs function invocation and so on. (In our case, it would be possible to define a Sabre type with few pre-defined expr values. A Runtime turns any value type to one of these exprs or combination of them then calls Sabre.eval(). This way Sabre needs to define eval rules for only few Expr types. And custom runtime implementations can decide how to turn any value type to expr type based on its own rules)

Joker being Clojure in Go, replicates a similar design. I have some ideas on a similar approach.

Also, I saw your PR in which you said the design of atoms had to change for your use-case. I would definitely like to hear about the requirements.

Finally, If delay here is blocking you, it might make sense to fork this repo or even copy and merge it into your project I think. I know this is not ideal and it's not how i wanted it as well. But considering my situation and possibility that there might be further delay, I don't want this to become a bottleneck.

[lthibault]

@spy16 I'm terribly sorry to hear about your father! I'm glad he's alright now, although you and your family must still be reeling from the shock. By all means take your time ... this can wait.

In the meantime, thanks for your response -- I'm all the more grateful for your insights given the situation. I'm going to proceed as follows for the time being:

• I'm going to point my go.mod at the reader branch I pushed last night. That should be enough to continue working for a bit.
• I'll read over your comment carefully and give this matter some thought on my end. I may push a few experimental branches as well.

Let me know when you're back in action and we can compare notes. I'll try to move some things around on my end so that I can be available for implementation work when you return.

P.S.: I'm more than happy to fill you in on the requirements for Wetware. Let's have that discussion over at #27, at your convenience.