Phillip Trelford's Array

POKE 36879,255

DDD East Anglia 2014

This Saturday saw the Developer Developer Developer! (DDD) East Anglia conference in Cambridge. DDD events are organized by the community for the community with the agenda for the day set through voting.

T-Shirts

The event marked a bit of a personal milestone for me, finally completing a set of DDD regional speaker T-Shirts, with a nice distinctive green for my local region. Way back in 2010 I chanced a first appearance at a DDD event with a short grok talk on BDD in the lunch break at DDD Reading. Since then I’ve had the pleasure of visiting and speaking in Glasgow, Belfast, Sunderland, Dundee and Bristol.

Talks

There were five F# related talks on the day, enough to fill an entire track:

Tomas kicked off the day, knocking up a simple e-mail validation library with tests using FsUnit and FsCheck. With the help of Project Scaffold, by the end of the presentation he’d generated a Nuget package, continuous build with Travis and Fake and HTML documentation using FSharp.Formatting.

Anthony’s SkyNet slides are already available on SlideShare:


ASP.Net was also a popular topic with a variety of talks including:

All your types are belong to us!

The title for this talk was borrowed from a slide in a talk given by Ross McKinlay which references the internet meme All your base are belong to us.

You can see a video of an earlier incarnation of the talk, which I presented at NorDevCon over on InfoQ, where they managed to capture me teapotting:

teapot

The talk demonstrates accessing a wide variety of data sources using F#’s powerful Type Provider mechanism.

The World at your fingertips

The FSharp.Data library, run by Tomas Petricek and Gustavo Guerra, provides a wide range of type providers giving typed data access to standards like CSV, JSON, XML, through to large data sources Freebase and the World Bank.

With a little help from FSharp.Charting and a simple custom operator based DSL it’s possible to view interesting statistics from the World Bank data with just a few key strokes:


The JSON and XML providers give easy typed access to most internet data, and there’s even a branch of FSharp.Data with an HTML type provider providing access to embedded tables.

Enterprise

The SQLProvider project provides type access with LINQ support to a wide variety of databases including MS SQL Server, PostgreSQL, Oracle, MySQL, ODBC and MS Access.

FSharp.Management gives typed access to the file system, registry, WMI and Powershell.

Orchestration

The R Type Provider lets you access and orchestrate R packages inside F#.

With FCell you can easily access F# functions from Excel and Excel ranges from F#, either from Visual Studio or embedded in Excel itself.

The Hadoop provider allows typed access to data available on Hive instances.

There’s also type providers for MATLAB, Java and TypeScript.

Fun

Type Providers can also be fun, I’ve particularly enjoyed Ross’s Choose Your Own Adventure provider and more recently 2048:

2048 

Write your own Type Provider

With Project Scaffold it’s easier than ever to write and publish your own FSharp type provider. I’d recommend starting with Michael Newton’s Type Provider’s from the Ground Up article and video of his session at Skills Matter.

You can learn more from Michael and others at the Progressive F# Tutorials in London this November:

DDD North

The next DDD event is in Leeds on Saturday October 18th, where I’ll be talking about how to Write your own Compiler, hope to see you there :)

TypeScript Mario

Earlier this year I had a play with Microsoft’s new compile to JavaScript language, TypeScript. Every man and his dog has a compile to JavaScript solution these days. TypeScript’s angle appears to be to provide optional static typing over JavaScript and some ES6 functionality while compiling out to ES3 by default. It provides a class based syntax similar to C#’s and seems to be aimed at developer’s attempting to scale out JavaScript based solutons.

Last year I ported Elm’s Mario sample to F#, which ended up looking similarly concise. I tried both FunScript and WebSharper for compiling F# to JavaScript, and both worked well:

mario

So I thought I’d try the sample out in TypeScript as a way to get a feel for the language.

TypeScript Interfaces

In F# I defined a type for Mario using a record:

// Definition 
type mario = { x:float; y:float; vx:float; vy:float; dir:string }
// Instantiation 
let mario = { x=0.; y=0.; vx=0.; vy=0.; dir="right" }

In TypeScript I used an interface which looks pretty similar syntactically:

// Definition
interface Character {
    x: number; y: number; vx: number; vy: number; dir: string
};
// Instantiation
var mario = { x:0, y:0, vx:0, vy:0, dir:"right" };

TypeScript transcompiles this to a JavaScript associative array using object notation:

var mario = { x: 0, y: 0, vx: 0, vy: 0, dir: "right" };

Composition

For me the cute part of the Elm and F# versions was using the record “with” syntax and function composition, i.e.

let jump (_,y) m = if y > 0 && m.y = 0. then  { m with vy = 5. } else m
let gravity m = if m.y > 0. then { m with vy = m.vy - 0.1 } else m
let physics m = { m with x = m.x + m.vx; y = max 0. (m.y + m.vy) }
let walk (x,_) m = 
    { m with vx = float x 
             dir = if x < 0 then "left" elif x > 0 then "right" else m.dir }

let step dir mario = mario |> physics |> walk dir |> gravity |> jump dir

I couldn’t fine either of those features available out-of-the-box in TypeScript so I resorted to imperative code with mutation and procedures:

function walk(velocity: CursorKeys.Velocity, character: Character) {
    character.vx = velocity.x;
    if (velocity.x < 0) character.dir = "left";
    else if (velocity.x > 0) character.dir = "right";
}

function jump(velocity:CursorKeys.Velocity, character:Character) {
    if (velocity.y > 0 && character.y == 0) character.vy = 5;    
}

function gravity(character: Character) {
    if (character.y > 0) character.vy -= 0.1;
}

function physics(character: Character) {
    character.x += character.vx;
    character.y = Math.max(0, character.y + character.vy);
}

function verb(character: Character): string {
    if (character.y > 0) return "jump";
    if (character.vx != 0) return "walk";
    return "stand";
}

function step(velocity: CursorKeys.Velocity, character:Character) {
    walk(velocity, mario);
    jump(velocity, mario);
    gravity(mario);
    physics(mario);
}

The only difference between the TypeScript and the resultant JavaScript is the type annotations.

HTML Canvas

TypeScript provides typed access to JavaScript libraries via type definition files. The majority appear to be held on a personal github repository.

Note: both FunScript and WebSharper can make use of these type definition files to provide types within F# too.

Among other things this lets you get typed access over things like the HTML canvas element albeit with some funky casts:

    var canvas = <HTMLCanvasElement> document.getElementById("canvas");
    canvas.width = w;
    canvas.height = h;

This has some value, but you do have to rely on the definition files being kept up-to-date.

Conclusions

On the functional reactive side TypeScript didn't appear to offer much value add in comparison to Elm or F#.

To be honest, for a very small app, I couldn’t find any advantages to using TypeScript over vanilla JavaScript. I guess I’d need to build something a lot bigger to find any.

Sample source code: https://bitbucket.org/ptrelford/mario.typescript

FParsec Tutorial

Back at the start of the year, I took the F# parser combinator library FParsec out for a spin, writing an extended Small Basic compiler and later a similar parser for a subset of C#. Previously I’d been using hand rolled parsers, for projects like TickSpec, a .Net BDD library, and Cellz, an open source spreadsheet. With FParsec you can construct a parser relatively rapidly and easily using the powerful built-in functions and F# interactive for quick feedback.

FParsec has been used in a number of interesting projects including FunScript, for parsing TypeScript definition files, and FogBugz for search queries in Kiln.

Like any library there is a bit of a learning curve, taking time to get up to speed before you reap the benefits. So with that in mind I put together a short hands on tutorial that I ran at the F#unctional Londoners meetup held at Skills Matter last week.

The tutorial consisted of a short introduction to DSLs and parsing. Then a set of tasks leading to a parser for a subset of the Logo programming language. Followed by examples of scaling out to larger parsers and building a compiler backend, using Small Basic and C# as examples.


Download the tasks from: http://trelford.com/FParsecTutorial.zip

Logo programming language

One of my earliest experiences with programming was a Logo session in the 70s, when my primary school had a short term loan of a turtle robot:

1968_LogoTurtle

The turtle, either physical or on the screen, can be controlled with simple commands like forward, left, right and repeat, e.g.

> repeat 10 [right 36 repeat 5 [forward 54 right 72]]

image

Abstract Syntax Tree

The abstract syntax tree (AST) for these commands can be easily described using F#’s discriminated unions type:

type arg = int
type command =
   | Forward of arg
   | Turn of arg
   | Repeat of arg * command list

Note: right and left can simply be represented as Turn with a positive or negative argument.

The main task was to use FParsec to parse the commands in to AST form.

Parsing

A parser for the forward command can be easily constructed using built-in FParsec parser functions and the >>. operator to combine them:

let forward = pstring "forward" >>. spaces1 >>. pfloat

The parsed float value can be used to construct the Forward case using the |>> operator:

let pforward = forward |>> fun n -> Forward(int n)

To parse the forward or the short form fd, the <|> operator can be employed:

let pforward = (pstring "fd" <|> pstring "forward") >>. spaces1 >>. pfloat
               |>> fun n -> Forward(int n)

Parsing left and right is almost identical:

let pleft = (pstring "left" <|> pstring "lt") >>. spaces1 >>. pfloat 
            |>> fun x -> Left(int -x)
let pright = (pstring "right" <|> pstring "right") >>. spaces1 >>. pfloat 
             |>> fun x -> Right(int x)

To parse a choice of commands, we can use the <|> operator again:

let pcommand = pforward <|> pleft <|> pright

To handle a sequence of commands there is the many function

let pcommands = many (pcommand .>> spaces)

To parse the repeat command we need to parse the repeat count and a block of commands held between square brackets:

let block = between (pstring "[") (pstring "]") pcommands
let prepeat = 
    pstring "repeat" >>. spaces1 >>. pfloat .>> spaces .>>. block
    |>> fun (n, commands) -> Repeat(int n, commands)

Putting this altogether we can parse a simple circle drawing function:

> repeat 36 [forward 10 right 10]

However we cannot yet parse a repeat command within a repeat block, as the command parser does not reference the repeat command.

Forward references

To separate the definition of repeat’s parser function from it’s implementation we can use the createParserForwardedToRef function:

let prepeat, prepeatimpl = createParserForwardedToRef ()

Then we can define the choice of commands to include repeat:

let pcommand = pforward <|> pleft <|> pright <|> prepeat

And finally define the implementation of the repeat parser that refers to itself:

prepeatimpl := 
    pstring "repeat" >>. spaces1 >>. pfloat .>> spaces .>>. block
    |>> fun (n, commands) -> Repeat(int n, commands)

Allowing us to parse nested repeats, i.e.

> repeat 10 [right 36 repeat 5 [forward 54 right 72]]

Parses to:

> Repeat (10,[Right 36; Repeat (5,[Forward 54; Right 72])])

Interpreter

Evaluation of a program can now be easily achieved using pattern matching over the AST:

let rec perform turtle = function
    | Forward n ->
        let r = float turtle.A * Math.PI / 180.0
        let dx, dy = float n * cos r, float n * sin r
        let x, y =  turtle.X, turtle.Y
        let x',y' = x + dx, y + dy
        drawLine (x,y) (x',y')
        { turtle with X = x'; Y = y' }
    | Turn n -> { turtle with A=turtle.A + n }
    | Repeat(n,commands) ->
        let rec repeat turtle = function
            | 0 -> turtle
            | n -> repeat (performAll turtle commands) (n-1)
        repeat turtle n
and performAll = List.fold perform

Check out this snippet for the full implementation as a script: http://fssnip.net/nM

User Commands

Logo lets you define your own commands, e.g.

>  to square
     repeat 4 [forward 50 right 90]
   end
   to flower
     repeat 36 [right 10 square]
   end
   to garden
     repeat 25 [set-random-position flower]
   end

garden

The parser can be easily extended to support this, try the snippet: http://fssnip.net/nN

Small Basic

Small Basic is a Microsoft programming language also aimed at teaching kids, and also featuring turtle functionality. At the beginning of the year I wrote a short series of posts on writing an extended compiler for Small Basic:

The series starts with an AST, internal DSL and interpreter. Then moves on to parsing the language with FParsec and compiling the AST to IL code using Reflection.Emit. Finally the series ends with extensions for functions with arguments and support for tuples and pattern matching.

It’s a fairly short hop from implementing Logo to implementing a larger language like Small Basic.

Parsing C#

A few weeks later as an experiment I knocked up an AST and parser for a fairly large subset of C#, which shares much of the imperative core of Small Basic: http://fssnip.net/lf

Check out Neil Danson’s blog on building a C# compiler in F# to see C# compiled to IL using a similar AST.

DDD North: Write your own compiler in 24 hours

If you’re interested in learning more, I’ll be speaking at DDD North in Leeds on Saturday 18th October about how to write your own compiler in 24 hours.