Phillip Trelford's Array

POKE 36879,255

Pattern Matching Database Records

Last week Jon Harrop showed me some interesting data access code (Jon and I often share the same Cambridge to London train). The code used pattern matching to match records returned from a SQL command, for example.:

let values = [for index in 0..reader.FieldCount-1 -> reader.GetValue index]
match values with
| [Date date; Price o; Price h; Price l; Price c] -> 
    yield date, o, h, l, c
| _ -> invalidOp "Record mismatch"

The code fragment above matches open-high-low-close (OHLC) field values read from an IDataReader. The first line creates a list of field values; these values are then matched using a List pattern with Active Patterns to unbox the objects to strongly typed values, i.e:

let (|Date|) (value:obj) : DateTime = unbox value
let (|Price|) (value:obj) : float = unbox value

Instead of constructing a list each time a pre-allocated array can be employed using the IDataRecord.GetValues method and an Array pattern:

seq {
    let values = Array.zeroCreate reader.FieldCount
    while reader.Read() do       
        reader.GetValues values |> ignore
        match values with
        | [|Date date; Price o; Price h; Price l; Price c|] -> 
            yield date, o, h, l, c
        | _ -> invalidOp "Record mismatch"
}

 

The OHLC values are returned as a tuple from inside a sequence expression which is the equivalent of a C# iterator block.

Pattern Matching

I think this is another great example of the power of pattern matching, a feature common to functional programming languages. Somewhat strangely it’s a feature missing from older and more established object-orientated languages like C++, C# and Java. However there is nothing intrinsically stopping these OO languages from supporting pattern matching. Pattern matching has been successfully implemented in newer moustache/curly-brace based languages like Scala, Nemerle and Kotlin. Progress in Java seems to be severely hampered by committees, just look at how long it took to get to Java 7. C# seems to be making more progress but I think may in part be hampered by the compiler being implemented in C++, which Project Roslyn may fix in the future.

Ordinals

The pattern matching approach makes use of the order of the fields. Another approach employed in examples in both Jon’s F# for Scientists and Don’s Expert F# book, is to get the field values based on their ordinals, i.e.:

yield 
    reader.GetDateTime(0), 
    reader.GetDouble(1),
    reader.GetDouble(2),
    reader.GetDouble(3),
    reader.GetDouble(4) 

String Literals

Yet another approach, used as an example in Professional F# 2.0, is to use the field names:

yield 
    reader.["Date"]  :?> DateTime,
    reader.["Open"]  :?> float,
    reader.["High"]  :?> float,
    reader.["Low"]   :?> float,
    reader.["Close"] :?> float

Dynamic Lookup

And now for a one line ORM which simply overloads F#’s dynamic lookup operator:

let (?) (reader:IDataReader) (name:string) = unbox reader.[name]

This really helps reduce the boilerplate:

yield 
    reader?Date, 
    reader?Open, 
    reader?High, 
    reader?Low, 
    reader?Close

For more information on this approach see Tomas Petricek’s excellent post:

Plotting

Now we’ve explored 5 different ways of doing exactly the same thing it’s probably time to do something useful like plotting.

Again Tomas has a tutorial for that:

Plotting a chart from inside F# interactive using Carl Nolan’s new FSharpChart library:

#load "FSharpChart.fsx"
open MSDN.FSharp.Charting

[ for date,o,h,l,c in getHistoricalPrices "MSFT" -> date, (h,l,o,c) ]
|> List.filter (fun (date, _) -> date >= DateTime(2011,08,01))
|> FSharpChart.Candlestick

Gives us a Candlestick chart for MSFT stock since August:

MSFT

Final Thought

With Type Providers in F# 3 a lot of complexity will disappear, as we get type safe data access with intellisense (rather than strings or ordinals) without the need for code generation!

The Rise of F#

Back in September 2008, Scott Hanselman ran a piece entitled the Rise of F#. The following year, in October 2009, Somesegar announced the release of F# in VS 2010. April 2010 saw F# 2.0 ship with VS2010. 6 months on F# goes open source under the Apache 2.0 license. Last month, F# entered the TIOBE top 20 for the first time, the first functional language to do so.

According to TIOBE the recent rise in popularity comes as no surprise:

Apart from being a nicely designed language, F# is available in the latest version of Microsoft's Visual Studio (2010).

A year before the August 2010 edition of the ThoughtWorks Technology Radar commented on the rise in interest of functional languages F#, Scala and Clojure:

Two characteristics of functional languages in particular are driving this interest, immutability with its implications for parallelism and functions as first class objects. While the introduction of closures to C# brings some of the latter capability, functional languages are almost synonymous with immutability.

Clearly functional programming has much to offer and interest in F# is rising rapidly. But before we get ahead of ourselves it should be noted that in the TIOBE index C# received a rating of 6.042%, while F# received a mere 0.604%. It therefore seems unlikely F# is going to overtake C# anytime soon, or for that matter Scala overtake Java. However,rather than thinking of F# as a competitor to C#, in many scenarios it is probably more appropriate to think of it is complimentary.

Don Syme remarks in the article Putting the ‘Fun’ into ‘Functional’, on F# 2.0:

F# is complementary to .NET stalwarts C# and Visual Basic .NET, which remain integral to the majority of .NET projects. But C# and Visual Basic programmers can use F# components in their existing applications, enabling developers to maintain their current investments and integrate F# on a component-by-component basis.

Sitting on the near horizon is F# 3.0 and the “magic” of Type Providers:

type Netflix = ODataService<"http://odata.netflix.com">

type SQL = SQLConnection<"Server='.\SQLEXPRESS'... ">

type EmeaSite = SharePointSide<"http://myemea/">

 

Jo Palmer describes F# Type Providers:

Type Providers are a unique mechanism for applying the benefits of strong static typing to external, dynamic data sources. For example, Type Providers enable IDE support for “Intellisense”-style completion on dynamic data sources while exploring their APIs. When the code is compiled, programmers can then enjoy static typing against these data sources with a variety of options for code generation.

F# already provides the kind of terseness experienced with dynamic languages while at the same time allowing the type safety and performance of a statically typed language. With F# 3.0 this capability is extended to accessing data and services.

Interested? Try F# interactively in your browser now.

Back to the now, the TIOBE ratings are based on skilled engineers world-wide, courses, third party vendors and results from popular search engines. Lets take a look at where some of those numbers come from.


Skilled Engineers

According to IT Jobs Watch the trend in demand for F# developer in the UK is up:fsharp-permanent-demand-trend

There is also plenty of anecdotal evidence of the use of F# in the Enterprise from case studies and user group membership.

Selection of F# case studies:

Selection of F# user groups:

At time of writing both the London and New York F# user groups had over 300 members!


Courses

 

In the UK, there are a selection of F# workshops now available through Skills Matter:

F# is also starting to be part of university curriculums around the globe:

With a good selection of books available in print:

Beginning F# Programming FSharp Book Professional F# 2.0 Real World Functional Programming F# for Technical Computing Expert F# 2.0

 

And books online:


Third Party Vendors

MSDN has a good article on F# Tools & Resources by Terrence Dorsey, which details some of the F# Open Source projects, including:

  • Fake – a build automation system
  • TickSpec – a BDD framework
  • FsCheck – a randomized testing framework

There are also a number of commercial products targeting F#:


Search

Google returns 3,720,000 results for “F# programming”.

If that’s a bit daunting a good place for the latest articles is the MSDN F# Developer Center.


Finally, something silly

Scott Hanselman ended his Rise of F# article with “something silly”, a 2D Tron-Clone game written in 182 lines of code, which you can now play online if you have Silverlight installed.

A recent post on the F# Snippets site gave us a Mine Sweeper in 99 lines of code.

FineSweeper

Update 23rd September 2011:

F# is the programming language of choice for discriminating hackers!

What makes a good step definition

Following on from last week’s What makes a good feature file article, this week it’s time to tackle what makes a good step definition. To automatically test that an application’s behaviour matches that defined by the scenarios in a feature file, a mapping is required. Step definitions provide that mapping from the lines in a feature file to the code that implements the feature.

Starting with a Given:

Given I have entered 0.1134 into the calculator

Implementations

Ruby step definition for use with Cucumber::

Given /I have entered (.*) into the calculator/ do |n|
  # Some Ruby code here
end

The code above specifies a regular expression used to match lines in the feature file and an anonymous function to execute.

C# attributed step definition compatible with both SpecFlow and TickSpec:

[Given(@"I have entered (.*) into the calculator")]
public void GivenIHaveEnteredNIntoTheCalculator(int n)
{
  // C# code here
}

Again a regular expression is specified, however this time the function is public and reflection is used for invocation.

F# attributed step definition compatible with TickSpec:

let [<Given>] ``I have entered (.*) into the calculator``(n:int) =
  // F# code here
}

F# allows white space and special charecters in method names enclosed with backticks. This removes the need for both an attribute and a function name.

TickSpec also supports anonymous functions as Ruby with Cucumber, i.e.:

Given "I have entered (.*) into the calculator" (fun [Int n] –>
  // F# code here
)

From a terseness point of view it looks like Ruby and F# the advantage for writing step definitions. But that’s only part of the story…

Scope 

By default in frameworks like Cucumber and SpecFlow step definitions are defined globally, so that they can be shared or reused. Global functions are usually fine for small programs, but for larger programs we usually group them into classes or modules for maintainability.

With SpecFlow and TickSpec it is possible to constrain the scope of a step definition or set of step definitions via a StepScope attribute:

[StepScope(Feature="Addition")]
public class AdditionSteps
{
  [Given(@"I have entered (.*) into the calculator")]
  public void GivenIHaveEnteredNIntoTheCalculator(int n)
  {
    // C# code here
  }
}

Here the step definition methods in the AdditionSteps class are only matched against the Addition feature. This allows the wording in the feature file to evolve without affecting other feature files, and it allows the actions in the methods to vary too. It also means that it is easy to find the step definitions that map to a feature file in one place.

Using a class level step definition scope is quite similar conceptually to using a View Model within the MVVM pattern, where the View Model is responsible for mapping from view to model. It adds a layer of indirection over binding directly to the model, but allows the application to evolve more easily over time.

Note: TickSpec can apply scope to both Step bindings and Event bindings like BeforeScenario and AfterScenario. There is currently was an open bug on this in SpecFlow (since fixed).

Fluent interfaces

So far the step definition examples have included a “code here” comment. This is where we arrange, act and assert. In the Given steps the preconditions are arranged. The When steps execute the actions. Finally the Then steps assert the outcome. This is almost identical to the steps you’d take in TDD for a unit test, except the parts are spread across multiple methods.

Note: The state for a scenario can also be initialized in a method marked with BeforeScenario attribute

For the arrange and act steps, DSL approaches like fluent interfaces can be useful for making step definitions more readable and maintainable.

In the arrange steps using the builder pattern is useful to build up the initial state:

public void constructPizza()
{
  pizzaBuilder.createNewPizzaProduct();
  pizzaBuilder.buildDough();
  pizzaBuilder.buildSauce();
  pizzaBuilder.buildTopping();
}

In the act steps a fluent assertion API is useful, for example the FsUnit F# library:

1 |> should equal 1
anArray |> should haveLength 4
anObj |> should not (be Null)

Summary

Using fluent interfaces inside the step definition functions can make them more readable and maintainable. For larger projects constraining the scope of step definitions helps make them more maintainable. Finally step definitions can sometimes be written more concisely in languages like Ruby and F#.