With week six behind us, we're on the home straight in Martin Odersky's Functional Programming Principles in Scala course. We have covered a number of new concepts which we've already started to apply to our work, and gone into more detail on the practical and theoretical principles of functional programming. This post forms part of an ongoing series on our experiences with the course, so if you've missed the others, check out our first impressions and our update 3 weeks into the course.

It's been a few weeks since our last update and in that time the course has moved on a long way. Week four introduced Scala's type system in detail, covering some advanced topics including Polymorphism, Generics and Variance. Week five formally introduced Scala's Lists (we had been working on idealised representations of the List type up to this point). The List type was used as a springboard to look into higher order functions and we were taken through Scala's implicits (a technique in which you can insert or convert data types non-explicitly). Implicits can allow developers to create flexible and powerful libraries, but – while they serve a clear purpose – they can also hamper the comprehension of a codebase. Implicits, by their very nature, are opaque to anything other than a detailed inspection of the code and our experience of implicits is that they can cause confusion in codebases written and maintained by multiple developers.

This week's lectures paid particular attention to Scala's for loop. Due to Scala's functional implementation, the loop syntax and behaviour is very different from the more familiar for loop found in languages like Java and C. In fact, Scala's for loop is just a shorthand for writing a combination of map, flatMap and filter calls on your collection. This sometimes leaves people asking what's the point in having this construct at all and we'd always overlooked the for loop as an unintuitive construct with unfamiliar syntax. The lectures and exercises take a bit of time to explain this relationship and outline some benefits to this approach. We found this week's assignment has highlighted that for loops can be particularly useful when the task is semantically one of generating a data structure from one or more input collections rather than reducing an existing collection. Also, looping through multiple dimensions can often be expressed with a for loop concisely, whereas writing out the collection operations explicitly can lead to some very opaque code.

The second half of the course has shown a noticeable shift in the scope of the assignments. The first few weeks felt like Scala with stabilisers, working with very limited subsets of Scala's functionality to solve the type of highly abstract problem that lends itself naturally to functional programming paradigms. These serve as a fantastic introduction to functional programming and Scala's syntax but, as we commented in our last article, this high level of abstraction was frustrating for some. The increased focus on practical aspects of the course has continued and we now find ourselves writing large programs to solve much more complex problems anchored in real-world examples. While these problems aren't necessarily going to come up in our day jobs, their complexity means that we're getting practice comprehending and manipulating Scala's data structures.

We're also delighted we are going into more depth on the abstract side of functional programming. The course includes more advanced videos that have been largely optional (they're compulsory for students taking the full version) and take us through the benefits of the close relationship between functional programming and maths. This relationship emerges naturally because functional programming is a pure abstraction of mathematical principles in software, whereas imperative programming is designed around the constraints of the underlying hardware.

One way to see how functional programming differs from programming in "classical" languages is in the way prospective developers are introduced to the concepts. Most C, Java and Python tutorials begin with a "Hello, world" example and move on to writing simple programs, introducing concepts as they go. Functional programming is very different – for some functional programming languages, even "Hello, world" can be a program that is built on some complex concepts because it is inherently impure (eg. it has side-effects). This course has followed the standard way to introduce functional programming by starting with pure, abstract programs to expose the developer to the underlying principles. You can then stack up more advanced concepts, all the while building an edifice of understanding with its foundations firmly rooted in basic axioms.

For example, across all types of programming, tests are a crucial tool for verifying that software behaves the way it ought to. Odersky shows us an alternative way to verify a program by applying the tools available to mathematicians. Using inductive methods, we verify the absolute correctness of some functionality on Scala's Lists with mathematical proofs. It is possible, although sometimes difficult or impractical, to come up with test data that completely captures the behaviour for part of a program in unit or integration tests. However, if we can get to a mathematical proof, we know that the function must work correctly for any possible data. It is a much stronger measure of its correctness, albeit correspondingly harder to arrive at.

We've got the last week ahead of us and at first glance it looks like we've got a tricky one in store, covering more of both the practical and abstract sides of functional programming and Scala. In the coming week, the extensive set of video lectures will introduce some new concepts and bring all that we've learned together, ready for us to create our largest program yet in the final assignment. Good luck and we hope to see you at graduation!