-Xlint:unchecked and -Werror for Java projects with Gradle

I was recently working in a codebase where there were chunks of Java code that used generic interfaces from the Apache Commons Collections project as well as java.util Collections in a type unsafe way, even though the interfaces/classes that were being used supported generics.

Our gradle build was emitting the following warning:

:compileJavaNote: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

This warning should be an error in my books. It’s possible to get this behaviour by adding the following to your build.gradle file:

tasks.withType(JavaCompile) {
  options.compilerArgs << "-Xlint:unchecked" << "-Werror"
}

This way, when our code is built everyone is aware when they’re using unsafe operations, and can either fix the issue or supress the warning with @SupressWarnings("unchecked") if it’s intentional.

Pedantic console for JavaScript tests

When running JavaScript tests that fire up a browser, it can be useful to ensure that console.warn and console.error aren’t being called, and to have your automated build fail if this starts happening.

If you’re using a framework like Jasmine, you can place the following file in your helpers directory to ensure that an exception will be thrown whenever a call is made to console.warn or console.error:

console.error = (function () {

  var originalConsole = console.error;

  function myError () {
    originalConsole.apply( this, arguments );
    throw Error('console.error called, failing test');
  }

  return myError;
})();


console.warn = (function () {

  var originalConsole = console.warn;

  function myError () {
    originalConsole.apply( this, arguments );
    throw Error('console.warn called, failing test');
  }

  return myError;
})();

I recently used this to locate all the places that a deprecated Moment.js feature was being used in a codebase.

Haskell stack, Yesod and Docker

The why

Over the break I’ve been working on a web app to replace a fairly old MS Access Database that I built for my Dad to use in 2009 (he has a mobile vet business).

This seemed like a good chance to try out Yesod, a web framework for Haskell. The Yesod philosophy is to leverage the Haskell type system wherever possible. For example, in the Hamlet templating language everything from generating URLs to including static files and generating forms is checked at compile time.

stack is a cross platform build tool for developing Haskell programs which (when combined with Stackage makes Haskell development much more enjoyable.

I’d recommend stack for any new Haskell project, and hopefully this post can point someone else in the right direction for a stack/Yesod/Docker project.

I built a very basic site with Yesod and stack locally on OS X, based off the yesod-postgres template (stack new mysite yesod-postgres). The yesod dev server makes this all very easy, just running stack exec -- yesod devel will keep Yesod running and recompiling the app whenever any changes are made.

Soon it was time to make the app available somewhere in order to start getting some feedback. At this stage, the most basic requirement for the hosting was that it be low cost. Digital Ocean came to mind (use this link for $10 credit), and I’m now using their one-click Docker droplet at USD $5 per month. This host can run both the Yesod frontend and the PostgreSQL database.

Binaries built locally on OS X won’t run on a Linux-based Digital Ocean droplet, and it was a bit early to spend $$ on something like Snap or Travis, so I needed a VM or container to run the build in.

The how

Set up

stack has support for Docker to run the build and package the application, but it isn’t currently supported when using boot2docker (see these issues), so I used a simple Vagrantfile to start up a beefy VM:

Vagrant.configure(2) do |config|

  config.vm.box = "puppetlabs/centos-6.6-64-nocm"

  config.vm.provider "virtualbox" do |vb|
    vb.memory = "4096"
    vb.cpus = 8
    vb.customize ["modifyvm", :id, "--ioapic", "on"]
  end

end

Centos 6.6 might seem like an odd choice for a development environment, but the Vagrant box was already on my laptop, so using it saved me the download time (bandwidth is a precious commodity in semi-rural Australia).

I didn’t bother setting up provisioning properly, but here are the commands that are needed to get stack working:

# Add FP Complete repo & install stack
curl -sSL https://s3.amazonaws.com/download.fpcomplete.com/centos/6/fpco.repo | sudo tee /etc/yum.repos.d/fpco.repo
sudo yum -y install stack

# Add EPEL repo and install Docker
sudo rpm -iUvh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
sudo yum -y install docker-io

# stack requires the Docker client to be able to connect
# to the daemon without root, so add vagrant user to dockerroot group
sudo gpasswd -a ${USER} dockerroot

# At this point, you'll need to log out and back in for
# the group change to take effect

# Setup iptables to allow Docker links to work
sudo iptables -t filter -A DOCKER -d 172.17.0.0/16 -i docker0 -j ACCEPT

Adding the vagrant user to dockerroot comes with all the usual security issues, but I’m not too worried about that here. There are some other suggestions in the stack documentation for how to handle this.

Running the build

So, stack and Docker are now both installed. Enabling stack’s Docker integration on the VM is as simple as modifying the ~/.stack/config file:

[vagrant@localhost ~]$ cat ~/.stack/config.yaml
docker:
  enable: true

This gives the advantage of using Docker to run the builds on the VM, while not impacting the yesod devel workflow on OS X.

By default stack will use the fpco/stack-build repository to obtain an image to run the build in. This suits me, as there is also fpco/stack-run for running the binaries once they’re compiled. The build image includes everything that’s necessary for building the whole of Stackage, whereas stack-run is more trimmed down.

Running stack build kicked off the build in the Docker container, which completed successfully as all required dependencies (e.g. libpq and pg_config) were already in the image. Thanks to FP Complete for setting up these Docker containers for everyone to use!

Packaging Docker image

stack’s support for creating container images is a relatively recent addition, and currently isn’t covered well in the documentation. There is a reference to the image configuration section here, which was enough for my needs. After added the relevant bits, my stack.yml looks something like:

resolver: lts-3.20

packages:
- '.'

extra-deps: []

flags:
  mysite:
    library-only: false
    dev: false

extra-package-dbs: []

# This isn't required as it's set in the user config on the VM
# docker:
#     enable: true

image:
  container:
    name: mdjnewman/mysite
    base: fpco/stack-run
    add:
      config: /app/config
      static: /app/static

With that in place, running stack image container produced a Docker container ready to be pushed to Docker Hub:

[vagrant@localhost vagrant]$ stack image container
Sending build context to Docker daemon 34.25 MB
Sending build context to Docker daemon
Step 0 : FROM fpco/stack-run
 ---> db9b2a858ef5
Step 1 : ADD ./ /
 ---> e682c572d7ed
Removing intermediate container 1ce35ed1ccea
Successfully built e682c572d7ed

Running a regular docker push published the image to the registry.

Deploying to Digital Ocean

After all these steps, the final package was in Docker Hub and it was just a matter of running the following commands on the droplet, and the site was live:

docker run --name=postgres -e POSTGRES_PASSWORD=$(uuidgen) -e POSTGRES_USER=postgres -d postgres:9.3

# Yes, I know UUIDs don't make the best passwords :)

docker run              \
    -d                  \
    -w /app             \
    --link postgres:postgres         \
    -p 3000:3000        \
    mdjnewman/mysite    \
    /bin/bash -c 'PGHOST=$POSTGRES_PORT_5432_TCP_ADDR PGPASS=$POSTGRES_ENV_POSTGRES_PASSWORD /usr/local/bin/mysite'

The last command is running the Yesod server, with environment variables set to the values provided by Docker (see here for info about setting configuration variables in Yesod).

The results

I’m very happy with the Yesod workflow, and stack’s Docker integration makes deploying a lot easier as I don’t have to worry about what packages are in my build and test environments. Seems like a good compromise between manually copying files around and having a full CD pipeline.

Using stack build with a LTS resolver and an isolated container is also the holy grail of repeatable builds!

There is still a lot that would need to be done (improving security, backups, logging etc) before this was closer to a production environment, but it’s good enough as a development region.

This whole process (including writing this post) from having something that I wanted to deploy to being able to view the live site took less than a day, and I was learning a lot along the way. Deciding where was the best place to host the site and waiting for Docker to pull the fpco/stack-build image took up most of the time!

Simon Peyton Jones on maintenance with strong types

People don’t like fixing type errors, but they are a lot easier to fix than runtime errors… The single biggest thing going for static typing is not so much that it helps you write your program in the first place, but it helps you maintain your program.

GHC itself is an example – it’s now 150,000 lines of Haskell & twenty years old and I regularly refactor it in pretty large scale ways. I just wouldn’t dare do that if I didn’t have static typing to keep me sane. It’s this enormous code base, chunks of which I’ve forgotten about, and yet I can confidently make systemic changes to it because I know the type checker is going to catch all the places that change needs to go.

Simon Peyton Jones, speaking on Functional Geekery Episode 11.

The part about GHC is anecdotal evidence, to be sure, but it makes the point nicely.

Why Functional Programming Matters

The following is the conclusion from a paper entitled ‘Why Functional Programming Matters’:

In this paper, we’ve argued that modularity is the key to successful programming. Languages that aim to improve productivity must support modular programming well. But new scope rules and mechanisms for separate compilation are not enough — modularity means more than modules. Our ability to decompose a problem into parts depends directly on our ability to glue solutions together. To support modular programming, a language must provide good glue. Functional programming languages provide two new kinds of glue — higher-order functions and lazy evaluation. Using these glues one can modularize programs in new and useful ways, and we’ve shown several examples of this. Smaller and more general modules can be reused more widely, easing subsequent programming. This explains why functional programs are so much smaller and easier to write than conventional ones. It also provides a target for functional programmers to aim at. If any part of a program is messy or complicated, the programmer should attempt to modularize it and to generalize the parts. He or she should expect to use higher-order functions and lazy evaluation as the tools for doing this.

Of course, we are not the first to point out the power and elegance of higher-order functions and lazy evaluation. For example, Turner shows how both can be used to great advantage in a program for generating chemical structures. Abelson and Sussman stress that streams (lazy lists) are a powerful tool for structuring programs. Henderson has used streams to structure functional operating systems. But perhaps we place more stress on functional programs’ modularity than previous authors.

This paper is also relevant to the present controversy over lazy evaluation. Some believe that functional languages should be lazy; others believe they should not. Some compromise and provide only lazy lists, with a special syntax for constructing them (as, for example, in Scheme). This paper provides further evidence that lazy evaluation is too important to be relegated to second-class citizenship. It is perhaps the most powerful glue functional programmers possess. One should not obstruct access to such a vital tool.

J. Hughes, “Why Functional Programming Matters,” Comput. J., vol. 32, pp. 98–107, 1989.

This paper was written twenty years ago – the more things change, the more they stay the same. I’d urge any programmer (whether currently interested in FP or otherwise) to read this paper if you haven’t already.