So it so happened that Ten News was on the T.V. at our place tonight (this is in Melbourne, Australia). Nothing remarkable until a story about 150 doctors issuing a warning that same-sex marriage poses “health risks”. The group behind this is called Doctors For The Family, which calls itself a “medical organisation to highlight the health aspects of marriage”. Ten just described them as “a conservative medical group”, a term that to me seems nonsensical. But I immediately thought “I bet it’s a Christian fundamentalist front”.

The Doctors For The Family web site is remarkably coy about who is behind it. But the group is run by the chap in the Ten story, who lists himself as the founder on his LinkedIn profile: Dr Lachlan Dunjey. From this and some other pages we learn some facts about Dr Dunjey that seem relevant:

• “He was the Senate candidate for the Christian Democratic Party at the 2004 federal election”
• Before scaring, sorry, educating us about the health risks of same-sex marriage he was campaigning for “freedom of conscience in health care”, a euphemism for permitting Christian medical professionals to refuse to perform abortions even when medically indicated.
• He says “we need to change people’s hearts and minds by bringing them into the Kingdom of God”
• He’s really upset about the depraved state of contemporary society, including such debauchery as alcohol being served at Christian wedding receptions.

Now, before I get accused of an ad hominem fallacy, it must be noted that none of this shows that Dr Dunjey’s claims are false. However, I do believe it gives us good reason to be very skeptical of his group’s claims, especially as to whether the term “medical” is merited.

The so-called medical evidence presented by Doctors For The Family seems very dodgy. Credible medical organisations generally don’t make claims about whether same-sex relationships are better or worse for children than hetero ones, simply because there’s no definitive evidence either way. Most studies seem to back the (to me, rather common sense) notion that when considering all the factors that effect child development, the sexual orientation of the parents doesn’t really register as significant.

Doctors For The Family is essentially another deceitful pseudo-scientific attempt to make fringe religious views appear scientific. Just like the Intelligent Design crowd, they start with a pre-determined conclusion and scour the globe looking for any scrap of data that supports that conclusion. In other words, they follow the scientific method, only exactly backwards. Just because this group is supported by some people with “Doctor” on their business cards doesn’t mean it deserves to be called “medical”.

Watching the Kony 2012 phenomenon unfold has been interesting. I’ve come to the conclusion that while the people behind they campaign are well-meaning, its not really useful or helpful. In other words, it’s pretty typical of campaigns to “raise awareness” these days.

It is a rather sad indictment on our mainstream media though. Kony 2012 did not reveal any facts that have not been public knowledge for some time. But get a slick and somewhat emotionally manipulative video to go viral and media outlets are suddenly falling over themselves to show their moral cred by promoting it. But the crimes of the L.R.A., while terrible, are not new. And horrifyingly, on the scale of violence and misery in Africa, the L.R.A. is actually small potatoes.

On the other hand, the reaction to Kony 2012 has definitely done some good. It has put the spotlight on a part of the world westerners usually pay no attention to, and started a genuinely useful conversation. People knowledgable about Uganda and surrounds have taken the chance to engage in the debate. To pick just one example, see the pieces by Michael Wilkerson in Foreign Policy and CiF.

The Arresting Officer

I can’t help but notice that many supporters of Kony 2012 seem to belong to another group: the “anti-war” crowd that has just spent roughly the last ten years decrying Western Imperialism and damning U.S. foreign policy as unmitigated evil. Am I the only one to find this ironic? If the world is to “stop” Kony, to which nation would the task fall? To which political leadership is the Kony 2012 campaign directed? Ban Ki-moon and his bureaucrats? Ha.

As the video shows, President Obama took action against Kony back in October last year. 100 troops were sent to advise the Ugandan forces who are hunting him down. Not huge, but it’s something, and doing more gets tricky very quickly. And while the video wants you to worry that the U.S. may pull out, there’s no indication of any plan to leave. At the time, there was no substantial public pressure to take action. There is no American interest at stake, no ulterior motive. Just a desire to bring one of the world’s worst war criminals to justice. How much time do you think Vladimir Putin has spent thinking about the plight of enslaved African children? What about Hu Jintao?

Why is the Kony 2012 campaign focused completely on U.S. policy? Or to switch to another example among many: when a Coptic Egyptian calls for aid for his beleaguered minority, why does he address his letter to the President of the United States? Why do the Syrians who are able to speak freely protest against Russia, not the great power some claim has “angered Muslims everywhere”?

Now, I’m not trying to defend every foreign intervention by the U.S. as noble or wise. My goal is just to give you something to think about next time you hear someone engage in cheap rhetoric about “U.S. Imperialism”.

Rush Limbaugh Bonus Feature

Just in case you missed this back in October, here’s Stephen Colbert covering Rush Limbaugh’s reaction to the President’s decision to go after Kony.

I’m going to attempt to explain my position on climate change, mostly for my own benefit, perhaps. The controversial and divisive nature of the climate change debate means that a staggering amount is written about it on a daily basis. What can an amateur like me hope to contribute that hasn’t been better covered elsewhere? I shall attempt to offer a useful perspective. But doing this right means doing it long, so sorry about that.

My Road to “Denial”

For most of my adult life, I have believed that human pollution of the atmosphere was causing destructive changes to the climate, and that this would lead to catastrophe if we didn’t mend our ways. I know I learnt about the greenhouse effect, and that anthropogenic CO₂ was making it stronger, in high school; no later than 1989, when I was 16. I recall playing the original Civilization PC game in circa 1991. In the late stages of the game, when industrialisation was in full swing, the planet would start heating up, resulting in some areas becoming deserts and others jungles. By adopting “low pollution” technologies, the climate change impacts could be reduced, but not entirely avoided.

I read Ben Elton’s books. I listened to what Greenpeace was saying. For a time I was a member of the Australian Democrats, who were the Greens before there were Greens. When I saw An Inconvenient Truth in 2005, I was thoroughly impressed, and – this is very embarrassing – considered buying a Prius. In the 2007 Australian federal election I voted Green and looked forward to the new Labor government finally doing something about climate change.

In other words, I was a Believer, someone who supported the environmentalist agenda. But not any more. I have actually surprised myself at how rapidly my views changed. I have seriously wondered: am I a total flake, lacking in any real conviction? I report, you decide.

My first break with environmental orthodoxy was when, in my capacity as an amateur science geek, I started reading about how nuclear power plants actually worked, in detail. It didn’t take very long to figure out that almost everything environmentalists said about nuclear power and radiation was complete garbage. Most nuclear advocates also push the consensus line on climate change, as they see it as an opportunity to resuscitate the industry in the west. But I found there were some people advocating nuclear power while also expressing skepticism of climate change claims. That seemed weird.

I was perplexed that the reality of Anthropogenic Global Warming (AGW) was not universally accepted. I started wondering what the skeptics were actually skeptical of. Did they doubt the planet was warming, or merely the influence of humans? I decided I needed to fully understand the skeptical arguments to better counter them. What I discovered was that the skeptical “side” actually consists of a somewhat maddening diversity of opinions regarding exactly which aspects of the consensus are wrong. The consensus side, by comparison, was remarkably uniform in its position.

For a layperson, getting into the contentious scientific issues of climate change is not at all easy. You’ll find arguments and counter-arguments over virtually everything, and it’s not easy to tell what is important and what can be ignored. The IPCC reports themselves are almost completely useless for gaining any understanding of the issues. I had expected that the reports were designed to inform the policy debate. But no, they’re purely technocratic – written by experts, for experts. The best approach I’ve found is to listen to what experts say the IPCC report says, then confirm that myself as well as I can.

Eventually, I was able to figure out what the skeptical case is mainly all about. The first thing to understand is that there is relatively little skepticism of AGW itself. Most skeptics accept that the evidence for AGW is strong, and that its very likely real. The skepticism is about the magnitude of the anthropogenic influence, and whether we are facing a catastrophe. A more accurate label for “Climate change skeptics” would “climate catastrophe skeptics”. No one (fringe religious nuts excluded) is denying that the climate is changing. One of the central planks of the skeptical argument is that the climate is always changing, and we should not necessarily be concerned when we observe this.

The basic argument for AGW goes like this:

1. The average temperature of the planet has increased since circa 1940
2. Anthropogenic CO₂ emissions have risen steadily since circa 1940
3. Adding atmospheric CO₂ makes the greenhouse effect stronger, which makes the planet warmer
4. Ergo, anthropogenic CO₂ emissions are warming the planet

When a consensus person is impressing you with the strength of their evidence, they are usually supporting the above points. And when they ridicule the “denial” of science by skeptics, they are usually saying that skeptics deny the above. The thing is that most skeptics accept the above points. Now, there are valid scientific debates about each of these items, believe it or not. It’s not all completely settled. But overall, there seems to be agreement across both skeptics and warmists that the above points have pretty solid evidence to support them. Skeptics have spent much effort trying to show that point 1 is not as strong as we’ve been led to believe; but while there is undoubtedly problems with the instrumental temperature record, the bottom line conclusion seems to have held up.

So, it’s case-closed? Shut down Hazelwood and cover the land in wind farms, right? No, it’s no where near that simple. We have a huge, genuinely incontrovertible, mass of evidence that the earth’s climate changes all the time, all by itself, without any human influence. Of course, whatever natural climate effects were present in the past didn’t just disappear when humans started burning coal, oil and gas. So all the climate observations we make involve some combination of natural and anthropogenic influence (“forcings” in the lingo). In order to know what we’re actually dealing with, we need to understand the strength of the natural vs the anthropogenic forcings. This is called attribution, and this is where it gets really messy.

It’s when I started looking into attribution that started doubting the consensus position. There’s a bunch of details involved that seemed very important and relevant to me, but that I’d never heard any environmentalist, greenie or consensus scientist even hint at. First thing: the greenhouse warming from CO₂ is logarithmic. This means that as CO₂ is added to the atmosphere, the impact of adding more CO₂ becomes less and less. The standard analogy is to think about painting over a window: the first coat blocks a lot of light, the second coat blocks less light, and so on. Now, I don’t want to over-emphasise the importance of this – the IPCC fully takes it into account – but it was a shock to me because I realised I didn’t really understand the fundamentals of AGW. My diet of environmentalist propaganda had led me to believe that all those megatons of CO₂ were going to cause Venus-like runaway warming. Nope.

Next surprise: the ice cores. I can only recall one slide in Al Gore’s entire presentation that dealt with attribution: the slide showing the temperature and CO₂ graphs from ice core data. He then said:

The relationship is actually very complicated but there is one relationship that is far more powerful than all the others and it is this. When there is more carbon dioxide, the temperature gets warmer, because it traps more heat from the sun inside.

And indeed the graphs on the slide show a very close correlation between CO₂ and temperature. But I learned that Mr Gore was deliberately misleading his audience with this slide. What did he mean when he said the relationship is “very complicated”? Well, what the ice cores actually show is temperature increases happen roughly 800 years before increases in the CO₂ level. This is not mysterious, it’s basic physics that as the oceans warm up they will release more of their dissolved CO₂ into the atmosphere. So it’s actually not that “complicated”, Mr Gore just didn’t feel like explaining it.

The ice cores were once one of the best pieces of evidence that CO₂ had a strong influence on temperature. Initially the resolution of samples was not so high, and it did indeed appear that CO₂ and temperature increased together. Then scientists were able to improve the resolution and see that temperature changed before CO₂. The consensus now tries to act like this was totally expected. The claim now is that the planet starts warming (due to a cause we can only speculate about), this causes more CO₂ to enter the atmosphere, which then amplifies the warming (this is somehow not speculation). I find the best way to evaluate claims like this is to imagine if the result had come out the other way; if the cores had shown temperature increasing after CO₂, do you think they would say “oh, we didn’t predict that!” Of course not, it would be (with some justification) trumpeted as evidence of CO₂ being a major driver of climate change. The ice core data presents other problems for theories of CO₂-driven climate catastrophe: despite the “amplification” going on, the warming somehow stops long before anything catastrophic happens. And when the temperature falls, CO₂ levels stay high for over a thousand years. A far simpler explanation for the ice core data is that CO₂ levels respond slowly to temperature changes, but do not significantly drive temperature changes.

Sensitivity and Feedbacks

Next shock: when it comes to the total greenhouse effect of the earth, CO₂ is only a minor contributor. The most significant player is water vapour, by quite a margin. So how exactly does CO₂ act as the climate bogey-man? (Someone familiar with climate science might say “duh, climate scientists have always acknowledged the importance of water”. That is so, but I had listened to a lot of alarmists and never heard it mentioned.)

At this point, I have to introduce the notion of climate sensitivity. This is not a thing that is part of the physics of the climate, it’s basically a rough measure of how much warming we get from increasing CO₂ levels. For laypeople who are mainly interested in policy implications, the sensitivity is usually expressed as the temperature increase caused by doubling CO₂, both relative to pre-industrial levels (the doubling will occur circa 2070 assuming current trends). The last IPCC report says the climate sensitivity is probably 3.3℃, but the range of possible values is 2.0℃ to 4.5℃. The skeptics think it is significantly lower. If you want the science of the climate change debate boiled down to its essence, it is the determination of this value.

Now, it turns out we can use laboratory experiments to work out a reasonably accurate (for climate science) value for how much warming we get from adding CO₂. A climate sensitivity of around 1.2℃ from CO₂ warming alone is widely accepted by both warmists and skeptics. Huh? That’s a lot less than 3.3℃.

The discrepancy is because the lab experiments measure the CO₂ effect in isolation, but in the real climate all forcings like this are subject to feedbacks. We know of the existence of many feedbacks, both positive and negative, in the climate system. The problem is we have very little idea of how strong any of them are compared to the others. One of the major feedback components is water vapour in the atmosphere. The IPCC says that when increased CO₂ levels make the planet slightly warmer, this causes the atmosphere to hold more water vapour. Water vapour is the strongest greenhouse gas, so the earth gets even warmer. Of course, I’m leaving out a huge amount of detail in all this, much of which I don’t completely understand. But I think it’s correct to say that water vapour feedback constitutes the bulk of the warming the IPCC says we’ll get beyond the 1.2℃ direct forcing.

I just want to take a quick aside to emphasise how completely, utterly, totally bogus it is to refer to CO₂ (or even worse, “carbon”) as “pollution”. If you accept the argument that the mere fact it’s possible to accumulate too much of something means it should be called a pollutant, then you have to be consistent. Most of the warming the IPCC predicts comes not from CO₂, but H₂O. Therefore, water is a pollutant, and a more significant one than “carbon”. Reductio ad absurdum.

The big issue with water vapour in the atmosphere is that it will sometimes form clouds. Clouds can both warm and cool the earth’s surface. This means that adding water vapour to the atmosphere produces both positive and negative feedbacks. I think it’s fair to say that there’s few aspects of the climate that we understand less well than clouds. The bottom line is we can only speculate about what the net water vapour feedback is. We can’t even say with any certainty whether the feedback is negative or positive overall.

So in order to know what the climate sensitivity is, we need to know how the feedbacks work. But we don’t know that, so we use various means to try to estimate the likely range. I think you can pick any value for the climate sensitivity between 0℃ and 6℃ and you’ll be able to find one or more peer-reviewed scientific papers that say that could be the value. The IPCC’s estimate of 2.0℃ to 4.5℃ seems to have been arrived at almost entirely via modelling. This range requires net feedbacks to be positive.

The sign and magnitude of the feedbacks is the difference between major catastrophe and an effect so small it cannot be noticed against natural variation. But the direct empirical evidence for a net strong positive feedback is virtually non-existent. Models, combined with the argument “we know of no other cause of 20th century warming” (I’m not making that up, they actually say that) are all that supports it.

On the other hand, there are increasingly good reasons to think the net feedback is slightly negative. For one, almost all natural systems that have reached a stable or meta-stable state are governed by negative feedback. The fact that the earth’s climate doesn’t constantly careen from one extreme state to the next indicates that negative feedback probably dominates (it’s also worth noting another factoid never spoken of by the consensus – over the entire history of the earth, today’s CO₂ levels are relatively low). And there’s the fact that the warming predicted is just not eventuating. For over ten years now, the various temperature records, surface, satellite, ocean, have all been flat. All while CO₂ levels have continued to rise.

The way the consensus is sold is by focusing entirely on the strong part of their theory – that human CO₂ emissions make the planet slightly warmer. But the chain of reasoning should be judged by the weakest link, which is that the warming will be catastrophic due to positive feedback.

Climate Models

Climate models are just computer programs. I don’t pretend to understand all the details of how they work, but I will attempt to give a fair summary. They are written to simulate what we know and what we think we know about the physics of the earth’s climate. They take into account all known climate forcings (solar activity for example) and all known feedback mechanisms. You end up with a big pile of parameters in the model, the values of which cannot be determined by observation with any great certainty. These parameters are set by a combination of simple assumption, combined with tuning the output of the model to match certain historical records. The IPCC (ok, not them directly, the scientists whose papers they rely on) then re-run the models with anthropogenic CO₂ left out, find that the results don’t match history, and from this conclude that their assumptions of positive feedback are correct and that CO₂ is the main driver of global warming. And this is how the value for climate sensitivity is arrived at – the models say that earth with CO₂ doubled is 3.3℃ warmer than earth with CO₂ unchanged.

You may be thinking that I’m either lying or mistaken here, because that sounds completely bloody crazy. It’s simply begging the question. Well I’m not lying, but I could be mistaken. Take a look at the critical IPCC chapter on attribution, especially pp 702-703 and decide for yourself if I’m being fair.

I haven’t yet heard a convincing argument as to why using models like this is evidence of anything. A telling sentence on page 702 referenced above is “Numerous experiments have been conducted using climate models to determine the likely causes of the 20th-century climate change”. The word “experiments” here is quite misleading, as in the scientific context it implies you’re doing something empirical. But all climate models do is answer the question “if I assume the climate works like this, what are the implications?” This is undoubtedly useful, but it can’t tell you if the assumptions you start with are correct. Model outputs are not observations.

But if the climate model produces results that match real-world historical data, doesn’t that prove the model’s assumptions are basically valid? No. A quote from John von Neumann sums it up: “With four parameters I can fit an elephant, and with five I can make him wiggle his trunk.” Climate models have many parameters. Like the relative strengths of various feedbacks. But the models also need to account for periods where CO₂ rose and temperatures fell (the 1970s for example). That’s where some of the negative temperature forcings come into play, the principle one being “aerosols”, which are mainly genuine pollutants like sulphates. We know that aerosols make the earth cooler overall (sometimes called “global dimming”) but, again, there’s huge uncertainty as to the strength of the effect. Because various positive and negative parameters are so poorly constrained by observation, it’s just a matter of tuning them until your model roughly matches the real-world data. The IPCC uses many climate models, each with different assumptions and producing slightly different future “scenarios”. Obviously they can’t all be correct, but they all match historical data to roughly the same degree.

The Bottom Line

The heart of the scientific debate about climate change is the issue of attribution. It is a real debate, concerned with real evidence. It is simply not true that the only people not going along with the consensus are few science-denying fools or paid shills for the fossil fuel industry. I didn’t learn about these issues on my own, there’s a great many knowledgable people who have come to doubt what climate change activists and the IPCC are telling us. There are very good reasons to doubt the consensus claim that almost all 20th century warming can be attributed to anthropogenic influence.

What should we do? Well, we can’t decide on a policy based on climate science alone, we have to examine the available options and their costs. That’s a whole new rabbit hole, so I won’t get into it now.

I would, however, like to address some of the common comebacks to the type of argument I’ve laid out above. When doubts about the consensus scientific position are raised, usually only professional scientists understand the details will go into bat and defend it. But most people who are advocating “action” on climate change aren’t really able to do this. Instead, it’s amazing how quickly they will declare the truth of climate science claims as basically irrelevant because of some other overriding reason.

1. We Should Look After the Environment

“It doesn’t matter if climate change science is wrong, because we have to look after the environment anyway”. This argument is actually at the heart of the climate change hysteria. Whenever the catastrophic climate change barrow is pushed, it is invariably linked to pollution and environmental problems generally. But our very real environmental challenges are not causing climate change, nor are they caused by it. Emissions from cars and coal power stations certainly are dirty and can cause environmental problems, but it’s not the dirty or toxic stuff that makes the greenhouse effect stronger. Neither do oil spills or the myriad of other ways we manage to screw things up. Whatever the merits are of the clean-up-the-planet argument are, it is a separate issue to what’s going on with the greenhouse effect. Alarming people with tales of climatic disaster is not an honest way to pursue a clean air agenda.

Australia’s “Carbon Price” is sold as the means to a “clean energy future”. So with all the nasty things emitted by fossil fuel combustion (there are many), we put a price on an emission that is not dirty or toxic, and was abundant throughout the biosphere before the first human drew breath. Yeah, that totally makes sense as a clean air policy.

Just a quick note on something you’ve probably heard about: that increasing CO₂ levels are making the oceans “more acidic”. The scare mongering about ocean acidification is stopped in its tracks by the elementary observation that the average pH of the oceans is about 8.14. That’s right, the world’s oceans are alkaline. So while it’s technically correct that increased CO₂ has made the oceans more acidic, it’s quite misleading. It’s more honest to say that the oceans are becoming more neutral, that is, less corrosive. Claims about the dangers of ocean acidification are just another desperate attempt to make CO₂ into a dangerous pollutant.

2. Fossil Fuels are Bad

Related to the previous argument, there is the “it doesn’t matter if climate change science is wrong, because fossil fuels are bad and we should stop using them”. There are indeed plenty of downsides to fossil fuels (and some major upsides too). However, this is again a separate debate from climate change. The various carbon pricing and emission reduction schemes were not sold on the basis that fossil fuels have downsides, they were sold based on predictions of drowned coastlines, dead coral reefs and endless droughts. If you want to make the argument for the negatives of fossil fuels, then make the argument on its own merits, don’t try to force your agenda with scare tactics about the climate. Oh and having a remotely practical alternative might help, too.

3. Idiots whom I hate are climate change skeptics

I’ve been really surprised how often I’ve heard some variation of this: “I can’t be a climate skeptic, because that would mean agreeing with: Tony Abbott, Andrew Bolt, Glenn Beck, assorted conservative religious nuts, et alia”. I have to admit that I have fallen victim to this line of “reasoning” myself in the past, and now do my best to guard against it. Forming your opinion on a matter based on the opinions of people you don’t respect is manifestly stupid. Need I say more?

4. We should act as a precaution

The argument goes: “because we can’t definitively prove that anthropogenic emissions are not going to destroy the world, we should play it safe and cut emissions”. This will often be backed up with some demagogic nonsense such as “we should not gamble with our children’s future”. The flaw in this argument becomes obvious when it is applied to an issue other than climate change: “because we can’t definitively prove that an asteroid is not going to destroy the world, we should play it safe and spend trillions of dollars on an asteroid defence system”.

A another deliciously silly argument I’ve often seen attached to this is that the “only” cost of such precautionary action is money. Right. Money. The thing we use to represent our available resources. The resources we use to sustain, grow and advance our civilisation. Oh, if that’s the only cost, I don’t know what all the fuss is about. The best part is that if you ask the person making this argument “should we spend more on education/health care/social justice/foreign aid/anything except defence”, they’ll say “yes” to every one without a hint of irony.

What’s lacking here is any assessment of risk and cost. Most environmentalists seem to avoid getting into any hard numbers concerning risk and cost, likely because their ideas do not come out well. That’s a topic for another post, however.

5. The Glaciers, they are melting!!!

This is the standard response Al Gore comes out with in the rare event he gets challenged on anything he says: (paraphrasing here) “Ice caps are melting, polar bears are dying, sea levels are rising, the reality of climate change cannot be denied!”. To the extent these all these claims are not exaggerations or equivocal or simply untrue, they do not say anything about attribution. The most they can tell us is what climate changes are occurring, but not why.

There is the ongoing, rather desperate, attempt to portray the changes we observe as “unprecedented”, as this would indeed be circumstantial evidence of an influence from human industrialisation unique in the world’s history. The problem is that we have very little idea of what is “precedented” in terms of climate changes. Considering the number of climate phenomena we have only been observing closely for a few decades, it is hardly surprising that we see changes without precedent.

A chap named ittayd has written a nice gentle tutorial on functional programming in Scala. My only minor complaint is that he calls the applicative functor “sequential application” function apply. While this is a logical choice, in Scala apply is used everywhere to implement plain old function application, so it could lead to confusion. I think it’s better to call it <*>, as in Haskell. Read on.

I found the tutorial interesting because it talks a bit about applicative functors, and how to give them a nice syntax in Scala. The one thing that has bugged me about applicatives is that using them in Scala always felt clunkier than Haskell. Ittayd comes up with a scheme I like, but says “we have one difficulty: we can no longer define apply inside Future”. He uses a standard object-pimping implicit conversion to get around this, and that works fine. However, it is possible to define <*> inside your thing-with-an-applicative-functor, using one of my favourite Scala features: generalized type constraints. To continue with ittayd’s example using Future:

trait Future[A] {
  …
  def <*>[X, Y](x: Future[X])(implicit ev: A <:< (X => Y)) =
  new Future[Y] {
    def get = Future.this.get(x.get)
    def isDone = Future.this.isDone && x.isDone
  }
}

The compiler will only allow <*> to be called on a Future that provides an appropriate function as its value.

Ittayd bootstraps things simply by lifting the function to be applied into Future, but personally I prefer the Haskell style of using <$>, aka “map”. In Scala, <$> is unsuitable for a few reasons, so I picked **::

trait Future[A] {
  …
  def **:[B](f: A => B): Future[B] = map(f)
}

Obviously, this is just the standard map method with a different name, what’s the point? Well, because the operator ends with a :, it’s right associative; and because it starts with an *, it has higher precedence than <*>; so you end up with:

(marry _).curried **: joe <*> jane

Or, if you subscribe to the slippery slope theory of gay marriage:

(gayMarry _).curried **: joe <*> bob <*> george <*> fido <*> aDonkey

This is reasonably close to the Haskell idiom:

gayMarry <$> joe <*> bob <*> george <*> fido <*> aDonkey

You may wonder why obsess over the best way of dealing with applicative functors? After all, most of the applicative functors you see have an associated monad, so we can use Scala’s nice for syntax. Yes this is true for many useful applicative functors, but not all of them.

Just a quick aside on termonology. In casual conversation, you may hear it said that “Option is a monad”. But it’s more accurate to say “Option has a monad”. Because many types only have one useful monad associated with them, we tend to think that the type itself is the monad, but that’s not quite right. This doesn’t just apply to monads, of course, the same goes for monoids, applicative functors, etc. For example, numeric types have two useful monoids: one for addition and one for multiplication.

Now consider a Validation type from this Tony Morris tutorial. You can write a useful monad for this type, which implies an applicative functor as well. However, the tutorial is concerned with a second useful applicative functor for Validation. This second applicative is only relevant for the case where the Validation is using a semigroup as its error type (the tutorial explains what a semigroup is), and it allows using the semigroup to accumulate a list of all the validation failures. The classic example of this is validating data entered by a user on a form with several fields. The “plain” applicative/monad will only record the first validation failure it finds, whereas the semigroup applicative will record all the failures.

Now, finally, to my point (I think). This applicative functor that uses semigroup does not have a corresponding monad. Which means that we can’t use it with Scala’s for syntax. Hence the desire for a nice syntax for applicative functors. It is most definitely useful.

Obligatory Scalaz reference: those guys have pretty much worked out the best way to do all this stuff in Scala. But I find I learn a lot by experimenting myself.

I’ve been using Scala’s Either type a bit recently, and found a couple of minor annoyances with it. So as a learning exercise I tried to make a variation of Either, and this is what I came up with. I also wanted to experiment with syntax a little. See what you think.

Either is a disjunctive type with two possible values: a left and a right. It places no meaning at all on left or right, but 99% of its uses in practice involve putting normal, expected values in right, and errors or other “exceptional” values in left. This means that almost all the time you end up getting the RightProjection, e.g.:

for {
    v1 <- foo(…).right
    v2 <- bar(…).right
    v3 <- baz(…).right
}
yield …

Whereas I’ve never used .left even once.

The second minor issue is that Either has some limitations with patterns in for expressions, because of how it is forced to implement the filter method. See this Stack Overflow question for the details.

So I created a type that is explicitly designed to handle the possibility of “exception” conditions. Because the word “exception” already has a well known meaning on the JVM, I called this type Anomaly instead. Actually, I ended up calling the type |: because I thought it worked nicely syntactically. Here’s a comparison with Either:

val e1: Either[String, Int] = Right(30)
val e2: Either[String, (Int, Int)] = Right((10, 2))
val e3 = for {
    i <- e1.right
    j <- e2.right
}
yield i + j._1 + j._2
val e4: Either[String, Int] = Left("error")

vs “Anomaly”:

val a1: String |: Int = Expected(30)
val a2: String |: (Int, Int) = Expected((10, 2))
val a3 = for {
    i <- a1
    (j, k) <- a2
}
yield i + j + k
val a4: String |: Int = Anomaly("error")

The |: operator is right-associative, which helps for the occasional cases where you have an Anomaly within an Anomaly, eg. String |: String |: Int is equivalent to Either[String, Either[String, Int]]. Of course, depending on your taste you could reverse the order of type arguments, using an operator like :|, or use a name instead of an operator.

There’s a cost to making the for expression pattern matching work like this, however. In addition to the Anomaly and Expected cases, I had to introduce a NoValue case object, which is what is returned when a filter does not match. This is annoying, because now folding or pattern matching must account for three cases instead of two. And unless you have explicit filters or patterns matching particular values, the NoValue outcome won’t come up, so it’s a pain to have to check for it. What I ended up doing was providing an alternate two-parameter fold method that calls error in case of a NoValue. You just have to be careful about only calling it when you know it’s safe.

All-in-all I’m still wondering whether this experiment is worthwhile, but here’s the code in case anyone is interested. There are other utility methods that could be added:

sealed abstract class |:[+A, +E] {
  def anomaly: A
  def expected: E
  def isExpected: Boolean = false
  def isAnomaly: Boolean = false
  def isNoValue: Boolean = false

  def map[X](f: E => X): A |: X

  def flatMap[X, AA >: A](f: E => AA |: X): AA |: X

  def withFilter(f: E => Boolean): A |: E = this
  final def filter(f: E => Boolean): A |: E = withFilter(f)

  def foreach(f: E => Unit): Unit = {}

  final def fold[X](fExpected: E => X, fAnomaly: A => X, fNoValue: => X): X = this match {
    case Anomaly(a) => fAnomaly(a)
    case Expected(e) => fExpected(e)
    case NoValue => fNoValue
  }

  final def fold[X](fExpected: E => X, fAnomaly: A => X): X = this match {
    case Anomaly(a) => fAnomaly(a)
    case Expected(e) => fExpected(e)
    case NoValue => error("Attempt to fold NoValue case")
  }

  final def join[X, AA >: A](implicit ev: E <:< (AA |: X)): AA |: X = this match {
    case Anomaly(a) => Anomaly(a)
    case Expected(e) => e
    case NoValue => NoValue
  }
}

final case class Expected[+A, +E](expected: E) extends (A |: E) {
  override def isExpected = true
  def anomaly: A = Predef.error("Not an anomaly")

  override def withFilter(f: E => Boolean) =
    if (f(expected)) this else NoValue

  def flatMap[X, AA >: A](f: E => AA |: X) = f(expected)

  def map[X](f: E => X) = Expected[A, X](f(expected))

  override def foreach(f: E => Unit) {
    f(expected)
  }
}

object Expected {
  def cond[A, E](test: Boolean, expected: => E, anomaly: => A): A |: E =
    if (test) Expected(expected) else Anomaly(anomaly)
}

final case class Anomaly[+A, +E](anomaly: A) extends (A |: E) {
  override def isAnomaly = true
  def expected: E = Predef.error("Not an expected value")

  def flatMap[X, AA >: A](f: E => AA |: X) = Anomaly[AA, X](anomaly)

  def map[X](f: E => X) = Anomaly[A, X](anomaly)
}

object Anomaly {
  def cond[A, E](test: Boolean, anomaly: => A, expected: => E): A |: E =
    if (test) Anomaly(anomaly) else Expected(expected)
}

case object NoValue extends (Nothing |: Nothing) {
  override def isNoValue = true
  def anomaly = Predef.error("Not an anomaly")
  def expected = Predef.error("No value")

  def flatMap[X, AA >: Nothing](f: Nothing => AA |: X) = this

  def map[X](f: Nothing => X) = this
}

A couple of days ago, Gavin King announced that he is leading a team at RedHat that’s creating a new JVM language called Ceylon. See here for most of what is known so far. The stated motivation, in a nutshell is that they like Java very much, but it has deficiencies that are preventing progress in key areas, and they are generally feeling “frustrated”. They don’t have a complete compiler yet, but they’re working on it, and I get the impression RedHat is pretty serious about it.

Gavin King’s slides provide a fascinating insight into the kind of thinking that remains so dominant in the Java community. Gavin King is well known in the Java world as the inventor/leader of Hibernate and various other JBoss projects. He certainly knows Java inside out, and his list of “frustrations” is astute, if incomplete. While I haven’t dug into all the details so far revealed about Ceylon, but it seems likely that they’ll succeed in their goal of creating “a better Java”. If given the choice between using Java and Ceylon, I could certainly see myself choosing Ceylon. But Ceylon is a terrible idea. The problem is that they are treating the symptoms, not the disease; and they’re repeating exactly the same mistakes that resulted in them finally hitting a dead-end with Java.

Possibly the best outcome of Ceylon is that it emphatically refutes the argument that “Java is all we need”. Few have put more hard labour into trying to make Java feasible than Gavin King and the JBoss folks. They and a few others have spent the last decade or so producing a staggering number of bloated tools, libraries and frameworks in order to keep Java going. Don’t get me wrong, there’s a lot of extremely useful Java libraries out there, but there’s also a huge amount of guff that exists solely to compensate for the impracticality of the Java language. Ceylon is the “main-stream” admitting, finally, that not all problems can be solved the Java way: more frameworks, more libraries, more tools, more code.

So Mr King and friends have finally hit the wall with Java. They’re a bit late to the party, but we should welcome them none the less. So why do I say Ceylon is a terrible idea? The short answer is: Scala. The long answer is, well, longer. My biased summary of the situation is this: after years of having to build more and more layers of increasingly baroque frameworks in order to make progress, they realise that they can no longer keep up with their competitors (C#) because Java the language is an unavoidable bottleneck. They identify certain essential abstractions that are needed to solve their immediate problems, and proceed to design a new language that provides those essential abstractions, while remaining as “familiar” as possible.

In other words, Ceylon is all about making “a better Java”. There is certainly support for a evolutionary approach like this. Some argue that Java itself was (in language terms) “a better C++”, so why not follow that successful approach now? Well, if that worked so well for Java, why are even its greatest fans now trying to replace it? I’m reminded of Linus Torvalds' reaction to “Subversion is a better CVS”. The problem with Subversion isn’t that is poorly designed or built; it isn’t. The problem with Subversion is that the goal itself just isn’t very useful.

This is how it’s going to go: Ceylon will be built, they’ll make a SDK library, then there will be Hibernate-Ceylon, Spring-Ceylon, WebFrameworks-Ceylon 1 through 63,854, Logging4Ceylon, Ceylon.util.logging, commons-logging-ceylon and Simple Logging Facade 4 Ceylon. All these will be an improvement over their Java equivalents. Then, in another five or so years they’ll hit some new pain points and realise that some of those other “academic” abstractions are actually useful in the real world after all. For example, Ceylon supports higher-order functions (but, inexplicably, not lambda expressions). If we could go back in time 5 years and ask Gavin King “how important do you think higher-order functions are?”, what would he say? My guess is “they’re not essential for Real Work™”.

But it’s still an improvement, so why not embrace it? Because we have an already have a much better option that’s already doing the job in the real world: Scala. Or look at the interesting stuff going on in .NET land. Ceylon’s goals look positively narrow and uninspiring by comparison. Of course, Gavin King was immediately asked “why not Scala?”. It is my sincere hope that this question will continue to be asked of him. His answer consists of the same misconceptions that Scala advocates have been trying to dispel since forever. Take this statement from Mr King, from an interview in InfoQ:

One important force that helped make Java so successful was the things Java left out. But you need to be very smart here: Java’s decision to leave out higher-order function support was enormously harmful. Now, obviously the Scala team have gone down a very different path here and have created a very feature-rich type system. As a general rule I would prefer to solve my problems with fewer, more general constructs, than with more, less powerful constructs.

This is the exact inverse of the truth. No one who has made a serious effort to understand Scala could make such a statement. “Fewer, more general constructs” is exactly what Scala is all about! My summary of Scala’s mission statement would be as follows:

• Scala aims to maximise the potential for useful abstractions, whilst:
  • giving assurance that your abstractions are correct (static typing)
  • allowing you to do what is necessary to get the performance you need
  • maintaining seamless interoperability with Java

Scala aims to be compatible with Java, but it’s about much more than just “a better Java”. Ironically, it seems as though Ceylon will not interoperate with Java as seamlessly as Scala does (e.g. Scala keeps null and type erasure). It’ll be interesting to see how that works out. To get an idea of how serious the Scala team is about powerful but practical abstractions, look at how they handle the tensions that crop up between abstraction and performance. Great examples of this are how Scala deals with the unpleasantness of arrays on the JVM, and also the @specialized annotation.

Scala is by no means perfect. The trade-offs it makes will not suit everyone fully. But I cannot help but scratch my head over statements like this (Gavin King, again):

But I guess I should mention that the number one technical problem that we simply can’t solve to our satisfaction in Java – or in any other existing JVM language – is the problem of defining user interfaces and structured data using a typesafe, hierarchical syntax.

I could very well be missing something, but this would seem like a problem that is right in Scala’s wheelhouse. Perhaps it may not be solved satisfactorily today, but it could done with far less effort than creating a whole new language (and new standard library, and new tool chain…). It is depressing to think of what these guys could achieve working with Scala instead of pouring resources into Ceylon.

You may wonder, “if you think Scala is so good, then use it and be happy, why worry about Ceylon?” Well, I worry because I think Ceylon is worse than Scala, but it could win anyway. That actually seems to be the more common outcome in these situations. I would much prefer a world with Scala jobs in demand than one with Ceylon jobs in demand. So, yes, it’s all about me being selfish. I would say to all Scala fans: don’t be afraid to be a little selfish and evangelise for the better outcome.

I’ve noticed this more and more: people lumping “climate change denial” and “evolution denial” (creationism) into the same category. The argument is that both of these are the result of the same religious conservative “anti-science” movement. In a very narrow sense, this is true; creationists generally do not seem to believe in anthropogenic global warming (AGW), for the same bogus reasons they don’t believe in evolution. But the reverse is not true: many (most I would say) AGW skeptics are not creationists. Lumping these two things together allows the environmentalists to dismiss any criticism of the established climate change narrative as the ravings of anti-scientific religious nuts. But there are some very significant scientific and economic arguments around climate change that are most definitely not in the same class as creationist arguments. Time for some de-lumping.

One major difference between climate change and evolution is how the evidence is understood and presented. It is now a familiar experience to see someone on TV telling us we need to immediately reduce emissions to avoid climatic catastrophe. In the rare event that such a person is asked to provide evidence of a human cause, the most common answer I’ve observed is along the lines of “ice caps and glaciers are melting, and the scientific consensus is that human CO₂ emissions are to blame.” Now, when was the last time you heard someone asked for evidence supporting evolution to respond with “life forms adapt to their environment and the scientific consensus is that Darwinian evolution is the cause.” This is not a small point. The fact that “consensus” is so regularly rolled out to support AGW, but is never used to support evolution shows that the levels of scientific understanding and debate are completely different.

No one talks about the “scientific consensus” for evolution because there’s no need to: evolution has a mountain of cold hard facts on its side, all confirmed by independent observation countless times. Climate research is very tough by comparison: our observational capabilities generally aren’t good enough to falsify competing theories. Instead, we have the IPCC saying that since we can’t establish the facts by direct observation, we’ll establish them by consensus. You can’t take the IPCC publications, repeat the experiments and confirm their results, because at the end of the day it’s just opinion.

My point here is not to show that the IPCC is wrong, but simply to show that the scientific process that it follows is fundamentally and necessarily different to that of evolutionary biology, or any other hard science for that matter. Questioning the consensus-derived “facts” of climate research is not the same thing as questioning the empirically verified facts of evolution.

Another difference I think is worth pointing out is how politics is so commonly mixed with climate science, in contrast to evolution science. Tell me if this scenario sounds familiar: well credentialed scientist goes on TV and says “my research shows that climate change is likely to significantly damage coral reefs…” and barely pausing for breath continues “so it’s clear we must act to cut CO₂ emissions.” This casual connection of minor scientific research to sweeping reorganization of the world’s energy production is now so common place that people don’t find it remarkable. And if you accept the climate change problem but doubt the solution you’ll likely still receive the “denier” pejorative. Evolutionary biology simply isn’t political in this way.

Skepticism regarding climate change is not comparable to the denial of the established facts of evolution. Climate research has a long, long way to go before it achieves the level of confidence we have in evolution. It is very wrong to think that all opposition to the climate change narrative is driven by Biblical literalism.