Tag Archives: R

HeatTraKR – A Knime workflow for exploring Australian climate data

Recently, I decided to crunch some data from the Australian Bureau of Meteorology (which I’ll just call BoM) to assess some of my own perceptions about how the climate in my home city of Brisbane had changed throughout my lifetime. As always, I performed the analysis in Knime, a free and open software platform that allows you to do highly sophisticated and repeatable data analyses without having to learn how to code. Along the way, I also took the opportunity to sharpen my skills at using R as a platform for making data visualisations, which is something that Knime doesn’t do quite as well.

The result of this process is HeatTraKR, a Knime workflow for analysing and visualising climate data from the Australian Bureau of Meteorology, principally the Australian Climate Observations Reference Network – Surface Air Temperature (ACORN-SAT) dataset, which has been developed specifically to monitor climate variability and change in Australia. The workflow uses Knime’s native functionality to download, prepare and manipulate the data, but calls upon R to create the visual outputs. (The workflow does allow you to create the plots with Knime’s native nodes, but they are not as nice as the R versions.)

I’ve already used the HeatTraKR to produce this post about how the climate in Brisbane and Melbourne (my new home city) is changing. But the workflow has some capabilities that are not showcased in that post, and I will take the opportunity to demonstrate these a little later in the present post.

Below I explain how to install and use the HeatTraKR, and take a closer look at some of its  outputs that I have not already discussed in my other post. Continue reading

Confessions of a climate deserter

For so long, climate change has been discussed in Australia (and indeed elsewhere) as if it were an abstract concept, a threat that looms somewhere in the future. Not anymore. In 2019, climate change became a living nightmare from which Australia may never awake.

While I prepared this post in the dying weeks of 2019 and the beginning of 2020, there was not a day when some part of the country was not on fire. As at 24 January, more than 7.7 million hectares — that’s an area about the size of the Czech Republic — have burned. Thirty-three people have died. Towns have been destroyed. Old-growth forests have burned. Around a billion animals have been killed. Whole species have probably been lost.

The effects were not only felt in the bush. Capital cities such as Sydney, Melbourne and Canberra endured scorching temperatures while choking in smoke. Newspaper front pages (except those of the Murdoch press) became a constant variation on the theme of red. The country entered a state of collective trauma, as if at war with an unseen and invincible enemy.

The connection between the bushfires and climate change has been accepted by nearly everyone, with the notable exception of certain denialists who happen to be running the country–and even they are starting to change their tune (albeit to one of ‘adaptation and resilience’). One thing that is undeniable is that 2019 was both the hottest and driest year Australia has experienced since records began, and by no small margin. In December, the record for the country’s hottest day was smashed twice in a single week. And this year was not an aberration. Eight of the ten hottest years on record occurred in the last 10 years.  Environmentally, politically, and culturally, the country is in uncharted territory.

Climate deserters

I watched this nightmare unfold from my newly adopted city of Melbourne, to which which I moved from Brisbane with my then-fiancée-now-wife in January 2019. As far as I can tell, Melbourne has been one of the better places in the country to have been in the past few months. The summer here has been pleasantly mild so far, save for a few horrific days when northerly winds baked the city and flames lapped at the northern suburbs. It seems that relief from the heat is never far away in Melbourne: the cool change always comes, tonight or tomorrow if not this afternoon. During the final week of 2019, as other parts of Victoria remained an inferno, Melbourne reverted to temperatures in the low 20s. We even got some rain. It was almost embarrassing.

Finding relief from the heat is one of the reasons my wife and I moved to Melbourne. Having lived in Brisbane all of our lives, we were used to its subtropical summers, but the last few pushed us over the edge. To be sure, Brisbane rarely sees extreme heat. In summer, the maximums hover around 30 degrees, and rarely get beyond the mid-30s. But as Brisbanites are fond of saying (especially to southerners ), it’s not the heat, it’s the humidity that gets you. The temperature doesn’t have to be much about 30 degrees in Brisbane before comfort levels become thoroughly unreasonable. Continue reading

Looking for letters

In the posts I’ve written to date, I’ve learned some interesting things about my corpus of 40,000 news articles. I’ve seen how the articles are distributed over time and space. I’ve seen the locations they talk about, and how this shifts over time. And I’ve created a thematic index to see what it’s all about. But I’ve barely said anything about the articles themselves. I’ve written nothing, for example, about how they vary in their format, style, and purpose.

To some extent, such concerns are of secondary importance to me, since they are not very accessible to the methods I am employing, and (not coincidentally) are not central to the questions I will be investigating, which relate more to the thematic and conceptual aspects of the text. But even if these things are not the objects of my analysis, they are still important because they define what my corpus actually is. To ignore these things would be like surveying a large sample of people without recording what population or cohort those people represent. As with a survey, the conclusions I draw from my textual analysis will have no real-world validity unless I know what kinds of things in the real world my data represent.

In this post, I’m going to start paying attention to such things. But I’m not about to provide a comprehensive survey of the types of articles in my corpus. Instead I will focus on just one categorical distinction — that between in-house content generated by journalists and staff writers, and contributed or curated content in the form of readers’ letters and comments. Months ago, when I first started looking at the articles in my corpus, I realised that many of the articles are not news stories at all, but are collections of letters, text messages or Facebook posts submitted by readers. I wondered if perhaps this reader-submitted content should be kept separate from the in-house content, since it represents a different ‘voice’ to that of the newspapers themselves. Or then again, maybe reader’s views can be considered just as much a part of a newspaper’s voice as the rest of the content, since ultimately it is all vetted and curated by the newspaper’s editors.

As usual, the relevance of this distinction will depend on what questions I want to ask, and what theoretical frameworks I employ to answer them. But there is also a practical consideration — namely, can I even separate these types of content without sacrificing too much of my time or sanity? 40,000 documents is a large haystack in which to search for needles. Although there is some metadata in my corpus inherited from the Factiva search (source publication, author, etc.), none of it is very useful for distinguishing letters from other articles. To identify the letters, then, I was going to have to use information within the text itself. Continue reading