This is a more technical post than I usually write, but it will be useful to some people. My political science research involves some natural language processing and machine learning, which I use to analyse texts from Japanese newspapers and social media – so one of the challenges is teaching a computer to “read” Japanese. Luckily, there are some tools out there which make this (relatively) straightforward.

For this guide, I’m using Python 3.5. My development system runs on macOS 10.12 and I deploy my code to a server running Ubuntu 12.16, so this guide will include commands for setting up the software on both macOS and Linux/Ubuntu. I don’t use any version of Windows so I don’t know how to set this up on Windows – if anyone can provide a set of Windows commands that achieve the same goal, drop me an email and I’ll add them to the blog post.

What are we trying to accomplish?

The first hurdle to doing any analysis of Japanese text is segmentation, or tokenisation; breaking it down into usable chunks. For European languages, words have spaces between them, so you can just divide everything up at the word boundaries (the spaces, commas, full stops and so on), which yields an array of words that we can use to calculate things like frequencies or co-occurrence matrices. Japanese, however, has no spaces in its text, so there’s an extra pre-processing step required before we can start using these text analysis approaches.

In essence, we want to turn a string like this…

"今日はいい天気ですね。遊びに行かない?新宿で祭りがある!"

… into an array like this…


["今日", "は", "いい", "天気", "です", "ね", "遊び", "に", "行か", "ない", "新宿", "で", "祭り", "が", "ある"]

… which a computer can process to figure out frequencies, co-occurrences and so on.

A Software Shopping List

That’s the objective. Here’s a quick list of what we’re going to use to get there. (The URLs are for completeness only; we’ll be downloading everything from the command line.)

System Software:
MeCab

Dictionaries:
MeCab-ipadic
MeCab-ipadic-neologd

Python Libraries:
mecab-python3

This is not a definitive list of every piece of software that can segment Japanese text. There’s a big range out there, from the lightweight tinysegmenter through to fully featured software like kuromoji. I’m using this setup for two reasons. First, in my experience it’s the best at handling text data sourced from the Internet. Second, I think it’s the best trade-off of simplicity versus accuracy. tinysegmenter is much easier to set up and use, but its output is unreliable; it often breaks apart words that are actually common phrases or proper names (the name of the present Japanese prime minister, 阿部晋三, is rendered by tinysegmenter as [“阿部”, “晋”, “三”], not the correct [“阿部”, “晋三”] or [“安倍晋三”]; the same problem occurs with the word for prime minister itself, 総理大臣, which comes out as [“総理”, “大臣”] when what you (probably) want is [“総理大臣”]). Mecab works nicely with Python, and it’s easy to set it up with an extensive dictionary of common phrases and neologisms so its output is very accurate.

One other thing; if you’re following this tutorial on macOS, I expect you to have a little bit of familiarity with the Terminal. If you don’t know your way around Terminal at all, your homework is to go and install the “Homebrew” package; we’ll be using it to install a lot of the rest of the software. There are detailed instructions on the homepage and once you’ve done that, you’ll be able to get cracking with installing MeCab and its dictionaries.

Installing MeCab

First, let’s get MeCab (the core segmentation software) and MeCab-ipadic up and running. (Again, for macOS users, this assumes that you have successfully installed the Homebrew package.)

macOS:

brew install mecab
brew install mecab-ipadic

Ubuntu:

sudo apt-get install mecab mecab-ipadic libmecab-dev
sudo apt-get install mecab-ipadic-utf8

That’ll take a while, but once it’s done, MeCab should be up and running. You can test it by typing “mecab” at the terminal; in the blank line it gives you afterwards, type some Japanese text, and press enter. The result should look like this:

[email protected]:~$ mecab
無事にインストール出来ました !
無事 名詞,形容動詞語幹,*,*,*,*,無事,ブジ,ブジ
に 助詞,副詞化,*,*,*,*,に,ニ,ニ
インストール 名詞,一般,*,*,*,*,インストール,インストール,インストール
出来 動詞,自立,*,*,一段,連用形,出来る,デキ,デキ
まし 助動詞,*,*,*,特殊・マス,連用形,ます,マシ,マシ
た 助動詞,*,*,*,特殊・タ,基本形,た,タ,タ
! 記号,一般,*,*,*,*,!,!,!
EOS

As you can see, it’s working nicely and correctly identifying parts of speech in this test sentence. Press Ctrl-C to get back to the terminal command line, and let’s continue.

The next step is installing mecab-ipadic-neologd working. This is a bit more complex, since we need to download the dictionary of neologisms and slang (vital for handling text from the Internet) and then recompile it for MeCab. First, we  install the tools used to download (“clone”) the most recent version of the dictionary, then we compile and install the dictionary itself.

macOS:

brew install git curl xz
git clone --depth 1 https://github.com/neologd/mecab-ipadic-neologd.git
cd mecab-ipadic-neologd
./bin/install-mecab-ipadic-neologd -n

Ubuntu:

sudo apt-get install git curl
git clone --depth 1 https://github.com/neologd/mecab-ipadic-neologd.git
cd mecab-ipadic-neologd
sudo ./bin/install-mecab-ipadic-neologd -n

You’ll need to type “yes” at some point in the install process to confirm that you’re okay with overwriting some dictionary defaults. Now let’s check that it worked. To use this dictionary at the command line, we need to specify it when we invoke MeCab:

macOS:

mecab -d /usr/local/lib/mecab/dic/mecab-ipadic-neologd/

Ubuntu:

mecab -d /usr/lib/mecab/dic/mecab-ipadic-neologd/

(Note the different paths – if you’re on Ubuntu you’ll also need to change the path given in the Python files below to match.)

If you can type some Japanese text and have it tokenised (as in the previous example), then everything is working.

Working with Python

The next step is to get MeCab talking to Python. Again, this tutorial assumes you’re using Python 3 (I’m on 3.5, but any recent version of Python should be fine); there is also a library for Python 2, but I haven’t used or installed it. If you’re an advanced Python user and are using a virtual environment setup for your packages, you should switch to that environment now. (All commands from here onwards should work the same on both macOS and Ubuntu.)

At the terminal, type:

pip3 install mecab-python3

(The command may be “pip“, not “pip3“, on your system, but a lot of systems use Python 2 internally for various things, and therefore rename the Python 3 tools with a “3” suffix to avoid clashes. Also, depending on your setup, you may need to “sudo” that command on Ubuntu.)

Now open your Python editor, whether that’s an IDE (I use PyCharm personally) or just a text window, and let’s see if this is working. Try the following code:

import MeCab
test = "今日はいい天気ですね。遊びに行かない?新宿で祭りがある!"
mt = MeCab.Tagger("-d /usr/local/lib/mecab/dic/mecab-ipadic-neologd")
print(mt.parse(test))

When you run this code, you should see the same output you got at the Terminal earlier. (Note that if you’re on Ubuntu, you’ll need to change the location of mecab-ipadic-neologd in that code to match the location we were using earlier; the same goes for subsequent code in this example.) Okay, so Python is now talking to MeCab, but how can we turn that output into something useful that we can use in our analysis?

import MeCab
test = "今日はいい天気ですね。遊びに行かない?新宿で祭りがある!"
mt = MeCab.Tagger("-d /usr/local/lib/mecab/dic/mecab-ipadic-neologd")

parsed = mt.parseToNode(test)
components = []
while parsed:
    components.append(parsed.surface)
    parsed = parsed.next

print(components)

This will give you the following output…


['', '今日', 'は', 'いい', '天気', 'です', 'ね', '。', '遊び', 'に', '行か', 'ない', '?', '新宿', 'で', '祭り', 'が', 'ある', '!', '']

… which is what we’ve been aiming for from the outset! The parseToNode function of the Tagger generates an object that you can iterate over to see each token in the text. The actual token is accessed as “.surface“; if you want to see details of the part-of-speech or pronunciation, you can access that as “.feature“.

You’ve now got a system that will take Japanese text input (like a tweet, a Facebook post, or a blog entry) and turn it into a list of tokens. You can then apply exactly the same techniques to those tokens that you would to (more easily segmented) European languages.

A Few Cautionary Words

Before turning this system loose on a large volume of tweets or other social media data, I would strongly suggest writing some code to clean up your input – in particular, code to strip out and separately handle URLs, image links and (on Twitter) @mentions. MeCab doesn’t handle these well – it breaks up URLs into little chunks which will throw off a lot of machine learning algorithms, for example.

One suggestion (and the one I use in my own research) is to strip non-text elements from the text before handing it over to MeCab, and then add things like URLs and images back into the dictionary that MeCab returns to you – either as the full URL (ideally the non-shortened version) and @name, or as a token referencing a table that you can look up later (“URL1248”, “USER2234”).

Also, if you’re planning on analysing hashtags, note that MeCab doesn’t understand those either (it’ll tokenise the # and the text separately), so you may also want to pre-process those.

Finally, it’s quite likely that you’ll want to strip some elements out of the list MeCab returns to you as well. It has a habit of returning empty elements at the beginning and end of text, which you can remove; depending on the analysis you’re conducting, you may also wish to strip out punctuation, or even certain parts of speech (in which case you should check the “.feature” segment of each node to decide whether to keep or dispose of each given token).

As Japan’s establishment and observers start to come to grips with the implications of a Trump administration for the country, one comment I’ve heard a lot is that this is a gift for Shinzo Abe and his inner circle. It clears the way for them to enact their long dreamed-of reforms, which would upend the post-war national order and return the country to a normalised military posture. Certainly, the almost certain cancellation of TPP is a blow to Abenomics, but in this view of events, Trump’s foreign policy stance is a huge boost for Abe’s militaristic agenda.

In one respect, that’s right. Trump’s election and the huge question mark it places over the stability and reliability of the 64 year old US-Japan security alliance does make Japan’s military normalisation vastly more probable. Even if the Trump administration does not, as seems very likely, demand an enormous increase in Japan’s financial contribution to the US military presence here (notably, Japan already pays a very large proportion of the US’ base costs) and talk down the nature of its commitment to the alliance, the unprecedented uncertainty his presidency introduces to the alliance will sway opinon among Japan’s political establishment and public alike. More significant military budget increases (not the small, conservative increases seen thus far) are almost inevitable; reform of Article 9 of the constitution, which looked all but impossible last week, may now be within the LDP’s grasp.

To call this a “gift” to the Abe administration, however, is a misreading of the reality. The Japanese government is not acting like the recipient of a long-desired gift; rather, it’s scrambling desperately to shore up and guarantee a continuation of the status quo. Abe was among the first leaders to speak to President-Elect Trump after his victory, not because he likes Trump’s positions but rather because Trump’s statements on Japan imply an uncertain and hence dangerous future for the East Asian order. Abe will also be one of the first world leaders to visit Trump and speak face to face with him in New York. In short, Abe and his officials are pulling out every stop and calling in every favour to ensure that this “gift” doesn’t actually materialise.

Why? Simply put, the LDP’s pursuit of remilitarisation and Article 9 reform has been the political equivalent of a dog chasing cars. It’s very fun and you get to make a whole lot of noise, but what on earth is a dog to do if one day it actually catches a car? It’s big, scary, hard and intimidating; the dog hasn’t the faintest idea what to do with it, and for all its chasing and barking, has never actually sat back on its hindquarters to come up with a plan of action for after the car is caught.

The LDP is a dog that just caught a car. Remilitarisation and the upending of the postwar order in Japan has been an issue that the LDP, and especially its more regressive wing (including Abe and much of his inner circle) has enjoyed harping on about for decades. Seriously, for all the talk of Japan’s right-wing shift or Abe’s nationalism, there’s nothing new under the rising sun; LDP leaders as far back as Nakasone in the early 1980s constantly banged on the same populist drum, and it was Koizumi back in 2001, not Abe, who reignited the whole mess around Yasukuni Shrine. Talking up military normalisation, making small, gradual changes (a peacekeeping operation here, a reinterpretation of legal advice there) to the existing order and muttering loudly about “masochistic” accounts of history or “correcting” other countries’ viewpoints is red meat for a certain portion of the LDP base. It’s a strand of populist rhetoric that has been a part of the LDP’s messaging since Nakasone, but in a relatively low-key way; the LDP knows that most of the Japanese electorate supports the constitution, dislikes the idea of overseas military engagements, rather likes the postwar order and will only tolerate this kind of populist rhetoric as long as it’s seen, broadly, as harmless letting off of steam by slightly daft nationalists.

The result is that for all the talk, all the noise and the chest-beating, there is no plan in place for Japan to take independent control of its national security. There is no plan for full normalisation of the Japan Self-Defence Forces into an actual national military. There is no roadmap for any of this. There isn’t even a coherent plan for reforming the constitution; the much-vaunted “ideal constitution” drafted by LDP right-wingers and posted online some years ago is a fever-dream of return to a dimly imagined glorious past that none of its authors ever imagined actually putting into practice. As for the JSDF, it is an impressive military force in its own right – extremely well-trained, well-organised and by far the most technologically advanced military force in Asia – but it’s designed to function in concert with the US military, fulfiling very specific roles alongside the much more capable and flexible US forces. Upgrading, repurposing and realigning the JSDF into a military capable of independent operation is an enormous undertaking – expensive, difficult and time-consuming.

That’s the car the LDP has been chasing, and has just caught. The most extreme (and thankfully unlikely) version of what happens next sees President Trump fulfil his most hardline electoral promise – vacating the US’ bases in Japan and pulling back US forces from East Asia generally. While the JSDF is a competent and well-equipped force, it is in no position to fill that vacuum; even on something as simple and vital as missile defence, Japan’s advanced AEGIS destroyer fleet relies upon the presence of US vessels to fill gaps in the shield through which a North Korean missile might slip, and the timescale for achieving operational independence in that regard alone is on the order of many years. A less extreme (and arguably the most likely) version sees the US remaining in Japan, but demanding more financial support for its presence, and appearing less robust in its commitment to defend Japan’s extremities – including sort-of contested (in a “China making up flat-out nonsense” sense that has become wearily familiar across Asia of late) territories like the Senkaku Islands. While that would reduce some pressure, ensuring that the US military would remain involved in Japan’s security, it will still demand essentially the same long-term response from the Japanese Govermment. If there is even the slightest doubt in the US’ willingness to defend Japan from attack, Japan has an absolute requirement to shift the posture of the JSDF – constitutionally, legally, technologically and strategically – to that of a normalised military capable of independent functioning.

Abe and his inner circle don’t want to do that. Sure, they’d like that to happen, but the passive voice is important here; it’d be great if someone else just went and did it overnight with a sprinkling of magical fairy dust, but the daunting, years-long project of turning around Japan’s strongly opposed public opinon, its constitution and legal system, its military stance and its governing institutions to allow such a change is a minefield the LDP never really planned on actually walking into.

That’s why Abe was so quick to lift the phone to President-Elect Trump, and why he’s been so quick to arrange to meet him in New York. The opportunity to reform Article 9 has never been greater, but that’s a distant second in Japan’s priorities right now; its most pressing and urgent priority is to head off the disaster that would be the US backing off even slightly from its commitment to the US-Japan alliance. Constitutional reform and military realignment is on the table, but it’s no gift; nobody in the LDP with any clarity of outlook is smiling at the prospect right now. Maintaining the status quo as much as possible will be the number one priority of Abe, his administation and any of his possible successors for the duration of the Trump presidency.

Donald Trump is, failing an electoral college miracle, going to be the next president of the United States of America. Countless words will be written about what that means for America, but as a resident of Japan and a scholar of Japanese politics, I’d like to talk a little about what a Trump presidency means for Japan.

This is a question whose implications extend far beyond Japan itself. While the UK (and Australia) make much of their “Special Relationship” with the USA, Japan has in many ways been America’s most steadfast and important ally in the postwar era. The security treaty between the two countries is a cornerstone of the geopolitical order and stability of East Asia; Japan’s development as an economic powerhouse, aided and abetted by the United States, created a bulwark against communism in Asia; its embrace of democratic values made it a template of Asian democracy in a century when that was often a rare commodity.

While Japan has generally viewed Republican administrations in the US as being more amenable to this relationship, the reality is that the US-Japan alliance – both the formal security alliance and the more complex mesh of economic and political arrangements that bonds the nations – has been supported and developed by both Republican and Democrat administrations since its origins in the late 1950s.

Donald Trump’s stated positions on foreign policy are a significant threat to that relationship. Trump’s statements throughout his campaign paint Japan not as a partner but as a global rival of the United States. He suspects Japan of currency manipulation to the detriment of the US, and explicitly opposes the Trans-Pacific Partnership free trade deal of which the US and Japan are the key players. He has also repeatedly railed against what he views as the United States being short-changed in its security arrangements with Japan, and has gone far beyond the US establishment – which has long pressured Japan to take a more proactive role in conflicts and peacekeeping efforts – by suggesting that Japan should arm itself with nuclear weapons to ensure its own security.

That platform blindsided the Japanese political establishment; it’s a blast from the almost-forgotten past of the 1980s, when Japan’s rapid development and aggressive overseas acquisitions spurred fears that it would overtake US economic dominance. However, it’s hard to say how much of the platform will actually make its way into US policy. TPP is almost certainly dead in the water, but what line the US will take on Japan’s supposed currency manipulation and on broader trade issues is unknown. The security alliance, meanwhile, is based upon a ratified international treaty which will limit the actions of even a US President with the House, Senate and Supreme Court all lined up behind him – but its implementation, and the scope of the security cooperation between Japan and the US, could certainly be influenced if the Trump administration takes the insular, isolationist approach to foreign affairs that seems most likely.

No matter what policy or structural changes ultimately result, however, a degree of damage has already been done. Alliances are not just built upon legal treaties; they also rely on relationships of trust and assurance – upon both parties believing, long-term, in the commitments made by the other. Moreover, in the case of security alliances in particular, it’s not just the parties to the alliance who have to believe in it; outside actors, looking on at the alliance, also need to believe that its commitments are firm and will be upheld if tested. If an outside actor feels that the alliance is a paper tiger – that its commitments would not be upheld under testing conditions, for whatever reason – it creates a flashpoint for conflict and a weakness in the structure that upholds regional or even global stability.

Trump’s election, after more than a year of anti-Japanese campaign rhetoric, will weaken both internal and external perceptions of the Japan-US alliance regardless of what actual policy changes result under his administration. The view of the USA as being an increasingly unwilling participant in the affairs of East Asia will only be enhanced by Trump’s similar reticence regarding America’s role in NATO in Eastern Europe; a narrative of US isolationism will take hold among geopolitical rivals who chafe at the existing order, including Russia and China.

That is a tremendously dangerous position for Japan to find itself in. From economic rivalry to unresolved wartime history issues, the country’s relationships with several of its neighbours are fractious – most notably with China, a burgeoning superpower with thinly veiled imperial ambitions that stokes anti-Japanese sentiment as a release valve for discontent among its populace, and with North Korea, a nuclear-armed failed state. This not a hypothetically dangerous position; in absolutely practical terms, Japanese planes and ships tangle with Chinese incursions into the country’s airspace and territorial waters on a daily basis, while North Korean missile launches become increasingly sophisticated and deadly with each passing month. The importance of the US in maintaining Japanese security in these regards is paramount. The US commitment to the defence of Japan deters China from escalating its confrontations into what it would expect to be a brief, glory-seeking conflict and seizure of Japanese islands. In an even more practical, nuts-and-bolts sense, the US’ AEGIS destroyers are an essential part of Japan’s missile shield, with the country’s own fleet, although more advanced than any comparable navy in East Asia, presently incapable of dealing with the latest generations of North Korea’s missiles without US support.

If China or North Korea view America’s commitment to Japan’s security as negotiable or softening, either party may attempt to test the waters in a way that could lead to a much broader and bloodier armed conflict. In anticipation of that, and reflecting Japan’s own dawning unease regarding America’s commitment to the alliance, it’s almost certain that the US establishment will now get its long-held wish, albeit in a way it never wanted or expected – with Japan pushing harder than ever to normalise and expand its military prowess in order to make up for perceived US weakness (or non-commitment) in the Asia-Pacific region.

Quite a few commentators on Japanese politics and policy would argue that this is a process which has already started, but as I’ve observed before on this blog, Japan’s military budget increases have actually been extremely limited in recent years, with the country continuing to treat the US alliance as the beating heart of its security arrangements. The possibility of a revision to the pacifist Article 9 of the postwar constitution, though desired by Prime Minister Abe and his inner circle, had looked very remote indeed – until Trump’s election. Now, it is guaranteed that America’s relationship with Japan and the depth of its commitment to Japan’s security would be a fiercely debated topic for the coming four years. Many moderates in Japan will likely conclude that while Japanese pacifism was wonderful while the country remained safe behind America’s shield, if that shield can no longer be fully relied upon (and if China and North Korea suspect that the shield is less impervious than it used to be) then Japan has an urgent, pragmatic need to arm itself, and to remove the legal restraints that might prevent its military from effectively defending the country.

It’s not just Japan’s security position and the likelihood of normalising its military role that will be heavily impacted by the Trump presidency. The Japanese government, assuming a Clinton victory, had sought to pressure the US on TPP by ratifying the deal this month, well ahead of the inauguration of any new president. With Trump taking the office, TPP is likely off the table, and with it goes one of the core pillars of Abenomics. How Japan will react is unknown, but it seems likely that the country will feel compelled to seek out alternative trade arrangements – a Plan B to shore up its troubled economic reform programme. A version of TPP negotiated between Japan and the other non-US signatories is one possibility. Closer ties with Russia are another, although Russia’s economy is something of a disaster and Japan’s bureaucrats may be worried about hitching their cart to that particular horse. A long-discussed Japan-EU deal might even be expanded, though for a full-spectrum deal, the EU wants Japan to look at things including abolishing its (grossly abusive and cruel) death penalty system, which would be a sticking point. None of these, though, would match the sheer volume of trade that would have been affected by the TPP’s liberalisation of the cross-Pacific trade between Japan and the US. Regardless of your view on TPP itself (personally, I think it’s a mess, with far too many self-interested parties involved in opaque negotiations that have ultimately yielded an over-complicated, ill-considered, under-researched and worryingly anti-democratic treaty – but it’s still probably better than the existing situation), this is a huge stumbling block for the plans for economic reform and recovery in Japan.

This is where we stand now, only hours after Trump was elected. We don’t know who his key appointments are, what his policies will be, or any other concrete detail – but when the USA sneezes, Japan catches a cold. Trump being President-Elect already has clear, powerful impacts on Japanese domestic and foreign policy. The country’s economic programme is facing a deep crisis. Meanwhile, the likelihood of “remilitarisation” (really, just a normalisation of Japan’s military to the same status as that of any other developed nation, but likely to stoke tensions in East Asia nonetheless) and constitutional reform just took a powerful shot in the arm. With Trump preparing to enter the Oval Office in January, Japan is for the first time since the 1950s being forced to consider that its future might not include a close US relationship – and that is, of necessity, going to yield a very different Japan.

Donald Trump’s supporters are the people who have been left behind by successive waves of economic change. They are the people who in the space of a generation have seen solid, stable manufacturing jobs turn into short-term, unstable, low-paid service industry jobs – or worse, into unemployment. They are the people who lost out to free trade – whose jobs went to China and India, and who gained nothing from what came back in return. They are the people who fell through the cracks of the economic recovery Barack Obama has overseen since 2008, just as they fell through the cracks of every recovery since the 1980s, and that has taught them to distrust statistics in general and politicians in particular. They’re voting for Trump because the current political status quo has given them nothing and perhaps burning it down will make things better in the end.

Or perhaps;

Donald Trump’s supporters are the people who have been unable to accept or adapt to the cultural change around them. Around a hard-core of white nationalist racists is a mass of support drawn from people who have seen the nature of their communities change due to immigration, equal marriage and secularisation. They are people who don’t understand why TV needs to have so many gay characters, especially in shows children might be watching, or why young people listen to hip-hop, and who feel like the values they grew up with – respect for authority and a sense of white, Christian identity – are under constant attack from an urban, liberal elite. They’re voting for Trump because he stands up and says the things they believe in blunt ways that make the dog-whistling of other Republicans seem cowardly.

Which of those statements is true? That’s become one of the biggest questions in political science – not just with regard to Donald Trump, but with regard to the rise of radical populism, both on left and right, across Europe and much of the developed world. The economic explanation has more devotees. The cultural explanation has a growing body of evidence behind it. Parties supporting both camps tug back and forth.

Support for Trump in economically depressed, traditionally Democrat-leaning towns of the Rust Belt and the North East is presented as a knock-out blow for the Economics camp. The Culture camp counters by noting that the average income of a Trump supporter is actually above the national average for people of their demographic group; those backing Trump are not, by any means, the economically hardest-hit of the nation.

Neither camp is entirely convincing. The reason for that is simple; neither factor, taken in isolation, can adequately explain support for Donald Trump (or for Europe’s radical populist parties, or for Brexit). By arguing for the supremacy of one argument over another, political scientists ignore two important things.

Firstly, political support is always a coalition, not a hive mind. There is space within Trump’s support base – currently standing, according to the polls, at between 38 and 40 per cent of the US electorate, or around 90 million voters – for people motivated by economic factors, people motivated by cultural factors, people motivated by a mixture of both and even, undoubtedly, people motivated by other things entirely (like the suggested group of evangelical Christian voters who have reportedly convinced themselves that God’s plan is to put Trump in office and then have him die or resign so that the right-wing Christian VP candidate, Mike Pence, can take his place). There is no theoretical requirement for a single factor to be able to explain a majority, or even a significant plurality, of a candidate’s support.

Secondly, economic hardship and cultural intolerance are not independent of one another. One of the things you always have to look out for in any kind of political research is correlation between the factors you’re investigating. Treating two factors as being independent, when they actually interact with one another or with other variables in some important way, can seriously break the model you’re trying to build.

That’s almost certainly what’s happening here. It’s not as simple as “poor people are more intolerant than rich people” – that’s both a gross oversimplification, and provably untrue in much of the data available. However, there are lots of ways in which those factors may interact that are a little more complex and worthy of consideration.

Take for example the urban-rural divide. Urban communities in the USA are overwhelmingly voting Democrat; rural communities generally go Republican. That pattern is true in polling for this election too; one of the most effective predictors for whether a county will vote Trump or Clinton is its population density. In part, that’s because cities have larger minority populations than the countryside; but urban white people lean more towards Clinton, too.

There are cultural differences between urban whites and rural whites, not least of which is that urban whites live alongside minorities (ethnic, sexual, religious and otherwise) and that experience has been shown to push people towards being more tolerant and liberal. In the UK, the biggest electoral successes for xenophobic far-right parties like the BNP and UKIP came in towns with the smallest numbers of immigrants; familiarity with the Other, well, stops them being the Other. That’s definitely a factor in the Trump support equation.

There are also, however, economic differences between urban whites and rural whites. Urban whites are not necessarily better-off than rural whites (indeed, they’re often less well off), but for the most part the economic opportunity available to them is greater. They live in diverse cities where the end of one kind of industry usually means the springing up of another. Metropolises do die on occasion, but it’s rare; mostly a period of decay following the decline of an old industry is followed by new businesses moving in to take advantage of lowered costs. Rural areas, though? Small towns and villages? When they die, they die. One factory closing can turn a small community into a ghost town; people who have the mindset and the opportunity to move to a city do so, and what remains is a zombie village, lurching forward through inertia but likely doomed to decay and decline forever.

In short, a rural white voter may have cultural differences from his urban cousin, but he also has a key economic difference; even if he’s doing fine personally (though again other factors come into play – is he doing as well as his father did, or worse? Perceptions of economic conditions are highly influenced by expectations, after all), he may be surrounded by a community in decline, driving past boarded-up windows every day, constantly reminded that his income and lifestyle is fragile, and fearful that his country has forgotten his kind of town even exists.

See how that imaginary voter ends up being a melange of both economic and cultural factors? My hypothesis is this; Trump’s voters are generally motivated by a little from column A and a little from column B. For those who harbour racist, xenophobic, homophobic or otherwise regressive social sentiments, they were willing to just roll their eyes at the multicultural liberals in the cities and calmly vote for mainstream, non-populist candidates as long as their communities remained economically stable and prosperous. These towns may not have been the most welcoming places in the world for minorities, but for the most part minorities were not blamed for major problems (a core underlying theme of radical-right populism, and of the Trump campaign specifically) because there weren’t really any major problems to blame them for.

On the flip side of the coin, you’ve got those who have been severely impacted by the economic shake-up of the developed world, and left behind by globalisation and technological progress, but who have not reacted by supporting Trump. That’s because they didn’t have latent regressive sentiments that could be activated by economic hardship (either their own, or that of their community as a whole). They live alongside minorities, or perhaps have minority groups represented in their own family and close friends; they’re college-educated, which in addition to giving them a different context in which to consider and critique Trump and Clinton’s competing claims, has exposed them to many minority groups (even the most conservative colleges have far more opportunties to meet and get to know people unlike yourself than small town communities generally offer). Though times are tough for them, they were never in the potential Trump voting camp, because they don’t have a culturally grounded anger at minority groups which they can shape into blame.


There’s another group worth mentioning; the “always-Republican” group of people for whom voting Republican is part of their identity (though they’d blanch at the thought of actually subscribing to identity politics, oh dear me no), for whom Clinton is too left-wing – and despite her progressive detractors, she is notably more left-wing and progressive than even Barack Obama on almost everything except foreign policy. For this group, and there seem to be a hell of a lot of them, voting Trump is partially about identity, and partially about ideology. They’ll happily admit that the man is a boor, a misogynist, a racist, even a fool; but they worry about the country’s direction under Clinton, and have convinced themselves that Trump will be effectively restrained by the Republican Party and by the nation’s democratic checks and balances, making him into a far more moderate-right leader than any of his own statements and positions have implied.

These people skew the data. They’re not really “Trump supporters”; they’d vote for anyone wearing the GOP badge, or perhaps simply for anyone standing against Clinton. They share some of his views but dislike how he expresses them (the dog whistling was just fine for them). They’re the Republican hardcore, inseperable in the data from the true Trump supporters, and they mess up the statistics; they’re likely older, and richer, and perhaps less religious and more educated than the kind of people who actually wear Make America Great Again baseball caps.

One might also observe that pretty much this exact demographic has also served the function of useful idiots ushering in their reign of every right-wing authoritarian strongman in history – relatively moderate traditional conservatives who give the reins of power to a demagogue in the mistaken confidence that they’ll be able to control him once he’s in charge – but that would be unkind. True; but unkind.


The 90 million or so people who currently say they support Donald Trump (assuming 100% turnout, of course; the actual vote figure will be far lower) is an enormous, diverse demographic. It’s not necessarily diverse in a traditional sense – it’s overwhelmingly white and a bit of a sausage party, with lots of traditionally Republican-supporting women seemingly crossing the aisle this year – but the backgrounds, reasoning and motivations of those in that group are incredibly disparate, which makes a bit of a mockery of attempts to find the Holy Grail, the one, true reason why people are voting Trump.

Perhaps confusing matters even more are the core supporters and true believers – the 13 million people who voted for Trump in the primaries (though some of those early votes were likely half in jest, and some of the late ones made through clenched teeth to keep the arguably even more odious Ted Cruz from the nomination), for example, or the thousands who turn up to his rallies around the country. Their motivations are perhaps more clear, and certainly more clearly telegraphed. Those who picked Trump from among a stable of boring but broadly moderate Republicans, and those who turn up to cheer him on at rallies, have an agenda that’s clearly cultural. A large portion of them are absolutely cheering on the turning back of the clock, the disenfranchisement of ethnic and sexual minorities who have “skipped the queue” at the expense of “real” Americans; they may have been activated by economic hardship, but the cultural base of their grievances, and the racist White Nationalism to which it has led them are extremely clear.

There aren’t very many of these people, though. Even if you accept that every single person who voted for Trump in the Republican primary was doing so on the basis of cultural regressivism and white nationalism – and I think that’s absolutely a ceiling figure, not a realistic estimate – that’s only 13.3 million people, in a country with 225.8 million eligible voters. About 6% of the population voted for Trump in the primaries. His gains from that figure may partially reflect an anger at cultural change, or an anger at economic instability, or more likely a mixture of both, but a much larger component is simple partisanship; Republicans voting Republican because they’ve always voted Republican, and/or because they don’t like the Clintons.

6%. That’s the deep, devoted core of Trump support. A higher percentage than that say they’re going to vote for the bumbling, comedic Libertarian candidate, Gary Johnson, on November 8, and yet we’re not falling over ourselves to understand why Johnson supporters are the way they are, what has motivated them, and what has activated them. (In fact, we’re also not falling over ourselves to ask why a majority of so many demographic groups are voting for Clinton; it’s only Trump that provokes this fascination.) It’s not that understanding this isn’t worthwhile; I’m a political scientist precisely because I want to understand everything, even while recognising the futility of the quest. Rather, it’s the assumption that the things that have driven and motivated those people should somehow be taken into account, listened to sagely, nodded at in understanding, and allowed to influence the future direction of a nation or even a planet.

6%. That’s smaller – perhaps less than half – than the percentage of the population from whom the 6% would like to remove the right to marry and the right to a family. It’s far, far smaller than the percentage of the population from whom they would deny the right to equal treatment under the law, or to schemes designed to rebalance historical inequalities. White Nationalism truly is White Supremacy; it assumes that a small group of white, conservative and under-educated men should be allowed to dictate the fates of far larger groups of people. This is a dark fantasy that is only fed and watered by earnest hand-wringing over their motivations and reasoning.

Of course, improving the economic conditions of everyone – yes, even racists, I guess – should be the role of goverment; the decline of communities and the installation of a safety net and alternative path for those impacted by globalisation and technological progress should be a priority. It shouldn’t be a priority because of Trump’s support; it should be a priority because it’s the right thing to do. Perhaps it will head off future waves of populism; the literature on cultural backlash suggests otherwise, but it certainly can’t hurt, especially if there really is some correlation between cultural intolerance and economic instability.

But when anyone talks about changing policies or slowing down the rate of social progress in order to attune to the desires of the Trump supporter, remember who we’re actually talking about; the 6%. Everyone else is a johnny-come-lately, a fairweather Trumper, a half-hearted enthusiast for anyone wearing a GOP badge on their lapel, a self-deluding cheerleader for a demagogue they assume, despite all evidence to the contrary, to be on some kind of leash (don’t you think that if the Republicans could control Trump, they’d have, well, done it by now?).

Understanding Trump’s support – in terms of cultural identity, in terms of the plight of economically depressed communities, or in whatever other terms are found to make sense – is important. Don’t let anyone tell you that the next step after understanding it should be pandering to it.

The remilitarisation of Japan is a popular theme for the international media. It gives a clear, dramatic narrative to international news coverage that might otherwise bore readers. In this narrative Japan’s leadership seek to cast off the shackles of the post-1945 world order, to rewrite the pacifist constitution, rebuild their military forces, inculcate hatred of their Asian neighbours, and adopt an aggressive, warlike stance towards Asia. Leading the charge is prime minister Shinzo Abe, with fellow members of the shadowy conservative/revisionist Nippon Kaigi organisation being given senior government positions from which to realise their militaristic goals.

Not all journalists or publications buy this narrative in its entirety – but either in full-blown “Abe is a Fascist!” form or in a more diluted manner, it has become the master narrative of Japanese politics in the international press. That narrative frame can be seen in coverage this week of the request by Japan’s Ministry of Defence for a 2.8% budget increase. “Japanese Government Urges Another Increase in Military Spending” reports the New York Times; “Japan Defence Ministry seeks Record Budget to Counter Chinese Threat” says The Guardian. Both stories, in common with most coverage of the budget request, emphasise that this is part of an ongoing process of (re)militarisation.

I don’t wish to single out the NYT or the Guardian, nor the writers of these articles (Mokoto Rich and Justin McCurry respectively) – my intention isn’t to bash their coverage, which is actually more even-handed and well-researched than a lot of other articles on this topic. Rather, I’m linking to those articles to demonstrate that even the better news outlets continue to support a narrative about Japan which deserves to be questioned more closely.

There are lots of questions to be asked about this narrative. We might ask why Nippon Kaigi, for all that some of its policy positions are unpleasant or ill-informed, is considered any more shady than other political lobbying groups. We might ask why an organisation portrayed in the media as a shadowy background powerbroker would have an extensive and informative website setting out its aims and policies, or media briefings with its leaders – including one fairly recently at the FCCJ, Tokyo’s foreign correspondents club. We might also ask why, following the recent House of Councillors’ election, media outlets almost universally reported that Abe’s government now had the votes necessary to reform the constitution, ignoring the fact that many of those who support constitutional reform (including the LDP’s coalition partner, Komeito) support entirely different proposed reforms to the LDP – not to mention that any reform would need to pass a referendum, too.

This week’s conversation is about military budget, though, so let’s look at military budget.

Graph of Military Budget in US$

Military Budget in 2014 US$

This graph shows the military budget of Japan and some of its neighbours over the past 20 years – from 1995 to 2015 – in millions of US$. For ease of comparison, all figures are normalised to 2014 US Dollars. Two things are immediately apparent.

First, Japan’s expenditure hasn’t changed much in 20 years. There were some large rises towards the end of the 1980s, not least because the United States demanded that Japan should pay more towards the cost of US bases on its soil, but since the mid-nineties Japanese expenditure has stayed fairly solid in US$ terms. In fact, its military budget is almost identical to that of Germany, and significantly lower than the United Kingdom – a smaller island nation in a significantly less turbulent part of the world.

Second, Japan’s neighbours are spending huge amounts on military expansion. China’s budget, three to four times greater than Japan’s and growing at 7 to 8% each year, is now second only to the United States (the US isn’t on this graph because it’s ridiculous – the scale required to show the US’ military spending, more than 10 times that of Japan’s, squashes all other countries into a multicoloured line bouncing along the bottom of the graph). Russia now spends double Japan’s budget. South Korea, with less than half the population but a more precarious defence situation, spends a comparable amount to Japan.

We have no data for North Korea, whose aggressive nuclear weapon and missile programs are one of the main reasons for Japan’s budget increases, much of which will be spent on improving missile defences.

Here’s a second way to look at the data.

Graph of Military Budget as a Percentage of GDP

Military Budget as a % of GDP

This graph shows the military budget of Japan, its neighbours and some other countries as a percentage of their GDP over the past 20 years. In some ways it’s a misleading chart – while China looks fairly flat on this graph, its GDP has boomed so the cash it spends on the military has grown enormously even without using a larger proportion of GDP. Japan, meanwhile, has had mostly stagnant GDP figures for the past couple of decades. With those caveats in mind, though, we can pick out some interesting things from this data.

We can see that Japan spends far, far less on defence as a percentage of GDP than pretty much any other major nation. Russia’s expenditure is off the chart (literally), while the USA, South Korea, the UK and China all spend over 2% of their entire GDP on the military. Japan doesn’t belong to the same category of nation at all; in fact, its GDP percentage spend is lower than Germany. The closest nation in the data set to Japan, by this measure, is the notoriously militaristic, sabre-rattling, neighbour-terrifying, aggressively warlike… Canada.

Incidentally, out of every single country in Asia and Oceania, only three spend less of their GDP on defence than Japan – Indonesia (0.9%), Mongolia (0.8%) and Papua New Guinea (0.6%). (If you’re interested, the lowest military budget as a percentage of GDP of any nation in the world was the 0.4% spent by Ireland, Guatemala and Nigeria. Famously neutral Switzerland spent 0.7%.)


None of this is to say that there aren’t some problematic things about Japan’s political trajectory. Abe and his close associates have troubling views on history, and there are valid fears that those views will drive his government towards policies which promote nationalism and xenophobia and erode international ties in East Asia. Much more worrying than anything about Nippon Kaigi is the extent to which his unprecedented dominance of the LDP has shut down intra-party competition and debate; the LDP used to be its own best opposition thanks to healthy competition between factions, which is now all but moribund. And yes, certainly, criticism is due of politicians (in all countries!) who can’t seem to control their childish urges to provoke their neighbours over historical or territorial disagreements.

The master narrative of Japan’s slide towards remilitarisation, nationalism and even fascism, however, just isn’t supported by the facts. Take constitutional revision; while being more seriously considered than at any point since the 1950s, it still has to clear many tall hurdles. More aggressive ideas for changing the constitution are not even supported unanimously by Abe’s LDP colleagues, let alone by the other parties whose support would be needed or the Japanese public who would vote in an eventual referendum.

The increases in the military budget, meanwhile, are eye-opening not as proof of militarism, but as proof of extraordinary restraint. Faced with enormous military build-ups in neighbouring nations – two of whom, North Korea and China, have carried out minor but overtly hostile actions towards Japan in recent years – Japan’s military spend remains modest. It has the third-largest economy in the world but spends less on its military than France or the UK – neither of whom experience either regular military / paramilitary incursions into their waters, as Japan does from China, or the testing of ballistic missiles aimed across their territory, as Japan does from North Korea. (Neither do the neighbours of France and the UK promote educational policies which distort historical fact to demonise France or the UK, tolerate rioters attacking outlets of French or British businesses, or broadcast endless TV shows dramatising often exaggerated accounts of French or British war crimes.)

In the face of these threats, and against a background of increasing pressure on the Japanese Self-Defence Forces to participate fully in international peacekeeping and reconstruction missions (Japan has been bashed for decades in the international community for sending cheques rather than physical assistance to stricken areas), the modest increases to Japan’s defence budgets suggest caution and restraint. Japan remains Asia’s most successful democracy and it relies for much of its security not on enormous military expenditure but on the strength of its diplomatic and economic ties around the region and the world. In the face of real concerns over regional stability in Asia, it would be helpful if the international press desisted from attempting to undermine that position for the sake of more dramatic headlines.