Introduction

In June 1971, the BBC broadcast an infamous documentary - 'fast moving and irreverent' as one commentator called it - on the Labour Party in opposition.1 It was entitled Yesterday's Men, a mischievous turning of the tables on the Labour Party, given that they had adopted precisely this phrase in their electoral advertisements in the general election of 1970 to refer to the members of the previous Conservative government. The documentary broke new ground in that it shed some of the deference which political interviewees had hitherto been accorded, and asked questions that until then had been considered taboo.

It also broke new ground in its very focus on the Labour Party in opposition. Media attention has for the most part been lavished on governments and their doings, and opposition parties have tended to be left in the shade. At the beginning of the programme, the presenter David Dimbleby muses as follows: 'Labour has now been a full year in the wilderness. Perhaps the worst is over; the harsh rediscovery of what it means to be out of power. It was a year to be got through rather than lived, a year when the other people made the running. Politics is severe on those who fail. They're left to kick their heels and wait, and watch the others have a go.' But is this all that political opposition is about, kicking one's heels, waiting and 'watching the others have a go'? The programme asks and seeks to answer this question, although it focuses more on the fate of the senior former Cabinet ministers and their personal fortunes rather than on the party as a whole and its work in opposition.

In books and studies on the Labour Party - both scholarly and popular - the focus has likewise tended to be on its time in government. In many ways, this is understandable. These were the years in which the party was put to the test of office. It is natural for scholars and commentators to ask how successful it was in achieving its goals, how competent it was in governing the country, to what extent it succeeded in its aim of changing society, whether it remained true to its principles and beliefs when tested in the crucible of government. But such questions are themselves dependent upon, and are only intelligible in terms of, other questions. How did the party determine its goals? How did it prepare for the task of governance? In what way did it seek to change society? How did it determine its principles and beliefs and how have these changed? Such questions can only be answered by turning instead to the time the party spent in opposition, during which, relieved of the task of governance, it had the leisure to engage in self-examination, institutional preparation and policy formation. These are vitally important tasks without which the party's time in government would be inexplicable.

In recent years, there has been some recognition of the importance of opposition as a subject of political study. In 2010, a Centre for Opposition Studies was established, now based at the University of Bolton. One of the co-founders, Nigel Fletcher, has edited How to be in Opposition: Life in the Political Shadows (2011).2 There have also been volumes studying the Conservative Party in opposition - Recovering Power: The Conservatives in Opposition since 1867 (2005) - and the party leaders in opposition - Leaders of the Opposition: From Churchill to Cameron (2012).3 All of this is to be welcomed, but it is striking that so little attention has so far been paid to the Labour Party in opposition. Where the Labour Party has been considered, this has tended to focus on the particular individuals who served as Leaders of the Opposition, rather than the experience of the party more generally. Such neglect is surprising for at least two reasons. First, because of the sheer number of years that the party has spent in opposition. In the 100 years since it first became the official party of opposition, it has spent only about a third of those years in government. The party is therefore much more used to being in opposition than in government, which makes an examination of its time in opposition all the more pertinent. This also, of course, raises the further question of why the party has spent so much of its time in opposition. Is this because it has squandered much of its time in opposition or more because of its failures in government? Again, this is a question that cannot be answered by an examination of the party's time in government alone.

Secondly, an examination of its years spent in opposition is particularly relevant for the Labour Party which understands itself to be a party of 'change', in contrast to the Conservative Party which, in principle at least, ultimately seeks to 'conserve'. The change that has been sought has been primarily to the economic system of capitalism, which is why the Labour Party has historically defined itself as a 'socialist' party. But the party has had to answer the fundamental question of what form this change should take. In the early years, this entailed the party asking whether it should seek to dismantle capitalism or to reform it. As we shall see, the 1930s were decisive years in which the party ultimately decided to reform capitalism. But this raised further questions over the nature and extent of this reform, prompting arguments that raged throughout Labour's time in opposition in the 1950s and, again, in the 1980s. In the 1990s, the party decided less to reform capitalism than to administer it more fairly, later expressing considerable dissatisfaction with this minimalist approach in the 2010s. These were vitally important decisions in the evolution of what the party meant by 'change' and they were all effected during periods of soul-searching in opposition. It would not be an exaggeration to say that its time spent in opposition has served to define the party. This, in turn, has determined the party's agenda at the times it has entered into government. And yet, the party's experience of opposition has not been accorded the attention it deserves. In the chapters that follow, we seek to rectify this neglect.

In the 100 years or so covered by this book, the country, society, politics and the Labour Party have changed enormously. Sensitivity to this has necessitated a chronological approach, and each chapter in what follows is devoted to a particular period of time that the party spent in opposition. But equally, one of the aims of this book is to think across and between these periods, to draw comparisons and to analyse thematically across the broad sweep of the century during which Labour has been one of the two major parties. In order to do this, we have adopted a common set of criteria by which Labour's time in opposition may be judged. Again, sensitivity to the differences between these periods has led us to apply these criteria in a malleable rather than a rigid way, thus allowing us to work with a common standard of assessment while avoiding undue distortion.

In a very broad sense, we may say that the duties of a party in opposition are of two main kinds - those that look outward and those that look inward. In terms of Britain's parliamentary system of government, the most important - and outward-looking - role of the main opposition party is to hold the government of the day to account. The Official Opposition in Parliament is meant to scrutinise, question and criticise government legislation. The existence of the opposition in Parliament is to ensure that there is a tangible body of critical opinion to which the government of the day is answerable. An effective opposition does not, of course, indiscriminately oppose everything the government does. Rather, it has to measure the government's legislative proposals against its own standards, policies and principles and demonstrate the ways in which such legislation is wanting. How to measure the effectiveness of an opposition in this respect is a difficult question. An opposition is at its most effective when it is able to force a government to change course or even bring a government down. There are some examples of an opposition being able to do both of those things. For instance, the Labour opposition in 1935 was able to persuade the government to change its plans for the provision of unemployment benefit, and in 1990 it helped to ensure a change of policy on the poll tax. In the late 1930s, it helped to secure the end of the policy of appeasement. It made a direct contribution to the fall of Neville Chamberlain as prime minister in 1940 and an indirect contribution to the deposing of Margaret Thatcher in 1990. But the very fact that an opposition is in a minority in Parliament means that the opportunities for forcing such radical government concessions are rare. Furthermore, it is usually able to achieve them only by harnessing wider public opinion or by exploiting discontent within the government party itself. More frequently, the success of an opposition in holding a government to account will be measured by the extent to which an opposition is able to persuade the wider electorate that they are a government in waiting who would be more effective than the current incumbents.

This, indeed, is the second of our outward-looking criteria. As much as looking at the government benches facing them in the House of Commons, an effective opposition must communicate successfully to the wider population outside Parliament itself, upon which it depends for its transformation from opposition to government....