Got a bad plan? The money will turn up anyway

After the drama surrounding the pre-investments, everyone thought that spending the revenues of the loan system was bound to go well. The report by the Netherlands Court of Audit on pre-investments at universities and higher education institutions is damning. In hindsight, it could only be established that a third of the investments that were made were pre-investments. Members of parliament and student unions are incensed and will do whatever it takes to prevent this from happening again with the revenues from the loan system.

Minister Ingrid van Engelshoven (Education, Culture & Science; D66) concurs. Clear goals must be set, and the plans of universities and higher education institutions need to be tested for feasibility and to ascertain whether the money is being spent on worthwhile projects.

The Accreditation Organisation of the Netherlands and Flanders (NVAO) will be responsible for checking up on this. ‘The NVAO is highly independent and knows how to oversee quality in higher education,’ said the minister in the parliamentary debate following the Netherlands Court of Audit's report. Nevertheless, the NVAO was not the first choice. The Inspectorate of Education, the Van Vught Commission (which assessed the performance agreements two years earlier) and an entirely new higher education authority had been considered first, but after protests from universities and higher education institutions, were not approached in the end. These institutions have been campaigning for years against what they see as excessively strict regulation, and are extremely concerned that their autonomy will be curtailed.

The NVAO is normally concerned with assessing the quality of studies but has been passed the buck this time as a form of compromise. Not much later, the so-called Sector Agreement on quality agreements was ceremoniously signed by students, administrators, and Education Minister Ingrid van Engelshoven. ‘The performance agreements triggered a lot of chagrin and mistrust,’ the minister said during the ceremony in April 2018. ‘It took a while to regain that trust, but it all worked out very well in the end.’

In the Sector Agreement, it states that every university and higher education institution must draw up a quality plan wherein it outlines what it intends to do with the revenues from the loan system. A panel of the NVAO, made up of former education administrators, students and staff from other universities and higher education institutions, examines this plan and decides which people from the relevant university or university of applied sciences they would like to speak to. They then send them an invitation.

‘Then you end up spending a day sitting in a small room, and every few hours a few people walk in and talk to you,’ says Bart Hekkema (pictured right). Hekkema is a history Master's student at the University of Groningen and a member of the NVAO panels that review the quality agreements. These panels assess the quality plans of universities or higher education institutions on the basis of three criteria: (1) Are the plans well-founded and do they contribute to improving education? (2) Were students involved in the development of the plan? and (3) Are the plans realistic and is it possible to monitor the effects?

‘Personally, I tend to pay particular attention to the question of whether students are actively involved in the plans,’ says Hekkema. He sees that the quality agreements have led to universities and higher education institutions doing more to promote student participation. ‘They often do this in innovative ways, with students who are not members of elected representative bodies.’

Rotterdam University of Applied Sciences is one such higher education institution that has adopted an innovative procedure, says Ron Bormans, chair of the executive board there. ‘The quality of the process is as important to us as the outcome for that process,’ he states. ‘We invested a lot of energy in this. During various stages, we have been in contact with students and faculty staff within the representative body, with our Supervisory Board, and with stakeholders such as the Rotterdam Port Authority and the Erasmus Medical Centre.’ Moreover, Bormans proudly mentions initiatives started by students from the representative body themselves, including the campaign ‘What should we do with €26 million?’ This is the amount that Rotterdam University will receive annually from the quality funds in 2024 and the years thereafter.

Ron Bormans.

This yielded a plan in which academic success and intensive, small-scale education are extremely important. ‘Our strategy is: The teacher is the one doing it, so we want more good teachers. We all share the realisation that academic success can best be achieved twit the help of high-quality teachers.’

(Below the infographic you can read more about how the Rotterdam University of Applied Sciences was not the only university of applied sciences that had a problem with its quality plan).

One of the criteria that the panels assess, and on which the plan submitted by the Rotterdam University of Applied Sciences was initially rejected, is whether the plan makes a substantiated contribution to improving education. According to Hekkema, the panels do not assess whether there will be any quantifiable quality improvements by 2024, but only whether the rationale behind the plan can be understood. ‘The assessment of the plans therefore has no predictive value for the actual realisation of an improvement in the quality of education.’

‘As a panel, it is really quite complicated to fully assess educational improvements. You cannot say that with any certainty about a lot of plans. Having more teachers does not necessarily mean better education,’ he says. ‘Some universities or higher education institutions are doing it right. They say, for instance, that they are hiring extra teachers so that fewer hours of study group sessions are held per teacher and there is more time for personal tuition, for example, when it comes to dissertations. At the end of the day, you don't know if that actually means better education, although it tends to be a strong indicator.’

Yet things went sideways anyway. ‘When the NVAO panel first came to us, I could already see in their eyes that it wasn't going to work out,’ says Bormans. ‘We were a bit too single-minded. It suited us, but the committee was clear: It had to be exactly within the lines of the national themes.’ The panel rejected the plan from the Rotterdam University of Applied Sciences. The NVAO upheld that, and the Minister adopted the NVAO's advice.

Bormans cites an example of one such criticism: ‘We had written down that we wanted to hire more teachers. That should have been: 'Rotterdam University of Applied Sciences is investing in more intensive, small-scale education,' he says as if he is dictating it. ‘In my view, something like that constitutes system dominance,’ says Bormans. ‘System reality triumphs over actual reality. I find that disappointing.’

That disapproval meant that the institution was subjected to a second round, just like sixteen other universities of applied sciences. ‘We used the second round to hone the plan and flesh it out for the longer term, but it was mainly about changing the wording,’ says Bormans. ‘I also think that the plan would have been approved the first time around if we had written it differently. To some extent, it does feel like you have to jump through a few hoops, but we still jumped through that hoop in the second round in our own way.’

Difficult criterion

Henriëtte Maassen van den Brink (pictured right), professor of Business Administration at the University of Amsterdam and up until 2018, professor of Evidence-Based Education at Maastricht University, also finds ‘quality’ a tricky criterion to measure. Like Hekkema, she has on several occasions been a member of the panels that assess the quality agreements. ‘It is complicated, but a critical and good panel can get a lot done.’ This is also due to the way universities and higher education institutions draw up their plans, she says. ‘Some send three or four A4s, so to speak, while others write a report that can go straight to the press.’

She also sees another problematic aspect, namely monitoring how the millions from the quality agreements are spent in practice. Although universities of applied sciences will have to account for how the money is spent in 2024, Maassen van de Brink says that this is simply 'impossible to do for plenty of institutions.' ‘This is especially the case if they adopt a lot of different measures and keep on reserving small amounts for them.’

Juggling with figures

Eindhoven University of Technology is a good example of this. The annual report for 2019 shows that half of the available funds (€3.5 million) were not spent. Of this, €3 million of student loan funds came from the government and half a million came from the university's reserves. The remainder of the student loan budget (€1.2 million) was added to the earmarked reserve for student loans. This already included €1.4 million that was not spent in 2018. In fact, TU Eindhoven did not spend all the money that was available that year either; at the time, it received €2.9 million from the government and had also added funds of its own.

This makes it difficult to tell where that €2.6 million in the earmarked reserve actually comes from. Is that money from the revenues of the loan system? Is it money from their own reserves? Or is it a bit of both, and if so, in what proportion? Now, this is not necessarily a problem, but if universities have not spent all the money by 2024, then they will have to give it back. Maassen van den Brink foresees a major problem when she looks at the plans. ‘To what extent will panels in 2024 be able to check how effectively the money has been spent?’

Bart Hekkema also has his reservations in some areas. ‘I personally find it absurd that the abolished student grant money flows into education and that universities and higher education institutions subsequently have had to invest a lot of time and money in the huge circus that that all entails. You can ask yourself whether it would be better to pay for study grants from that amount, especially since it is not clear whether the money from the quality agreements will be made available on a structural basis.’

Nonetheless, he does see that the revenues from the loan system are at least being well spent. ‘We have to be honest: Many universities and higher education institutions have really wonderful plans whereby they are genuinely trying to improve education.’

Accountability

It is precisely external control that is essential for this, according to Maassen van den Brink. ‘Scrutiny from outside forces universities to properly account for and provide insight into how public funds are spent,’ she feels. Maassen van den Brink is well aware of this, as she was also a member of the committee that assessed whether universities of applied sciences were complying with the performance agreements.

‘In my view, a straight line can be drawn from the performance agreements to the quality agreements,’ she says. ‘The process is broadly similar. After all, agreements were made between universities, universities of applied sciences and the ministry,’ says Maassen van den Brink. ‘The main difference is that the quality agreements also involved students and faculty staff, whereas performance agreements do not.’

Watered down

Maassen van den Brink also sees the same trend amongst the performance agreements. Where there used to be a heavy fine for not meeting the targets, this was watered down during the process. ‘As a committee, you get fed up with it in no time. When we subsequently visited universities of applied sciences and universities, we noticed that some had a bit of a laissez-faire 'let it go, it‘ll all work out’ attitude. They knew that they could simply smooth over the commission's more negative remarks with the minister later on. The same thing is now at risk of happening again with the quality agreements.’

She is referring to Minister Van Engelshoven's decision to ignore the negative advice from a panel in the cases of the Delft University of Technology and the Hanze University of Applied Sciences, and still award them money from the quality agreements. The minister did this after she had invited the administrators of these universities and universities of applied sciences to the ministry and they had been able to convince her of their plans in person, bypassing the independent NVAO.

According to Maassen van den Brink, this poses the risk that the process will once again become 'ineffective.' ‘It seems that ultimately, no one stands to lose a single cent, while some plans raise questions about how you can gauge the effectiveness of their spending,’ she emphasizes. ‘Some administrators think: 'We'll see in good time, and if things don’t work out, we can just go to the minister.' This undermines both the agreements that we made and the trust of students who had to forfeit their student grants.’

With the cooperation of Altan Erdogan, Yvonne van de Meent and Laura ter Steege.

This story is the seventh instalment in a series made possible by the Dutch Journalism Fund (SVDJ) and various editors at higher education institutions and universities in the Netherlands.

Read the final article in this series on our site next Wednesday, March 3.