Auditing Academic Processes Down Under

June 1, 2017

This article summarises an ACUA webinar presentation prepared by the author and delivered in May 2017. The subject was inspired by a two part article written by Sonya von Heyking in the ACUA Journal entitled Auditor Turned Faculty Member: Lessons Learned in which Sonya poses the question:

All colleges and universities exist to educate and develop students. You will find this statement, in one form or another, in every school’s mission. And yet, what proportion of the average internal audit plan directly addresses how this education is achieved? (ACUA Journal Winter 2016 edition)

As a quality reviewer of many university internal audit functions in Australia and the USA, my general observation is that there is, on average, very little core business auditing going on. There appears to be reluctance, for various reasons, for internal auditors to tackle subject matter that appears to be out of their comfort zone. Why is this so?

My observations in the USA have generally been that the constraints of demand for compliance focused audits in an environment that has tight resource limitations limits opportunities for internal audit plans to tackle the areas I am talking about; academic process audits. The situation is only marginally better in Australia, although the much earlier take up of broad based risk management concepts has meant that these audits have certainly been considered more widely. There may well be other reasons that internal auditors have avoided tackling such audits such as; backgrounds based in State audit teams (external) or accountancy, acknowledgement of a lack of in-depth knowledge of academic processes, or management pushback.

The bottom line is that academic process is what a college or university is about and these processes are entwined in the thing that Sonya talks about in her article; the provision of an educational experience for students. This is core business and should be included in the internal audit plan universe for regular inclusion. If the university doesn’t get this right it fails on its mission and objectives – that’s risk with a very capital ‘R’.

WHAT ARE ACADEMIC PROCESS AUDITS?

I’ve included a vast array of audit activities in this category and they range from the R&D process in establishing academic programs, through the marketing of those programs and the attraction of students into them. It includes not only the admissions process, but the underlying academic scrutiny associated with determining a potential student’s right to enter the program. Then, throughout the entire student life there are academic processes such as assessment (term work and examinations), student support (from an academic perspective), complaints and grievances (and resolution and process improvement), through to graduation and beyond.

These academic processes are supported by a plethora of policies, procedures and guidance, and in some cases, regulation – the stuff of auditors! However, one must be careful when entering the murky waters of academia. It is all too easy to fall back into the position of a compliance auditor and checklist our way through the forest without recognising that the trees might not be in great shape in supporting that fundamental proposition, that we are educating students. In support of this notion, it is often wise to step back and consider the overarching objectives of what the process is trying to achieve. That is not to say that, for example, such things as the physical security of examination papers is unimportant, but if the paper was not constructed in a manner that provided the best education outcome for the students, they may as well be unsecured!

In order to provide some understanding of academic process audits that could be considered, here is a list of some that have been completed at the University of South Australia over the past ten years or so:

Policy A35A – Quality assurance and improvement: Programs, courses and teaching arrangements

There are going to be a plethora of academic policies in every institution and these provide fertile grounds for audits. Consider those relating to academic quality and accreditation processes.

Distance education

Many institutions still offer academic programs through external modes of study. From enrollment through to assessment, these alternate forms of delivery can be audited to ensure consistency with on campus processes within a well-controlled environment.

“However, one must be careful when entering the murky waters of academia. It is all too easy to fall back into the position of a compliance auditor and checklist our way through the forest without recognising that the trees might not be in great shape…”

Offshore teaching

This may not apply to every institution, however it is key to those that rely on this form of delivery to ensure agreements entered into with offshore partners are sound and overseen by solid governance processes.

Student exchange

The primary focus for us was on the Study Abroad program, but others do exist. Student safety, academic quality and student outcomes are key points to consider.

Credit transfer

We looked at this some years ago and are repeating an audit in 2016/17. Credit can be viewed as a marketing tool, to attract prospective students, but also from an academic perspective of providing like- for-like ‘discounting’ of academic programs. Every piece of credit granted is foregone revenue to the organisation and processes need to ensure that the granting of credit does not impact student outcomes.

Assessment

This audit looked at the over-arching policy framework for assessment across the university. Rather than being compliance focused, the audit looked at the framework for ensuring assessment was constructed under a sound framework in order to achieved desired outcomes.

Student complaints

The reporting, escalation, assessment, follow up and reporting of student complaints is necessary in ensuring satisfactory student education experiences. Complaints might encapsulate everything from the food in the cafeteria to harassment and other complaints, and academic process complaints, remarking issues, etc.

Post graduate research supervision

Academic and research standards in Australia demand that controls be in place in this area. Inadequate supervision and oversight of research students can result in undesirable outcomes for both the student and the institution, but also any subjects that might be part of a research project.

Open Universities Australia

This audit focused on the specific operations of an agreement with this consortium.

Examination boards/moderation of assessment

This audit required a narrow scope as we fairly quickly found there were inconsistencies in approach across the university that were being addressed through an external review. The plan was to look at how moderation of assessment occurred.

Equivalence of assessment

This looked at assessment as applied to onshore delivery and offshore delivery. Given criticism in the media over the years about Australian institutions providing ‘degrees for dollars’ to international students, this audit looked at one method of dispelling those accusations by ensuring that assessment was equivalent.

Course information

New consumer protection laws in Australia meant that the University needed to be sure that none of its course information that was available to prospective students was non-compliant, for example, stating that ’the completion of a particular degree guaranteed employment’.

Professional Experience program (sabbatical)

This involved a review of our programs to ensure that objectives were set and that outcomes not only benefited the individual academic, but the institution as well. It also touched on travel arrangements and any safety risks that might emerge through non-standard bookings, etc.

Student feedback, and later, action plans (closing the loop)

This pair of complementary audits looked at our student satisfaction survey results and then how actions were developed to tackle persistent issues.

New programs

This took a look at innovation in the academic space, where it came from and how it was exploited.

Marketing our programs

This audit examined the connection between our centralised marketing department and the efforts of the academic units in direct marketing their programs to ensure consistency and efficiency.

Third party teaching arrangements

Numerous instances occur where a third party assists us in delivering academic programs, this includes partner institutions. We examined the arrangements and performed a risk assessment.

Academic governance

Like any governance audit, this examined academic governance to ensure that a policy framework, oversight and consistent approach existed.

Adjuncts

Adjuncts are mainly unpaid positions, however resources are often applied through other means; access to and provision of facilities such as laboratories, computers and equipment, office space; telephone and email; casual positions tutoring; etc. We looked into the cost/benefit of adjunct arrangements.

Examination processes

As a key form of assessment, we focused on the controls surrounding examinations from the setting of the papers through to marking and recording of grades.

Scholarships

This audit ended up being the creation of an inventory of every possible form of scholarship from direct funding, through tuition waiver, etc. We attempted to determine whether the application of scholarship ‘funds’ was aligned to strategic education foci.

Professional accreditation

In Australia this is a key component for marketing academic programs, so that graduates can join a professional association and start a career. We examined the risks associated with the process.

Fraud risk assessment: teaching and learning processes

This fraud risk assessment was quite complex and covered everything from assessment to plagiarism. The review was broad and could occupy an entire article on its own.

In closing, I’d like to encourage all college and university internal audit functions to consider including academic process audits in their ‘universe’. Deliberately aligned to risk and management expectations, true value add can be the result.

About the Author

Phillip Armanas

Phill Armanas is the Manager: Audit and Risk Management at the University of South Australia, based in Adelaide. Phill has more than 30 years internal audit experience in private and public institutions and has conducted several quality reviews under the IIA international standards. Phill will not disclose his age but adds that if he had a dollar for every year, he would be a wealthy man.

Articles
Auditing Academic Processes Down Under