How to Improve Performance in Public Sector Auditing Through the Power of Big Data and Data Analytics? – The Case of the Republic of North Macedonia

The accelerated process of digitalization and continuous development, and implementation of technology is a challenge, but also an opportunity to support the auditing profession. The ability to conduct more efficient, cost-effective, and effective audit engagement stems from the benefits of modern and practical computer-based solutions. The development of Internet-based technologies creates new standards and new ways of generating, analyzing, collecting, and archiving data. Big data is a fast-growing market that, according to ResearachAndMarkets (2022) is estimated to reach $273.4 billion in 2026, where data analytics software tools will be the market leader. According to a DataReportal (2022)’s report, more than 62.5% of the world's population is connected to and uses data online, while about 98% of the information stored worldwide is electronic. Modern audit engagements include examining clients who actively use cloud computing technology, big data, and data analytics to stay competitive in today's intense environment. In order to the audit engagement be carried out in more efficient and effective way, public sector auditors as „watchdogs” of public funds, must always be one step ahead of the audited entities, to have full insight into all data, and understand the operation and environment of the entity. The quality of audit engagements can be improved by using data analytics tools (Botez, 2018).

The purpose of this paper is to elaborate the application of big data and data analytics as one of the most advanced disruptive technologies in public sector auditing. The paper is structured in three conceptual sections, where at the beginning a brief introduction is given to the problem and the popularity of the covered topic, then qualitative research is made which includes a review of the relevant literature on the application of big data and data analytics in auditing and what are the key benefits of these technologies that can improve transparency, and accountability. The third section includes the qualitative research on the perception and attitudes of the public sector auditors, that are part of State Audit Office (hereinafter SAO), which is Supreme Audit Institution in Republic of North Macedonia, and the internal auditors in the public sector in the Republic of North Macedonia. The conclusion identifies the summarized results of the quantitative and qualitative research on the key benefitsand opportunities of big data and data analytics for public sector auditing, throughthe automation of routine procedures, complete and continuous audit, and greater effectiveness and efficiency in audit engagements.

According to CIPFA (2020), data can be defined as a set of non-random facts observed through inspection or research, while information can be defined as data that has been processed and thus gained greater meaning or usefulness.

Due to the complexity of the technologies and for a better understanding of the essence, it is of great importance to distinguish the terms, i.e., special areas that are closely related and complementary - data science, big data, and data analytics.

Data science compared to big data and data analytics is a broader field that includes everything related to data processing, cleaning, compilation, and analysis (Botez, 2018).

Big data is an expression for mass databases or quantities of data that have a large, diverse, and complex structure, which create difficulties for storing, analyzing, and visualizing them, but also carry huge potential for sequential processes, revealing hidden indicators and secret correlations, as well as generating results that influence decision making (Balios, Kotsilaras, Eriotis, & Vasiliou, 2020; Sagiroglu & Sinanc, 2013).

Data analytics is a process of examining data sets in order to draw conclusions about the information they contain, increasingly with the help of specialized systems and software (Deloitte, 2019; Botez, 2018; Appelbaum & Vasarhelyi, 2017). It is a process that uses data visualization, statistical and quantitative analysis, explanatory and predictive modeling of factual information, leading to an integrated and properly informed decision-making process (Deloitte, 2019). Applying data analytics to obtain evidence in the audit process is the science and art of identifying and analyzing suspicious and disputed transactions, deviations, irregularities, and other useful information about the subject matter of the audit through analysis, modeling, and visualization to plan or conduct an audit (IAASB, 2016; AICPA, 2015). According to AICPA (2017), the data analytics process is conducted in several stages and steps, starting with (1) data collection, followed by (2) cleaning and (3) manipulation to reduce the data to the required and useful format, which can then be (4) analyzed and (5) reduced to a visual format. Through the process of analysis of raw data, certain conclusions can be reached to identify suspicious irregularities, trends, current situations, comparisons, as well as information about events that are likely to occur in the future.

Тhere are several types of data analytics, which are interconnected and together represent the whole cycle of a detailed process of data analytics. They are descriptive, diagnostic, predictive, and prescriptive analyses (KPMG, 2016).

The benefits of using audit data analytics are (Earley, 2015; Cangemi, 2016):

- Testing 100% of the population, instead of using a sample;

- Better understanding of the client's work and its environment through the use of data from different sources;

- Increased effectiveness and efficiency in audit engagements;

- Automate recurring audit activities;

- Continuous audit;

- Greater focus and detection of fraud.

In addition to the benefits of audit data analytics, there are also limitations to the widespread implementation of the technology, such as (Earley, 2015; Balios, Kotsilaras, Eriotis, & Vasiliou, 2020):

- Implementation costs;

- Education and training to acquire the necessary skills;

- Lack of law/Standard for application of data analytics in auditing;

- Increased expectation gap;

- Unequal and unfavorable circumstances for the implementation of data analytics;

- Data security and confidentiality.

Auditing can benefit significantly from adopting big data technology, regardless of whether the client company is using it or not (Gepp et al., 2018). For the audit profession, the advancement of technology and the accelerated digitalization process is causing a new evolutionary phase in which many ideas and concepts that were once discussed can now be realized.

Developments in artificial intelligence, data analytics, blockchain, and other modern technologies have a significant impact on auditing and finance (ICAEW, 2018). Globally, governments, companies, professional auditors, and regulators are increasingly focusing on the impact and potential of technology. The private sector is often the initiator and is at the forefront of innovation, while the public sector tends to lag behind (Ellul & Buttigieg, 2021).

In the process of digitalization, a drastically larger amount of data is created, especially in electronic form, which means that the mass of data is enormously large and consequently, the use of a sample becomes an insufficiently good practical solution. The development of Computer Assisted Audit Techniques (CAATs) facilitates data extraction, sorting, and analysis procedures (Appelbaum & Vasarhelyi, 2017).

According to Gepp et al., (2018), there are many opportunities to use big data techniques in audit engagements, especially when advanced rigorous analytical procedures are performed in combination with traditional audit techniques and on which professional judgment is based.

Data analytics technology can be applied at all phases of the audit process (Stensjo, 2020 and CPA Canada, 2016), including planning the audit, conducting tests of controls and risk assessment, conducting analytical procedures and tests of details, as well as evaluating the results and forming an audit opinion.

Rakipi et al., (2021) tested the relationship between data analytics tools and internal audit functions, and they conclude that the advanced data analytics techniques positively affect the work of internal auditors and specifically their ability to identify internal control deficiencies and possible management fraud.

Governments are investing heavily in IT because of the enormous benefits that IT brings to public operations and services. Ahmi et al., (2014) indicate that a large number of public officials rely on IT to make decisions without understanding the essence of how computers work, so in order to reduce all the risks associated with the use of computers, internal auditors in the public sector should take steps in advance to ensure that the data provided to the decision-maker is reliable.

Cong et al., (2016) describe asset management as a process that is largely data-driven and that data is one of the key elements of asset management to support well- informed decisions. In their project to identify the benefits and limitations of big data in asset management, Cong et al., (2016) point out that public entity can be described as "data rich but information poor". Hence, big data and data analytics in public sector auditing are necessary to improve the management of public assets by recording, archiving, storing, searching, sharing, and analyzing data on state-owned physical assets to gain better and continuous insight, i.e. data that will be transparent, understandable, automated, and visual.

At the 23rd INTOSAI Congress in 2019 (INTOSAI, 2019), two resolutions on data and data analytics were prepared, encouraging SAIs to start the process of applying modern technologies and to respond effectively to the development opportunities that technological progress offers. According to Lewis et al., (2014), the adoption of advanced data analytics tools is a fundamental step in modernizing public sector audits and thus will enable the profession to add more value in terms of public sector asset management but also in terms of transparency, and accountability.

Data analytics has already become an integral part of the audit process in the National Audit Office (NAO) in the UK (Kelly, 2020). The author promotes the application of data analytics in public sector auditing through the successful practical application of data analytics tools in NAO and emphasizes how this technology assists auditors in their work, through: (i) increased productivity in auditing operations by automating routine procedures and enabling large amounts of data to be handled; (ii) creating new and deeper knowledge by linking the data and the causes of certain anomalies and inconsistencies; (iii) greater quality of audit engagements through consistency, avoiding mistakes or delays, and enabling tests to be performed on entire populations rather than through sample selection, thus overcoming the work traditionally.

During the accelerated process of digitalization, a drastically larger number of data is created, especially electronically, which means that the mass of data is enormously large and consequently the use of a sample becomes an insufficiently good practical solution. In this regard, Appelbaum & Vasarhelyi (2017) point out the need to apply big data analytics and testing the entire population, instead of selecting a sample with a smaller data volume that leads to increased audit risk. Audit data analytics can be applied to a wide range of audit engagements, including financial statement audits, performance audits, compliance audits, and non-assurance engagements, such as expert and advisory services (Cangemi, 2016).

There is a growing awareness in the public sector of the potential value that can begained from big data and data analytics as governments generate and collect hugeamounts of data through their day-to-day operations. The benefits of big data in thepublic sector auditing can be grouped into three main areas: (1) advanced analytics,through automated software tools and algorithms;(2) improvements in the effectiveness and efficiency of audit engagements and (3) ensuring greater transparency and accountability for the work of government, ministries and localauthorities (Munné, 2016).

We conducted a quantitative research through a survey questionnaire on the perception and attitude of the public sector auditors who are part of the SAO and the internal auditors in public sector departments. The survey questionnaire was web- based by sending an access link to all active external and internal public sector auditors, and consisted of 30 questions that required 6-8 minutes to answer. Most of the questions, with the exception of the questions where they had to answer "yes" or "no" and choose from the listed options, were asked on a 4-point Likert scale. In the research we used a 4-point Likert scale, of which 1 - I do not agree at all, 2 - I do not agree, 3 - I agree and 4 - I completely agree, in order to avoid the neutral answer and to see if the respondents agree or not with the statements for the improvement of audit engagements. According to Joshi & Pal (2015) and Chyung et al., (2017) the 4- point Likert scale is a type of forced scale that encourages users to express their opinions because there is no "safe" neutral option, and researchers use it to get a specific answers.

The questionnaire was structured to identify the following segments:

1. To what extent are public sector auditors familiar with and use big data (BD) and data analytics (DA) in audit engagements?

2. What are the most significant benefits and limitations of big data and data analytics?

3. Which of the key factors for big data and data analytics application: automation, effectiveness and efficiency, trust, continuous and complete audit are most important to auditors and can enhance public sector audit engagements?

Key factors were identified based on a relevant literature review and research already conducted in the field. For each factor, multiple questions were asked in shuffled order, and respondents expressed their views on how much they agreed with the statement.

The processing of the collected data was done through several statistical methods and tests, using SPSS software:

- Cronbach’s alpha to identify internal consistency in respondents ’responses.

- Descriptive statistics to perceive the aggregate statistics that quantitatively describe the answers of the respondents.

ANOVA analysis to examine the differences in the mean values of responses by the state auditors that are part of the SAO versus the internal auditors in the public sector departments.

The process of digitalization in the public sector has accelerated in the last two to three years, especially under the influence of the Covid-19 pandemic, which generates many opportunities for improvement, but also potential abuses and manipulation in the work of public entities. External and internal public sector auditors must harness the power of big data and effectively and efficiently extract all relevant information contained in it, in a timely manner.

The questionnaire was conducted in the period from February to May 2021 and was answered by 77 public sector auditors. The whole population, i.e. the total number of auditors to whom the questionnaire was sent was 222, from where the response rate is 34.7%. As of 2021, the number of public sector auditors part of the SAO was 90, while the number of internal public sector auditors at the central level was 132. The responsiveness is not large enough, but given the fact that this is an initial research from which early results are drawn and we strive to identify the attitude and perception for big data and data analytics in the domestic practice, it represents a satisfactory level of responses and is a good basis for further in-depth research.

Table1 presents the descriptive characteristics of the respondents that are part of the research where it can be seen that 31 out of 77 are external auditors, i.e., publicsector auditors that are part of the SAO, and 46 are internal auditors employed by various public sector entities at central level,including ministries,and other government departments and agencies.

According to many authors, Cronbach's Alpha is considered a measure of certainty, and the coefficient α must be greater than 0.7 (α ≥ 0.7) for the answers to be considered relevant and acceptable for further analysis. For the calculation of the Cronbach's Alpha indicator are excluded the questions that identify the characteristics of the respondents and two questions that are elaborated separately and refer to the benefits and limitations, where the respondents had the opportunity for multiple choice and additional field to add their personal opinion. Cronbach's Alpha is 0.835 (presented in Table 2), which means that the results can be considered relevant and appropriate conclusions can be drawn.

The process of digitalization in public sector entities is taking a big and accelerated steps, but it can still be concluded that it is at an early stage and we expect a period in which public sector entities need to be synchronized and continuously improved.

Chart 1 summarizes the responses of public sector auditors to the level of knowledge of big data and data analytics and their application in audit processes. The mean value of responses is 2.75 out of a maximum of 5, which indicates that auditors generally do not have sufficient theoretical and applicative knowledge. Public sector auditors have an mean value of responses of 2.97, while internal auditors 2.61, which indicates that the state auditors who are part of SAO have a greater knowledge of the application of big data and data analytics in audit engagements.

When asked if they use advanced data analysis tools, as can be seen in Chart 2, only 24 public sector auditors or 31% said that they use advanced tools in their work, while 53 or 69% have not used any advanced tools so far. Although an equal number of public sector auditors as part of SAO (12) and public sector internal auditors (12) have answered that they use sophisticated data analytics tool, it can still be concluded that external public sector auditors are more up to date with technology due to the percentage share in the number of respondents, i.e. 39%, unlike the internal audits where only 26% have given an affirmative answer.

Public sector auditors in the Republic of North Macedonia have a low level of knowledge and, accordingly, little applicative use of big data and data analytics, which suggests that more education, training and workshops are needed to promote the benefits of advanced technologies, and also greater pressure on public institutions is needed to invest more in modernizing and digitizing audit processes.

Auditors were asked to select two to a maximum of three benefits and limitations that they consider to be of paramount importance for the implementation of big data analytics software solutions in Macedonian audit practice. Table 3 presents the auditors' responses, where the benefits and limitations are ranked according to the number of responses received, i.e. according to their significance.

Although all the listed benefits are complementary and related, respondents believe that increased effectiveness and efficiency in work is the most crucial benefit of big data and data analytics and perhaps the best argument to motivate them to implement advanced software solutions. In terms of limitations, there is an almost equal number of answers regarding the necessary education and training to acquire the necessary skills, the high implementation costs that are a real constraint in the public sector, and the insufficient degree of digitalization in the public entities.

The total value of the answers to the questions ranges from 1 to 4, where 2.5 is considered an average and is a limit by which the auditor's answer is measured whether he agrees and expresses a positive attitude (>2.5) or not agrees and expresses a negative attitude (<2.5). The total average answer, derived through all the mean values of the questions set on 4-point Likert scale is 2.98, which is a higher value than 2.5 and it can be concluded that in general, auditors have a positive attitude and perception of big data and data analytics in public sector auditing. In other words, domestic practice auditors agree that data analytics can improve, modernize, automate, and digitize the audit process. The descriptive statistics of th individual questions are shown in Appendix A.

In question no. 14 the auditors were asked at what phase of the audit process do they think that big data and data analytics would contribute the most, where most of the answers refer to the phase of conducting analytical procedures and test of details (28 responses) and the phase of conducting control tests and assessing the risk of material misstatement (26 responses).

According to the purpose of the research and review of relevant literature, we were able to identify 4 key factors that can improve the performance of public sector auditing. In order to identify which of the following factors has the highest mean value of the answers per respondent and consequently to determine which of the factors has the greatest importance and impact, we analyzed several questions grouped according to the respective factor, as shown in Table 4.

The results for the mean values of the answers are shown in Chart 3. The results show that the automation of routine procedures and the use of advanced software tools instead of certain manual procedures are the key factor that public sector auditors consider to be the most important and that through automation auditors will be able to devote more time to more important aspects of working with which will improve the overall performance of audit engagements.

All factors have an average value of responses greater than 2.5 which indicates that all are important and through greater automation, greater effectiveness and efficiency in the work, trust in the generated data and conducting a complete and continuous audit, auditors will be able to optimize work and with limited resources to conduct many more audit engagements compared to the traditional way.

As can be seen in Chart 3 and Appendix B (ANOVA test) where a comparison is made between the public sector auditors that are part of SAO and the public sector internal auditors there is no significant discrepancy. Hence, it can be said that public sector auditors generally have consistent views on big data and data analytics, which is a good indicator because their cooperation and compliance is important in taking new and innovative steps to improve the performance of audit engagements.

From the conducted ANOVA analysis, it is worth mentioning that there are three questions where there are significant differences in the responses between the internal and external public sector auditors. Only in Q16 there is a significant difference in the answers between the groups, while in the other two questions (Q28 and Q29) the difference arises from the different answers within the group. Public sector auditors who are part of SAO are neutral on whether there must be a separate ISSAI or law governing the application of data analytics (mean = 2.58), while public sector internal auditors find it extremely necessary to there is adequate regulation (mean = 3.13).

All respondents with mean values of the responses greater than 3.4 believe that in the near future it will be necessary for auditors to possess appropriate IT skills and that is to implement new study programs or subjects in which the technology will be studied and its application in auditing.

The fast-growing digital world, together with the presence of smart devices, changes the daily way of life and requires all information to be available anywhere and anytime. Today, every organization strives to implement technology in the processes that becomes a necessary factor for long-term success and achieving the set goals. Auditing as a profession is not immune to the digitalization process. The current pandemic Covid-19 is forcing organizations and professions around the world, including Republic of North Macedonia, to change the way of working and to adapt to the digital world and new way of working.

Big data and data analytics represent new evolutionary phase in auditing and upgrade the profession from manual tests and physical controls to digital and automated auditing, where most technical audit procedures will be performed through computer software, while the auditor will focus on those areas and aspects where it is necessary to judge, evaluate, etc.

The results of the research conducted on public sector auditors in Republic of North Macedonia shows that both external and internal public sector auditors do not have sufficient knowledge and experience in applying big data and data analytics in audit engagements, but it is very important that we identified their positive attitude towards technologies, and they agree that advanced analytics tools can improve audit performance. When analyzing the key factors arising from the application of big data and data analytics in the audit process, we can conclude that the automation of routine audit procedures, that traditionally take a lot of time and processing resources, is most important from the point of view of respondents, and also the increased effectiveness and efficiency in audit engagements that can be achieved through the modernization of audit procedures. However, there are no drastic differences in terms of key factors, which can be said that public sector auditors agree that big data and data analytics contribute to improving the performance of audit engagements through automation, more reliable and accurate data generated by computer software instead of manually generated, the ability to receive timely and up-to-date information in real time and the maximum utilization of available resources can achieve great effectiveness and efficiency in auditing.

The application of data analytics in public sector auditing results in a multifaceted benefit, where in addition to better audit performance, the performance of public services provided by the government and public sector entitles will be improved. Directly and indirectly, public sector auditors influence better management of public funds, greater accountability and responsibility, and thus better policies and services provided to citizens.

A significant limitation in our research stems from the level of digitalization in domestic practice and the different levels of maturity and acceptance of change, where awareness and propensity for IT solutions is not at the required level, nor at a level compared to more developed countries. The use of a 4-point Likert scale is intended to obtain specific answers and avoid a neutral response and attitude, but in terms of statistical data processing and the formation of a hypothesis testing model is a weakness and limitation.

To the best of our knowledge, this paper is the only one that explores the potential of big data and data analytics in public sector auditing in the Republic of North Macedonia and provides a good basis for further detailed research on this issue. We believe that this issue is extremely important for the development of the auditing profession, but also for the work of the government and public sector entities. We will continue this research with additional research methods, such as interviews and workshops that will include a number of external and internal public sector auditors and representatives of relevant ministries and public sector entities that are policy makers to discuss and research the benefits that technologies bring and what is the best way for an appropriate and easy transition process from traditional to modern auditing engagements. We also want to encourage state authorities to think and organize professional education, workshops and events that will encourage responsible persons, as well as public sector auditors to accelerate the process of digitalization and modernization in audit engagements.