Post-implementation Evaluation of a Digital Dictation System in a Large Health Service Using HOT-Fit Framework

Abstract

Objective: The objective of this study was to evaluate a digital dictation system (DDS) that has been implemented in a large health service. The data collected in this study was used to understand the overall uptake and performance of the system and consequently improve the quality of care delivered by the organisation.                          

Methods: A mixed method research design was used in this study. Clinicians and Medical Transcriptionists using the DDS across the two campuses of the health service over a period of four months, were surveyed to assess the extent to which the implementation of the DDS is fulfilling its purpose. In addition, system usage statistics, project implementation documents and user support emails were also analysed.

Results: This study utilised an existing comprehensive and validated evaluation framework, the Human, Organisation and Technology Fit (HOT-Fit) framework. Human fit: 79.55% (n=35) of Clinicians and 33.33% (n=2) of MTs reported an overall satisfaction with the DDS. Organisation fit: The document analysis revealed that the DDS selected aligned best with current organizational IT strategies and was an easy fit with existing practices. Technology fit: An overall satisfaction of 53.49% (n=23) from the Clinicians and 16.67% (n=1) from the MTs was reported on the DDS. Out of 22 issues lodged regarding the system, 77.27% (n=17) issues met the Service Level Agreement (SLA).

Conclusions: The overall findings of the study suggest that, the DDS was a good fit within the organisation in terms of Human fit and Organisation fit. In terms of Technology fit there existed some technical issues on the end-user level due to the system being new to the end users. To overcome this and facilitate the smooth functioning of the DDS, effective communication with the vendor and other relevant stakeholders was recommended so that end users i.e. the Clinicians and MTs understand the system, its functionality and their role in providing timely and high-quality information for clinical care.