This study attempts to analyse the Critical Failure Factor (CFFs) for Enterprise Resource Planning (ERP) system implementation in one of the small and medium enterprises (SME), located in Sabah, Malaysia. Enterprise A has been chosen where the case study is applied to better understand how this firm has implemented its Mind Your Own Business (MYOB), a computerized accounting system software as its ERP system. The case study is carried out based on four phases as suggested by Ganesh et. al 2010 which are (i) planning, (ii) implementation, (iii) stabilization and (iv) improvement. The results show that, lack of communication within the firm during initial stage of project implementation, lack of information system knowledge and lack of vendor supports while implementing, stabilizing and improvise the software are the major failure factors for Enterprise A. Throughout this study, we suggest some recommendations to Enterprise A in respond to the critical failure factors faced by Enterprise A.