Quality Evaluation of a Web-Based Life Insurance Claims Portal Using ISO/IEC 25010

Figure Ilustration AI

FORMOSA NEWS - Depok - Study Exposes Quality Deficits in Web-Based Life Insurance Claim Portals Using Data Triangulation Framework. A comprehensive software quality evaluation of a major web-based life insurance claim portal in Indonesia revealed significant functional and reliability limitations despite achieving high speed performance. Conducted by researchers Melati Indah Petiwi and Singgih Jatmiko from Universitas Gunadarma, the study examined operational data and user experiences between January 2025 and January 2026 before being published in August 2026 in the Formosa Journal of Computer and Information Science. The findings matter because digital insurance claims directly affect policyholders' financial well-being during critical life events, yet systemic technical issues continue to generate high volumes of customer complaints across the financial sector.

Background and Context
The Indonesian Financial Services Authority (OJK) set a strategic target for digital channels to handle 45% of insurance services by 2027. However, Indonesia's overall life insurance penetration remained low at 1.19% of GDP in 2023, well below the Southeast Asian regional average of 3.5%. While companies have rushed to digitize their services, delays and lack of transparency in online claims processing remain top sources of consumer dissatisfaction. To assess whether digital claim platforms meet technical and user requirements, researchers evaluated the primary claims portal of an anonymous Indonesian life insurer, designated as PT XYZ, using the internationally recognized ISO/IEC 25010 software quality framework.

Research Methodology
The authors designed a quantitative data triangulation framework combining three complementary streams of evidence: historical operational data, active system testing, and user perception metrics.

• Historical Ticket Analysis: The study analyzed a saturated sample of 144 problem tickets logged on PT XYZ's Jira system over a 12-month period to identify real-world production defects.
  
• Black-Box Testing: Evaluators executed 152 functional test scenarios across seven core portal modules to test system inputs and outputs without examining the underlying source code.
  
• Task-Based Evaluation (TBE): A single trained evaluator completed five critical tasks across 10 repeated trials under controlled conditions to measure processing efficiency without user-skill variance.
  
• User Perception Survey: A System Usability Scale (SUS) survey was administered to a representative sample of 250 active policyholders who had submitted claims through the platform.
  

Raw measurements were standardized and converted to a 5-point Likert scale (ranging from 1 for "Very Poor" to 5 for "Very Good") across four ISO/IEC 25010 quality characteristics: functional suitability, performance efficiency, usability, and reliability.

Key Findings
The overall evaluation demonstrated that while system speed and capacity perform exceptionally well, foundational functional and interface defects undermine the user experience.

• Functional Suitability (Score: 3 / Sufficient): Functional defects accounted for 61.8% of all logged problem tickets (89 out of 144) and contained all seven "Critical" severity bugs identified during the study period. Although controlled black-box testing yielded a 98.03% success rate, edge cases in real production environments generated recurring operational failures in claim status tracking, data validation, and payment confirmations.
  
• Performance Efficiency (Score: 5 / Very Good): System time behavior and capacity received top ratings. Evaluators achieved an average task completion rate of 1.0925 goals per minute, with 100% of tasks meeting efficiency benchmarks.
  
• Usability (Score: 3 / Sufficient): Policyholders surveyed reported an average System Usability Scale (SUS) score of 68.52 out of 100, placing the portal in the "OK" or marginally acceptable category. Respondents reported difficulties navigating complex forms and expressed a frequent need for technical assistance.
  
• Reliability (Score: 3 / Sufficient): Mean Time Between Failures (MTBF) averaged 40.56 days. However, system recovery showed severe bottlenecks: the Mean Time To Repair (MTTR) for fault tolerance incidents reached 55.5 days, far exceeding standard corporate IT resolution benchmarks.
  

Implications and Real-World Impact
The study demonstrates that traditional pre-release testing methods, such as basic black-box testing, fail to catch critical functional errors that emerge under actual operating conditions. For financial service providers and regulatory bodies, the findings highlight that high processing speeds do not compensate for functional software defects that risk miscalculating policy benefits or delaying payouts. Implementing continuous operational ticket monitoring allows tech teams to address high-risk defects before they disrupt claims processing. Short-term operational fixes recommended by the authors include enabling auto-save functions on multi-step forms to prevent claim data loss, correcting broken validation rules, and streamlining incident diagnostic runbooks to lower system repair times.

Author Profile
Melati Indah Petiwi, M.Kom. is a researcher affiliated with Universitas Gunadarma in Indonesia, specializing in software engineering, system quality evaluation, and information system management.
Singgih Jatmiko, Dr. is an academic and researcher at Universitas Gunadarma with expertise in computer science, software architecture, and digital system optimization.

Source
Melati Indah Petiwi, Singgih Jatmiko. Quality Evaluation of a Web-Based Life Insurance Claims Portal Using ISO/IEC 25010. Formosa Journal of Computer and Information Science (FJCIS), Vol. 5, No. 2, 2026, hlm. 325–340
DOI : https://doi.org/10.55927/fjcis.v5i2.17100
URL: https://journal.formosapublisher.org/index.php/fjcis