Das J, Holla A, Das V, Mohanan M, Tabak D, Chan B. In urban and rural India, a standardized patient study showed low levels of provider training and huge quality gaps. Health Aff (Millwood). 2012 Dec; 31(12): 2774-84, doi10.1377/hlthaff.2012.1356

There is little evidence on the quality of care patients receive in clinics in resource-poor settings. This study reports on the quality of primary care services in the public and private sectors in India, using about 926 clinical interventions between 305 medical care providers and 22 unannounced standardised patients in rural and urban settings.

The study included 248 out of 256 possible patient-provider interactions from a convenience sample in urban Delhi collected in 2009, and 677 out of 738 possible interactions amongst selected providers who accounted for 80% of all healthcare visits from households in 60 villages in three districts of rural Madhya Pradesh collected in 2010.

The standardised patients were recruited from the local communities where the study was undertaken and trained for 150 hours to consistently portray the emotional, physical and psychosocial aspects of the cases and accurately recall interactions. They were coached to avoid invasive medical examinations in order to avoid any potential harm, and to retain the medicines dispensed to them. The standardised cases presented with three common clinical scenarios: unstable angina, asthma, or dysentery of a child (without the child’s presence). Quality measures included adherence to defined” case-specific checklists” of essential and recommended care; the likelihood of correct diagnosis, and the appropriateness of treatment. Interactions were documented within one hour of the occurrence and medicines dispensed were saved and recorded whenever possible.

Results

In rural Madhya Pradesh, 11% of the providers had a medical degree and 67 % had no medical training. In public sector clinics, 63% of the interactions were with providers without a medical training. On an average, consultation time was 3.6 minutes and one third of the essential questions and examinations were completed. In only 30.4% of times was the correct treatment protocol followed. Also, 41.7% of the time, unnecessary or harmful treatment was prescribed. Only 12% of providers could give a correct diagnosis. Among those who provided a diagnosis, there is a direct relation seen between completion of the checklist and giving a correct diagnosis.

Multiple linear regression analysis of three measures of quality – percentage of recommended questions and examinations, relative risk of giving any diagnosis and relative risk of giving correct treatment – showed that qualification of the provider, infrastructure, and the patient caseload had little significantassociation with any of the quality measures. In rural MadhyaPradesh, in private clinics, the mean checklist completionrates were 6.81% points higher than in public clinics; butwith respect to the other two quality measures there was no significant difference.

In Delhi, 52% of private sector providers in the sample had a medical degree and all the public sector providers in the sample did. The average consultation time was 5.4 minutes in Delhi, and the rates of correct diagnosis (21.8%) and treatment (45.6%) were both higher than in Madhya Pradesh. But the essential checklist adherence in Delhi was 31.8% as against 33.7% in Madhya Pradesh.

The study concludes that training alone is not a sufficient arbiter of quality care. Both trained and untrained doctors were equally likely to not provide a diagnosis or correct treatment. (The paper also finds that “untrained providers in the private sector were more likely to adhere to checklists and were no worse in their treatment protocols than their public sector counterparts”.) (p 2781 )

Comment

The paper discusses the very important aspect of quality of care within the existing healthcare delivery systems in rural Madhya Pradesh and urban Delhi. It has used the standardised patient as the means of obtaining data.

The ethical dilemmas associated with the method, namely mystery/simulated patients, have been documented elsewhere (1, 2). However, the authors have failed to mention these potential dilemmas in the paper, thereby leaving it unclear as to what the methodological necessities were which outweighed the ethical dilemmas involved in the use of mystery/simulated patients.

The paper,s key finding is that adherence to checklists and providing diagnosis and accurate treatment are not shaped by the training received and there is reason to believe that untrained private sector providers were more likely to adhere to checklists. It is possible that the protocols that facilitated effective training of standardised patients were the training materials used by various providers to educate themselves; alternatively, the finding is an artefact of the training given to the standardised patients. If they were trained to note certain actions to be performed by the providers, they would. As they were not blinded to the qualifications of the providers they would note them more for those non-qualified. But any additional actions undertaken by the provider would not have been noticed or reported.

In Exhibits 4 (p2780) and 5 (p2781) of the paper, the authors have represented the marginal effects of three regression analyses of which the first regression depicts possibly the unstandardised regression coefficients. These unstandardised regression equations cannot be compared across the same model and they cannot be used as a marker of their magnitude of relevance. For this, one uses the standardised regression coefficient. Either this should have been clarified or such depictions” that give a false impression that private sector provisioning is more important than qualification or facilities and equipment” should have been avoided. Such a representation is erroneous and misleading.

As two-thirds of the scenarios involve chronic conditions, the study results may not be generalisable for communicable diseases.

The authors recommend reforms in the way medical degrees are awarded, in addition to better payment mechanisms and monitoring and denser peer network, the last possibly referring to more interactions, formal and informal, across providers in an area which will serve as a check on improper or ill advised medical practice. This conclusion is valid only if we know why the current set of observed providers did what they did. For this reason, it would not be prudent to use this study for generalisations about the determinants of quality of medical care in India.