Original paper
Development of Parents’ of Children with Hymenoptera Venom Allergy Quality of Life Scale (PoCHVAQoLS)

Introduction

Hymenoptera venom allergy (HVA) is a potentially life-threatening disease and may have a considerable impact on the health-related quality of life (HRQoL) of patients. This impact is related to the patients’ awareness of their vulnerability and their understanding of its implications, as well as to the severity of symptoms of the disease in a given individual. In the case of children, the disease may also affect the quality of life (QoL) of their parents. The parents, feeling responsible for their children’s life and health, undertake some actions, which, in their opinion, will protect the child from being stung, or at least from the most dangerous consequences of a sting. These actions may include being overprotective of their children, talking to them about their allergy and possible consequences of being stung, or restricting their activities. Above all else, the parents should provide their children with appropriate treatment. Younger children may not be aware of a danger related to a sting, so these actions to a greater extent may be a burden for their parents. Though in adolescents this burden may decrease, the anxiety of parents about their children may continue to remain high.
In 2002, the first tool for measuring HRQoL in adult patients with wasp venom allergy was published – the Vespid Quality of Life Questionnaire (VQLQ) – but until 2010, there was no such tool for children and their parents [1]. The first adaptation of VQLQ for Polish children and their parents was published in 2010, whereas its adaptation for Polish adolescents appeared in 2012 [2, 3]. The VQLQ, and its Polish versions, were indexes consisting of 14 items, which were chosen based on the impact methodology. The items, based on results of content validity analysis, could be divided into 4 domains: anxiety, caution, limitations and discomfort, but the VQLQ authors presented the scale as a one-dimensional measurement tool [1]. Factor analysis conducted on Polish adaptations of VQLQ supported the results of content analysis and revealed that the index measured 3 dimensions of paediatric patients’ HRQoL and 4 dimensions of parents’ QoL, but the limitations dimension was measured by one item only [2, 3]. The same authors also showed that different determinants affected each dimension of QoL in Hymenoptera venom (HV) allergic children and their parents, what supported the thesis that QoL of these groups was a multidimensional construct [4, 5]. This oriented our efforts to develop scales measuring QoL in HVA patients and their parents based on valid measurements of particular dimensions of the children’s and parents’ QoL.
In 2013, the first scale for children with HVA – Children’s Hymenoptera Venom Allergy Quality of Life Scale (CHVAQoLS) – measuring 6 dimensions of their HRQoL: anxiety, caution, limitations, discomfort, received support and feeling of safety, was published [6]. Though HRQoL of children with HVA is more important for physicians, QoL of their parents should not be neglected. As shown by our previous work, anxiety and caution of the parents and their HVA children correlated positively; however, the cross-sectional design of our study did not allow for answering the question whether increasing anxiety in children increased the anxiety level in their parents, or else the causal direction went the other way. Moreover, increasing levels of parental caution increased their level of discomfort, so it is probable that the child’s treatment will increase QoL of its parents. No similar relationship was observed in children, possibly due to differences in measurement tools characteristics [4].

Aim

This paper presents the development of the QoL scale for parents of children with HVA, including an analysis of the scale theoretical validity (i.e. answering the question whether the scale measures the latent variables it is intended to measure) and reliability (i.e. answering the question whether the results obtained using the scale precisely reflect variability of the measured latent variables).

Material and methods

Study sample

The study sample included 70 parents of children referred to an allergy centre at the University Children’s Hospital of Krakow, Poland, with a history of systemic reactions to insect stings. The consultations of the children were held during the period from 2000 to 2010. Informed written consent was obtained from the parents. The study took place from May to July 2010 and was approved by the Jagiellonian University Bioethical Committee (KBET No. 67/L/2007, signed 28 June 2007).

Methods

The multistage development process of this scale started from creating sets of items (called domains), which provided indicators available for selection to subscales measuring particular dimensions of parental QoL. The subsequent steps of this process included the selection of items, testing psychometric properties of particular subscales and multidimensionality of the created scale. All the stages of the development process were described in detail elsewhere [6].

Statistical analysis

In the paper introducing CHVAQoLS, we introduced a quasi-Likert response format for items measuring phenomena non-observed in all the respondents. Multiple Correspondence Analysis (MCA) was performed to check whether in the parents sample items for which responses were worded in a quasi-Likert format scale behaved as ordinal variables, as do items with a typical Likert response format [6, 7]. All the correlations between the items included in a particular domain were assessed using the Kendall’s tau-b correlation coefficient [8]. The unidimensionality of the scales constructed to measure the particular domains were tested with principal component analysis (PCA) [9]. The multidimensionality of the scale was tested using the multitrait matrix method [10]. The reliability of the subscales in terms of the Classic Test Theory (CTT) with respect to their internal consistency was tested with the use of the Cronbach  coefficient and with respect to the Item Response Theory (IRT) with the rho coefficient [11, 12]. The IRT characteristics of particular items and subscales were tested based on Mokken analysis procedure [12]. The homogeneity of the subscales and scalability of the items were tested using Mokken analysis procedure and measured with the Loevinger H coefficient [13]. The null hypotheses were rejected at 0.05 level. All the analyses were performed using SPSS 15 statistical package, except Mokken analysis, which was done with MSP 5 statistical software.

Results

Sample characteristics

The surveyed parents were at the mean age of 40.7 years (SD = 6.6), with a predominance of mothers in the sample (78.6%). The majority of the parents were married (92.9%) and lived in villages (60%). They mostly completed secondary education (42.9%); of the remaining parents, a similar percentage of subjects were university graduates (27.1%) or had a vocational (or lower) level of education (30%).

Response format

Figure 1, originating from MCA, shows that the first two options of the quasi-Likert response format (used in the support subscale as well as the limitations and discomfort subscales), related to a lack of stimulus and lack of reaction to the stimulus, respectively, are located at coordinates different enough along the 1st dimension of the solution, which explains a higher percentage of variance of the set of variables included in the analysis. The result, consistent with that obtained during CHVAQoLS development, indicates that variables with a quasi-Likert response format behave as typical Likert scaled items, and two options related, for instance, to lack of behaviour causing limitations and to the same behaviour but causing no limitations, can be treated as different levels of limitations [6].

Item selection

The initial pool of 56 items, collected based on the existing scale for parents of HVA children and on information obtained in talks with parents accompanying their children during clinical consultations was prepared. The items were divided into 6 domains. Particular domains included 8 items for measuring anxiety, 9 items for measuring caution, 7 items for measuring limitations, 12 items for measuring discomfort, 9 items for measuring support provided to children and 11 items for measuring feeling of safety. Table 1 presents arithmetic means (column A) and standard deviations (column B) of all the items from the initial pool grouped in particular domains. Column C of Table 1 presents the number of salient (i.e. higher than 0.7) correlations for each item. The analysis of correlation matrix for every domain separately was the basis for excluding items with salient correlations, what led to diminishing the number of items available for creating subscales to: 5 items for anxiety, 7 for caution, 2 items for limitations, 7 for discomfort, 6 for support and 9 for safety.
Principal component analysis conducted for each domain separately (except the limitations domain due to 2 items left in the domain at this step) allowed for removing items which were too low-correlated with the first principal components (factor loadings presented in Table 1, column D), what led to retaining 5 items for anxiety, 6 for caution, 7 for discomfort, 4 for support and 6 for feeling of safety. Mokken analysis conducted separately for each of the five domains (excluding limitations due to 2 items left in the domain at this step of the analysis) showed that the scalability coefficient H for the subscales including all the items from the domain exceeded 0.5 for all the domains except feeling of safety. Similarly, the Hi scalability coefficients for items available for creating particular subscales were higher than 0.5 for all the items from all the domains except safety. In the case of this domain, all the items with the exception of one were characterized by the Hi coefficients lower than 0.5 (Table 1, column E).
After applying Mokken analysis, the following variables were preselected to the pre-final version of particular subscales: A1, A2, A3 and A9 for measuring anxiety, C1, C2, and C9 for measuring caution, L2 and L6 for measuring limitations, D2, D7 and DB for measuring discomfort, S2, S3 and S6 for measuring support provided to children, and F2, F3, F6 and F7 for measuring feeling of parental safety. The multitrait matrix analysis showed that item A9 correlated with the discomfort subscale within one standard deviation of its correlation with the anxiety subscale, so it was removed from the scale. Items C1, C2 and C9 were too high-correlated with the anxiety, safety and discomfort subscales, respectively, so they were replaced by C5, C6 and C9 items, which constituted an internally consistent subscale. Since the limitations subscale consisted only of 2 items, we decided to exclude item D2 and combine the limitations and discomfort items into one subscale.
Table 2 presents psychometric properties of the developed subscales and their items. Column F shows the values of Cronbach  coefficients for the final version of particular subscales in the sample of parents of children with HVA (in the row containing the name of the subscale), and the value of  coefficient after removing the item from the subscale (in the row containing the particular item). Column G presents the values of rho reliability coefficients (in the row containing the name of the subscale) and estimation of convergent validity of a particular item estimated as the correlation between the item and the summary score of the subscale it belongs to, after correction for overlap (i.e. after removing the item from the subscale score – in the row containing a particular item). Column H contains estimation of discriminant validity of the items calculated as the correlation between the item and the subscales that do not contain that item, whereas column I presents scalability coefficients, obtained from Mokken scaling analysis, for the subscales and their items.

Subscales divergent validity

Correlations between particular subscales (excluding feeling of safety) ranged from 0.37 to 0.45, suggesting a moderate relationship between particular dimensions of parental QoL and supporting the thesis that the extracted dimensions of QoL were independent. Correlations of safety with other subscales ranged from –0.14 (not significantly with caution) to –0.50 (Table 3).

Reliability and scalability

Internal consistency of three subscales was higher than 0.8, whereas for two other subscales, the  coefficient was markedly higher than 0.7. None of the items caused an increase in the value of the  coefficient when removed from the subscale it belonged to (Table 4).
In terms of IRT, reliability of the subscales was very close to that estimated in terms of CTT for all the subscales (except feeling of safety, which was not evaluated in these terms) ranging from 0.75 to 0.84. Only for the anxiety subscale was the reliability coefficient rho markedly lower than the  coefficient. If the feeling of safety subscale was evaluated in terms of IRT, the rho coefficient for this subscale would equal only to 0.65 (Table 4).

Discussion

This paper presents the development of the QoL scale for parents of children and adolescents with HVA. Our aim was to create a short scale (up to 5 items per dimension) intended to measure 6 dimensions of QoL of this group of parents. Hence, we decided to base the entire process of selecting the items for particular subscales from the initial item pool on their intercorrelation pattern, with regard to reliability of the chosen sets of items and their correlation patterns with items selected to other subscales [10].
We formulated the scale items with no evident referral to the illness the scale is dedicated to (similarly to CHVAQoLS and contrary to the way it was done in VQLQ), as we assumed that such feelings as anxiety about insects, feeling of safety with respect to possible stings and their consequences, or finally, behaviours leading to avoiding stings and discomfort caused by these feelings and behaviours, were related not only to the diagnosis of HVA, but also existed in a healthy population, and their extent was associated with psychological characteristics of an individual [1, 6]. Such characteristic features are shaped by personal experiences, knowledge derived from one’s experience or from observation of people from the individual’s environment; hence, we wanted to develop a tool that could be applied in the future to study both the parents of children with and without HVA.
We created the tool consisting of five subscales, because during the selection process, we noted that in the limitations domain, too many items overlapped, so we decided to combine the domains of limitations and discomfort. It could result from similarity in the wording of particular items measuring limitations, so future efforts for measuring this dimensions in parents of children with HVA should be directed at taking into account a broader range of their problems. On the other hand, it could also result from the similarity in meaning of these two latent variables.
We selected the items separately for each subscale; however, the multitrait analysis proved that all the items of the scale except one correlated with other subscales weaker than its correlation with its own subscale minus one standard error. However, objections in this matter did not markedly affect the discriminant validity of the subscales. Similarly to CHVAQoLS, the presence in the subscale of one item correlated too strongly with one of the other subscales, was efficiently suppressed by other items, whose discriminant validity was obvious [6]. In spite of the objections concerning divergent validity of the items, correlation between any two subscales did not exceed 0.5, what is distinctly lower than the 0.7 threshold usually considered as indicating redundancy between variables [14]. The highest correlation between the developed subscales was obtained for anxiety and safety, what may actually be an effect of the similarity in the meaning of the measured latent variables (some may treat caution as a way of reducing their anxiety).
Correlations between the anxiety and caution subscales of the created tool are weaker than these obtained for the Polish adaptation of VQLQ for parents of children with HVA, what means that our scale measures two independent, although to some extent related, latent traits. Correlations of the limitations and discomfort subscale with other subscales are stronger as compared to these of the limitations subscale and weaker than these of the discomfort subscale included in the Polish adaptation of VQLQ for parents of HV allergic children. This resulted from the different meaning of both subscales of VQLQ adaptation, in which the limitations subscales addressed limitations imposed by parents on their children, whereas the discomfort subscale addressed the parental knowledge about their child’s HRQoL. In the newly created tool, it strictly addressed parental feelings and behaviours. Additionally, in the Polish adaptation of VQLQ for parents, limitations are measured with one question only, what makes the meaning of that question very broad and in consequence very subjective, while in our scale, the assessment of limitations is based on some particular criteria [2].
Although we created a short scale for measuring particular dimensions of QoL of parents of HVA children, the internal consistency of the majority of the subscales was higher than 0.8, whereas the Cronbach  for other subscales distinctly exceeded 0.7, what is considered as a minimum acceptable value for group comparisons [15]. As the subscales were very short, we obtained a slightly lower reliability coefficient as compared to the anxiety and caution subscales of the Polish adaptation of VQLQ for parents of children with HVA. That adaptation measured anxiety with the reliability of 0.90, whereas we obtained the value of 0.84, both for 3-item subscales. In the case of the caution subscale, we obtained  equal to 0.75, whereas in the adaptation of VQLQ for parents of children with HVA,  reached 0.86, both for 3-item subscales. In the case of discomfort and limitations measurement, we obtained a lower value of internal consistency coefficient (0.82 for a 3-item subscale) than in the adaptations of VQLQ for parents of children with HVA; however, in that version of the index, the discomfort subscale consisted of 7 items, and its reliability equalled 0.98 [2]. However, this particular subscale could include overlapping items, similar to the VQLQ adaptation for adolescents [16].
As to the psychometric properties of the subscales estimated in terms of IRT, we obtained reliability coefficients close to those measured in terms of CTT for all the subscales except anxiety. Rho coefficients were higher than 0.8 for the limitations and discomfort subscale, and for the support subscale. Reliability of the safety subscale was not assessed in terms of that theory, because the results of Mokken analysis showed that homogeneity coefficients for this subscale and scalability coefficients for most of its items were lower than 0.5, what we considered a minimal accepted value; however, other authors allow scalability coefficients higher than 0.3 [12]. If we considered the feeling of safety subscale as a hierarchical one, its reliability would equal 0.65, what is below the threshold of 0.7, what should be treated as the acceptability level for all reliability coefficients. We cannot compare these values with other scales measuring QoL in parents of HVA patients, as they were not evaluated in terms of IRT [2]. Scalability coefficients for the developed subscales were also very high – for all the subscales and for all the items, they were equal to or higher than 0.5 – which threshold indicates scales with a strong hierarchical relationship [12].
We encountered, however, some difficulties that demanded making decisions that might markedly influence the quality of the developed tool. First of all, we had to choose whether the development of the scale should be based on impact methodology, which is often used in the case of rare diseases (such as HVA – 0.5%, food allergy – 2% or even asthma – 5%) [1, 17–19], or on methodology the origins of which come from psychological research on traits of personality, which is often used in HRQoL research [20–22]. The main drawback of development methods omitting the use of factor analysis is lack of possibility to properly assess multidimensionality of the created tool, what makes the interpretation of the scale scores unclear [2, 3]. The use of impact methodology may also lead to redundancy between items (i.e. intercorrelations higher than 0.7). Although the authors of VQLQ stated that between the items of VQLQ there was no redundancy, in at least one Polish adaptation of VQLQ redundant items occurred [1, 16]. However, the fundamental objection that may be raised against impact methodology is that selection of the items concerns properties (importance and relevance) other than those, which will be later interpreted (i.e. items’ scores).
In the case of the multitrait method, similarly as in development of CHVAQoLS, we decided to loosen the condition of distinguishing between the items’ convergent and divergent validity from two standard errors, as proposed by Hays, to one standard error. This decision was made because the estimation of standard error depends on sample size, and in our small sample, this would lead to treating many items as not divergently valid; however in the parents sample, the majority of the items would fulfil the original condition [10].
As we wanted to avoid the use of conditional items addressing actions or feelings, which could not been reported by all the parents, we had to solve another difficult problem associated with the response format for subscales containing such items. A solution – items with a quasi-Likert response format – has been presented in the development of CHVAQoLS [6]. The results of MCA presented in the current paper supported the results obtained in the case of CHVAQoLS: in the space defined by variables coming from different domains, answer options of the quasi-Likert response format addressing, for example, lack of discomfort related to non-avoidance of places where insects occur and feeling no discomfort in spite of avoiding such places are located at different coordinates on the dimension which explains the highest percentage of variance of the analysed items set. Similarly as for CHVAQoLS, we may conclude that the difference between these two options of a quasi-Likert response format related to lack of discomfort is similar to that observed between other answer options, which are formulated in a typical Likert format, and should be treated as different levels of the measured latent variable. A similar decision was reached with respect to items from the support subscale [6].
The main limitation of the presented study addresses the same issue as in the case of the development of CHVAQoLS – a too small sample size than that necessary to perform either exploratory (which requires a number of observations of at least to 5 times the number of variables included in the analysed set of items) or confirmatory (at least 200 observations) factor analysis to directly prove multidimensionality of the scale [9]. We planned to create a scale including about 20 items, what required our enrolling at least 100 parents of children with HVA. We were able to collect a sample of 142 parents of children aged 10 to 18 years, diagnosed in the University Children’s Hospital in Krakow because of HVA. Unfortunately, they were mostly accessible by mail only, due to their place of residence being up to 250 km away from Krakow. We wanted to develop a scale that would be valid for the entire sample of parents of young HVA patients (irrespectively of the Mueller grade of systemic allergic reaction of their children), so the questionnaires had been sent to all the available parents (also to those of children with large local reactions). The number of questionnaires that had been sent back did not let us gather a sample the size of which would be sufficient to analyse multidimensionality of the P^oCHVAQoLS using PCA. Nevertheless, we tried to reach the goal by some indirect means, i.e. by conducting multitrait scaling [10].
The response rate obtained in our study is typical for mail surveys, even conducted in large epidemiological studies, in which it usually does not exceed 50%, but sociologists treat this number as providing reliable and valid data [23, 24]. Moreover, the use of a mail survey may not ensure to all the parents the same conditions at the moment of filling the questionnaire, although we may assume that they were answering the questionnaires in circumstances better reflecting their everyday emotional status, what could render the obtained results more reliable.
The next important limitation of the study concerns the fact that we were not able to provide cross-sectional validation, because the entire available sample was used to develop the scale. The cross-sectional validation allows for confirming the psychometric properties of the scale in respondents not involved in the development process, because a repeated use of the tool in the same sample may increase reliability and validity estimations [25]. In the case of HVA, as a rare disease, a number of patients qualified every year for treatment is rather small – even in large medical centres, this number reaches about 10 persons per year (whereas the catchment area may have up 300 km in diameter). Considering this, the duration of gathering a sample required to conduct exploratory (at least 100 parents of HVA patients) or confirmatory (at least 200 parents needed) factor analysis would be very time-consuming and its duration may be estimated as at least 10 years.

Conclusions

P^oCHVAQoLS is a highly valid and reliable scale for measuring QoL in parents of HVA children and adolescents. The tool will provide deeper insight into this matter, especially when used together with CHVAQoLS [6].

Acknowledgments

The study was conducted in the Department of Paediatrics, Polish-American Institute of Paediatrics, Jagiellonian University Medical College, Krakow, Poland. The work should be attributed to the Jagiellonian University Medical College, Chair of Epidemiology and Preventive Medicine, Department of Medical Sociology, Krakow, Poland

Conflict of interest

The authors declare no conflict of interest.

References

1. Oude-Elberink JNG, de Konchy JGR, Golden DBK, et al. Development and validation of a health-related quality-of-life questionnaire in patients with yellow jacket allergy. J Allergy Clin Immunol 2002; 109: 162-70.
2. Brzyski P, Cichocka-Jarosz E, Tobiasz-Adamczyk B, et al. Adaptation of Vespide Allergy Quality of Life Questionnaire for Polish children with venom allergy and for their parents. Przegl Lek 2010; 67: 1237-42.
3. Brzyski P, Cichocka-Jarosz E, Tobiasz-Adamczyk B, et al. Theoretical validity and reliability of Vespide Quality of Life Questionnaire in Polish adolescents with Hymenoptera venom allergy. Postep Derm Alergol 2012; 29: 8-13.
4. Cichocka-Jarosz E, Tobiasz-Adamczyk B, Brzyski P, et al. Health-related quality of life in children with Hymenoptera venom allergy treated with VIT and quality of life of their Barents. Przegl Lek 2010; 67: 1243-8.
5. Cichocka-Jarosz E, Brzyski P, Świebocka E, et al. Health-related quality of life in Polish adolescents with hymenoptera venom allergy treated with venom immunotherapy. Med Sci 2012; 8: 1076-82.
6. Cichocka-Jarosz E, Brzyski P, Tobiasz-Adamczyk B, et al. Development of children’s hymenoptera venom allergy quality of life scale (chvaqols). Clin Transl Allergy 2013; 3: 25.
7. Garson GD. “Correspondence analysis”, from Statnotes: Topics in Multivariate Analysis. Retrieved 01/07/2010 from http://www2.chass.ncsu.edu/garson/pa765/ statnote.htm.
8. Kendall MG. A New Measure of Rank Correlation. Biometrika 1938; 30: 81-9.
9. Garson GD. “Factor analysis”, from Statnotes: Topics in Multivariate Analysis. Http://www2.chass.ncsu.edu/garson/pa765/statnote.htm.
10. Hays R, Fayers P. Evaluating multi-item scales. In: Assessing quality of life in clinical trials. Fayers P, Hays R (eds). Oxford Medical Publications, New York 2005; 41-54.
11. Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrica 1951; 16: 297-334.
12. Molenaar W, Debets P, Sijtsma K, Hemker BT. MSP, a program for Mokken Scale analysis for Polytomous items, user’s manual, version 3.0. Iecprogramma. Groningen 1994.
13. Loevinger J. The technique of homogeneous tests compared with some aspects of ‘scale analysis’ and factor analysis. Psychol Bull 1948; 45: 507-30.
14. Juniper EF, Guyatt GH, Streiner DL, et al. Clinical impact versus factor analysis for quality of life questionnaire construction. J Clin Epidemiol 1997; 50: 233-8.
15. Nunnaly JC, Bernstein ICH. Psychometric theory. III ed. Mcgraw-Hill, New York 1994.
16. Brzyski P. Aspekty prawne i metodologiczne użycia skal badawczych w badaniach epidemiologicznych. Przegl Lek 2012; 12: 1287-92.
17. Juniper EF, Guyatt GH, Feeny DH, et al. Measuring quality of life in children with asthma. Qual Life Res 1996; 5: 35-46.
18. Juniper EF, Guyatt GH, Feeny DH, et al. Measuring quality of life in the parents of children with asthma. Qual Life Res 1996; 5: 27-34.
19. Flokstra-de Blok BMJ, Dunn-Galvin A, Vlieg-Boerstra B, et al. Development and validation of the self-administered Food Allergy Quality of Life Questionnaire for adolescents. J Allergy Clin Immunol 2008; 122: 139-44.
20. Ware JE, Snow KK, Kosinski M, et al. SF-36 health survey: manual and interpretation guide. The Health Institute, New England Medical Center, Boston 1993.
21. Beck AT, Steer RA, Brown GK. Manual for Beck Depression Inventory (2nd Ed.). Psychological Corporation. San Antonio TX 1996.
22. The Kidscreen Group Europe. The Kidscreen Questionnaires. Quality of life questionnaires for children and adolescents. Pabst Science Publishers. Lengerich 2006.
23. Babbie E. The practice of social research. 9th ed. Wadsworth/Thomson Learning. Belmont 2001.
24. Calnan M, Wadsworth E, May M, et al. Job strain, effort-reward imbalance, and stress at work: competing or complementary models? Scand J Public Health 2004; 32: 84-93.
25. Hornowska E. Psychological tests. Theory and practice. Wydawnictwo Naukowe SCHOLAR, Warsaw 2009.