Willingness of Online Access Panel Members to Participate in Smartphone Application-Based Research

There are limited academic activities on the methodological aspects of smartphone research applications. This book chapter focuses on this niche area of research. VeVa (Véleményem Van) is an online / mobile hybrid research panel which has been running since 2013 in Hungary. VeVa is already suitable for online research, and its smartphone research application will be available for panel members from 2015. Download of the application will be part of the recruitment process as well. The VeVa panel aims to convince both recent and future panel members to download and use its research application. An online survey was carried out in October 2014 in the VeVa panel to investigate who is willing to download the research application and why. A survey was conducted to evaluate nonresponse bias with an analysis of those panel members who are not willing to download the application. In this book chapter we analyze the motivations related to application download, comparing app accepters, app rejecters and uncertain respondents. This is followed by a detailed analysis of the three groups to detect differences in their characteristics. We have identified 19 significant variables and found smartphone application usage patterns as the most important explanation. There are only slight differences between the groups in regard to socio-demographic variables and to social and other further analyzed factors. How to cite this book chapter: Pinter, R. 2015. Willingness of Online Access Panel Members to Participate in Smartphone Application-Based Research. In: Toninelli, D, Pinter, R & de Pedraza, P (eds.) Mobile Research Methods: Opportunities and C hallenges of Mobile Research Methodologies, Pp. 141–156. London: Ubiquity Press. DOI: http://dx.doi. org/10.5334/bar.i. License: CC-BY 4.0. 142 Mobile Research Methods


Introduction: exploring smartphone research as a methodological topic
Due to the more frequent use of mobile phones in conducting surveys, new methodological questions have appeared in the world of research. Studies of the last few years have focused on questions such as the role of mobile phones in sampling (Andrews, Russell Bennett & Drennan 2011;Terhanian & Bremer 2012), the impact of smart technology on research (Mace 2012), mixed-mode and multiple devices (Callegaro 2013), mobile web surveys (de Bruijne & Wijnant 2013;Mavletova 2013;Mavletova & Couper 2013) and the effect of the use of mobile devices in a web panel (de Bruijne & Wijnant 2014). These research papers primarily engaged with the use of mobile phones in web surveys or other more traditional research methods. But mobile phones can be used to participate in research not only by voice or via browsers, but using dedicated smartphone applications for research purposes (and that even offline). There are many smartphone research applications in commercial market research. However, not so many research projects done with applications have compared them with online surveys (CAWI 32 ) or other current quantitative methods (CAPI,33 CATI 34 or PAPI 35 ). This is still true even though mobile research (MAWI 36 or MAPI 37 ) can be held as the fifth biggest research method besides the former mentioned ones (Snaith 2009). Both in MAWI and in MAPI mobile devices are mainly used in traditional ways and not with smartphone applications (of course there are a few exceptions, for example smartphone applications for data gathering with interviewers, e.g. droid Survey Offline Forms on Android or iSurvey Offline Surveys & Data Collection Forms on iOS).
Use of smartphone research applications is not so common in the academic sphere, despite the fact that these kinds of applications have been available for more than five years (for a first typology of 54 different research applications on iPhone and iPad see Michelson 2010 38 ). This is probably the main reason why research methodologists have not yet explored smartphone applications as an important topic of investigation on research and other methodological activities. While using mobile applications for data collection researchers should answer the following questions: 1) what are the characteristics of respondents with smartphones compared to those who do not have this device, 2) who is willing to download a research application among them and who is not willing, and finally 3) who participates in research (completes the tasks and questionnaires)?
This paper deals with these questions and applies the example of VeVa (Véleményem Van), a Hungarian online research panel, and its new research application which will be introduced in 2015. Based on Michelson (2010)'s classification, the VeVa smartphone research app is mainly for surveys. However, it can be applied for diary, ethnographic and location based research as well. The targets are consumers (members of the VeVa panel), the access is closed (available only for members: a user needs a username and password to login) and the app can be downloaded for free.
VeVa is an online / mobile hybrid research system in Hungary that has been running since 2013. Its explicit goal is to allow members to participate in traditional online surveys as well as in research through a smartphone application. Online research projects have been carried out since early 2014. The system will be ready for smartphone application research in 2015, with the involvement of panel members as respondents. Wide-scale internal testing of research applications has been running since 2014 with the help of panel members. The panel already had 15,000 members at the beginning of 2015, before the introduction of the research application. The long-range goal of VeVa is to collect 50,000 members, but only after the research application becomes available in app stores 39 and the download of the application becomes integrated into the recruitment process. In order to make the integration as smooth as possible preparative research was carried out in October 2014 among VeVa panel members. This study investigated the attitude of smartphone owners, in terms of their willingness to download the application. It also examined how the application will be used in various research types in the future (e.g. surveys, diaries, ethnographic studies, location based research projects and short mobile surveys). This research on attitudes had several objectives. First, it seeks to understand the characteristics of those panel members who would download the research application and to identify their interests in certain research types. Second, it seeks to see if there is any difference between those panel members who would download the research application and those who would not. Third, it aims to find out what is the reason behind the 'rejection' attitude and what arguments can be provided to convince those who are not willing to download the app. Fourth, it seeks to find out what the recommendation of this research could be with regard to the integration of the research application into the VeVa panel recruitment process, as well as the expected use of the research application. In this chapter I shortly present the results of this research and try to answer the above questions.

Background and hypotheses: an exploratory research
The research was designed primarily for exploratory purposes. We wanted to examine what variables have correlations with willingness to download the research applications. Three groups of these variables were identified: sociodemographic variables, smartphone and application using habits, and other background variables. The first two are obvious choices, but we had to define the third groups. Biler, Šenk and Winklerová (2013) found that the non-technical parameters of religion, use of shopping or of travel discount cards, and charity had a significant impact on participation in a study with GPS devices in the Czech Republic. We therefore included background variables in our questionnaire based on our hypotheses that willingness to download the research app may increase with the possession of loyalty cards and with frequency of legal gambling activities by respondents. We also assumed that research app downloading preferences may have a connection with free-time activities, with religion and with political orientation.
One of the explicit goals of this research was to understand the concerns of smartphone owners regarding the download of the research application. We investigated what would be the relevant answer to these concerns to be capable to adjust the recruitment process later based on this feedback.

Data collection
The online survey research was carried out in October 2014 among VeVa panel members on their attitudes toward downloading a smartphone research application. 40 The sample size was relatively big (N = 2028) compared to the total size of the panel (~15,000). This survey has taken into account the usual response rate in VeVa online surveys (~30%), and the actual response rate was 26%. The bigger-than-usual sample size was intentional because we planned to form different groups with the respondents to address further questions during the survey study. We used a quota method for sampling from the panel (gender, age and region quotas) in order to make our findings follow the structure of the Hungarian adult (older than 18-years-old) population.
The questionnaire contained eight sections altogether, which were the following in the order of sections (see Figure 1): 1. 'Section S' was submitted to everyone and contained only one question to screen out those who do not have Smartphones and who therefore cannot download the research application even if they would like to (these participants skipped most of the sections and completed only the final 'Section B'). 2. 'Section SO' was submitted only to Smartphone Owners and contained 14 questions. Part of the questions sought out basic data about smartphone devices, e.g. for how long the respondents had had smartphones, the types of operation systems and brands of their smartphones, and the respondents' application downloading habits in general. The other part of the questions focused on participating habits in online research with mobile devices. 3. 'Section D' contained only one screening question about whether the respondent would Download the dedicated research application of the VeVa online panel in the future. 4. 'Section P' was submitted only to those smartphone owners who responded Positively and reported they were willing to download the application (this group was referred to later as 'app accepters'). This section had seven questions: how often and for how long they would participate in smartphone research; what would motivate them to download the research application; what was their preferred types of incentives; and what was their possible future participation in short surveys, location based research, diary, ethnographic studies, passive measurement 41 and research triggered by a smartphone sensor. 5. 'Section U-N' was submitted only to those smartphone owners who were Uncertain or Negative about whether they would download the research application of VeVa. It contained only two questions. In the first one we showed eight different statements (possible concerns or barriers) about the research application, and respondents were asked to rate them on a scale of 1 to 4. Then we showed our concrete replies to those concerns/ statements which had been rated as 3 or 4 by the respondent earlier. After this we asked in the second question whether they had changed their minds and would download the research application of the VeVa panel.
6. 'Section C' was submitted to those smartphone owners who were previously uncertain or negative, but whom our arguments Convinced and who had changed their minds and now replied that they would download the research application. 'Section C' also contained seven questions, but  it was a shorter version of 'Section P' (the reason for the shortening was to decrease the possible attrition rate because of grid questions and the longer path of this group in the questionnaire). 7. 'Section R' was submitted to those smartphone owners who were previously uncertain or negative and whom our arguments could not convince. They finally Rejected the download of the application (this group is referred to later as 'app rejecters' and 'uncertain'). This section contained two questions. In the first one we showed eight possible reasons for rejecting the research application, and respondents could choose a maximum of three and rank them. The second question was an open question for those who had chosen other, non-listed reasons for rejection in the first question, so that they could specify their reasons with their own words. 8. 'Section B' was submitted again to everyone (including those who were screened out in the initial 'Section S'). It contained eight Background variables used for later analyses: how often and for what reason did respondents replace mobile phones; possession of loyalty cards; regularity of legal gambling; regularity of going out; preference for passive or active freetime activities; religion (religious or non-religious); and political orientation (left or right).
Socio-demographic variables (such as age, gender, region, settlement type, education, income etc.) were imported from panel variables based on replies given by respondents in a former recruitment process, hence we did not need to ask them again and the questionnaire could be shorter.

Accepters, rejecters and uncertain respondents: motivations for participation and reasons of rejection
61% of respondents (N = 1227) had smartphones within our sample. 21% of smartphone owners (N = 257) had already used their mobile devices to fill in 'traditional' online questionnaires earlier, but not a smartphone research app. Altogether, 10% of panel members 'filled in questionnaires of market research companies' by mobile, which is a relatively high percentage; however, it is in line with the international literature (Bosnjak et al. 2012;Jue 2015). 42 About 42% of smartphone owners reported they were willing to download the research application (accepters), 35% rejected it and 23% were uncertain (N = 1227). 42 It means that those members of VeVa who regularly fill in online market research questionnaires by smartphone represent 10% of the full panel (61% of panel members have a smartphone, 21% of them have filled in online questionnaires by mobile, and 78% of these 21% of smartphone using respondents replied in market research projects).

Motivation of accepters
What would motivate the first accepters (N = 510) to download the application? Motivations would be the following ones for them -with multiple responses option: 80% chose incentives (prizes); 47% were interested in innovations (we may say that it is the impact of novelty 43 ); 43% would like to have impact on products and services by expressing opinions in smartphone research projects; 37% like to fill in questionnaires; and 36% would like to tell their opinion. It means that two of the top three motivations mentioned are classical ones of the type which also motivated our panel members to join the panel itself earlier, and only the second argument (interested in innovations) is something which has a strong connection with the nature of the smartphone research application.

Reasons for rejection
The reasons for uncertainty or rejection in the first round given in 'Section U-N' (N = 717) were -as proportions of those who rated the given reason with 3 or 4 on a 1-4 scale: • there is not enough free time to participate in mobile research: 61% • there is not enough information to decide about the use of the research application: 53% • an expectation that use of the research application can cause extra costs (most probably because of limited mobile broadband plan of respondents): 45% • would participate only in some research projects but less likely in others: 44% • afraid that use of the research application would heavily drain battery of smartphone: 43% After asking about the reasons for rejection we gave respondents specific information regarding their previously chosen concerns. Therefore, in ' Section U-N' 27% of rejecters and uncertain respondents could be convinced to download the application, but 28% of them remained uncertain and 45% again rejected the download of the research app.
The main reasons for rejection in the second round (N = 502) were -as cumulated percentage of mentions as first, second and third reasons: • there is not enough free time to participate in mobile research: 63% • there is not enough information to decide about the use of the research application: 38% 43 The VeVa smartphone research application is the first of its kind available to Hungarian respondents, hence it may have some novelty to panel members.
• other: 37% (N = 134). Small screen size (N = 36), prefers computer to fill in questionnaires (N = 21), smartphone is used mainly for voice calls (N = 13). • afraid that use of research application would heavily drain battery of smartphone: 32% • would participate only in some research projects but less likely in others: 30%

Significant correlation between downloading research application and other variables
Altogether, 57% of smartphone owners were convinced in the first and second rounds to download the smartphone research application of VeVa, 27% rejected the app permanently and 16% remained uncertain at the end. So the majority of the sample was open to the research app. Every sixth respondent could be convinced with more information and more relevant arguments. And only approximately one quarter was reluctant to participate in smartphone application based research.
We have compared these three groups -accepters, rejecters, uncertain -in order to see which variables show significant correlation with willingness to download the research application. We wanted to identify significant differences between these three groups and find explanations of why respondents accepted or rejected the application or remained uncertain. Table 1 summarizes the results of significant correlations between variables and willingness to download the research application. We did not find any variable with a strong correlation, and we found only four variables with medium levels of correlation. Every other variable had a weak or very weak correlation.

Smartphone using habits
The following variables have a significant correlation with the three groups and the willingness to download the application: 44 • medium correlation: • how often respondent downloads applications (more regular app downloaders are open to research app, majority of non-downloaders reject it) • total number of applications on smartphone (non-linear connection) • number of applications regularly used (the more apps used, the more willingness to download the research app) • how often respondent used GPS on smartphone (more openness to research app if GPS is more often used)  Summarizing the findings we found that general application using habits are the utmost determining factors for the research app downloading preferences: if someone often downloads applications, uses them regularly and already has plenty of apps on their smartphone, they will more likely download the research app and give it a chance. However, these connections are still medium strong only. Former survey filling experiences also help to convince respondents that it is worthwhile to download the application.

Socio-demographic profile
Beside the variables analyzed above certain socio-demographic variables also have a correlation with the willingness to download the research application. It is important to see what kind of socio-demographic differences we have between the three groups: • weak connection: age (with increasing age the willingness to download the app decreases); monthly personal income (respondents with no personal income are more open 45 ); labor market status (students are more open to the research app, while pensioners are less open) • very weak connection: main earner (more willingness to download the app if respondent is not the main earner in the household) • no connection: gender; region; settlement type (i.e. capital city, county seat, city, village or other); education; marital status; main shopper of the household; number of children younger than 18 in the household; monthly household income; size of household; and subjective income status.

Discussion and conclusion
There are not enough experiments yet in the academic sphere on the methodological problems of running projects with smartphone research applications. That is why this research and these results could be interesting and useful for a wider audience. The main results of the research carried out with VeVa panel members on their willingness to download the research application of the panel are described below.

Reasons for rejection: 'more information is needed'
The reasons for rejection show that the panel members do not have enough information about the nature and functionality of the VeVa smartphone research application. We need to provide more information before asking to download the real app. Participation in mobile research is usually less time consuming than filling in a traditional online questionnaire (a mobile questionnaire is rarely longer than 8-10 minutes, while a traditional online questionnaire is rarely shorter than 8-10 minutes). From this perspective 'there is not enough free time' is a fear rather than a real barrier, especially if we keep in mind that these respondents are already members of an online research panel and actively participate in online research, so that they surely have free time to spend on research questions. 'There isn't enough information' as the second argument also convinced us that more information is necessary, and some respondents feel the same. Misbelief concerning extra costs was also derived from lack of information: the application is built in a way that means it works offline as well and uses only WiFi Internet connection by default. Of course respondents can change the settings and use 3G or 4G (mobile broadband) if they wish. Occasional participation in research as justification to reject the app download seemed to be strange. Participation in VeVa research projects has always been voluntary, and this is highlighted in every research invitation e-mail. This would not be different in projects run by smartphone research application. However, respondents need to be reassured that the rule of volunteering will not be changed with introduction of the app. The last reason of fast battery drain is also based on lack of information -however, one needs to try the application before this concern can really be disproved.
In the first round 42% of respondents were willing to download the VeVa smartphone research application, but after giving relevant information to the other participants, a further 15% were convinced. Meanwhile, the ratio of app rejecters decreased from 35% to 27%, and in the case of uncertain members it decreased from 23% to 16%, so it was possible to win supporters for the application from both groups.
In the main reasons for rejection in the second round, 'not enough free time' remained almost the same as in the first round (63% compared to 61% in the first round). This is possibly a good choice for some respondents to reject the application in a polite way, but this result also shows that our argument about volunteer participation was not convincing.
'Not enough information' decreased from 53% to 38%, which is understandable: those respondents who felt the received extra information was enough most probably were convinced and did not need to reply to the questions in this section. However, the fact that information scarcity is still the second barrier in the second round warns us that certain groups need more information about the application in advance. It is worthwhile to investigate this topic further, possibly with qualitative research methods.
'Other' as third reason in the second round also highlighted an important message: we did not think about every concern when we designed the questionnaire, so this result was very important. It would be a good idea to add the three new insights (small screen size, prefers computer to fill in questionnaires, smartphone is used mainly for voice calls) to the original list of eight arguments when we integrate the download of the application into our recruitment process.
The fact that a few arguments were not among the top five shows that these factors are less important for the majority of rejecters, and more important for researchers and VeVa system designers. However, it does not mean that these concerns can be neglected. A smartphone research application can be used for observation only with the consent of users. Personal data and responses must be used with care and under clear rules in a research environment. Finally, the application needs to be user friendly and simple to use. We would suggest that these factors represent a certain kind of minimum expectations (of respondents) that research apps must fulfill per se.
Considering the reasons for rejection provided, it is clear that prior to download more information is needed about the research application for the panel members. But most probably it is also true that it is worthwhile to choose carefully whom to inform, when and with what kind of information about the application and its expected use. Doing it the right way can improve the conversion rate and mitigate the fear arising from the uncertainty of the possible downloaders.

Slight differences only in socio-demographic profile; and smartphone using habits as decisive
It is an important lesson from the research that there is little difference between app accepters, app rejecters and uncertain respondent from a socio-demographic point of view. This is good news for the broad applicability of research application in the future. Nineteen variables have been identified during the research as significant ones, and they have medium, weak or very weak connection to the willingness groups. Nearly half of these variables are related to smartphones and their use. Smartphone application related habits (e.g. how often someone downloads applications, number of regularly used apps, total number of apps on smartphone) have the biggest impact on the willingness to download the research app. Social factors such as religious or political orientation, leisure time preferences or gambling habits have only weak connections to the three groups, so these factors most probably will not distort the sampling process and the results of future smartphone application based studies.
The results regarding the socio-demographic profiles mean at least two things: first, we need to find other variables if we want to explain why certain panel members are open to our smartphone research app and others are uncertain or reject it. Second, this result is a very good news from a smartphone research perspective: smartphone application based research can be carried out without the fear that participants are completely different in socio-demographic categories from those smartphone owners who do not want to participate in this kind of research. Socio-demographic characteristics of app accepters are somewhat, but not very, different from those of rejecters or uncertain respondents: they are a bit younger and more likely to be students with no personal income.
From the other 15 variables (with weak and very weak correlations), 5 are related to smartphones and 1 is technical. Altogether, nearly half (9 out of 19) identified variables are smartphone centered. Four variables are socio-demographic (two of them are economic) and five are social. From the previously incorporated social background variables, five proved to be a good decision and have correlation to the willingness of application download. However, these relationships are weak or very weak. Social factors are far less decisive than the already mentioned smartphone ones, or even than the socio-demographic ones. This can be considered again as good news. Research application accepters are only slightly different from app rejecters and uncertain respondents in their political views, religious attitudes, leisure time preferences or gambling activities.
Finally, it is quite sure that there are countless topics for smartphone application related research in the future. In the short term a possible topic can be, for example, the impact of different incentives on the willingness to download the application. The impact of other advantages when downloading the application (and the measure of the difference between the conditions) can also be an important variable. Finally we may focus on the investigation of privacy concerns and other factors with different wording at application download and with a measure of the impact of such different wording on the conversion rate.