Campus Units

Industrial and Manufacturing Systems Engineering, Landscape Architecture, Kinesiology, Virtual Reality Applications Center, Human Computer Interaction, Aerospace Engineering

Document Type


Publication Version

Accepted Manuscript

Publication Date


Journal or Book Title


Research Focus Area(s)

​Operations Research, Ergonomics and Human Factors




Facilitated-voluntary geographic information (f-VGI) is a promising method to enable systematic collection of data from residents about their physical and social environment. The method capitalizes on ubiquitous mobile smartphones to empower collection of geospatially-referenced data. It is important to evaluate the validity of user-generated content for use in research or program planning. The purpose of this study was to test whether the aggregated environmental (“bikeability”) ratings from novice community residents converges with ratings from experts using a robust research-based, paper audit-tool (the established Pedestrian Environment Data Scan (PEDS) tool). Equivalence testing statistically showed overall agreement between the composite ratings of bikeability within the novice group. Agreement in categorical ratings between novices and experts were examined using the summary agreement index, which showed substantial agreement across the 10 locations rated by 11 novices using an f-VGI mobile application and four experts using PEDS; variability depended on the nature of the specific questions asked. Results reveal overall substantial agreement between novice and expert ratings for both composite scores and individual categorical ratings. However, additional research is needed to refine the methodology for use in formalized research applications.


This is a manuscript of an article published as Kalvelage, Kelly, Michael C. Dorneich, Christopher J. Seeger, Gregory J. Welk, Stephen Gilbert, Jon Moon, Imad Jafir, and Phyllis Brown. "Assessing the validity of facilitated-volunteered geographic information: comparisons of expert and novice ratings." GeoJournal (2017), doi:10.1007/s10708-017-9781-z. Posted with permission.


The final publication is available at Springer via

Copyright Owner

Springer Science+Business Media Dordrecht 2017



File Format


Published Version