Measuring Stewardship
Quick Stick: Ask Town of Vail to adjust the Stewardship surveying…
Acknowledge and measure the differences between the current context and regular use of language/concepts surrounding ‘environmental stewardship and the care of our wildlife and natural resources” ( Source: Resolution 22 a.k.a. East Vail Workforce Housing condemnation action)
Use specific language for more actionable insights
Provide definitions to avoid knowledge bias
Use smaller point scales & more ranking questions to avoid extreme/neutral bias
Try shorter surveys to capture larger audiences
Summarize response data by building personas
Is this a sticky idea? Share it and ask your friends what they think. Think it should stick? Email your comments to the Vail Town Council at towncouncil@vailgov.com
Avoid Demand Characteristics Bias
People tend to try to deduce what the goal of a survey is so they can give the “right” answer within the context of your study. This is critically important when measuring public sentiment related to environmental stewardship in Vail immediately following contentious action to condemn the East Vail Workforce Housing.
The condemnation resolution cites ‘environmental stewardship and the care of our wildlife and natural resources’. These are all key concepts and priorities that should garner overwhelming support from respondents taking an optional survey about stewardship.
Measure demand characteristic bias by asking separate questions that acknowledge the difference between the current context and the regular use of these concepts. This is a great opportunity to measure public sentiment about a contentious issue, while also capturing feedback on important concepts that use the same language. Conservation should not be controversial.
Suggestion:
“The Town of Vail has initiated condemnation proceedings to acquire the East Vail Workforce Housing parcel from Vail Resorts citing our commitment to environmental stewardship and the care of our wildlife and natural resources.
In the following section(s) we will ask you about your feelings specific to the condemnation action; we will then ask you about your feelings about these same concepts aside from the condemnation action. “
…then ask the same set of questions.
Avoid Non-specific language
Non-specific language confuses respondents and clouds data. Specific language provides insights that are actionable.
Instead ask: What specific qualities would this type of visitor have? What actions or behaviors distinguish them from other visitors? What are our values as residents? What actions or behaviors signal our values to others?
Specific:
High-spending visitors who contribute most to the local economy
come at less busy times
from other parts of Colorado
Remote workers staying for less than 30 days (*Note sales tax doesn’t apply to stays over 29 nights*)
Non-specific:
Visitors care about protecting our natural resources
wish to leave a light carbon impact when they travel
share respect for our local way of life
disrespect for the local way of life
Conservation and celebration of cultural heritage
Respectful visitor behaviors
Quality of life
Question 18: Do you or a close family member work in tourism, hospitality, or a related field?
What is a related field?
The ski industry (snowmakers? lifties? instructors?)
Retail (boot fitters? Sports apparel?)
Property management
Restaurants
Avoid Knowledge Bias
In order to focus the measure on respondents feelings, standardize the respondents knowledge. Provide a definition for a topic, then in a separate sentence ask about feelings:
Diversity is the presence of differences that may include race, gender, religion, sexual orientation, ethnicity, nationality, socioeconomic status, language, (dis)ability, age, religious commitment, or political perspective.
Equity is promoting justice, impartiality and fairness within the procedures, processes, and distribution of resources by institutions or systems.
Inclusion is an outcome to ensure those that are diverse actually feel and/or are welcomed. Inclusion outcomes are met when you, your institution, and your program are truly inviting to all. To the degree to which diverse individuals are able to participate fully in the decision-making processes and development opportunities within an organization or group.
Avoid Acquiescence (Agreement) Bias
People tend to default to being agreeable. Agreement bias can be exasperated by respondent fatigue and confusion. Combat this by asking the question again with inverse scaling and replace non-specifics with actionable.
“Greater cooperation between Vail Resorts and the Town of Vail would go a long way toward addressing tourism-related pressures.”
What does this mean if the user identifies with one of the four 4 levels of disagreement?
LESS cooperation …. would go along way toward addressing tourism-related pressures?
Greater cooperation … would NOT go a long way toward addressing tourism-related pressure?
“More frequent communication between leadership at Vail Resorts and the Town of Vail would help anticipate and mitigate overcrowding”
Use specific actions and measurable outcomes.
‘greater cooperation’ -> more frequent communication between leadership
‘would go a long way’ -> help anticipate and mitigate
‘addressing tourism-related pressure’ -> overcrowding
Avoid Extreme/Neutral Response Bias
Most of the questions use a 10 point scale to evaluate multiple concepts independently. It should be a 4 point scale for independent evaluation - or a priority ranking for related concepts.
A 5-10 point scale with a 60% neutral response range fails to distinguish between users that
don’t care about the topic
don’t understand the question
are genuinely undecided
are fatigued by the length of the survey
Summarize Results with Personas
Compiling large amounts of user data into actionable insights is best accomplished by building personas.
Useful Personas are
an archetype of a group of users
created by identifying themes from user research
reads like a real person, but doesn’t represent one specific person
helpful in understanding similarities and differences between users
class side from General Assembly
