Proposal - Probolsky Research - 2019-08-16Costa Mesa Sanitary District
Proposal for:
Citizen Survey Services
Prepared. for:
Nabila Guzman', ManagementAnalyst I
August t6, 2019
C� �3,�00
rPRO BOLSKY Market and Opinion Research:
RESMCH Elections and Public Policy
L,
Probollsky Research
3990 Westerly Place Suite 185
Newport Beach, CA 92660
Newport Beach (949) 855-6400
San Francisco (415) 870-8150
Washington DC (202) 559-0270
Table of Contents
A. SUBMITTAL LETTER..........................................................................................................................................1
B. DESCRIPTION OF FIRM.....................................................................................................................................2
C. PROPOSALS......................................................................................................................................................3
PARTIAL GOVERNMENT CLIENT LIST.................................................................................................................................. 3
D. ORGANIZATIONAL STRUCTURE........................................................................................................................ 5
E. REFERENCES & DESCRIPTION OF EXPERIENCE...................................................................................................7
SIMILAR PROJECTS AND REFERENCES................................................................................................................................ 7
WesternCommunity Energy................................................................................................................................ 7
Cityof Newport Beach......................................................................................................................................... 7
DublinSan Ramon Services District..................................................................................................................... 7
OrangeCounty Sanitation District....................................................................................................................... 7
OrangeCounty Storm Water............................................................................................................................... 7
SantaMargarita Water District.......................................................................................................................... 7
ValleyWater........................................................................................................................................................ 7
F. PROJECT OVERVIEW.........................................................................................................................................8
RESEARCHOBJECTIVES................................................................................................................................................... 8
DueDiligence....................................................................................................................................................... 8
ResearchGoals.................................................................................................................................................... 8
RESEARCHMETHODOLOGY APPROACH............................................................................................................................. 9
Statistically Valid Multi -mode, Multi-lingual Survey Methodology..................................................................... 9
In -person and Online Focus Groups................................................................................................................... 11
Availabilityand Capability................................................................................................................................. 11
G. DETAILED WORK PLAN...................................................................................................................................12
SURVEYPROCESS........................................................................................................................................................
12
SampleStratification.........................................................................................................................................12
InitialKick-off Meeting......................................................................................................................................
13
StaffReview.......................................................................................................................................................13
Conductingthe Survey.......................................................................................................................................
14
DataProcessing.................................................................................................................................................
14
Verbatim Responses to Open -Ended Questions................................................................................................
14
DataAnalysis.....................................................................................................................................................15
Reporting...........................................................................................................................................................
15
TIMELINE...................................................................................................................................................................
16
SurveyTimeline Example...................................................................................................................................16
H. COST PROPOSAL............................................................................................................................................17
APPENDIX/ADDITIONAL INFORMATION.............................................................................................................18
REPORTINGEXAMPLES................................................................................................................................................. 18
H. COST PROPOSAL.........................................................................................................SEE SEPARATE ENVELOPE
ELEMENTS INCLUDED IN PRICING.......................................................................................................SEE SEPARATE ENVELOPE
Probolsky Research Newport Beach (949) 855-6400
3990 Westerly Place Suite 185 San Francisco (415) 870-8150
PROBOISKY Newport Beach, CA 92660 Washington DC (202) 559-0270
RESEARCH
SURVEY PRICING OPTIONS................................................................................................................SEE SEPARATE ENVELOPE
IN -PERSON OR ONLINE FOCUS GROUPS (OPTIONAL) ..............................................................................SEE SEPARATE ENVELOPE
Probolsky Research Newport Beach (949) 855-6400
3990 Westerly Place Suite 185 San Francisco (415) 870-8150
PROBOLSUNewport Beach, CA 92660 Washington DC (202) 559-0270
RESEARCH
A. Submittal Letter
August 16, 2019
Nabila Guzman
Management Analyst I
Costa Mesa Sanitary District
290 Paularino Ave
Costa Mesa, CA 92626
SUBJECT: Consultant Services for Citizen Survey Services
Nabila:
Thank you for the opportunity to once again offer our research services to Costa Mesa Sanitary District.
We look forward to helping the District evaluate ratepayer opinions of the District's rate structure for
solid waste and sewer services and assess awareness of solid waste and sewer programs. We look
forward to helping the District improve programs and achieve high levels of satisfaction.
Having conducted thousands of market and opinion research projects among customers, residents and
voters for government agencies, Probolsky Research has unmatched experience researching and
messaging on community satisfaction and waste services.
Market research on solid waste and recycling and wastewater has been a central practice area for our
firm for more than 20 years. This includes work on behalf of governments such as the City of Newport
Beach, Rossmoor Community Services District, Orange County Sanitation District, Irvine Ranch Water
District and waste haulers and solid waste consultants like HF&H. Additionally, I previously served as an
Orange County Waste and Recycling Commissioner.
We love measuring data for comparison over time (trend analysis) and look forward to conducting this
analysis using our 2016 survey as a baseline. Our firm is adept in employing multiple methodologies
(telephone, online, in -person, video) in our research, involving both traditional research tools and new
innovative technology that lead to insightful, statistically significant results.
I am an authorized representative of the firm. Our firm has both the staff availability and financial stability
to complete this work for the District and we are available to start work immediately. You can count on
me to be at District Headquarters, in -person, as much as needed, as we design the research, and after
our research is complete to develop the ideal reporting to meet the District's needs. This proposal shall
remain valid for a period of 60 days from the date of submittal. We look forward to working on this
important project.
Sincerely,
Adam Probolsky
President
PRU�ISKY Newport Beach
RESEARCH
San Francisco
PRINCIPAL CONTACT:
Adam Probolsky, President
Probolsky Research
3990 Westerly Place Suite 185
Newport Beach, CA 92660
adamp@probolskyresearch.com
Telephone: 949-855-6400
FAX: 949-855-6405
Washington DC
B. Description of Firm
Established in 1992 (27 years in business) and organized as a Limited Liability Corporation, Probolsky
Research LLC specializes in market and opinion research. We are a woman -owned, Latina -owned
Certified California Small Business Enterprise (#1758330).
We are a medium-sized firm with a staff of nine. We believe the District will benefit from our size and
organizational structure.
Our firm has zero bureaucracy. All Probolsky Research staff is cross -trained, every member of our team
will be intimately familiar with the District's project details, and we will all work on the project together.
This ensures continuity, timeliness, and quality of our work product. This is something that a firm with
just one or two staff members cannot accomplish.
While Adam Probolsky will serve as project manager and the District's point of contact, you have a large
pool of experienced staff with broad government research experience that can assist you at any time.
Our research services include:
• Telephone surveys • Community meetings and outreach
• One-on-one interviews • Social media surveys
• Employee surveys 0 Focus groups
• Online surveys • Field Focus GroupsTM
• Multi -mode surveys • Public Listening ToolsTM
We are independent, non -partisan researchers. Our mission as researchers is to deliver our clients
accurate results that are both actionable and easy to understand.
We do not charge for travel. We value in -person meetings and presentations, and we do not charge for
travel or any travel -related expenses.
Our work does not end after we deliver our final report. We place no limitations on our availability for
meetings (whether in -person or by telephone), and no limit on updated or customized reporting and
never charge extra for those services.
Probolsky Research understands ratepayer satisfaction, solid waste and waste water.
For more than 20 years, our firm has worked directly for public agencies and industries on solid waste,
recycling and waste water issues. Most of this work relates to understanding attitudes and household
awareness. We believe this diverse client base offers us a greater understanding and ability to deliver
quality results to the District.
From Costa Mesa Sanitary District's anaerobic digester to the latent waste collection powers of the
Rossmoor Community Services District, the Orange County Water District's treated water outflow ocean
pipeline, Irvine Ranch Water District's Michelson Water Recycling Plant, the City of Newport Beach's
move to outsource their solid waste collection after decades of doing it in-house, to our work on behalf
of dozens of waste haulers throughout the country, our depth of experience is unmatched.
Firm president Adam Probolsky was appointed to serve on his county's Waste and Recycling
Commission (2007 — 2009) because of his background in waste and his deep knowledge of how the
industry interacts with customers.
��ISKY Newport Beach San Francisco Washington DC
RESEARCH
C. Proposals
Probolsky Research has worked with thousands of clients assessing public opinion and customer
satisfaction. Below is a partial list of government clients. Contact information for several references are
below in section E.
Partial Government Client List
Town of Avon, CO
Bear Valley Community Services Dist.
California Coastal Conservancy
City of American Canyon
City of Beaverton, OR
City of Brisbane
City of Colton
City of Colton Water Department
City of Corona Dept. of Water/Power
City of Del Mar
City of Diamond Bar
City of Encinitas
City of Fontana
City of Garden Grove
City of Grand Terrace
City of Huntington Park
City of Irvine
City of Laguna Niguel
City of Loma Linda
City of Loma Linda Water Division
City of Los Angeles (Harbor City)
City of Mission Viejo
City of Napa
City of Newport Beach
City of Norco
City of Perris
City of Pismo Beach
City of Placerville
City of Pomona
City of Redlands
City of Redlands Utilities Department
City of Rialto
City of Riverside
City of San Bernardino Water Dept.
City of Santa Ana
City of Santa Clarita
City of San Clemente
City of Sierra Madre
City of Stanton
City of Twentynine Palms
City of Upland
City of Yorba Linda
Coachella Valley Water District
Coachella Water Authority
Costa Mesa Sanitary District
Contra Costa Water District
County of Orange
County of Kern
Desert Water Agency
Dublin San Ramon Services District
East Valley Water District
Eastern Municipal Water District
Elsinore Valley Municipal Water District
Great Parks of Hamilton County, OH
Huntington Beach Union H. S. District
Indio Water Authority
Irvine Ranch Water District
Joshua Basin Water District
Jurupa Community Services District
Los Angeles County Sanitation District
Los Angeles Unified School District
Marin Municipal Water District
Mesa Consolidated Water District
Mission Springs Water District
Municipal Water District of O. C.
Nevada Department of Transportation
Nevada Irrigation District
Newhall County Water District
O. C. Stormwater Program
Orange County Cemetery District
Orange County Fair
Orange County Sanitation District
Orange County Transportation Authority
Orange County Water District
Paramount Unified School District
Rialto Water Services
Riverside Highland Water Company
Riverside Public Utilities
Riverside Water Symposium
Rubidoux Community Services District
San Bernardino County Flood Control
San Bernardino Municipal Water Department
PROBO M 3
RFH Newport Beach San Francisco Washington DC
L
San Bernardino Valley Municipal Water District
Upper San Gabriel Valley Municipal Water Dist.
San Bernardino Valley Water Conservation Dist.
Victor Valley Water District
San Gorgonio Pass Water Agency
Water Replenishment Dist. of Southern CA
Santa Ana Watershed Project Authority
West Valley Water Conservation District
Santa Clara Valley Water District
West Valley Water District
Santa Margarita Water District
Western Municipal Water District
Temescal Valley Water District
Yorba Linda Water District
Texas Department of Transportation
Yucaipa Valley Water District
PROBOISKY Newport Beach San Francisco Washington DC
RESEARCH
D. Organizational Structure
This organizational chart includes the Probolsky Research team that would be assigned to this project.
Laura Flores Victoria Griffin Gabby Benitez Alyssa Tutterow Shalom Veffer Samantha Calloway, LCSW, MSW
Research Analyst Research Analyst Research Analyst Research Analyst Production Director Clinical Social Worker
Unique to Probolsky Research: Our team is cross -trained in all aspects of the research process
and every team member will be involved with our work for the District. Our philosophy in a cross -
trained team is what ensures we deliver accurate results in a timely manner.
Desiree Probolsky, MBA, CEO
Desiree has more than a decade of branding, market research and strategy development experience.
She works with our broad client base spanning business, government, political, and non-profit sectors.
A credentialed English teacher and former legislative staffer for the California State Senate, Desiree
earned her MBA from the Paul Merage School of Business at the University of California, Irvine and her
undergraduate degree at California State University, Fullerton.
Adam Probolsky, President
Project Manager
Adam has acted as pollster and strategic advisor on hundreds of research projects for local, county and
statewide governments and citizen outreach efforts.
Additionally, Adam has been a key advisor to the firm's clients on matters of public policy, legislation
and business strategy. Adam was both a planning and finance commissioner for the City of Irvine.
As a past chairman of the Irvine Valley College Foundation, he has overseen the fundraising and the
award of hundreds of thousands of dollars in scholarships to students and funding for college police. He
was also a member of the Orange County Waste & Recycling Commission where he oversaw landfills,
recycling programs, waste hauling companies and power generating facilities.
PRUBO)�KY Newport Beach San Francisco Washington DC
RESEARCH
He previously served as a board member and development committee chairman for Jewish Family
Service of Orange County, a non-profit organization that serves families in need, and volunteered for
nearly eight years with the Orange County Sheriff's Department as a spokesperson.
Adam served on the Environmental Oversight Committee for the Orange County Transportation Authority
where he allocated over $700M in public funds used for the preservation and restoration of natural
lands.
Adam Probolsky serves as the project manager on all research projects our firm conducts and oversees
all of the work that is carried out by staff. Adam will be the point person to represent Probolsky
Research, as well as the primary contact person for the District during the project period.
Samantha Papurt Calloway, LCSW, MSW, Clinical Social Worker
Samantha is a Clinical Social Worker with Probolsky Research. Samantha graduated with her B.A. in
Sociology from Brandeis University and her Master of Social Work from the University of Southern
California. She is a licensed clinical social worker with a deep understanding of the human experience.
Her expertise with accessing the emotional drivers behind resident behaviors brings a new dimension to
the research process and informs our outreach.
Laura Flores, B.A., Research Analyst
Laura is a Research Analyst with Probolsky Research and is a graduate of the University of South
Florida. She is involved in all aspects of the diverse research projects our firm conducts.
Victoria Griffin, B.A., Research Analyst
Victoria is a Research Analyst with Probolsky Research. She is a graduate of the University of California,
Irvine. Victoria is involved with all types of research projects we conduct for clients. Victoria also has
experience working on political campaigns.
Gabriela (Gabby) Benitez, B.A., Research Analyst
Gabby is a Research Analyst with Probolsky Research. She is a graduate of the University of California,
Irvine. Gabby is involved with all research projects our firm conducts.
Alyssa Tutterow, M.P.P., Research Analyst
Alyssa is a Research Analyst with Probolsky Research. Alyssa graduated with her B.A. in Environmental
Biology and International Studies from Washington University in St. Louis and with her Master of Public
Policy from University of California, San Diego. Alyssa is involved in all research projects our firm
conducts including statistical and geographic analysis. She also has public service experience working
with local governments and non-profit organizations through the Peace Corps and AmeriCorps.
As a Peace Corps Volunteer, Alyssa implemented a comprehensive household waste management
survey in rural Peru and provided results to the regional government.
Shalom Veffer, Production Director
Shalom is an accomplished story teller. For more than a decade, he has been producing compelling
video and graphics. Shalom is involved in all aspects of the messaging content, motion graphics,
presentation graphics, printed materials, and video we produce for our clients.
PROBOLM 6
RESEAIltCH Newport Beach San Francisco Washington DC
L
E. References & Description of
Experience
Below you will find past projects/references similar in scope and size as the Scope of Service described
in the RFP. All of these projects were completed on time and within budget.
Similar Projects and References
Western Community Energy
Project Name: Ratepayer Survey
Agency: Western Community Energy 13390 University Ave. Suite 450, Riverside, CA 92501
Client contact information: Barbara Spoonhour 1 951-405-6760 1 bspoonhour@wrcog.us
City of Newport Beach
Project Name: Community research (focus groups and surveys)
Agency: City of Newport Beach 1 100 Civic Center Dr., Newport Beach, CA 92660
Client contact information: Tara Finnigan 1 949-644-3035 ( tfinningan@newportbeachca.gov
Dublin San Ramon Services District
Project Name: Ratepayer research (Focus group and survey)
Agency: Dublin San Ramon Services District 17051 Dublin Boulevard, Dublin, CA 94568
Client contact information: Sue Stephenson 1 925-875-2295 1 stephenson@dsrsd.com
Orange County Sanitation District
Project Name: Engagement survey (survey)
Agency: Orange County Sanitation District 110844 Ellis Avenue, Fountain Valley, CA 92708
Client contact information: Lori Karaguezian, Principal Staff Analyst 1 714-593-7087 1
Ikarag_uezian@ocsd.com
Orange County Storm Water
Project Name: Resident research (Field Focus GroupsTm and surveys)
Agency: OC Stormwater 12301 N Glassell Street, Orange CA 92865
Client contact information: Richard Boon, Chief of Stormwater Program 1 714-995-0670
richard.boon@ocpw.ocqov.com
Santa Margarita Water District
Project Name: Ratepayer research (focus groups and surveys)
Agency: Santa Margarita Water District 126111 Antonio Parkway, Rancho Santa Margarita, CA 92688
Client contact information: Jim Leach, Director of External Affairs 1 949-459-6601 jiml@smwd.com
Valley Water
Project Name: Flood outreach research (focus groups and surveys)
Agency: Santa Clara Valley Water District 5750 Almaden Expressway, San Jose, CA 95118
Client contact information: Gina Adriano 408-630-2853 1 gadriano@valleywateL=
PROBOMY Newport Beach San Francisco Washington DC
RESEARCH
L
F. Project Overview
Research Objectives
Due Diligence
We begin every research project by understanding the history of how our client got to this moment in
time. We have already collected and reviewed open -source information including the Costa Mesa
Sanitary District Strategic Plan, as well as recent Rate Studies. Having previously conducted a
Community Survey the District in 2016, we are already familiar with many of the unique needs of the
District.
You cannot overwhelm us with background information and data. We are experts at parsing through
solid waste and financial reports, legislation, news clips and other District documents and finding
relevant data. We believe this is an essential part of ensuring we can be strategic partners. There is
nothing peripheral about our work.
Research Goals
Specifically, our research will:
• Identify key issues of importance among Costa Mesa Sanitary District ratepayers regarding
services and rates
• Determine ratepayer perceptions of the District, services and facilities
• Measure ratepayer satisfaction with current rate structures
• Assess community support among emerging concerns
• Profile respondents demographically/geographically throughout the District such as by age,
gender, home ownership, and by geographic area within the District
• Capture how sentiments have changed over time for trend analysis (see example — Figure 1 -
below)
• Establish a roadmap with key recommendations for staff to utilize for program improvements
PRUBOI.SKY Newport Beach San Francisco Washington DC
RESEARCH
L
53.8 % say Beaverton is on the right track
Question 2: Do you think that the City of Beaverton is on the right track or on the wrong track?
53.8 %
Right track
27.3%
19.0%
Wrong track Unsure/Refused
Figure 1: Comparison data over time (Beaverton, OR)
�s o%
61-0% w 0% tie 0%
see%
—*-Right track
o%
:wor —*—Wrong track
2es% v 3%
20 U'Yo
20.Uyn 'J ,../ "Unsure/Refused
190
1b U`Yy 130
ia.Wap, I'll,
15U% 160"iu 16./%
y o% 12 UYb 11.7%
2006 2008 2009 2012 2013 2014' 2016 2019
Research Methodology Approach
Statistically Valid Multi -mode, Multi-lingual Survey Methodology
Given the unique needs of the District, we recommend using a multi -mode methodology for the surveys,
meaning that we will conduct the surveys via telephone and online using our live professional
interviewers and secure online platform. A multi -mode approach maximizes the accuracy and reach of
the research (especially when contacting hard -to -reach groups within the District), increases
participation rates, and minimizes response bias. Our multi -mode approach has been successful over
the course of hundreds of market research projects — it encourages broad participation.
We recognize that some individuals are more likely to respond to an email and complete a survey online.
We have also seen some limited evidence that those responding online (without the intervention of a live
interviewer) may offer different responses to questions. For those reasons, we recommend including a
portion of each survey online. We provide unique reporting analysis of all survey modes, comparing
online responses to telephone responses (landline and mobile phone).
We are fully committed to completing this survey exclusively by telephone if the District does not
want us to us to complete aportion of the survey onlinea portion of the survey online.
Here is an example of how we present the results from our multi -mode reporting:
PROBOLSKY Newport Beach San Francisco Washington DC
RESEARCH
Support by survey mode and reported income
Question 8: WCE program will improve the local economy by investing in energy efficiency projects and local renewable energy projects that will create local jobs.
Does knowing this make you more likely to support or more likely to oppose [CITY] participating in Western Commmity Energy, or does it make no difference to you?
Mom likely to support
More likely to oppkwe
Phone [NETJ
..
t' %/1�,';
Landbne
iitrrr I
/r �!�r
2.1%
Mobile
,j!(�
Online
■ More likely to support
■ Makes no d ifference
Under $50,000
fni
Unsure/Prefer not to answer
■ More likely to oppose
$50,0004100,000 t
j%
$100,000-$200,000
Over$200,000
r �
35
Figure 2: Survey mode results example (Western Community Energy)
We believe that conducting research in the language most comfortable to respondents is critical in
obtaining accurate results in a statistically valid survey and is one of the best ways to alleviate language
barriers. We recommend including Spanish and Vietnamese in both the telephone (and online portions of
the survey if this option is chosen).
Online participants can choose their preferred language (English, Spanish or Vietnamese) at the onset of
their online experience. We initiate all interviewing calls with live-U.S. based interviewers in English and
switch to Spanish only upon request or when a clear language communication issue presents itself.
Our online and telephone data collection systems are integrated internally for seamless data integrity
and reporting purposes. We use security measures to ensure that participants are only able to
participate in the survey once by generating just one token for each resident contacted.
We email invitations to encourage participation in online surveys and provide a secure link for the
individual to access their survey. Each link is associated with an individualized token that is designated
for the specific person. Our online survey platform is easy to use and allows participants to complete our
surveys from any device: computer, tablet or mobile phone. This only applies if the online
recommendation is chosen.
After launching the online portion of the survey, we will contact residents on their landlines and mobile
phones. We place no limit on the number of completed responses via mobile phone, meaning that we
match the percentage of residents/customers who use mobile phones exclusively or just prefer to
respond on mobile phones in general.
For the statistically valid multi -mode survey, we will survey 500 ratepayers as requested in the RFP
which will yield a margin of error of +/-4.4% with a confidence level of 95%.
r
BOISKY 10
RESEARCH Newport Beach San Francisco Washington DC
In -person and Online Focus Groups
Figure 3: Example of in -person focus group
Focus groups can be conducted in -person or
online and will help identify how residents
talk about District services and allows for the
testing/refining of ideas and messaging prior
to development of the survey. While insiders
have a significant depth of knowledge on
solid waste and waste water -related issues
and rate structures, we often uncover new
concerns from ratepayers at these groups.
Focus groups will specifically help to capture
group vernacular and tone, helping the
District to match ratepayer language when
communicating with them via public
outreach.
For in -person focus groups, we will recruit with the goal of having 12 participants in each group to
generate meaningful feedback. We will serve a buffet dinner to help encourage participation and
maintain the energy level in the room. In -person focus groups last 90 minutes and would be held at a
location convenient to ratepayers.
For online focus groups, we will recruit with the goal of having 20 participants. Online focus groups can
be conducted over the course of several days and allow for deeper conversations with each participant.
They also allow participants to upload pictures and videos and allow us to introduce new concepts and
ideas as new issues are discovered. Adam Probolsky will serve as moderator for both in -person and
online focus groups.
Group participants are recruited to reflect the demographic and geographic diversity of the District.
In advance of groups, we will develop a discussion guide in collaboration with the District. Reporting
includes analysis, summary report, transcripts and edited video with participant demographic overlay.
Our focus group pricing is comprehensive and includes all costs for participant recruitment and
compensation, facility rental and catering, discussion guide design, audio- and video -recording,
transcription, and reporting and analysis.
Once we conduct the focus group (whether in -person or online), we are able to deliver reporting within
3-5 days. We recommend conducting focus groups before a survey. As we develop the focus group
discussion guide, we also develop a preliminary draft of the survey, and once we deliver the focus group
report with the results, we take those results and use them to inform the final survey, amending the
survey questionnaire as necessary.
Example footage of a recent in -person focus group:
https•//www probolskyresearch com/probolsky-research-focus-group-example-footage/
Availability and Capability
We have met or exceeded the client timeline for every project we have mentioned in this proposal and
delivered on expectation within budget for each as well. Our firm has the staff availability to complete
this work on time and within budget. We have a 27 year record of profitability and financial stability with
a diverse client base (corporate, government, and non-profit).
r
BOLgCY 11
R]F�JEARC)EI Newport Beach San Francisco Washington DC
L
G. Detailed Work Plan
Survey Process
Sample Stratification
Probolsky Research's stratified random sampling methodology ensures survey results represent the
demographics of District ratepayers.
When determining an appropriate sample size, we consider utility and cost. While larger samples can
reduce the margin of error, surveying large numbers of ratepayers is not only costly, but unnecessary.
And there are diminishing returns in reducing the margin of error of the survey results as the sample size
increases (see Figure 4). A sample size of 500 is robust and will be more than adequate to fulfill the
District's goals for the survey and yields a margin of error of +/-4.4% at a 95% level of confidence. This
will also allow for statistically reliable comparisons among subgroups of residents (e.g. by demographic
and geographic variables).
Relationship of Margin of Error to Sample Size
20%
18%
16%
14%
L.
O
sL. 12%
W
w-
O 10%
C
.M
8%
Sample Size = 500
6% Margin of Error = +/- 4.4%
4%
2%
0%
0 100 200 300 400 500 600 700
Sample Size (Number of Respondents)
Figure 4: Plotted Margin of Error
800
900 1000
The statistically valid sample will be developed from a combination of consumer and government
databases, to be decided in discussion with staff, and then enhanced with publicly available consumer
data that will ensure that we have the most up-to-date contact information for ratepayers.
PROBOLW 12
R]FSEARCH Newport Beach San Francisco Washington DC
Unique to Probolsky Research: We match the demographics of residents within the District. This
means we do not use mathematical calculations to weight results after we have completed the
survey —we capture real responses of the right number of each gender, age group, race, housing
type, geographic location, etc. This is critical to accuracy. Weighting of results in surveys of this
nature is unnecessary and could compromise the accuracy of the data.
Initial Kick-off Meeting
Probolsky Research will meet with District staff to develop the themes and essential questions the
survey needs to answer. We will review background information, agree on a timeline and make
recommendations for methodological approach.
Using the information gleaned from focus groups
(if pursued), meetings, conference calls,
background information provided by and
requested from the District, past research and
our own open -source preliminary research of
current performance measures and survey
results, Probolsky Research will select question
types (open ended, multiple choice, debate
block, etc.). Then, we will construct each
question with care, using simple language. We
will also pay particular attention to answer
choices, ensuring that we do not have too many
possibilities, nor too few (running the risk of
overlooking key data). We will order the
questions to limit biasing results.
Open-ended questions are an excellent way to
capture the unknown and allow survey participants to express themselves beyond the typical multiple
and rank choice questions. We expect to include open-ended follow-up questions in the surveys,
especially to capture ratepayer concerns.
We recognize that the District encompasses single family and multi -family homes as well as rental units
and homeowners and we will match our sample accordingly and may need to develop different
questions for these groups. We look forward to discussing this with staff.
Upon beginning the contract, each key personnel team member mentioned earlier in this proposal will be
ready and available to perform the tasks required for the project. Given our cross -trained philosophy,
key personnel will have the time necessary to deliver on their responsibilities with this project. All key
personnel team members will be involved with this project. Our internal processes include continuous
progress reporting to the District and evaluation of project milestones. This system also allows for quick
response to any District request by any Probolsky Research staff and effective time management. At
least one Probolsky Research staff member is on -call and available to the District at all times, 24/7/365.
We believe in regular and open communication and a collaborative working relationship. We also know
that District staff is busy, so we work hard to limit the need for District staff time and make ourselves
available on your schedule.
Staff Review
Probolsky Research will provide draft survey questions. District staff will have the opportunity to review
and comment. We will remain available to attend any additional meetings in order to discuss the survey
r
BOI.SItY 13
RESEARCH Newport Beach San Francisco Washington DC
questionnaires and address any questions/concerns. Following meetings or conference calls, in which
we discuss proposed modifications, Probolsky Research will incorporate agreed -upon changes. We
recognize that this may require several rounds of revisions. Once we produce final drafts, we will submit
it for final approval before the surveys are fielded.
Conducting the Survey
Once the questionnaire has been finalized, we professionally translate the questionnaire into Spanish
and Vietnamese, create phonetic pronunciations of names for interviewer training, and establish quotas
to match the ratepayer universe model and program the quotas to match demographics, along with the
surveys, into our software utilized by our professional live telephone interviewers (and for online if option
is chosen).
After programming, surveys are tested for logic and presentation. Once this initial testing has been
completed, we conduct a soft -launch, also known as a pre -test, of the surveys and collect 20 completed
interviews. Once the pre -test collection period is complete, our team meets to go over the results and
address any problem areas that come to light, adjusting the surveys as necessary. If any adjustments
are made, we exclude the pre -test responses from the total number of responses needed.
Once the pre -test is complete, (we then email invitations to ratepayers to complete their survey online
and wait 24 hours before we begin the live telephone interview phase if this option is chosen). Live
interviewing can take place from 9AM - 8PM on Monday - Thursday, and 10AM - 4PM on Saturday and
Sunday. Online users have the option to respond to their survey at their convenience, 24/7 (if this option
is chosen).
We implement a strict procedure when contacting potential respondents. In order to obtain the adequate
number of completes and eliminate bias, we will make five attempts to each prospective participant
before deeming them unresponsive and removing them from the contact list. Non -respondents will not
affect the accuracy of the surveys.
We conduct quality control checks on a regular basis throughout the survey process. We check for
correct skip patterns, randomization and rotation, and completeness of responses. We also actively
monitor to ensure that respondents will match the demographic and geographic make-up of ratepayers.
Data Processing
Once the surveys are complete, we will process and review the data for errors and inconsistencies and
code responses to open-ended questions.
Verbatim Responses to Open -Ended Questions
Once we code the responses to open-ended questions into categories, we analyze the tone and word
choices and overlay demographic data for each response. This demographic overlay will provide the
District with a deep understanding of the opinions of ratepayers by age, ethnicity, geography, gender,
housing type, etc. when reading their comments. Here is an example:
Female, Latina, 40-54, English, Westside
Question: What is the most important issue facing your community today?
Answer. Trash in the streets
Figure 5
PROBOLSU 14
�� Newport Beach San Francisco Washington DC
Data Analysis
We complete comprehensive statistical analyses of the research results, utilizing software programs,
including IBM SPSS, to conduct deep statistical testing, such as multiple regression analysis. This helps
us discover and present statistically significant results - beyond the broad opinions - and understand
the specific factors that contribute to attitudes and beliefs of District ratepayers. Such analyses are
crucial in developing guiding recommendations and strategies that will inform the District's future. We
can also analyze how performance measure data collected compares to previous survey data in
responses and audience type even though this may be limited for this first survey.
Once we have run all of the analyses on the results, cross tabulations are developed, graphics are
generated, and other elements of the report are prepared.
Reporting
Our reporting is comprehensive, but never complicated. Our reporting is presented in an easy to read
format with common language and immediately usable by decision makers.
The report for each survey will include:
1. PowerPoint presentation, communicating key findings - focused on usable and actionable data.
2. Report on results including survey background and results, methodological description,
questionnaire, survey toplines (displaying the aggregate percentages of responses to each
question), an executive summary with key findings, trend analysis, conclusions and
recommendations, cross -tabulations, and open-ended question responses with demographic
overlay.
Unlimited in -person meetings, presentations, and conference calls.
We will provide draft and final reports to the District and amend the draft reporting in consultation with
staff before delivering the final report via email.
We include unlimited post -research consulting and meeting time to allow us to participate in strategy
sessions and meetings and collaborate with the District on an ongoing basis.
PROBOLSKY 15
RESEARCH Newport Beach San Francisco Washington DC
L
Probolsky Research can meet any schedule you require for each performance measure survey. We are
capable of starting our work for the District immediately. While we can certainly complete the work and
deliver reports in less than 30 days, we are ultimately flexible and will meet any timeline the District
requires.
Below is a general example of an individual survey timeline that can easily be modified.
Survey Timeline Example
Day 5-8
Discussions
Day 1 Discuss changes to draft
Initial Meeting or survey
Cont Call
Day 2-4 1 Day 9-10
Day 29
Draft Results Presented
Day 11-25 Present draft results and
Fielding meet to review results
Survey is conducted before finalization
Day 26-28 Day 30+
Discovery & Draft
Survey
Report Development Final Results
Development
Finalization/Translation
Results analyzed and on -going
• Background information
Finalize/translate
report is developed Consultation
collected
questionnaire
Final results
• Survey questionnaire
presented and
developed and
availability for
delivered
unlimited ongoing
consultation &
presentations
Research results are kept private and released only at the District's direction.
We take data integrity seriously. All client materials are hosted in a secure digital cloud
environment.
We use 256-bit Advanced Encryption Security to keep your data secure. This is the same level of
encryption used by the U.S. Government.
PROBO)�][i� Newport Beach San Francisco Washington DC 16
RESEARCH
L
H. Cost Proposal
Please refer to the cost proposal in the separate, sealed envelope as requested in the RFP.
PROBOLSKY
RESEARCH Newport Beach Sari Francisco Washington DC
17
Appendix/Additional Information
Reporting Examples
Public safety is the most important issue
In Indian Wells, among residents
0—bon V"1.ft mod k'o t.l Wue%.vY—e -ft lodr�i
�uae w,.nam.
Top issue chart example
City of Beaverton - Community Survey 2019
Verbatim Responses: Question 1
Gentler: FarnMe, 18.29, WMWCmmaslan, Leng% of "me living In Beaverton: 6 p om or ioee, Child in home
Question: Wner Is Ow most in pwtant Issue Wing vow communitY today?
Answer: HameAssness
Gender: Female, 60.69, WWWCsuaalem Length of dtm living In Beaverton: 11 be 15 yews, Ho dniltl "fn home
Question, Whet Is the most krom end Issue "your �MnitI' may?
Answer Tries and homkinmlwas
Gondar Maio, 70+, VAtkoiCaucesion, Length of *m living M Beavwton: 11 to 15 years, Child In hams: Refused
Question: What Is the moat Importer! Issue lacing Your ownmunnity today?
Answer. Urban plan" to mAide Beamton mn w than w4te it as population Increases change the po"hon
disblbutlon and Its m0tfon &* to m4Ior ho*mys.
Gender Maio, 50-%, Mthta/Caucaslan, Length of tme WAng in eeavertom 10 years or mars, No children In home
Question: Whet Is fhe trust Important Issue Aw ft your commoft ty today?
Answer Tmft
Gender Fm We 60.69, Allen, Length of rim living In Beaverton: 16 yeas or mom No cNWren in hom
Question: Whet is flu most Impotwrf Issue /aa V your cm mhxWy today?
Answer: Traffic and roads
Gender: FemWe, 70+, Length of 1Am WAV In Beevelom 16 yms «more, No childfsn In honk*
Question: What la flu most inporteM Issue taring Your community today?
Answer DnW Home*ss
Open-ended response example
PROBOLSKY
RESEARCH Newport Beach
L
Message testing chart example
Infographic example
Results Memorandum
ram w.,, wcwa,w
anwwwwr n..wu
r� waw.w..nww,ewd.e
hae;.n. ora„e Vawna.w rwi.,,o. seoom w. o. war e.qm
M.Ijorllp: Hoandeanhraw is a CS 6" (o Oakland
cahmew. mmwrasw w auawa w usw aoero ra.w.a+caanwaaro wa.
awW M rAY Cvscr MC W 1W mew .oev� ry M dr_CSMeWJt lOF1e en 6rineYw
moa9awYnaelaRasY/ie.Rehnnnv.4aGSr.
Voter. Mated Offidele BrUgl Note Hoes[" to Oo l
Results memorandum example
San Francisco Washington DC
18
C
H. Cost Proposal
Elements Included in Pricing
Based on our experience conducting ratepayer surveys for solid waste and waste water agencies and
our past research for the Costa Mesa Sanitary District, we have prepared the following pricing. While we
present several pricing options, we are ultimately flexible and look forward to creating the perfect
research plan together with District staff.
Meetings, presentations, and custom reporting are all included in our pricing. We do not charge for
expenses or travel. Our pricing is not -to -exceeded and includes:
• Surveying ratepayers on landlines, mobile phones (and online if option chosen)
• Focus group among ratepayers (if option chosen)
• Stratified random sampling accurately representative of Costa Mesa Sanitary District ratepayers
• English, Spanish and Vietnamese language interviewing and translation
• No limit on in -person meetings or conference calls, consultations, and presentations
• Comprehensive reporting including presentation and graphics - focused on usable data
Survey Pricing Options
Number of Questions/Time Universe Number of Margin of Error Cost
pprox. 35 questions/20 minutes Costa Mesa 500 +/-4.4% $33,700
Sanitary
District
Approx. 35 questions/20 minutes Costa Mesa 300 +/-5.8% $24,100
Sanitary
District
ratepayers
Languages: English, Spanish and Vietnamese
Level of confidence: 95%
Mode: Telephone (landline and mobile phones) and online (if option chosen)
In—person or Online Focus Groups (Optional)
PROBI
Newport Beach San Francisco Washington DC
RES