Why this survey?
I published the survey to all current members of Virginia Beach EMS. The goal of the survey was to establish baseline measurements in three topic areas:
- Vehicle shuttling, because this is in my Division's area of responsibility
- Quality management, because this is also in my Division's area of responsibility
- Fatigue, because assessment of provider fatigue is now a national-level recommendation that we want to comply with
The breakdown of invitations was as follows:
Here's the breakdown of responses:
The breakdown of individual people who completed the survey was:
Encouraging results?
The data showed that when members are asked to move a vehicle to or from the city garage, they overwhelmingly (96%) do it. That struck me as a very encouraging rate of cooperation, and it's about the same whether volunteer or career.
Or selection bias?
But surveys like this are prone to a problem called "selection bias". That's when the answers to the survey are not representative of the group as a whole.
For example, among volunteer members, 67% said they were asked to move a vehicle to or from the city garage in the past six months. Extrapolating this percentage to a system with approximately 1006 volunteers, we'd see that 674 different volunteers had been asked, and at a cooperation rate of 96%, 647 different volunteers actually moved a vehicle in six months time.
Although this is possible, I find it highly unlikely. I strongly suspect that a small core group of volunteers do most of the vehicle shuttling, apart from when we enlist duty crews for this chore. Several Squad Fleet Coordinators have told me as much, bluntly.
(About 94% of career members had been asked to move a vehicle, and they complied 100% of the time. No surprises here since compliance is mandatory and they're moving units while they're on the clock.)
The following statement earned the following average score:
"I AM SATISFIED with the way EMS vehicles get moved to and from the city garage."
If we actually have managed to spread the shuttling chore across so many members with such high cooperation, never unfairly burdening the same people over and over again, then what's not to be satisfied about?
I suspect that a lot of people who really do shuttle vehicles responded to the survey -- and a lot of people who don't shuttle vehicles did not respond to the survey. So the answers are not representative of the group as a whole. Selection bias.
Attitude
The following statement earned the following average score:
"I FEEL ENGAGED in the effort to get EMS vehicles the preventive maintenance and repairs that they are supposed to get at the city garage."
Coincidentally, the "engaged" score was the same as the "satisfied" score. The most popular answer to the "engaged" statement was "Somewhat". The next most popular answer was "Not at all", and those two responses accounted for the majority.
I find this a bit disappointing, and have some thoughts about the cause, but I think the wise thing to do is to sit back and see if this grade changes over time.
The feeling of engagement tended to be lower among volunteers and higher among career members. The opposite was true for satisfaction.
What next?
To confirm and overcome (or disprove) the selection bias, we would have to choose a different way of sampling answers from the group as a whole. One technique might be to randomly go get the answers we need by visiting unsuspecting duty crews, instead of waiting for the answers to come to us online. But this would be very time and manpower intensive.
Instead, I'll just make this request:
Please tell me (by reply email) what's preventing you from feeling "very much" satisfied and engaged with the way EMS vehicles get moved to and from the city garage.Maybe you can suggest some interventions we can try before measuring these things again.