Update on Our Open Badges

Our open badge project is moving forward - we now have 8 badges credentialing specific training and operational experience.  As we have been developing these badges, we have effectively defined a future architecture for our badges with 5 categories:

(1) general training badges - tied to our four level operational qualification program - ranging from entry to advanced level.

(2) specialized incident commander training - tied to completion of our three levels of training for incident commanders in The Virtual Emergency Operations Center - ranging from basic to advanced level.

(3) skills credentialing - verifying specific skill performance critical to operation of The Virtual Emergency Operations Center.

(4) service and experience - in 100 hour increments in actual disasters or exercises for our staff and in varying increments of numbers of reports for our disaster observer micro-volunteers.

(5) professional certification - the Certified Technologist in Virtual Emergency Management (CTVEM) - capstone level integrating elements of the other four badge categories.

On The Importance Of Disaster Exercises

This is an update and expansion on an earlier blog post on exercises based on our overall experience in 2014.

We  do some level of activation numerous times a year.  When we look at the data from 2014 we participated in 17 events, including:

• Responses for actual events - ranging from upgrading our alert status to Communications Watch to full Activation.
• External exercises - supporting the organizations to which we provide services.
• External day of ... exercises - the Great Shake Out is an example, drills that increase awareness of specific hazards and that allow us to practice our operations in the hazard scenario.
• Internal exercises and drills - intended to work on specific technical issues or to train our staff either in specific response actions or in responding to different types of disasters.
• Orientation seminar exercises - in our case as presentations of our capabilities to potential supported organizations or agencies.

Why do we, as volunteers, want to do that much work?  Each exercise that we designed required a full work-up, drafting a master sequence of events list, preparing inputs, doing an after action review, etc.  Some organizations count themselves lucky to do one tabletop exercise a year.

We respond to actual events because they are actual, and the organizations we support want our assistance.  So the simple answer to why is so that we will be able to respond when called.  But it is more complex than that, and the answer lies in what exercises and drills do.

One categorization of exercises divides them into discussion based and operations based events.  The discussion based events include:

• Orientation Seminars - to acquaint participants with new plans, concepts, resources, etc.
• Workshops - to achieve a specific goal or develop a specific product, such as a plan or procedures.
• Tabletop Exercises - to assess plans and policies and develop common approaches to problems.
• Games - to explore decision making in a team environment and to evaluate the results of decisions.

The operations based events include:

• Drills - testing a specific function by actually performing it.
• Functional Exercise - staffs do actual work but movement of field units is simulated to test and evaluate command center operations.
• Full Scale Exercise - with operations by all components of the system to implement plans, procedures, agreements, etc. developed in other exercises.

From this list we use orientation seminars (as part of new member Initial Qualification training), drills (to work specific functions in The Virtual Emergency Operations Center), and functional exercises (for a full function response to a scenario).

There is a long standing division in the exercise literature about what exercises are about.  Some argue that they are for training - you hold an exercise to train people to work as a team in an otherwise unfamiliar environment.  Others argue that training is something you  do in a classroom or training facility, and that exercises are only to evaluate whether the training has done its job and the procedures are appropriate.  We view this distinction as an exercise in pedantic pettifoggery.   Perhaps if you have a multi-million dollar budget, a large exercise staff, and essentially unlimited time, it might make sense to build firewalls around the purpose of an exercise.  But as a small non-profit organization every exercise has to train and test.

But it is more than simply training and testing.  One of the principles of training is that trainees do well things that are both practiced and recent.  Hint - this applicability of this principle does not stop when initial training ends.  Practice with repetition of key tasks builds both speed and quality of execution.  This is true in sports (a major league baseball pitcher who only threw the ball in games, but never in practice would not be a pitcher very long, no matter how many balls he threw in Little League, high school, college, and the minor leagues), or for that matter, any physical skill.  It is also true in mental skills such as decision making.  

But practiced skills must also be recently used to be at their top form.  I once had a job where I controlled tactical fighter aircraft in air combat training.  I was hand-picked for the job because of my proficiency as a ground controlled intercept controller.  I controlled thousands more tactical intercepts with more aircraft involved under higher pressure than the average controller in a tactical control unit.  And yet on Monday morning, having been off the radar scope for two days, I was noticeably less proficient.  The lesson of the war story is that if you work one disaster event a year (real or simulated) you will be less proficient the further you are from that one event, and certainly less proficient than if you work 17. 

 

We Take Key Skills For Granted ...

... and we really should not.  In running an emergency operations center there is often a baseline assumption that the people staffing the facility know how to do certain basic things.  Anyone can make a good log entry or can complete a formatted report or follow an emergency operations plan, as examples.  But experience suggests that clearly is not true.

 

A number of years ago, I was one of the search mission coordinators on a search for a missing aircraft en route from Florida to points north.  We flew hundreds of search aircraft sorties low level in dangerous summer weather looking for the aircraft, at some risk and considerable cost over a two week period.  And the last day we had planned to search, one of our aircrews made the find.   As we were cleaning up the room we had used to coordinate search operations, we found a torn piece of a legal pad with information on it from a witness that pinpointed the crash within 2 miles on a several hundred miles long flight path.  It had slipped down behind a desk. A scribbled date and time indicated that this information was received on the first day of the search.  Someone clearly did not know how to manage witness reports and ensure that the information entered the system.

 

And a number of years ago, I was serving as an observer for a county level disaster exercise.  The Fire Battalion Chief and Emergency Medical Services Captain (both of whom I knew to be highly competent leaders on scene) staffing the emergency services position were presented with a problem.  The dialog went something like this:

 

Fire - "I don't know what to do about this, what do you think we should do ...?"

EMS -  "I don't know either, what do you think?"

Fire -  "I just don't know, maybe we ought to call the Chief"

EMS - "yes, that makes sense, lets call the chief."

Me, trying to be helpful - "well what does the plan say to do in cases like this?"

Fire - "what plan?"

EMS - "a plan?  There is a plan?"

Me, trying to control myself, and pointing to the red, three ring binder, a foot and a half in front of them, clearly labeled EMERGENCY OPERATIONS PLAN in very  large white letters - "well that binder has PLAN on it, so maybe that will help ..."

Fire and EMS pull the plan towards themselves and open the binder.

Fire - "there is a lot of stuff in here."

EMS - "yeah, I can't figure out where to look."

Me, trying to be helpful - "well I can see that there is a tab there that says ANNEX K. EMERGENCY MEDICAL SERVICES ... maybe there is something in there."

Fire, opening up Annex K - "boy, there sure are a lot of words in here."

EMS - "yeah, there are too many words, we better call the chief and ask him what he wants us to do."

Fire, dialing the chief's number - "yeah, the chief will know what to do."

Me, pulling the last handful of hair out of my head, and writing furiously on the comment sheet ...

 

This is not a ding on either fire or emergency medical services personnel.  It could have been anyone.  In years of working in emergency operations centers, I have seen very few staff members actually open up a plan and apply the planning to their problem.

 

We assume that people can do a wide variety of tasks.  And these tasks are important.  In the first case, doing the job correctly in establishing a log and recording the witness information in it, could have saved 13 days of flying search aircraft sortie after search aircraft sortie.  In the second not knowing what to do or where to find guidance delayed the response.  In a real event, with the chief unavailable, what would the two officers have done?

 

The important lesson is that training is important, but a lot of training misses the mark.  The search staffs were trained and all were certified in their jobs.  The fire and EMS officers were very well trained in their day to day jobs.  But no one had sat down, put pencil to paper, and sketched out exactly what unimportant little details they needed to know to be competent in the command/coordination center environment.  Something as simple as writing a good log entry takes ability and knowledge.  So look for the small details, train on those, and your performance will improve.

 

 

Badges? Open Badges? What is this?

In the classic Western comedy Blazing Saddles, the Bandidos utter the famous line "Badges, we don't need no stinking badges."  So why then are we as a disaster response organization starting a badge program?

 

The answer lies in what modern open badges do.  Traditional verifications of skills have been based on formal education or on worker progress through apprenticeships in trades.  Skills learned through self-study, individual practice, and other learner based approaches were undocumented.  To some degree the explosion of certifications that started in the 1960s in the United States addressed this by creating credentials that documented competency in a wide variety of semi-professional forms of work.

 

However, these certifications mirrored the traditional academic verifications with formal structure and rigid requirements.  In our field, for example, more than one professional certification as an emergency manager specifies that the required documentation must be submitted in a three ring binder, no document protectors are allowed, and that the information must be in exactly the same order as the requirements are listed in the instructions.  That certainly demonstrates competency in preparing an application packet following highly detailed instructions in a highly bureaucratic environment, but its value in documenting the ability to save lives and protect property is open to question.

 

The wider problem is that professional certifications are heavily weighted toward activities that individuals in full time employment with a supervisor who values certification and adequate funding for travel can complete.  A requirement that a certification applicant must have attended certain professional conferences (attendance is a common "contribution to the profession") of a set number of days effectively rules out anyone who does not have employer support, and that means it rules out volunteers.

 

More dangerously, such certifications tend to substitute easily measured events for demonstrated performance of skills.  For example, attending an ungraded training course is assumed to demonstrate that you can perform to an acceptable level the skills it presents in a lecture or short activity setting.  At the same time there is a strong tendency toward traditionalism in learning - classroom courses are valued over online instruction.  And there is an inbuilt prohibition against giving credit for training received more than a certain number of years ago, in emergency management typically five years.  Finally, actually understanding what the certification represents may be difficult and represent a journey through multiple sources.

 

So we have looked carefully at the open badge movement.  First, we are a volunteer organization, so we need verification of capabilities that volunteers can do.  Second, we need portability and easy access, so that the credential and its requirements are readily available to the individual and to those who need to verify capabilities.  Third, we need to recognize the actual skills the individual can perform and the level of those skills.  We need immediacy, so that our credentials represent capabilities when they are demonstrated.  And finally, we need to understand the scope and trajectory of our volunteer's work in building skills, not just a delimited snapshot of a five year interval.  Open badges allow us to do all these things.

 

Our first step is to badge our four existing training levels because these are based on a mix of training, experience, testing, and skill demonstration and are easy to capture.  The next step will be to start to badge specific competencies needed for Internet based disaster work.

 

And if you are wondering what a badge is ...  Badges are simply electronic symbols with embedded metadata that document the achievement of competencies and provide information on who issued the badge, the requirements for its award, and the level of skill that shows.  We are using Achievery as our badging system, and recommend the use of Mozilla's Backpack as a framework for building an electronic badge resume. 

And a disclosure note - the author, Dr. Walter Green, taught for 13 years in an academic emergency management degree program, wrote his dissertation on emergency management certification, at one time held 6 national, state, and professional association emergency management certifications, and wrote the requirements package for Virginia's Professional Emergency Manager certification program when it was introduced.

Why a Micro-Nation for Disaster Testing and Training?

We have just completed the scenario handbook for a fictitious country for use in training our staff and testing our systems to ensure that we are ready to do our mission in real disasters.  It would seem that there are plenty of real high hazard areas and scenarios that we could use for this purpose.  So why did we do all the work to build a micro-nation?

 

As we looked at it, there are a number of very good reasons:

 

(1)   The Virtual Emergency Operations Center generates Tweets when we activate, change alert status, etc.  We realized that some are retweeting these as though they were real events (even though they are clearly marked Exercise).  If one goes viral, people will be asking government officials about the terrorist attack, flood, earthquake, etc.  And we will get complaints that we are creating unnecessary concerns, hurting real estate values, impacting tourism, etc.

 

(2)  All of our staff come to this work with their own experiences and built in assumptions about how emergency response works, who the resources are, etc.  Having a fictitious country allows us to build resources and design geography that requires attention to the actual details, not to assumptions.

(3)  We wanted to be able to work a wide variety of disaster scenarios, some quite rare, many geographically delimited.  With a scenario country we can design scenarios that work unusual problems that would not be found in the jurisdictions we support.

 

(4)  And we wanted to be able to test our systems with data so that we can see what the resulting displays look like, make sure we have the right options in drop-down menus, etc., not just rely on "test" as our best entry.

 

So welcome the micro-nation, the Republic of Perilo ("perilo" means danger in the constructed language Glosa).  We expect it to be a busy place.  To visit, simply become a member of our team, but remember that citizenship requires commitment to work.

 

A New Set Of Baseline Capabilities

We are in the process of rolling out a new set of forms and views for The Virtual Emergency Operations Center.   This set will be a baseline set for the services we offer supported organizations, including those we support now and new supported organizations in the future.

Why a baseline set?  And what does this mean?  Let's take the what question first.  The baseline set consists of 12 forms used to enter data online, and 12 views that display the entered data for use by organization staff to view information to alerting, task and resource tracking, planning, and reporting.  It is in effect a complete set of emergency operations center functions in one place on the Internet.  Some key facts:

• The baseline set is installed in a functional area for the organization within The Virtual Emergency Operations Center.
• Some forms generate messages that are automatically sent for notifications and warnings and reports.
• Some forms, with their associated views, serve as status boards.
• Some forms are designed for use by teams on site to report back conditions and progress.
• Access is protected by a variety of security measures to ensure that privacy is preserved.

 

The baseline set gives us a rapidly deployable and completely functional package that can be provided to supported organizations with minimum set-up, test, and training time.  Equally important is the capability that it gives us to customize to meet the needs of supported organizations.  Additional forms and views can be added easily to meet specific needs.  The order in which information is collected and the selections available through drop-don menus and check boxes can be easily customized to meet the particular operational practices of the supported organizations.  The result is the ability to deploy an easily understood and fully capable system that can be used to meet immediate needs for information management and the ability to tailor that system to the individual needs of a supported organization.

Why Exercises?

Why do we do exercises?  For that matter, what is an exercise?  Let's take the second question first.

Exercises are simulations, in our case of disaster events.  Generally they are conducted to:

(1) train and/or evaluate staff in the performance of their emergency jobs under realistic conditions,

(2) test equipment  in the roles in which it will be used - including hardware and software, and

(3) test plans and procedures for emergency response.

Testing in this context implies both evaluating whether or not something works and identifying ways in which performance can be improved.  Although there is a theoretical difference between training and testing, most exercises that we do serve for both training and testing.

Our exercises can be internal or external.  Internal exercises involve only our staff and the Virtual Emergency Operations Center.  External exercises involve and are conducted by other organizations that we support.

There are five commonly accepted types of exercises.  Two of these, orientations and tabletop exercises, are not really applicable to our virtual environment.  We do conduct and participate in:

(1) drills - these focus on specific subsystems, functions, or procedures within the Virtual Emergency Operations Center and may be done with limited staffing. 

(2) functional exercises - these involve most or all functions and systems of the virtual Emergency Operations Center and exercise a complete staff.

(3) full scale exercises - these normally involve the full system including the movement of resources and the performance of simulated on scene hands-on tasks.  For us, external exercises, even if they are functional for the agency conducting them, are full scale exercises for us.

So why do we do exercises?  There are three key reasons.  First, we use them to train new members on the actual job they will be doing when we are activated.  Our operations are complex, and for someone who has never been through an event they can seem difficult to perform.  Training takes away the newness and makes the unfamiliar familiar.

Second, we use them to maintain proficiency.  Experience shows us that if you do not regularly operate our systems you play catch-up during the first part of your duty shift in a major exercise or disaster.  And playing catch-up is stressful, causes errors, and results in slow and inefficient work.  That is why we run at least one drill or functional exercise each month.

Third, we are constantly modifying our software, checklists, job aids, and emergency operations plans to reflect new capabilities, new taskings, and lessons learned from previous events.  Testing these in drills and functional exercises are the only way we can be sure that they will work when we need them.

Finally, as a virtual organization, the only time we regularly "see" our members is in exercises or actual disasters.  Exercises help us get to know each other, see what each of us can do well, and build the teamwork and shared confidence needed when the water is rising, the wind is blowing, and other organizations need our help.