The Audit Advantage
Quality system auditing can improve safety, operational integrity
By Louis A. Sorrentino III, Vice President, SH&E
June 2001
The practice of quality system auditing has emerged as one of the most effective diagnostic tools available to all aspects of aviation operations, from air carriers to airports, from ground handlers to airframe manufacturers. Here’s a look at the implementation and merits of such a system.
Say the word "audit" around April 15th and watch what happens:
faces pale, panic begins to set in, and horror stories race through people’s
minds. Now, say "quality system audit" and most people unfamiliar
with organizations and programs such as the European Joint Aviation Authority
(JAA) and the JAR OPS 1 standard, similar to our FAA’s FARs (Federal
Aviation Regulations), will assume you are referring to some form of customer
service quality issue or maybe even quality control. Not so.
Within the aviation community, the words
audit and, more recently, quality system audit have taken on an entirely
new meaning among management and operations personnel.
In fact, with the advent of the JAA JAR-OPS
1, the benefits of an effective quality system have yet to be fully realized
by many of those foreign air carriers required to comply with the standard.
JAA JAR-OPS 1.035
So what is quality system and quality systems
auditing? The JAA in JAR –OPS 1.035 states that a quality system
shall be established by an operator "to monitor compliance with,
and the adequacy of, procedures required to ensure safe operational practices
and airworthy aeroplanes."
Further, it states that "The Quality
System must include a Quality Assurance Programme that contains procedures
designed to verify that all operations are being conducted in accordance
with all applicable requirements, standards, and procedures."
So for those operators that have a quality
system in place, such as the one outlined in JAR OPS 1.035, one could
assume that the operator is conducting quality audits or oversight of
the technical functions that support flight operations. In most cases,
that assumption would be correct, but like most industries, our aviation
industry is not insulated from operational pressures such as over-extending
line and staff responsibility, inefficient internal tasking, cost-cutting
efforts, expense management, staffing constraints, restricted travel budgets,
time and the lack thereof.
To ensure operational integrity, the Quality
System Audit has emerged as the most effective tool to determine an Operator’s
level of compliance to both the regulations it operates to and the technical
programs it has developed to perform specific functions. And to complicate
matters, there are many ways in which an audit can be performed.
Auditing History
Most of us familiar with auditing recall
a rigid, bureaucratic compliance checklist that either you passed or failed
— yes or no; black or white. The style of that audit, the methodology,
set the tone, allowing very little in the way of understanding corporate
culture, operational constraints, management styles, and so forth.
For the most part, it was merely based on
a set of regulations or compliance indices. Usually little tangible feedback
was offered from the auditors, because again they were auditing, checking
to see if a program or element was evident in writing or that the correct
string of words came out of the respondent’s mouth.
In most cases, the appropriateness and effectiveness
of the program or element being audited was never determined because that
was not the focus. There was no requirement for an evaluation of the way
in which different programs or departments (e.g., flight operations and
maintenance) worked together.
Process Auditing
Process auditing emerged as the methodology
from which a thorough understanding of the administrative and technical
elements of an organization’s operation could be attained following
examination and evaluation. The intent is to determine the appropriateness
and effectiveness of those technical and administrative elements that
support the operation.
Process auditing is looking at the entire
organization, the way it is organized, the training, the culture, the
way in which different departments communicate internally and the meaningfulness
of the corporate safety statement or mission statement, to name just a
few.
Over the past few years, I have been the
beneficiary of working with some very special individuals and companies
dedicated to operational integrity and improving the safety performance
of the industry. It is within this environment that the concept of systematic
process evaluation has grown.
At SH&E, we have developed and refined our
evaluation/audit methodology called safety architecture, based on industry
best practices observed over the years and elements of ATOS, the FAA’s
program for operational integrity known as the Air Transportation Oversight
System. ATOS is a systematic approach to FAA certification and surveillance
oversight, using safety principals and risk management to ensure that
air carriers have safety built into their operating system.
The safety architecture methodology also
incorporates ISO 9001 quality system auditing concepts and uses detailed
checklists, skilled technical auditors, and selected evaluation techniques
which enable auditors to perform forensic assessments of the technical
areas they are auditing.
The heart of the safety architecture methodology
is a detailed examination of the safety structure (goals, scope, culture,
organization) that supports the technical programs (systems, controls,
and work process) and what if any risk management system (safety designed
into operation, data collection, risk analysis, and warning thresholds)
is in place to analyze operational data and decisionmaking processes.
This process provides a comprehensive look at the entire company: departments
that have any role in the decisionmaking processes of technical operations.
Comprehensive Approach
Why look at the entire company, rather than
focusing on just the flight department or the contracted services provider?
It is because people outside of what you and I would consider to be the
traditional department have direct and indirect decisionmaking authority
which effects how that department performs its function. Many times, well-meaning
executives are making decisions that can affect the operational integrity
of a technical department without a full understanding of the ramifications
and downstream effects that their decisions may have.
This is one reason why FAA mandated that
air carriers should have a comprehensive and effective safety department
as well as a director of safety reporting to the chief executive (see
Dec. 1995 Final Rule, "The Commuter Operations and General Certification
and Operations Requirements"). In essence, it is so decisions can
be made with complete knowledge and understanding of the potential safety
repercussions in a given scenario.
Just consider a typical domestic commercial
flight: The aircraft is touched by crewmembers; maintenance providers;
fueling companies; catering; baggage and cargo handlers; the U.S. Mail;
freight forwarders; security screeners; cleaners, lavatory and potable
water; deice/anti-icing; bus services; and many more. Our past practice
of focusing on just the main three — flight ops, maintenance, and
ATC — in accident scenarios leaves an entire population out there
that can easily be attributed as the proximate cause of a failure mode
(accident).
Recent history has shown that focusing on
the pointy end, the folks in the cockpit, is no longer the only area where
decisionmaking skills and process integrity need to be evaluated. We have
more than enough examples of this in the aircraft accident case studies
and NTSB reports.
Ground Services
Another example of the benefits of a program
audit can be found in the ground handling business where many of the services
provided by ground handlers are not regulated by FAA. Companies offering
these services may believe that they have a comprehensive program in place,
only to find out, by participating in a program quality audit, that they
lack certain performance and integrity elements.
For instance, a company may perform the
physical ramp handling, but lack a defined program outlining the standard
that it will train its employees to, and possibly miss important elements
that would comprise a successful program such as what constitutes acceptable
training, qualifications of instructors, types of training offered and
timelines for each module, content of training module, instructors’
guides by training module, test development, test grading processes, certification
processes, recordkeeping, and data tracking.
In this particular example, while a standard
or regulation defining the exact training course material for, say, ramp
handlers may not exist, there are industry best practices that have emerged
as the best, from a safety and efficiency point of view, based on current
knowledge.
Airlines are looking for program quality
and integrity practices behind the scenes, as an integral part of the
service a company is offering. If it isn’t written down, as in a
defined comprehensive program, than how can it be managed or evaluated?
About the Author
Lou Sorrentino is vice president and partner at SH&E, an international air transport consultancy based in Cambridge, MA. He is responsible for its safety practice, and also serves as managing director of the FlightSafety Foundation’s Internal Evaluation Program. He is a commercial pilot and serves on the FlightSafety Foundation’s Int’l Advisory Committee, which promotes flight and ground safety initiatives within the global aviation community and the National Cargo Security Council. He can be contacted at [email protected].
