Q.How did you measure the results of soft-skills training?

A.In answering that question Dr. Genie Z. Laborde and colleagues in several corporations filled a 300-page book, Tooting Your Own Horn: How To Measure Soft Skills Training.

Following is the book's introduction—a summary of the research techniques and results discussed. 
 
 

A Case Study: Instruments, Surveys, Interviews, Figures from

Sprint
IBM
Pacific Bell
FPL
Chase Manhattan Bank
M&M Mars
Dell Computer Corporation
7-11
Allstate Insurance
Nissan
Genie Z. Laborde, - Ph.D. - I.D.E.A. 
JoAnn Garner, M.B.A. - Sprint
Kay Best - IBM
Linda McGregor - Pacific Bell
George Flury - FPL
Cliff Saunders, Ph.D. - Bell Northern Research
 Chuck Romero - Sprint
Lindsay Collier - Eastman Kodak
Donna Potignano - FPL
Barbara Dabney - FPL
Dell University Staff - Dell Computers

The secrets of conducting corporate research to measure changes from soft-skills training.

TABLE OF CONTENTS
Preface 

Introduction 

Chapter 1 Bottom Line Impact 

Chapter 2 Long Term Impact: IBM and Chase Manhattan Bank 

Chapter 3 Research 

Chapter 4 Meeting $aved 

Chapter 5 Management Practices 

Chapter 6 Team Building 

Chapter 7 Humor 

Chapter 8 Time $aved—Money $aved 

Chapter 9 Common Language 

Chapter 10 National Sales Record Set 

Chapter 11 Attitude Changes 

Chapter 12 Changes/Success 

Chapter 13 Energy/Fun 

Chapter 14 Use of Other Seminar Skills 

Chapter 15 Super Question5 and Metaphors: Two Skills Going in Opposite Directions 

Chapter 16 Manipulation/Dovetail 

Chapter 17 Other Measurable Indicators: Customer Complaints, Retentions, Replacements, and Activation 

Chapter 18 I-E Locus of Control 

Chapter 19 ASTD Panel Report: FPL, Sprint and Pacific Bell 

Chapter 20 Synopsis from Pacific Bell study by Linda McGregor 

Chapter 21 Evaluations from Dell Computer 

Chapter 22 Health and Interpersonal Skills 

—Peter Kizilos, Training Magazine

Companies today do not have the time, the personnel, or the resources to monitor people carefully. People have to manage themselves. Besides, people perform better when they manage themselves. 

—John Naisbitt and Patricia Aburdene, Re-inventing the Corporation
 

     I began designing and conducting research as a sales tool for our seminars fifteen years ago. My clients sometimes conducted their own research. Years later, usually when the trainer retired, I would receive a copy. Recently I've become aware of how scarce soft skills research is. So this book is a case study of justifying training dollars for our clients. At the same time the instruments and approaches can be valuable to any training group. Thus this book and its title. Research is expensive- in time and money. However, anecdotal evidence is not enough to justify "soft" skills being designated "hard" skills. Statistics probably won't change soft skills to hard skills either, but these statistics indicate that soft skills do make a difference in important areas for business. This book contains enough "indicators" to challenge the connotation of "soft" skills as unimportant for the business world. 

     Research indicates a three-day training in interpersonal skills, management practices, sales strategies, the retention figures and/or the bottom line of Sprint, IBM, Chase Manhattan Bank, Pacific Bell and FPL. The trainings tested teach interpersonal skills which are similar to those skills cited by Peter Senge as essential for creating a learning organization. More on this under Theory 2 in the Author's Note. 

     Change is hard to prove. Change in a positive direction is even harder to prove, and change that increases dollar return on investment is the most elusive of all. In this book we cannot prove this connection, but we can "indicate strongly," and we will. We will use 27 separate research studies. Two were conducted by IBM, one by Sprint Corporate, 19 by International Dialogue Education Associates, one by Pacific Bell, one a panel report, one by Chase Manhattan Bank, one by Dell Computer Corporation and one, a review of the literature. The studies collected data from more than 1,250 employees. 

     In a study conducted for the Labor Department of Motorola Inc. estimates that they earn $30 for every $1 invested in employee training. For "soft" skills, this book's research indicates the figures might be even larger. 

     Do the people involved think the skills they learned in the training had a significant role in the improved productivity and sales? We'll use their own words to answer this. Their exact words are one of the reasons for this book. 
 
 

      What would explain the evidence, presented here, that people changed their beliefs and their behaviors in only three days of training? 

THEORY 1: LEFT BRAIN / RIGHT BRAIN / WHOLE BRAIN

    It's been known for some time that we have two brains, a right and a left, and that these two brains process information differently. The left brain processes language and the right brain processes pictures, collages of data, the present moment and joy. What is not widely known is that the left brain-and our culture is mostly left brained-processes small amounts of information close up. . . a worm's eye view, so to speak. It's as if one half of our brain is a computer programmed by a worm. The right brain processes large amounts of information into patterns. This computer is programmed for a bird's eye view, a mountain top view-the big picture. 

     Because our culture is so left-brain oriented, business people tend to problem solve with one solution, rather than stepping back and looking at the big picture, discovering many options, then choosing the best. We use small amounts of information to think about, then take action. In our own culture, people who can readily use their right brain are known as creative, often as creative geniuses. They may or may not be geniuses, but they do process their perceptions in a different way from most. Interestingly enough, there is an attraction to right brain thinking that is seductive. If you've done it a few times, you want to do it more and more. The problem is that most people don't know how to set it in motion. They have blinding flashes--intuitive insights, then they revert to "normal" left brain thinking. Right brain thinking is normal, too. 

     Left brained people find this whole area and the people in it "weird." What they don't know is they can do it, too. We all have both left and right brains, but most of us don't know how to use them. Also, the right brain is seen to be the door to the unconscious mind, while the left brain uses conscious information and small amounts of information, viewed up close. Whether the right brain is the door or the unconscious mind itself is controversial. 

     Both left and right brains have certain advantages, but the biggest difference occurs when you can use both brains together. It may be a rapid shuttling back and forth or it may be both brains are synchronized. Here is the biggest secret of all. When both brains work together, extra energy becomes available. This extra energy can be used to create success. 

     The attraction of right brain thinking is that when it "backs up" the left brain thinking, the extra energy available with this kind of processing is astounding. It's as if, once you begin to use both brains, to look for and find lots of options to a "next move" then the energy to pursue one or more of these options is generated. The energy seems to be intrinsic to the process of two brained thinking. Why is this? I don't know; however, this was pointed out as early as 1945 by Robert Desoille, whose work on the Waking Dream contributed to Roberto Assigioli's work, Psychosynthesis. Psychosynthesis is one way to learn right brain processing. There are many ways. 

     This book teaches some of these ways. Coincidentally, it also teaches how to carry out the advice of the management experts. These experts tell you what to do, but in most cases are not able to tell you how to do the "what." By using two brain thinking, their advice really does work. This book supplies many of the missing pieces for business management and gives first-hand accounts from people who have used these "missing pieces" and found they work. 

     This insight about left and right brain processing as applied to business practices came ten years after I began teaching the processes presented here. I knew the processes worked to make me more effective. When I taught them to business people, they became more effective. I actually did not realize how much more effective until I began gathering information for this book. I heard story after story of success attributed to the three day seminar, Influencing with Integrity^®, and my response was, "I don't think one three-day seminar can cause so much change in so many people." I believe in education, and I believe in what I teach, but my natural skepticism was greater than my ego, in this case. What I finally came to believe is that the skills we teach are tapping into the processes natural to the right brain, and thus the energy of the two-brain approach is tapped. This vortex of energy from the two-brain processing creates successful changes in behavior. 

     This vortex of energy also allows humor and play to appear in the work environment. Humor is not present without an excess of energy. If you are struggling to survive, it's hard to laugh. 0ne of the clues as to what was happening with the group of executives, directors, managers, supervisors, and sales representatives presented here was the repeated mention of increased humor, fun and play. The formula is simple: 
 

 TWO BRAINS PROCESSING => ENERGY => HUMOR

      In the next chapters are ways to tap into right brain processing and to activate two brain processing. I knew that the twelve skills I taught changed the habitual patterns of my students. What I did not realize was that I was teaching them to interrupt their left brain habitual thinking with right brain patterns. Whenever they actually did remember to try out the right brain processes (the 12 skills presented the seminar Influencing with Integrity^®), then both brains were engaged, and this special vortex of energy was tapped. This energy usually led to success and more success. 
 

THREE OTHER THEORIES

     While I'm attracted to the "whole brain-energy-success" theory, there are other possible explanations of what happened at Sprint, Pacific Bell, Chase, FPL, and other large corporations. 
 

THEORY 2: CREATING A LEARNING ORGANIZATION

     Another theory is that by teaching the twelve process skills (p. 1-10), we are setting up a "learning organization," to use Peter Senge's term. The process skills do give you an "itch" to learn more. They sort of "prime" the brain and get it going, like a splash of water in a rusty pump. In "The Fifth Discipline" Senge lists the five disciplines needed for a learning organization: Team Learning, Mental Models, Personal Mastery, Vision, and Systems Thinking. I am sure that "team learning" occurs during the seminar itself and feedback says this usually continues. Our training module, Maps of Reality (p. I-11 ), overlaps with Senge's ideas of "Mental Models." "Systems thinking" is defined, explained and referred to again and again in the training, and "Personal Mastery" is both emphasized and taught in the State of Excellence Module (p. I-11). Senge's idea of a "Vision" is implicit in the Outcome Setting Module (p. I-11 ), and the Dovetailing Module (p. I-11). 

     So even a quick appraisal indicates the essential components of creating a "Learning Organization" are taught in the Influencing with Integrity^® seminar (IWI). Maybe the extra energy and success are the natural result of learning to learn. I was impressed enough with Senge's book to include his idea as one of the reasons soft skills are important to business. 
 

THEORY 3: LEARNING ESSENTIAL SKILLS

     One of these theories is inspired by a July 29,1991, Fortune magazine article, "The Trouble with MBAs" which focused on the skills needed in today's changing market-place, skills which are not now taught. This quotation cites the position of this article in regard to the lack of essential skills among MBAs. "The Trouble with MBAs: Business schools aren't giving them the skills employers need-- leading, creating, communicating-so companies are doing it themselves, or looking elsewhere." 

     Later the article continues: "Corporate recruiters complain that MBAs lack creativity, people skills, attitude for teamwork, and the ability to speak and write with clarity and conciseness-all hallmarks of a good manager." 

     Perhaps, what our trainings do is supply those needed skills. The extraordinary successes talked about in these interviews may simply be the result of having skills that are needed for success in today's changing marketplace. 
 

THEORY 4: FLOW FOR SUCCESS

     A third theory could be found in Flow: The Psychology of the Optimum Experience by Mihaly Csikszentmihalyi of the University of Chicago. Impressive research over 20 years indicates that flow experiences are found in all cultures when people are sufficiently challenged and adequately skilled. Maybe the communication, sales, negotiation, and interpersonal skills we train are enough to meet the daily challenges at Sprint, IBM, Pacific Bell, Chase, Dell and FPL and the energy of the flow experience then creates success. 

     Or maybe the changes are a combination of all four of these ideas. Why not decide for yourself? 

     The communication and interpersonal skills which are taught in IWI grew from studies about perceptions; the roles that our eyes, ears, fingers, skin, and emotions play in our work-life, often without our awareness. 

The first concept is that each person seems to prefer one sense over the other two major senses. Smells and tastes are considered here to be part of the kinesthetic or feeling sense. This concept becomes a skill when you know your own preference, can determine someone else's preference, and tailor your language to their preference. This often means translating from your favorite language to a sensory language foreign to you. This takes some practice, which we do in the class. 

The second concept is that people display their perceptual preferences, not only in their language, but also in their body postures, their eye movements, hand gestures and even breathing patterns. This concept becomes a skill when you can recognize and use this information in an ongoing conversation. Rapport is quickly established when you can recognize the other's perceptual system and match it. 

The third skill is setting your goals in see, hear, and feel terms so that the natural processes of the brain can help you obtain your outcomes. 

The fourth skill evolves from understanding that each of us has a different map of the way the world is. Our "map" is the result of our experiences and the way we have coded these experiences in the electrical-chemical memory bank. Our different maps cause the misunderstandings and conflicts that beset a business day. 

The fifth skill is improving the ability to see and hear so that we notice information we are now deleting from our awareness. 

The sixth skill is the awareness of rapport and non-rapport with methods of setting up rapport, if need be. 

The seventh skill uses all six of these concepts and skills to create the synergy of a high-energy exchange between people. 

The eighth skill is a set of five questions to gather enough information and not too much. 

The ninth concept is Pavlov's stimulus-response, which is how the brain learns new information. This concept becomes a skill when you can associate two different experiences quickly and easily and bring these experiences to awareness consciously 

The tenth skill is being able to move into a peak performance state using the stimulus-response method quickly and easily. 

The eleventh skill is being able to create a perfect metaphor for a business communication that requires great sensitivity. 

The twelfth skill concept is win-win plus, called dovetailing here. This becomes a skill when you can implement it at every stage in a business interaction. 

These twelve skills are powerful enough to cause the changes recounted by these employees of Sprint. 

HOW SPRINT BECAME PART OF THE STUDY

     The series of events that led to my company's training over a thousand people in a three-day communication seminar inside Sprint are surprising. The whole thing began when a rising young executive in the retail business made a serious career mistake. Overnight, he found himself in a basement cubicle, having been moved from his large corner office on the l2th floor. 

     What was his mistake? He talked when he should have listened. While he was sitting in his cubicle, staring at the wall, he was handed a copy of Influencing with Integrity by a sympathetic friend. 

     Having no duties, all his assignments had been moved as well, he began to read. Over the next few months, he read it sixteen times. He yellow marked, he underlined, he memorized whole sections. 

     After six months, he had managed to turn around a few projects in the retail corporation by personal power alone. He had no authority, but he made his newly learned skills available to the project leaders, and he gradually regained his reputation as a problem solver. At that point, he decided the time had come to move on. 

     In his job interview with Sprint, he bargained to attend one of our public seminars before he accepted the new job. He got the agreement from the interviewer and within six weeks was in California attending the training. 0n the last day, we had a drink together and he said, "I'm going to give you Sprint as a client." 

     I gave an involuntary yawn. I had heard this line before. Lots of people get excited about the new skills they have learned and think they can sell the training to their company. Ninety-nine out of one hundred can not. This young man could and did. 

      Within six weeks, the head of HRD had contracted a pilot program. He himself had not seen the training. The young man had sold him sight unseen, very unusual. The HRD Manager set up a severe test for me, however, to be sure the program did all that had been promised. 

     He knew communication was a problem in Sprint. It's a problem in most corporations. So he brought five people from five different groups who had a history of ongoing conflicts. Twenty-five people spoiling for a fight. About half of them were not told they were coming to the seminar on Tuesday morning until Monday afternoon. Since many of them were from the Information Systems group, when they heard "Communication," they thought it was a new computer network system. They had important tasks waiting and found themselves in a seminar focused on skills to talk to people. They had no need to talk to people; their job was interacting with the pluses and minuses of computers. 

     Two days before the training I had taken my seven year old ice- skating and had broken my wrist in three places. I was not in my peak performance state. In fact, my orthopedic surgeon had forbidden me to get out of bed. Even so, I arrived in Kansas City, with great hopes, and in three days all 25 people in the Sprint training were talking to each other. They were not only communicating, they were saying things like, "I never really understood your situation before. Now I have some ideas about how we can work together." At the end of the training, the HRD Director asked for the price to train 16,000 people. We almost missed our flight to give him a price, but he was not able to sell it to management. He did schedule classes however, and our in-house instructors trained about three hundred managers over the next two years. We also trained one division of about 700, all the way from the top to the sales representatives. 

     What happened to the young man who "gave" me Sprint as a client? In three years, he has been promoted from internal consultant to manager to group manager to vice president. He arrived in California and took me out to lunch when the vice president level was announced. He now has an office with a wall of windows. 
 

HOW IBM BECAME PART OF OUR RESEARCH

     About five years after our company's inception, I was travelling a lot, conducting seminars. I had two employees at the time and one was new, the office manager. The company office was in a cottage in my garden and the office manager thought her job was to protect me from being disturbed by anyone, including clients, when I was at home. No one had told her this. She had decided this on her own. One afternoon about 4:00, she phoned me from across the garden to say "This man from IBM has been here the last two afternoons, hoping to meet you to discuss your seminar. He'll be here for the last time in 15 minutes. He's returning to Atlanta tonight. Are you interested in meeting with him?" - "IBM. Yes." 

     In our discussion, Bob, (not his name) told me he'd heard about my book, Influencing with Integrity, and asked his secretary to locate a copy in Atlanta. Two days later, she finally found one in a bookstore across town. 

     Traffic was bad; the day was hot. The drive was long, but the bookstore had held his copy. In fact, they had one more copy. He buys his, heads back to the office in even heavier traffic and realizes he should have bought both copies. He turns around, fights traffic back to the bookstore and the other copy has already been sold. 

     Bob was teaching a course for IBM employees who had a new product they wanted IBM to manufacture and sell. The many people in several different departments of IBM, who had to approve a new product were hard to sell. New ideas were being lost for lack of communication skills. The inventors and designers needed these skills to promote their new products. A university had been paid to design a training. Bob was teaching it and the training was not successful. He wanted to add some of our seminar material in his training, about a day and a half of our material. 

     I told him about the five year lease of our videos, and he returned to Atlanta to get approval. In the meantime, I agreed that he could "borrow" a day and a half of our seminar for one dollar so I would retain my ownership of the material. I think IBM spent thousands in attorney's fees to send me a check for $1.00. Bob loved teaching the seminar. The outcomes were what he wanted. 

     Four weeks later, I had a phone call from another IBMer in Tennessee who wanted to use two days of our material in a Negotiation training he was leading for engineers. He didn't know Bob, but I suggested they pay for the video lease together. We leased to corporations and they both worked for the same one. The contract was being considered at higher levels. No approval yet. 

     Then I got a call from Bob saying he had a letter from the head of his division, saying, "No Neuro-Linguistic Programming is to be taught inside IBM." Bob said he'd resign and work with me. His New Design Sales Trainings with our material were going so well, he didn't want to stop teaching our skills. 

     I suggested he not quit his job, but instead, send a book to the person who wrote the letter. 

     Eventually, after a face-to-face meeting we sold the letter-writer on the usefulness of our skills and five years later IBM conducted its own research, presented here. 
 

HOW CHASE MANHATTAN BANK BECAME PART OF OUR STUDY

     Well, actually, Chase does not consider the newsletter interviews research. When I pointed out that interviews conducted by an impartial third person are considered highly valuable as research, I was told, "It's only for an internal newsletter 'article.'" Yes, well, the numbers are small but the results are impressive. One-year-later interviews are long-term research and highly valued, by their editor, by me, and, maybe, by you. Those interviewed not only remembered what they had learned, but they were still using the skills and found them essential for their job of auditing. 

     The secrets of legitimate research are usually buried under the obscurantism of academic writing, but there are only a few necessary facts that you need to know to design sophisticated research. You will find these facts here and in the following pages, which contain copies of all the instruments and questionnaires used. 
The above is an excerpt from "Toot Your Own Horn". 
 

LEVELS OF EVALUATION

      Donald L. Kirkpatrick has written a book, Evaluating Training Programs: The Four Levels, in which these widely accepted criteria are presented. 

• Level 1: "Smile Sheets." Trainee responses. Important because if you don't like the course you are not likely to use the material. Did trainees like the course? 
• Level 2: What did the trainees learn? 
• Level 3: Do behaviors change on the job? 
• Level 4: Did behavior changes improve business results? 

27 STUDIES: THIS BOOK IS BASED ON 27 DIFFERENT RESEARCH PROJECTS: SOME PROJECTS HAVE AS MANY AS 11 SEPARATE DATA COLLECTIONS FROM INTERVENTIONS OVER TIME.

1. DESIGN l: MAILED SURVEY DESIGNED AND ADMINISTERED BY SPRINT PERSONNEL.

     The first study in 1989, was designed, administered and collated by corporate training personnel under the direction of Janice Schoenwetter and JoAnn Garner. Some questions gathered Level 4 data about bottom-line impact from training. The individual comments, which were written on the mailed form by Sprint employees, are presented separately here from the interviews. 

     Genie Z. Laborde, who led the later research studies, did not know of this Sprint research until months after it was completed. The questionnaire and study itself is presented in Chapter 3. 

     Levels I.II.III.IV 
 

2. DESIGN 2: INTERVIEWS FOR PATTERNS OF CHANGE - CONDUCTED BY GENIE LABORDE WITH KEY EXECUTIVES, MANAGERS, AND EMPLOYEES.

     The second study began with interviews from the three directors of a division we later trained. These three interviews, plus interviews with the direct reports shaped the instrument called Opinion Surveys, which Genie then administered to the top people in this division. 

     Levels I and III. 
 

3.DESIGN 3: INTERVIEWS FOR QUOTATIONS AND ANECDOTES.

     Genie also continued to interview 43 additional people from all levels, who had been through the Influencing with Integrity^® training. These last interviews were primarily for a training video she produced with Sprint, and are called Ditto Ripples because the subjects mentioned usually had already been initiated and encouraged by members of the key group. The key group interviews were longer, more in-depth, and give a bigger picture so these are given more space. The Ditto Ripple quotations are presented to show that the changes noticed at the director and manager level, also were being noticed and repeated at the supervisory and salesperson level as well. 

     Levels II and III. 
 

4.DESIGN 4: CHASE MANHATTAN BANK: OBJECTIVE THIRD PARTY INTERVIEWS

     The editor of an in-house newsletter at Chase Manhattan Bank conducted interviews with eight of the participants of the Communication Excellence seminar more than one year after the seminar. The interviews were about changes as a result of the seminar. A copy of this article is included in Chapter 2. 

     Levels II and III. 
 

5. DESIGN 5: MAILED QUESTIONNAIRE TO PARTICIPANTS FOR LEVEL 3 RESEARCH AT IBM (SUBJECTIVE ANSWERS).

     This questionnaire was designed by IBM to gather Level 3 data from the participants about their use of the seminar skills on the job one year later. 
 

 6. DESIGN 6: MAILED QUESTIONNAIRE TO MANAGERS OF PARTICIPANTS FOR LEVEL 3 RESEARCH AT IBM (OBJECTIVE ANSWERS).

     Level III. 
 

7.DESIGN 7: OPINION SURVEYS: INSTRUMENT ADMINISTERED AT SPRINT. (BOTH SUBJECTIVE, ON SELF, AND OBJECTIVE, ON OTHERS).

     This instrument grew out of the in-depth interviews and was mailed, or handed to the 15 members of the key group and other managers. Eleven returned the instrument. Levels II and III. 
 

 DESIGN 7: (SAME INSTRUMENT, DIFFERENT AUDIENCE).

     The same Opinion Survey, created by Sprint feedback, was administered at FPL (Florida Power & Light) with trainers and managers to ascertain whether the same changes were noticed in another corporation where we had trained approximately 1,000 people in Customer Service. This FPL Opinion Survey is also included in Chapter 3. A composite of the Sprint FPL results is also included in Chapter 3. 

     Levels II and III. 
 

8.DESIGN 8: I-E LOCUS OF CONTROL LEVENSON'S VERSlON OF ROTTER'S INSTRUMENT FROM SOCIOLOGY.

     Conducted with 10 groups in the U.S., two in Russia, one in Mexico. This sensitive instrument evolved from theories in sociology and measures deeply held beliefs that affect behavior, responsibility and productivity. The first version was designed by Julian Rotter and there are more than 1500 separate studies using versions of his instrument. More information on this can be found on pages 18-1 to 18-20. The companies contributing data are: Southland Corporation (7-11), Cernitin America, Allstate Insurance, Multi-Management Co., M&M Mars, Interceramic and Nissan. 
 

9.DESIGN 9: I-E LOCUS WITH THREE SEPARATE CONTROL GROUPS.

     Linda McGregor, in her job as Sales Training Manager at Pacific Bell conducted her own study using this instrument. See pages 19-1 to 19-7. 

     Levels II and III. 
 

 10. DESIGN 10: PANEL DISCUSSION AUDIO-TAPED FOR QUOTATIONS. AMERICAN SOCIETY FOR TRAINING AND DEVELOPMENT PANEL

     Six representatives of corporations who are experts in their field delivered their own research and experiences around the G. Laborde seminars and these presentations were audio-taped. Relevant excerpts from the audio tapes are presented on pages 21-1 to 21-7. 

     Levels II and III. 
 

 11.DESIGN 11: BOTTOM LINE RESULTS

     Actual figures and opinions are both useful in this area; however, actual figures carry more weight. This is Level 4 research and indeed, difficult to elicit from clients. Sprint's corporate research gathered Level Four data. See chapter one. 

     Level IV. 
 

12.DESIGN 12: SMILE SHEETS

     Evaluations by participants immediately following a seminar are designated Level One research. This level reflects whether the students liked the course or not. If they liked the course, they are more likely to use the skills presented. The research presented in this category was collected by Dell Computers. We also gathered class evaluations from each corporation on our client list. Most of these have been retained in our files at our corporate office. 
 

13.DESIGN 13: REVIEW OF OTHER STUDIES

     Compilation of multiple related studies conducted by other Researchers. This review is focused on the statistical correlation between interpersonal skills - relationships - health and longevity. 
 

IF YOU ARE DESIGNING RESEARCH

     The essential know-how of research is revealed in the chapters which contain examples of 13 specific types of research. All instruments that were used to gather the data are included in this book. 
 

 A RECAP OF POSSIBLE RESEARCH DESIGNS AND LEVELS:

1. SURVEY FOR PARTICIPANTS GATHERING DATA ON THE SUBJECT'S SUBJECTIVE AND OBJECTIVE EXPERIENCES. 

     When using questionnaires, the responses are returned without contamination of data by the presence of an interviewer. 

     Example: Sprint. 

     Levels I,II,III, and IV. 
 

2. INTERVIEWS FOR PATTERNS OF CHANGE

     Interviewing of subjective experience of participants is useful, if the numbers are large enough. One or two or even ten people is not enough, and the information is considered anecdotal. When thirty-plus people are interviewed, this is research. Subjective accounts are very powerful in indicating change. 

     Example: Quotations in Chapter 1. 

     Levels II and III. 
 

 3. INTERVIEWS FOR QUOTATIONS AND ANECDOTES.

     The author conducted numerous video interviews which are an excellent source of information. Example: Quotations in Chapter l. 

     Levels II and III. 
 

4. OBJECTIVE PARTY INTERVIEWS

     When the interviewer does not have a personal or professional stake in the answers, the information gathered has less contamination and more credibility - a legitimate method of data gathering. Example: Chase Manhattan Bank quotations in Chapter 2. 

     Levels II and III. 
 

5. QUESTIONNAIRE TO GATHER SUBJECTIVE INFORMATION ON TRAINING.

     Example: Level 3 questionnaire sent by IBM in Chapter 2. 

     Levels II and III. 
 

 6. QUESTIONNAIRE TO MANAGERS GATHERING SUBJECTIVE INFORMATION ON DIRECT REPORTS.

     Example: Level 3 questionnaire sent by IBM, Chapter 2. 

     Level III. 
 

7. OPINION SURVEY FOR OBJECTIVE AND SUBJECTIVE INFORMATION

     Objective information from managers, colleagues and observers adds another level of credibility to research findings. 

     Example: Chapter 3. 

     Levels II and III. 
 

8. INSTRUMENTS CAN PRODUCE POWERFUL STATISTICS.

     If the instruments are completed by 30 or more, you have useful and impressive research. The statistical results of pre and post instruments, compared with a control group, which indicate a shift or trend are difficult to discount. Your intervention (the training) has a strong probability of being the cause of the shift or the trend. To study our trainings, we've used the I-E locus of Control pre- and post-seminar as well as the Communication Congruence Inventory from University Associates. 

     Example: I-E Locus in Chapter 18. 

     Levels II and III. 
 

9. USE OF THREE DIFFERENT CONTROL GROUPS - PACIFIC BELL STUDY BY LINDA MCGREGOR.

     If you are gathering statistical data, you must be aware of the proper use of control groups. Control groups, When used properly will counter the Hawthorne effect. The Hawthorne effect is named from a series of studies that indicate any change, shorter hours or longer hours, can increase productivity at a manufacturing site. The way Ms. McGregor used control groups, to eliminate the Hawthorne effect is that she: 

1. administered the instrument at one time then again at a later time, roughly equivalent to the pre-and post- time of your test group. 

2. administered the instrument, but the group attended another training with similar but different content. Pre- and post- measurements are used. 

3. administered the instrument pre-and post-, but the group attended another training with decidedly different content. 

     Example: Pacific Bell Study 19-1 to 19-7. 

     Levels II and III. 
 

10. PANEL DISCUSSIONS WITH EXPERTS WHO ARE AUDIO-TAPED

     Panels are another source of valid information on the effectiveness of interventions. The experts draw not only from their own personal experiences but also on the experiences of the groups they represent. Research measuring change in human beings is difficult but possible. 

     Levels II and III. 
 

 11 INTERVIEWS USING A QUESTIONNAIRE.

     This approach if designed correctly and administered correctly, has the advantage of gathering figures. The disadvantage is that the information gathered may be contaminated, either positively or negatively by the attitude and nonverbals of the interviewer. We all respond to the person with whom we are communicating. Some of us like the other's approval and some of us consistently go polarity to what the other wants. 

     This can skew the results. 

     Levels II and III. 
 

12. MONEY TALKS: BOTTOM LINE RESULTS 

     Bottom line figures that show a trend are impressive as research. Money saved, time saved, estimates of money earned as a result of using new skills all affect bottom line. 

     Levels II, III and IV. 
 

13. REVIEW OF LITERATURE

     Research conducted by others can be compiled to build a strong case. Here health research, relationship studies and skills of communication indicate a correlation between immune system function, strong ties, and communication skills. 
 

NUMBERS

     Research is driven by numbers. What many people do not know, however, is that 33 people going in one direction are enough to indicate a trend. So if you use random selection and interview 33 people to discover a similarity, you've got research. Yes, the purists will want control groups, and matched populations, but this, in a practical world, is frosting on the cake. If the 33 are part of a written survey, without a live interviewer, even better. The live interviewer can contaminate as explained above. Also, statistics are not infallible, as you probably know. Here's one example: When Elvis Presley died there were 26 Elvis impersonators. In three years, there were 233 impersonators. In five years there were 1,397. Using statistical projections, by the year 2000, one in five U.S. citizens will be an Elvis impersonator. Be careful of relying too much on figures. Use your eyes, ears, feelings and common sense to make decisions about trainings. As one of the Sprint responders wrote on his survey in response to a request for an estimate of dollar amounts earned from the learned skills of the seminar, "Even as a manager, I think this is a peculiar question and wonder why we are so afraid to try things just because we intuitively know they are the 'right' things to do!" 
 

TIME OF RESEARCH

SPRINT:

     Most of the key-people interviews took place 6 to 12 months after the IWI training. The bottom line results, in the interviews, refer to those from August 1989 to February I990. The first interviews began in February 1990 and continued through the remainder of the year. The bottom line results from the Sprint Corporate Research are from 1989. 

PACIFIC BELL: 1988

FPL: 1990 

CHASE: 1987

IBM: APPROXIMATELY 1988

NAMES

     The real names of the ASTD panel members are used as are the real names of the Chase Manhattan Bank interviewees. In most cases the names of the personnel at Sprint have been changed. In a few cases, like Chuck Romero, the real names have been retained.