Lessons Learned: Is Training Without Generalization Effective?

by Adrienne Robek and Alicia M. Alvero
Queens College and The Graduate Center of the City University of New York

Although most articles are accounts of successful research studies, I prefer to think of the present article as an account of a “learning experience.” In 2002, my advisor at the time and I came up with a study to train staff at an applied behavior analysis (ABA) school for children with autism. Unfortunately, we did not get the results we had hoped for. We were never able to obtain experimental control over the behavior of the staff members. In the spring of 2004, I decided to take an introductory organizational behavior management (OBM) class and now that the semester is almost over, it is very clear to me why our study did not work. The following is a summary of what the study consisted of and why it was a “learning experience.”

Many of the staff in the ABA school where the study was conducted knew how to conduct discrete trial teaching, but were not familiar with incidental teaching. Discrete trial teaching is when the three-term contingency (A > B >C) is applied to teaching new skills and is primarily teacher-initiated. The “A” is the antecedent in the form of the instruction provided by the teacher. The “B” is the behavior the student performs and the “C” is the consequence provided to the student by the teacher. Incidental teaching is a method of teaching that occurs in a naturalistic and/or unstructured setting and is child-initiated (Hart & Risley, 1975). Incidental teaching can be used to teach a number of skills but is most often used to help facilitate language acquisition.

The purpose of my study was to teach teachers to train their assistants to implement incidental teaching in an attempt to increase the amount of appropriate requests made by three school children. A multiple-baseline across subjects design was used. During phases 1, 2, and 3 the staff were pulled out of their classrooms and trained by the experimenters in an empty classroom. The baseline phase (phase 1) consisted of giving the teachers a definition of incidental teaching and then verbally instructing them to use incidental teaching with the children in their classrooms. Data were collected on the behavior of the teachers assistants and the children during regular class time. The scores for this phase were predominantly zeros for all subjects (see Fig.1).


Figure 1

Phase 2 was the training phase. Teachers were given a training packet (see Table 1) and the experimenter reviewed the materials with them.

Table 1
How to Conduct Incidental Teaching

Next, the experimenter modeled five different incidental teaching scenarios for the teachers with another experimenter. These five teaching scenarios were all modeled during each training session. While the experimenters were modeling incidental teaching, they verbally explained what they were doing step-by-step. After this modeling took place, one of the experimenters conducted incidental teaching with a child who was attending the school but was not taking part in the study. The teacher and one of the experimenters collected data on the experimenter who was working with the child in order to ensure the teacher could accurately assess correct incidental teaching. Data collection stopped once the teacher and the experimenter had an inter-observer agreement (IOA) of 100% for three consecutive sessions (see appendix). Subsequently, the teacher conducted incidental teaching with the same child until the child achieved 100% accurate performance for three consecutive sessions. Again, data were collected in class, and the scores for this phase were all zeros for all subjects (see Fig. 1).

Phase 3 was the assistant training phase. During this phase, the teachers trained one of their classroom assistants using the same training method that the experimenter used to train the teachers. After the assistants completed their training, the experimenters collected data on the frequency and accuracy of incidental teaching conducted in the classroom by the teachers assistants. Once again, the scores for all participants were zeros (see Fig. 1). Phase 4 was the feedback phase. In this final phase, the experimenters gave feedback to the teachers on the behavior of their assistants. The teachers relayed this feedback to their assistants. We only collected data for one teachers’ assistant (Jackie) during this phase and her scores were all zeros.

During all phases, data were collected on the assistants’ rate of classroom incidental teaching. Data were also taken on the rate of appropriate requests made by the children. Throughout all phases of this study, the children’s scores were also predominantly zeros (see Fig. 2). Although the teachers and assistants were trained on how to conduct incidental teaching, they rarely ever incorporated it into their classroom routine. The staff had clearly shown that they knew how to conduct incidental teaching but did not use it in their classrooms.


Figure 2



Perlow (2001) cited a procedure by Maher (1983) as a possible solution to this problem. The procedure is called DURABLE and is an outline of how to successfully apply an intervention. “D” refers to meeting with management and discussing the proposed intervention. “U” stands for understanding whether the organization is truly dedicated to having a successful intervention. In the case of our study, management was very enthusiastic about their staff learning a new skill. Despite their enthusiasm, it is possible they did not consider whether or not the teachers and assistants could fit incidental teaching into their schedule. In addition, there were no consequences for performing this new skill. The consent form signed by all participants in the study clearly stated that the outcome of the study would have no consequences on their standing at the ABA school. They were not disciplined in any way for failing to implement incidental teaching into their classroom routine. This was a requirement of the institutional review board (IRB) at the university which sponsored the study. Likewise, the staff were not given any tangible reinforcers for engaging in the target behavior. They were simply given verbal praise. “R” is for reinforcement of the target behavior by the experimenters and management. The participants were praised for engaging in the target behavior during training, but because they never engaged in the target behavior outside of training, the opportunity for reinforcement did not occur. “A” stands for acquiring prerequisite conditions for successful implementation. We clearly stated to all participants that the purpose of the study was to teach staff members how to conduct incidental teaching, hoping it would result in appropriate requesting behavior from the children. “B” refers to building and stresses the importance of having support from management, which we had. “L” is for learning which means teaching participants in a way that will enhance their ability to engage in the target behavior. This is one of the areas where our study fell short. We taught the participants how to emit the target responses in one setting, but never taught them how to emit those same responses in a different setting. Lastly, “E” represents the evaluation of the intervention to see if it was successful. As our results clearly indicate, our intervention was not a success.

In retrospect, it was not that the staff never learned how to conduct incidental teaching – it was that there were no consequences to motivate them to do it in their classrooms. Furthermore, no one assessed whether or not it could feasibly be done. For example, the staff may not have had time in their classroom schedules to implement this new teaching method. The skill which was taught to them in one surrounding (an empty classroom) was never taught to them in a different surrounding (their classroom). The antecedent in training was different than the antecedent in the real world. There were also no reinforcers in the natural setting. Simply praising someone on a job well done may not be the most effective way to change their behavior. Our training sessions may have been DURABLE but the natural environment was not. When a staff member is pulled out of their classroom and has nothing else to do except incidental teaching, praise may function as a reinforcer. When that same staff member is back in their classroom with competing activities, it is very possible that praise is not a strong enough reinforcer to bring about a change in behavior. Although we successfully taught the teachers and assistants new skills, we did not ensure motivation to perform the new skills outside of the training session.

References

Hart, B. & Risley, T. R. (1975). Incidental teaching of language in the preschool. Journal of
Applied Behavior Analysis, 8, 411-420.

Perlow, R. (2001). Training and development in organizations: A review of the organizational
behavior management literature. In C. M. Johnson, W. K. Redmon, & T. C. Mawhinney
(Eds.), Handbook of organizational performance: Behavior analysis and management.
Binghamton, NY: The Haworth Press, Inc.