SECOND-ORDER SCHEDULE

SECOND-ORDER SCHEDULE

Primary Disciplinary Field(s): Behavioral Psychology; Operant Conditioning

1. Core Definition

The second-order schedule of reinforcement represents a sophisticated arrangement within the framework of operant conditioning theory, designed to maintain behavior through long chains of responding without continuous primary reinforcement. Fundamentally, it is a timetable of reinforcement wherein the units counted towards the delivery of the ultimate, primary reinforcer are not individual, discrete reactions, but rather the completions of a specific, defined reinforcement routine or component schedule. This arrangement differs markedly from simple schedules (such as Fixed Ratio or Variable Interval) where every required response or interval directly precedes the primary reward. In a second-order schedule, the overall schedule reinforces the completion of a sequence of actions, with each completion often signaled by a brief, non-primary stimulus which serves as a conditioned reinforcer.

The defining characteristic is the hierarchical organization: the component schedule operates repeatedly, and only after a specified number of component completions (or components occurring over a specified time) does the terminal primary reinforcement become available. For instance, an organism might be required to complete five separate Fixed-Ratio 10 sequences before receiving food. The structure allows researchers and trainers to sustain high rates of responding over extended periods, relying heavily on the motivational strength derived from the conditioned stimuli associated with the completion of each unit, rather than the primary reward itself. The secondary reinforcer links the immediate action to the delayed primary outcome, bridging the temporal gap between the necessary response and the ultimate satisfaction of a need state.

This type of schedule is particularly valuable for studying how complex, sustained behaviors are maintained and how conditioned reinforcement influences response persistence. It highlights the power of internally or environmentally cued achievements, where the successful conclusion of a small task provides sufficient reinforcement (the brief stimulus) to motivate the immediate commencement of the next task in the larger sequence. The structure effectively models real-world situations where significant rewards are contingent upon the completion of numerous sub-goals, emphasizing the importance of sequential behavior chains in achieving distant objectives.

2. Relationship to Simple and Compound Schedules

To fully appreciate the complexity of the second-order schedule, it must be contextualized alongside simple and other compound schedules. Simple schedules, such as Continuous Reinforcement (CRF), Fixed Ratio (FR), Variable Ratio (VR), Fixed Interval (FI), and Variable Interval (VI), involve only one rule relating response to reinforcement. Compound schedules combine two or more simple schedules. Second-order schedules are considered a highly specific form of a chained schedule, but with a critical difference regarding the nature of the reinforcement delivered within the chain.

In a strict chained schedule (Chain FR 5, FR 10), the organism must complete the first schedule, which leads to a change in the environment (a new stimulus) signaling the start of the next schedule, until the final response yields the primary reinforcer. In contrast, the second-order schedule maintains a single primary schedule throughout the experiment, but the reinforcement delivered at the end of each component unit is purely a brief, conditioned stimulus (S), not a change in the environment signaling a new response requirement. This structure, often denoted F(R-S), where the primary reinforcement (R) follows a fixed number (F) of component completions (S), emphasizes the motivational efficacy of the secondary reinforcer in its own right, irrespective of environmental changes.

The key distinction lies in the concept of the response unit. In most simple schedules, the response unit is a single lever press or peck. In a second-order schedule, the response unit is the completion of an entire component schedule (e.g., 50 consecutive pecks under an FR 50 component). The overall schedule, which dictates when the primary reinforcement is delivered, governs the frequency with which these complex units must be completed. This overlay mechanism is powerful because it allows a researcher to study the maintenance of high-effort, sustained behavior that cannot be easily studied using schedules that rely on frequent primary reinforcement.

3. Notation and Structural Variations

The notation for second-order schedules is crucial for understanding their specific operation. The general form typically uses brackets or parentheses to denote the relationship between the primary schedule and the component schedule. The component schedule is usually one of the four basic schedules (FR, VR, FI, VI), and the overall schedule can also be one of these types. The general structure is represented as Overall Schedule (Component Schedule: Conditioned Reinforcer). For example, a common structure is FR 5 (FR 10: S).

In this example, the organism must complete five separate component schedules (FR 10) before receiving the primary reinforcer (R). During each of the five component completions, the conditioned reinforcer (S) is delivered immediately after the tenth response. The component schedule (FR 10) means 10 responses are required to complete one unit. Thus, 50 responses are required in total for R, but the conditioned stimulus S is delivered five times during the process. This structure facilitates the study of response patterns characteristic of the primary schedule (FR 5) but scaled up, relying on S to sustain the behavior through the long overall requirement.

Variations exist based on the combination of interval versus ratio schedules. A particularly important variation is the FI (FR 50: S) schedule, where a fixed interval must elapse before the primary reinforcer is available, but the components completed within that interval (each requiring 50 responses and yielding S) sustain the responding. Researchers often use ratio-based components (FR or VR) because they generate high and consistent rates of responding, making the delivery of the conditioned stimulus S reliably frequent, which is essential for maintaining its associative strength. The selection of the specific component schedule determines the local rate and pattern of responding, while the primary schedule dictates the overall temporal structure of the experiment.

4. The Role of the Secondary Reinforcer (Conditioned Stimulus)

The mechanism by which the second-order schedule successfully maintains responding over long delays to primary reinforcement is entirely dependent on the effectiveness of the secondary reinforcer (S). This stimulus is typically brief and unobtrusive—such as a short tone, a momentary light flash, or a click—given immediately upon the completion of each component unit. Its power is derived from its association with the eventual delivery of the primary reinforcer (R), established through repeated pairings, making it a conditioned reinforcer.

The conditioned stimulus serves several crucial functions. First, it acts as a marker or signal, confirming to the organism that a sub-goal has been successfully achieved, thereby providing immediate feedback that counters the effects of delayed reinforcement. Second, the secondary reinforcer maintains the motivational momentum necessary to initiate the next component sequence. Without this intermediate reward, long schedules requiring many responses before primary reinforcement typically lead to extinction or significant response pausing, a phenomenon known as “ratio strain.” The consistent delivery of S prevents this strain by reinforcing the component sequence itself.

Research has demonstrated that the characteristics of S are critical. If the conditioned stimulus is too weak, poorly paired with R, or delivered inconsistently, the structure of the second-order schedule collapses, and behavior degrades. Therefore, experimental rigor demands careful calibration of the S to ensure its strength as a reinforcing agent remains high throughout the session. The use of second-order schedules thus provides an excellent methodological tool for quantifying the strength and maintenance of conditioned reinforcement, a central topic in the study of motivation and learning.

5. Applications in Behavioral Modification and Training

The principles underlying the second-order schedule have profound implications for applied behavioral analysis (ABA) and complex animal training. In real-world contexts, most significant achievements—such as completing a college degree, building a complex physical structure, or mastering a professional skill—are inherently second-order processes, requiring the completion of many small, reinforced steps before a final, large reward is attained.

In animal training, particularly with species like marine mammals or working dogs, second-order schedules are invaluable for developing long behavioral chains. Trainers use “bridging stimuli” (like a whistle or clicker, which are powerful conditioned reinforcers) to immediately signal the correct completion of a sub-task. For example, a dog learning a complex search pattern might receive a click (S) for correctly clearing one quadrant of a room, even though the primary reward (R, food or play) is only delivered after the entire search pattern is completed. This immediate feedback maintains high motivation and precision through the entire sequence.

In human contexts, educational and therapeutic environments often implicitly rely on second-order structures. A student receiving high marks (S) on a series of assignments (component units) is reinforced immediately, sustaining the effort required to eventually earn the diploma (R). For individuals undergoing behavioral therapy, especially those dealing with addiction or impulse control, setting up intermediate goals that yield measurable, immediate social or personal reinforcement helps maintain adherence to a long-term plan that eventually yields the primary, life-changing reward. The schedule provides a template for structured goal decomposition and long-term motivation.

6. Experimental Evidence and Response Patterns

Experimental research using second-order schedules, pioneered largely by researchers like B.F. Skinner and later detailed by investigators such as Kelleher and Gollub, has provided critical insights into response dynamics under complex contingencies. A significant finding is that the response pattern observed under the overall schedule often mirrors the characteristic pattern of that schedule, even though the primary reinforcement is delivered rarely.

For instance, if the overall schedule is FI 5 min (FR 10: S), the overall pattern of responding tends to exhibit the classic “scallop” shape associated with Fixed Interval schedules: a period of low or zero responding immediately following R, followed by a gradually accelerating response rate as the interval approaches completion. However, within the accelerated phase, the local responding is sustained at a high rate by the consistent reinforcement provided by the component schedule completions (S). The organism learns not just the overall timing, but also that sustained effort is required between the secondary reinforcements.

Furthermore, research on second-order schedules has been instrumental in demonstrating the enduring power of stimulus control. When the conditioned reinforcer (S) is delivered, it often acts as a momentary discriminative stimulus (S^D) for the initiation of the next component schedule. The maintenance of steady, high rates of responding under very lean primary schedules (e.g., VI 60 min (VR 50: S)) highlights the remarkable efficacy of conditioned reinforcement in sustaining behavior over durations that would be unsustainable using simple schedules alone. These experiments confirm that behavior is not purely driven by immediate need reduction but by sophisticated associative processes.

7. Debates, Complexity, and Limitations

While highly effective, the second-order schedule structure is subject to several theoretical and practical debates. One major point of contention centers on the exact nature of the conditioned reinforcer (S). Some researchers argue that S functions purely as a discriminative stimulus, signaling the opportunity to continue responding toward the primary goal, rather than possessing independent reinforcing properties. Others maintain that S acquires true reinforcing value, capable of supporting behavior in the absence of the primary reinforcer, much like a primary reward. The consensus leans toward the latter, emphasizing the motivational impact of S, but the fine distinction remains an area of ongoing research.

A practical limitation is the complexity of implementation. Setting up a highly effective second-order schedule requires rigorous control over environmental variables and meticulous calibration of the component and primary schedule parameters. If the overall schedule is too lean (i.e., too long between primary reinforcers) or the component schedule is too demanding, the conditioned reinforcer may lose its associative strength, leading quickly to ratio strain and behavioral breakdown. Therefore, the successful application relies heavily on established behavioral history and the inherent reinforcing potential of the chosen primary reward.

Finally, there is a debate regarding the generalizability of response patterns observed under these schedules. While they successfully model the maintenance of long behavioral chains, the artificial nature of the discreet, often mechanical S (tone or light) may not perfectly replicate the often-subtle, intrinsic conditioned reinforcement (e.g., feelings of accomplishment or cognitive feedback) that sustains complex human behavior in natural environments. Nevertheless, the second-order schedule remains a cornerstone methodology for analyzing the hierarchical organization of learned behavior.

8. Further Reading

• Operant Conditioning (Wikipedia)
• Conditioned Reinforcer (Wikipedia)
• Reinforcement Schedule (Wikipedia)