previous next

Chapter 11: Adaptive Stream Management

Adaptive Stream Management (ASM) rules describe a streamed datatype to RealServer, helping it make intelligent decisions about how to deliver that datatype's packets efficiently and robustly. File format and broadcast plug-ins define the ASM rules, which, in their simplest form, assign predefined property sets such as "priority" and "average bandwidth" to groups of packets. If RealServer needs to resend packets, for example, the packet priorities tell it which packets are most important to redeliver.

In their advanced form, ASM rules allow plug-ins to modify the delivery of packets based on changing network conditions. Each ASM rule can have an expression that defines a condition. For example, one rule's expression might define the condition in which available client bandwidth is from 5,000 to 15,000 bps and packet loss is less than 2.5 percent. If that condition describes the client's current network connection, the client subscribes to the rule. The properties defined within the rule then help RealServer stream the packets effectively. If network conditions changes, the client can subscribe to a different rule.

To implement ASM, the file format plug-in defines one or more rules (a "rule book"), each of which contains a set of properties and, optionally, an expression. When a client requests a file, the file format plug-in sends the client a stream header that contains the rule book. The client evaluates the rules' expressions against its own properties, such as its available bandwidth, and subscribes to one or more rules. It does not subscribe to a rule, for example, if the expression specifies a necessary bandwidth that is higher than the current network connection.

The file format plug-in begins streaming the file after the client transmits its subscription choices back to RealServer. The plug-in associates each packet with a rule, sending to RealServer only the packets for the subscribed rules. If there are three possible rules, for example, and the client subscribes only to the first two, the file format plug-in should not produce packets associated with the third rule.

Definitions

Rule
A set of properties and, optionally, an expression that helps RealServer transmit a group of packets intelligently. Each packet is associated with one rule.

Rule Book
A set of rules that describes a media stream. Each stream has one rule book.

Property
Information that helps RealServer stream packets associated with the rule intelligently. For example, the mandatory informational property "AverageBandwidth" tells RealServer the average bandwidth needed to stream packets of this rule effectively.

Expression
An optional part of a rule that defines a condition the client evaluates. The client subscribes to the rule if the condition fits its current network connection.

Subscribe
To pick a rule from a rule book and receive packets associated with that rule. The client can subscribe to any number of rules at any time.

Unsubscribe
To stop receiving packets for a rule.

Interfaces

The following interfaces are used with ASM:

Example of ASM Interfaces in Use

The following example explains the interactions of the ASM interfaces. The scenario involves two datatypes, A and B, requested by a RealSystem client. The file format plug-in for each datatype contains multiple ASM rules to which the client can subscribe.

  1. The client requests from RealServer a presentation containing datatype A and datatype B.

  2. The file format plug-ins for datatype A and datatype B send the stream headers to RealServer.

  3. RealServer transmits the rules, which are defined by the ASMRuleBook variable in the stream headers, to the client.

  4. By comparing the rules' expressions to its own network connection, the client picks rules to subscribe to for each stream.

  5. The client queries for IRMAASMStreamSink on the rendering plug-ins. If the plug-ins implement this interface, they can receive notice of rule subscriptions and perform necessary actions when the client subscribes to a rule that contains a custom property not defined in RealSystem.

    Additional Information
    See "Custom Properties".

  6. If the ASM rules for datatype B contain properties not defined in RealSystem, the rendering plug-in for datatype B calls IRMAASMStream::AddStreamSink on the client to set up a stream sink and receive notifications of rule subscriptions.

  7. The client calls IRMAASMStreamSink::OnSubscribe(<rule number>) for the datatype B rendering plug-in to notify it of the client's rule subscription.

  8. The client calls IRMAASMStream::Subscribe(<rule number>) for each stream to transmit the rule subscription information to RealServer.

  9. RealServer calls IRMAASMSource::Subscribe(<rule number>) to transmit the client's rule subscription choice to each file format plug-in.

  10. The file format plug-in begins to send RealServer packets associated with the subscribed rules.

    Note
    When a broadcast plug-in implements ASM, RealServer always subscribes to all rules. For each client, RealServer internally filters out packets associated with unsubscribed rules.

During the presentation, the client or the rendering plug-in (if it monitors rule subscriptions) can change the rule subscription because of changing conditions in the client connection. Here is an example:

  1. The client determines that, due to changing conditions in the connection, it must drop a rule subscription for datatype B.

  2. The client calls IRMAASMStreamSink::OnUnsubscribe(<rule number>) to notify the rendering plug-in for datatype B.

  3. The client calls IRMAASMStream::Unsubscribe(<rule number>) to relay the unsubscribe action to RealServer.

  4. RealServer calls IRMAASMSource::Unsubscribe(<rule number>) to transmit the rule change to the file format plug-in for datatype B.

  5. When the client determines that it can reinstate the rule, it calls IRMAASMStream::Subscribe(<rule number>) and starts off the subscription process described above. RealServer then waits for a packet with the ASM_SWITCH_ON flag and starts resending the packets for the reinstated rule.

ASM Rules and Properties

A file format or broadcast plug-in defines a "rule book," which is a set of rules for streaming packets of its datatype. Each rule can optionally have an expression that the client evaluates when determining which rules to subscribe to. Each rule sets a relative priority for its packets and contains information that helps RealServer determine how best to stream the packets, as well as how to handle certain situations that may arise. Here are examples of two simple rules:


AverageBandwidth=12000,
Priority=7;

#16000 <= $Bandwidth,
AverageBandwidth=4000,
Priority=6;

The client always subscribes to Rule 1 and always receives the Rule 1 packets. It subscribes to Rule 2 and receives the Rule 2 packets only if its total available bandwidth is at least 16,000 bps. Note that this does not mean that Rule 2 packets stream at 16,000 bps. If Rule 1 packets currently consume 12,000 bps and total available bandwidth is 16,000 bps, for example, Rule 2 packets stream at 4,000 bps. The different priority values indicate that if packets need to be resent or dropped, Rule 1 packets take priority over Rule 2 packets.

RealSystem predefines properties such as "Priority." Note, however, that the architecture also supports custom properties known only to the rendering and the file format plug-ins. The following sections explain the predefined rule properties.

Predefined ASM Rule Properties
Property Value Property Type

Bandwidth

INT32

Expression Variable

PacketLoss

Float

Expression Variable

Priority

1-10

Priority factor

AverageBandwidth

INT32

Informational

AverageBandwidthStd

Float

Informational

TimeStampDelivery

Boolean

Directive

WaitForSwitchOff

Boolean

Directive

Marker

Boolean

RTP Marker Bit

Additional Information
See "Custom Properties". See "Creating a Rule Book" for rule syntax descriptions and examples.

Expression Variables

The RealSystem client can subscribe to ASM rules by evaluating each rule's expression. If the rule has no expression, it is active at all times and the client always receives packets associated with that rule. The expression can contain the following variables, which are known to RealSystem. You cannot use other predefined ASM properties or custom properties defined in your file format and rendering plug-ins in an expression.

Additional Information
See "Custom Properties".

Bandwidth

This variable triggers subscription to the rule based on the client's available bandwidth. The rule book needs to cover the continuous range of possible bandwidths, from the minimum acceptable bandwidth to "infinite" bandwidth. For example, the bandwidth properties in the expressions for the following two rules leave a gap for client bandwidth between 16000 and 20000 bps:


#(12000 < $Bandwidth) && ($Bandwidth < 16000),
. . . ;
#(20000 < $Bandwidth) && ($Bandwidth < 24000),
. . . ;

But this set of rules is acceptable:


#12000 < $Bandwidth,
. . . ;
#16000 < $Bandwidth,
. . . ;

This set of rules is also acceptable:


#(12000 < $Bandwidth ) && ($Bandwidth < 16000),
. . . ;
#16000 < $Bandwidth,
. . . ;

Note that the two sets of acceptable rules above cause different client actions. In the first set, the client subscribes to Rule 1 if bandwidth is above 12000 bps. It subscribes to Rule 1 and Rule 2 if bandwidth is above 16000. With the second set of rules, the client subscribes to either Rule 1 or Rule 2, but not both, depending on its available bandwidth.

PacketLoss

This property is a percentage that specifies a range of packet loss. A rule book that uses this variable typically states it in an expression such as:


#(12000 < $Bandwidth < 16000) && (20.0 < $PacketLoss)

Priority

RealServer uses the Priority property to determine how best to stream a presentation to a client. The property takes a value from 1 to 10 that sets the relative priority of the packets associated with the rule. The highest priority, 10, is reserved for packets that are essential to a presentation. RealSystem does everything necessary, including stalling a client presentation, to deliver these packets. Use this priority only if losing a packet in the stream invalidates a presentation. Packets that are critical but not essential, such as key frames, should use priority 9.

The remaining priorities, 8 to 1, are relative rankings that help RealServer decide what packets to resend first and which to drop if needed. Audio data would typically fall in the range of 6 to 8. Video p-frames would typically be ranked at priority 5 or 6. Lower rankings indicate data than can be dropped without damaging the presentation severely.

Informational Properties

Informational properties help RealServer determine how best to stream packets associated with the rule. The rule book expresses them with this syntax:


<property> = <value>

AverageBandwidth

AverageBandwidth is an estimate of the average bits per second of the packets streamed for this rule.

Note
All rules must provide a value for this property, unless TimeStampDelivery is set to True in the rule.

AverageBandwidthStd

This optional parameter indicates an estimated standard deviation of the average bandwidth. Data with a stable bit rate, such as audio, should have a standard deviation of 0. Data with variable bit rates, such as video, should have a value appropriate to the datatype.

Directives

Directives place conditions on serving packets for the rule. Most directives are Boolean values that tell RealServer to perform certain actions while the rule is active. The rulebook expresses directives with this syntax:


<property> = TRUE|FALSE|<value> 

TimeStampDelivery

When the Boolean property TimeStampDelivery is set to True, RealServer delivers packets in step with the presentation timeline, and does not attempt to send packets in advance of their rendering time. This is useful for datatypes such as RealText that consume small amounts of bandwidth and have highly variable bandwidth usage patterns. It is not to be used for audio, video, or other datatypes that consume high bandwidth and require initial buffering.

If TimeStampDelivery is set to True, AverageBandwidth cannot be used in the rule. When TimeStampDelivery is set to its default value of False, AverageBandwidth must be set in the rule. Note, however, that one rule could use AverageBandwidth, while another rule uses TimeStampDelivery.

WaitForSwitchOff

This Boolean property has a default value of True. In this case, after RealServer notifies the file format plug-in that a client has unsubscribed from a rule, it waits for a packet with the flag ASM_SWITCH_OFF before it stops sending packets for that rule. If WaitForSwitchOff is set to False, RealServer stops sending packets as soon as it receives the unsubscribe request from the client. RealServer discards any further packets for that rule that the file format plug-in may send.

Additional Information
See "Creating Stream Packets".

RTP Marker Bit Property

If the file format supports RTP payloads that make use of the marker bit in the RTP header, it must create the ASM rule book to identify which ASM rules map to RTPpackets with the marker bit set. By convention, if a rule book is to contain only two entries, rule 0 corresponds to marker bit off and rule 1 to marker bit on. Since the ASM rule book requires the AverageBandwidth (or TimeStampDelivery) property to be set, the minimum rule book for an RTP payload making use of marker bits looks like this:


Marker=0,AverageBandwidth=<half of total bitrate>;
Marker=1,AverageBandwidth=<other half of total bitrate>;

where <other half of total bitrate> = <total bitrate> - (ULONG32) (<total bitrate> / 2).

Once the mapping has been established, the file format plug-in assigns the ASM rule numbers to packets to match the desired RTP marker bit setting.

The renderer performs the reverse of this mapping. It receives the ASM rule book through its IRMARenderer::OnHeader method and parses the ASM rule book to determine which ASM rules correspond to RTP packets with the marker bit set. For datatypes that do not support ASM switching and therefore contains only two rules in the ASM rule book, by convention, rule 0 corresponds to marker bit off and rule 1 to marker bit on.

Note
When the RealPlayer receives RTP packets from a server that is not taking advantage of ASM, it forms the ASM rule book always mapping the packets without a marker bit set to ASM rule number 0 and packets with a marker bit set to ASM rule number 1.

Additional Information
See the table "RTP Payload Types".

Custom Properties

Rendering plug-ins can monitor rules that are not predefined in RealSystem. RealSystem ignores any property it does not know about, passing the property value to the rendering plug-in. If a file format plug-in defines custom properties, its corresponding rendering plug-in becomes a stream sink with IRMAASMStream::AddStreamSink. The RealSystem client then uses IRMAASMStreamSink to notify the rendering plug-in of rule subscriptions. The plug-in can thereby take the appropriate actions when a rule contains a custom property.

Note
You cannot use custom properties as expression variables.

Creating a Rule Book

You can define or generate a rule book within the file format plug-in code or within another source that the plug-in reads. The latter method is useful if the rules for delivering the datatype are subject to change. The plug-in passes the rules to RealServer as a single, long string in the file stream header. The following are the syntax rules for the ASM rule book:

The following example shows a simple rule book defined within the file format plug-in code:


    const char* pRuleBook =
{
/* Rule 0 */
" \
#$Bandwidth < 16000, \
AverageBandwidth=12000, \
AverageBandwidthStd=0, \
Priority=7; \
"
/* Rule 1 */
" \
#16000 < $Bandwidth, \
AverageBandwidth=16000, \
AverageBandwidthStd=0; \
"
/* Rule 2 */
" \
Marker = 0; \
"
};

Rule Book Expressions

Rule expressions can employ the following C-style operators. The operators follow the standard C++ evaluation order, which you can modify with parentheses. Expressions can be arbitrarily complex.

ASM Rule Book Operators
Operator Evaluation
> greater than
< less than
= greater than or equal to
<= less than or equal to
== equal to
!= not equal to
&& AND
|| OR

Rule Examples

Example 1: RealAudio Sample Rules

The following table shows several rules defined for RealAudio streams. The various rules are used by the client to switch between different Dolby codecs.

RealAudio Sample ASM Rules
Rule # Low
Bandwidth
bps
High
Bandwidth
bps
Average
Bandwidth
Average
Bandwidth
Std
Priority
0 0 16000 12000 0 7
1 16000 20000 16000 0 7
2 20000 40000 20000 0 7
3 40000 80000 40000 0 7
4 80000 0xffffffff 80000 0 7
5 (Used for Events) 5 3000 9

Example 2: 28.8K RealVideo Sample Rules

The next table defines rules for delivery of RealVideo over a 28.8Kbps connection.

RealVideo Sample ASM Rules
Stream # Rule # Low
Bandwidth
bps
High
Bandwidth
bps
Average
Bandwidth
Average
Bandwidth
Std
Priority
1 0 (audio) 0 0xffffffff 8500 0 7
2 0 (video keyframes) 0 0xffffffff 4000 5000 9
2 1 (video P frames) 20000 0xffffffff 7500 1000 5


Copyright © 2000 RealNetworks
For technical support, please contact supportsdk@real.com.
This file last updated on 05/17/00 at 12:50:20.
previous next