Commonwealth Scientific and Industrial Research Organisation CSIRO, Australia

Locked Bag 17 North Ryde NSW 2113 Australia +61 2 9325 3141 Silvia.Pfeiffer@csiro.au http://www.cmis.csiro.au/Silvia.Pfeiffer/

Commonwealth Scientific and Industrial Research Organisation CSIRO, Australia

Locked Bag 17 North Ryde NSW 2113 Australia +61 2 9325 3133 Conrad.Parker@csiro.au http://www.cmis.csiro.au/Conrad.Parker/

Commonwealth Scientific and Industrial Research Organisation CSIRO, Australia

Locked Bag 17 North Ryde NSW 2113 Australia +61 2 9325 3156 Andre.Pang@csiro.au http://www.ict.csiro.au/

This document specifies a syntax for addressing time intervals within time-based Web resources through URI queries and fragments. It suggests a Best Current Practice (BCP) for any time-based Web resource for which temporal subparts may be requested and retrieved. This enables, e.g., direct access to a clip of a video stored on a Web Server, or direct jumps to a specific event within a piece of music. The syntax is not restricted to audio or video Web resources, but covers all kinds of Web resources that contain time-continuous information. The URI query specification of this syntax implies that a Web server is capable to extract the given time subsections from the Web resource and serve that as another complete Web resource of the same type. Specifying a time point or interval through a URI fragment in contrast does not deliver a subpart of the Web resource - instead it is up to the application and the resource type as to what action is invoked. Examples are a fast forward to a time point, or a zoom into a time interval. This document is a proposal for a Best Current Practice. The authors strongly encourage anybody interested to explore the implementation of the syntax on different types of Web resources and contribute any findings. Current implementations work with Annodex and CMML format Web resources over HTTP. An implementation over e.g. RTP/RTSP for Internet streaming and for other protocols has not been explored yet. The syntax is also being explored within the ISO/MPEG standardisation body for addressing into MPEG-4 or MPEG-7 format resources. Also note that RealNetworks is using an different syntax for the query case for RealMedia over RTP/RTSP. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

Many resources on the World Wide Web represent information that relate to a timeline of events. Time-sampled recordings of sound, temporally interleaved audio-visual streams, or data files containing time series data are examples. This document describes a syntax for addressing temporal offsets and time intervals in such resources. The aim is to make it simple to incorporate infrastructure into the Web/Internet which supports the addressing and retrieval of temporal subparts of time-based Web resources, as well as the automated processing of such subparts e.g. for reusing them. Files containing commands for creating time-continuous experiences (such as SMIL files for authoring interactive multimedia, or VRML files for authoring 3D worlds) cannot be addressed in this manner - however, if one particular user interaction (i.e. one experience) is recorded, the recording contains information that relates to a single timeline and can be addressed with the manner described in this specification.

The Generic URI Scheme defines two manners in which subparts of Web resources can be addressed: queries and fragments. Queries are given through extending the Web resource's address by a string that is separated through the special character "?". Fragments are given through extending the Web resource's address by a string that is separated through the special character "#". RFC 2396 specifies that URI queries identify a resource and thus a Web server has to process the query returning a fully defined resource. There is no generically defined structure to URI queries. However, the format of a CGI query string where name-value pairs are separated by the special character "&" is a commonly used format. The Common Gateway Interface, or CGI, is a standard for external gateway programs to interface with information servers such as HTTP servers (see http://hoohoo.ncsa.uiuc.edu/cgi/). When defining a common manner in which temporal subparts of a Web resource can be addressed, it is important to be compatible with this common CGI format. RFC 2396 also specifies that URI fragments identify a secondary resource by reference to a primary resource and that fragments get interpreted client side, i.e. the Web infrastructure consisting of Web servers and Web proxies will generally ignore the "#" extension of URIs. This basically implies that a fragment provides a particular view on a resource and the view is defined through the type of resource and the client side application. Both, URI query and URI fragment identifiers have traditionally been defined for specific media types or even specific services. This is especially true for URI queries where the query string may consist of name-value pairs that contain the particular fields and field entries of a particular form that resides on a particular server and gets processed by a particular server extension. For the temporal addressing that is proposed in this document, it is important that every Web server that interpretes the query parameter reacts uniformly to it. The particular resources which are covered in this manner are however not defined in this document - when a media type gets registered for the Web/Internet, the authors must explicitly define if a Web resource of that media type can be addressed through the time-interval URI addressing scheme defined here.

URI query strings are specified after the special character "?". A temporal query parameter defines one or more intervals of time. Time is given with respect to specific time schemes. The default time scheme is "npt" (normal play time), in which a time point is specified in seconds through a floating point number with an arbitrary temporal resolution. The available time schemes and their specifications are described further down in this document. The BNF for a temporal URI query parameter is: All temporal intervals actually specify start and end times. If no end timespec is given, it defaults to infinity, which for a file means the end of the file or for a stream the end of the stream. Overlapping temporal intervals MUST be interpreted by merging the intervals into one. It is not valid to give several temporal URI query parameters. They all need to be wrapped into a single specification. Examples are: http://www.foo.bar/video.anx?t=15.2 --- video.anx is transferred from 15.2 seconds into video.anx to the end of the file/stream. http://www.foo.bar/video.anx?t=15.2-18.7 --- video .anx is transferred from 15.2 seconds into video.anx to 18.7 seconds. http://www.foo.bar/video.anx?t=15.2-18.7,23 --- video.anx is transferred from 15.2s to 18.7s and from 23s to the end of the file/stream. http://www.foo.bar/video.anx?t=15.2-18.7,17.4-30.1 -- video.anx is transferred from 15.2 seconds into video.anx to 30.1 seconds. http://www.foo.bar/video.anx?t=15.2-18.7&t=23 --- invalid query parameters - MUST create an error message.

URI fragment specifiers are given after the special character crosshatch ("#"). A temporal URI fragment defines a specific temporal view onto a Web resource consisting of one or more intervals of time. Again, time is given with respect to specific time schemes as defined below. Again, the default is "npt". The BNF for a temporal URI fragment identifier is (reusing the temporal-intervalspec of the URI query section above): As with temporal URI query parameters, all temporal intervals are specified through start and end times, the default end time being infinity, which may map to the end of a file or stream. Also, several temporal URI fragment identifiers are not allowed. Examples are: http://www.foo.bar/video.anx#15.2 --- specifies a view on the section between 15.2 seconds into video.anx and the end of the file/stream. http://www.foo.bar/video.anx#15.2-18.7 --- specifies a view on the section between 15.2s and 18.7s of video.anx. http://www.foo.bar/video.anx#15.2-18.7,23 --- specifies a view on the section between 15.2s and 18.7s, as well as 23s to the end of the file/stream. http://www.foo.bar/video.anx#15.2-18.7,17.4-30.1 -- specifies a view on the section between 15.2s and 30.1s of video.anx.

A time scheme is a short identifier for a type of time specification. The following general time schemes are specified in this document. Further time schemes are expected to emerge and should probably be registered through IANA. "npt" (Normal Play Time; base of seconds as used in the RTSTP standard) "smpte" (Society of Motion Picture and Television Engineers time codes as specified in the SMPTE time and control code standard) "utc" (Universal Time Code giving wall-clock time as specified in the ISO 8601 standard). Specification as BNF: Thus, the available time schemes are: NPT time has a second or subsecond resolution. It is specified as H:M:S.h (npt-hhmmss) or S.h (npt-sec), where H=hours, M=minutes, S=second and h=fractions of a second. Negative values are not allowed. Specification as BNF: SMPTE time codes are optimized for frame level accuracy. SMPTE is specified as HH:MM:SS:FF, where HH=hours, MM=minutes, SS=second, FF=frames. The drop-frame algorithms for calculating the exact times can be found in the references SMPTE standard. Negative values are not allowed. "smpte-24=" SMPTE time with a 24 fps basis "smpte-24-drop=" SMPTE time with a 24/1.001 fps basis "smpte-25=" SMPTE time with a 25 fps basis "smpte-30=" SMPTE time with a 30 fps basis "smpte-30-drop=" SMPTE time with a 30/1.001 fps basis "smpte-50=" SMPTE time with a 50 fps basis "smpte-60=" SMPTE time with a 60 fps basis "smpte-60-drop=" SMPTE time with a 60/1.001 fps basis Specification as BNF: UTC time has a second or subsecond resolution. It is given as YYYYMMDDTHHmmss.hhZ, where Y=year, M=month, D=day, H=hour, m=minute, s=second, h=subseconds (one-hundredth of a second). Specification as BNF: Examples for time-interval URI specifications with different time schemes are:

The semantic interpretation of time specifications given with any of the schemes depends on the resource. With every resource there are two associated timebases: a UTC timebase which may e.g. specify the creation time of the resource, and a playback timebase used for display in a user agent while viewing the resource. The playback timebase of a resource defaults to 0 seconds if the resource has no other timebase associated with it. For example, with professional video production, the first frame of video of a program normally refers to a SMPTE timebase of 01:00:00:00, not 00:00:00:00. This practice arose from the requirements of program production and analog videotape recording technology, and it has subsequently become a uniform, almost ironclad practice worldwide. Associating such a practice to a digital video resource requires a way to store that timebase with the resource, which may or may not be possible, depending on the content type of the resource. Examples: If a resource has an associated timebase of 3600 seconds, and the given temporal fragment offset is 4000 sec, a seek time 400 sec into the resource is requested. If the timebase is given as clock time 20001010T142211.23Z and the temporal offset specified is 20001010T145511.23Z, the time 33 minutes into the resource is requested. The UTC timebase of a resource defaults to non-specified. Associating such a UTC timebase with a resource requires a way to store that timebase with the resource. For example, for a resource that is a file on a server, it may be chosen to be the time of storage of that resource. The semantic interpretation of temporal intervals depends on the time scheme. Unless specified differently, the temporal intervals given are closed intervals, i.e. they start at the first time point and finish at the second time point: [time_from;time_to]. For SMPTE timecodes, however, it is conventional to express such temporal intervals as IN and OUT times for editing. Thus, the IN time specifies the first frame that is included in the interval and the OUT time specifies the first frame that is not included in the interval. Therefore, a SMPTE interval is specified as [time_from;time_to[, which explains the additional frame in the above example.

The temporal URI interval specifications are intended to be used on resources that represent a specific timeline. An example of such resources are most resources of MIME types "audio/*" and "video/*". A retrieval action on a URI that includes a temporal URI query parameter MUST result in a time-continuous resource that is reduced to the given temporal intervals. As per HTTP standard, if the URI including the temporal query parameter cannot be resolved by the server, the server has to return a "404" error message. Time-continuous resources often come with high bandwidth requirements, so serving out only the queried time interval avoids unnecessary network load. For example, a 1 hour Digital Video (DV format) requires about 25 GB (MPEG-2 reduces that to about 3 GB). It also ignificantly reduces the delay for the user agent for receiving relevant data.

Servers that support the temporal query format MUST implement a retrieval action of time-continuous resources with such query specifications by serving the requested resource from the temporal offset onwards. For many time-continuous resources - especially when in compressed format - this means that the server has to parse the structure of the resource and construct another valid resource from the original resource's header information and data frames. If a server cannot perform the fragment offset, it MUST return an error as otherwise the user agent cannot identify if the offset action was performed or not. It is expected that over time more servers and client applications understand and handle the temporal URI query parameters and thus enable direct networked access to content in time-continuous resources. Also Web proxies may begin to understand such temporal offsets and can exploit them for caching.

[TBD]

This specification does not create any new security issues beyond the ones already specified for URIs in RFC 2396.

draft-pfeiffer-temporal-fragments-01: Extension of the number of available SMPTE time-schemes. Many thanks to Bill Miller and Oliver Morgan of the SMPTE for their input on these changes. Deleted "start" and "now" as time specification values. Extension of the temporal fragment addressing to also address temporal intervals, not only time points. Added section that includes some key points of discussion where the existing URI standard contradicts the use of fragments for time-continuous data. draft-pfeiffer-temporal-fragments-02: Full compatibility with existing URI standard specification is achieved by using both, query and fragment specifications for addressing. Restriction of specification to http scheme (Web) only, as usage on other protocols (such as rtp/rtsp) still needs to be explored. All addressing is interval based because an offset really implies a view onto the document from that point onwards.

Harvard University

29 Oxford Street Cambridge MA 02138 US +1 617 495 3864 sob@harvard.edu

World Wide Web Consortium

MIT Laboratory for Computer Science 545 Technology Square Cambridge MA 02139 US +1 617 253 5702 +1 617 258 8682 timbl@w3.org

University of California, Irvine

Department of Information and Computer Science University of California, Irvine Irvine CA 92697-3425 US +1 949 824 7403 +1 949 824 1715 fielding@ics.uci.edu

Xerox PARC

3333 Coyote Hill Road Palo Alto CA 94304 US +1 650 812 4365 +1 650 812 4333 masinter@parc.xerox.com

Columbia University

Dept. of Computer Science 1214 Amsterdam Avenue New York NY 10027 US schulzrinne@cs.columbia.edu

Netscape Communications Corp.

501 E. Middlefield Road Mountain View CA 94043 US anup@netscape.com

RealNetworks

1111 Third Avenue Suite 2900 Seattle WA 98101 US robla@real.com

The Society of Motion Picture and Television Engineers

595 W. Hartsdale Ave. White Plains NY 10607 USA smpte@smpte.org

International Organization for Standardization

1 rue de Varembre Case Postale 56 Geneva 20 1211 CH central@iso.org

Commonwealth Scientific and Industrial Research Organisation

Locked Bag 17 North Ryde NSW 2113 Australia + 61 2 9325 3100 + 61 2 9325 3200 Silvia.Pfeiffer@csiro.au http://www.annodex.net

Commonwealth Scientific and Industrial Research Organisation

Locked Bag 17 North Ryde NSW 2113 Australia + 61 2 9325 3100 + 61 2 9325 3200 Conrad.Parker@csiro.au http://www.annodex.net

Commonwealth Scientific and Industrial Research Organisation

Locked Bag 17 North Ryde NSW 2113 Australia + 61 2 9325 3100 + 61 2 9325 3200 Andre.Pang@csiro.au http://www.annodex.net

Commonwealth Scientific and Industrial Research Organisation

Locked Bag 17 North Ryde NSW 2113 Australia + 61 2 9325 3100 + 61 2 9325 3200 Silvia.Pfeiffer@csiro.au http://www.annodex.net/

Commonwealth Scientific and Industrial Research Organisation

Locked Bag 17 North Ryde NSW 2113 Australia + 61 2 9325 3100 + 61 2 9325 3200 Conrad.Parker@csiro.au http://www.annodex.net/

Commonwealth Scientific and Industrial Research Organisation

Locked Bag 17 North Ryde NSW 2113 Australia + 61 2 9325 3100 + 61 2 9325 3200 Andre.Pang@csiro.au http://www.annodex.net

The authors greatly acknowledge the contributions of Andre Pang and Andrew Nesbit in developing this syntax. We also thank Bill Miller and Oliver Morgan of the SMPTE for their contributions and Dave Singer from Apple Computer Inc. for his. The many comments on this document from the URI discussion group at the W3C (uri@w3.org), especially the comments from Larry Masinter, Al Gilman and Martin Duerst and others have strongly shaped the current format. Many thanks also to the Audio/Video Transport working group at the IETF (avt@ietf.org), foremost to Magnus Westerlund, for picking up the discussion around how to use the scheme on RTP/RTSP. This is now not covered in this document, but should be explored in the future, possibly in a different document. Last but not least thanks go to the ISO/MPEG working group with whom harmonisation in addressing formats is still pursued - thanks to Ian Burnett, Myriam Amielh, Ernest Wan, and Gerrard Drury.