Content Production Technologies

Content Production Technologies Edited by Fumio Hasegawa Tohoku University of Art and Design, Japan Haruo Hiki Japan Int...

Author: Fumio Hasegawa | Harou Hiki

70 downloads 1801 Views 4MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!

Report copyright / DMCA form

DOWNLOAD PDF

Content Production Technologies Edited by Fumio Hasegawa Tohoku University of Art and Design, Japan Haruo Hiki Japan International Cooperation Agency, Uruguay

Content Production Technologies

Trademarks Microsoft, Windows, Windows Media, and Windows NT are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. EJB, Java, J2EE and JSP are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States and other countries.

Content Production Technologies Edited by Fumio Hasegawa Tohoku University of Art and Design, Japan Haruo Hiki Japan International Cooperation Agency, Uruguay

Copyright # 2004

John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex PO19 8SQ, England Telephone

(+44) 1243 779777

Email (for orders and customer service enquiries): [email protected] Visit our Home Page on www.wileyeurope.com or www.wiley.com All Rights Reserved. No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, exept under the terms of the Copyright, Designs and Patents Acts 1988 or under the terms of a licence issued by the Copyright Licensing Agency Ltd, 90 Tottenham Court Road, London W1T 4LP, UK, without the permission in writing of the Publisher. Requests to the Publisher should be addressed to the Permissions Department, John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex PO19 8SQ, England, or emailed to [email protected], or faxed to (þ44) 1243 770571. This publication is designed to provide accurate and authoritative information in regard to the subject matter covered. It is sold on the understanding that the Publisher is not engaged in rendering professional services. If professional advice or other expert assistance is required, the services of a component professional should be sought. Other Wiley Editorial Ofﬁces John Wiley & Sons Inc., 111 River Street, Hoboken, NJ 07030, USA Jossey-Bass, 989 Market Street, San Francisco, CA 94103-1741, USA Wiley-VCH Verlag GmbH, Boschstr. 12, D-69469 Weinheim, Germany John Wiley & Sons Australia Ltd, 33 Park Road, Milton, Queensland 4064, Australia John Wiley & Sons (Asia) Pte Ltd, 2 Clementi Loop #02-01, Jin Xing Distripark, Singapore 129809 John Wiley & Sons Canada Ltd, 22 Worcester Road, Etobicoke, Ontario, Canada M9W 1L1

British Library Cataloguing in Publication Data A catalogue record for this book is available from the British Library ISBN 0-470-86521-0 Typeset in 10/12 pt. Palatino by Thomson Press (India) Ltd., New Delhi, India Printed and bound in Great Britain by Antony Rowe Ltd., Chippenham, Wiltshire. This book is printed on acid-free paper responsibly manufactured from sustainable forestry in which at least two trees are planted for each one used for paper production.

Contents Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii About the Editors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

ix

About the Contributors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

xi

Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii Abbreviations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv

1

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1

I.1 I.2 I.3 I.4 I.5 I.6

. . . . . .

1 3 5 6 6 8

What is a Large-scale Archive?. . . . . . . . . . . . . . . . . . . . . . .

9

1.1 1.2 1.3 1.4 2

3

Shortage of Video Content . . . . . . . . . . Preservation . . . . . . . . . . . . . . . . . . . . . . Distribution . . . . . . . . . . . . . . . . . . . . . . Production . . . . . . . . . . . . . . . . . . . . . . . New Content Production Environment . New Content Distribution Environment .

. . . . . .

. . . . . .

. . . . . .

Need for an Archive. . . . . . . . . . . . . . . . . . . Example of a Preservation-oriented Archive. Example of New Type-oriented Archive . . . Future Video Archives . . . . . . . . . . . . . . . . .

. . . . . .

Archive-correlated Technology Standards . . . . . . . . . . . . . .

55

3.1 3.2 3.3 3.4

55 60 78 81

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . . . .

36 44 47

. . . .

. . . .

. . . . . .

2.1 Storage of Video Material . . . . . . . . . . . . . . . . . . . . . . . . 2.2 How to Search for Speciﬁc Material in an Archive . . . . . 2.3 Content Production Using Metadata. . . . . . . . . . . . . . . . . . . .

. . . .

. . . . . .

33

. . . .

. . . .

. . . . . .

Content Production from Digital Archives . . . . . . . . . . . . .

. . . .

. . . .

. . . . . .

12 15 20 31

. . . .

. . . .

. . . . . .

. . . .

Standards Related to Archives . . Standards Related to Metadata . . Standards Related to Production Reference . . . . . . . . . . . . . . . . . .

. . . .

. . . . . .

. . . .

. . . .

. . . .

. . . .

vi

CONTENTS

4

Experiment for Content Production with Content ID and MPEG-7 . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1 4.2 4.3 4.4 4.5

5

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. 84 . 88 . 107 . 115 . 122

Content Distribution Environment . . . . . . . . . . . Proposal for a New Content Production System. Study on New Content Distribution . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

129 133 143 163

Recap and the Future . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165 6.1 6.2 6.3 6.4

7

. . . . .

New Content Production and Distribution Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 5.1 5.2 5.3 5.4

6

Archive System . . . . . . . . . . . . . . . . . . . . Application of Watermarking Technology Veriﬁcation Tests . . . . . . . . . . . . . . . . . . . Retrieval Methods . . . . . . . . . . . . . . . . . . Editing Process . . . . . . . . . . . . . . . . . . . . .

83

Archives . . . . . . . . . . . . . . . . . . . . . Content Production from Archives . Experiments . . . . . . . . . . . . . . . . . . New Environment . . . . . . . . . . . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

166 169 173 177

Utilization and Systematization of Video Assets. . . . . . . . . 183 7.1 7.2 7.3 7.4

Building a Video Content Business. . . . . . . . . . . . . Local Industry Policy and Archives . . . . . . . . . . . . Digital Content Creation. . . . . . . . . . . . . . . . . . . . . Policy and Guidelines Based on Original Creation Activity in Yamagata . . . . . . . . . . . . . . . . . . . . . . . 7.5 Activity Guidelines of the YDCC . . . . . . . . . . . . . . 7.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . 183 . . . . 184 . . . . 185 . . . . 185 . . . . 186 . . . . 192

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193

Preface Recently, the main concern with technology development seems to be shifting away from network infrastructure provision to multimedia content distribution. This may well be caused by the popularity of the Internet and the diversity of networks, such as xDSL, and mobile phones. Broadband networks, like that at 100 Mbps, even reach homes, and the Internet is becoming more popular and a more effective communication tool. It is not just text data or still-image data that are distributed through the Internet, but audio data and movie data are as well. For broadband users, high-quality video can be served up without difﬁculty, and for narrowband users, such as users of mobile phones, shrunken movie pictures can be delivered. Clearly, distribution services depend on the network infrastructure. The mobile phone is entering the third generation of mobile communications, in which global services and broadband communication will be available. In these circumstances the demand for high-quality video is strongly growing as original sources for distribution at various levels of quality through various networks. However, the production of video content is not so easy because of the cost, the need for on-location ﬁlming, and so forth. If stored archive material could be used, then the situation would take a turn for the better: there is a vast quantity of video material and broadcast content in archives that could be shared among users. In such an environment, content production will become higher quality and less expensive. The Yamagata Video Archive Research Center (YRC) was established to investigate the possibility of content production using an archive by the Telecommunications Advancement Organization of Japan (TAO). At the YRC we developed several technologies that relate to preservation, distribution, and production as follows:

viii

PREFACE

preservation – a large-scale (petabyte order, denoting a factor of 1015) video archive for multi-purpose use; distribution – digital watermarking to embed metadata in video content; production – remote editing for facilitating the use of archive and editing facilities remotely. In addition to these fundamental developments we studied the following processes and made them practical: content management with Content ID proposed by the Content ID Forum (cIDf); retrieval and editing processes in Moving Pictures Expert Group (MPEG-7) format. Concurrently with the YRC project, the Yamagata Multimedia Development Promotion Association was established in 1999; it has supported the R&D of the YRC project by the acquisition of video materials. Through their efforts, the amount of video material exceeded the 2,000 mark. These videos not only include traditional and cultural heritage material but also local events. The purpose of the archive is content production. This book, Content Production Technologies, describes the project, existing archives, technology standards, experiments, and the new environment in seven chapters. They are mostly based on the results of our actual developmental and experimental work. I hope this book realistically portrays this work and is useful for the reader. Fumio Hasegawa

About the Editors Hasegawa, Fumio Tohoku University of Art and Design (email: [email protected]) Fumio received an MSc from the University of Electro-Communications and joined the Shimizu Corporation in 1974. In 1986 he received a DEng from the University of Tokyo and was a visiting researcher at MIT (Massachusetts Institute of Technology). In 1989 he became an assistant professor at the University of Tokyo. He was the Leader of the R&D project at the Yamagata Video Archive Research Center (YRC) of the Telecommunications Advancement Organization of Japan (TAO) from April 1999 to March 2003 and is now the senior director of the graduate school at the Tohoku University of Art and Design. Hiki, Haruo Japan International Cooperation Agency, Uruguay (email: [email protected]) Haruo graduated from the University of Electro-Communications and joined the Sony Corporation VTR Development Division in 1965. For 20 years he worked on many aspects of R&D for video camera, VTR and video-editing system in PAL, SECAM, and NTSC, which included R&D in Paris for 7 years. Then he worked in the personal computer ﬁeld for 5 years and in the digital interface standardization ﬁeld for a further 5 years. He was a senior researcher at YRC/TAO from April 1999 to March 2003, where he was in charge of network-based editing technology in the R&D project, ‘Fundamental Technology of Broadcasting Program Production with Large-scale Video Archives’. He is now a free lance journalist for archival production.

About the Contributors Ishibashi, Yoichi Yamagata Digital Content Center for Research & Promotion (email: [email protected]) Yoichi graduated from the University of Electro-Communications and joined Matsushita Electric Industrial Co. in 1971, where he worked on the development of professional videotape recording (VTR) and was in charge of archive system solution. In March 2001 he joined the Yamagata Video Archive Research Center (YRC) of the Telecommunications Advancement Organization of Japan (TAO) as Subleader. The R&D project at the YRC is called ‘Fundamental Technology of Broadcasting Program Production with Large-scale Video Archives’. He is the administrative director and the technical managing director at Yamagata Digital Content Center for Research & Promotion. Ito, Manabu Yamagata Digital Content Center for Research & Promotion (email: [email protected]) Manabu graduated from the Chiba Institute of Technology with a major in Electric Engineering and joined Yamagata National Electric in 1994, where he was engaged in the design and distribution of audiovisual (AV) systems. He joined YRC/TAO as a researcher in 1999 and is a senior researcher at Yamagata Digital Content Center for Research & Promotion. Kogure, Takuyo Matsushita Electric Industrial Co. (email: [email protected]) Takuyo received a BSc from Shizuoka State University, Japan and joined Matsushita Electric Industrial Co. in 1962. He moved to the

xii

ABOUT THE CONTRIBUTORS

Acoustic Research Laboratory of this company in 1969. Since then he has been engaged in the R&D of digital audio and video technologies. He established a multimedia research laboratory – the AV/Information Research Center – when he was with Asia Matsushita Electric and became the director of the Center between 1990 and 1995. He was a researcher at YRC/TAO from June 2001 to March 2003. Tanno, Yoshikazu Yamagata Digital Content Center for Research & Promotion (email: [email protected]) Yoshikazu graduated from Sagae Technical High School and joined the Toshiba Corporation in 1983. After he majored in Electric Engineering at Toshiba Gakuen, where he ran the telecommunications house, he joined Yamagata National Electric and worked on the network design and management of cable television. He was a researcher at YRC/TAO from April 1999 to March 2003 and is now a graduate student at the Graduate School of Engineering Science of Osaka University.

Acknowledgements I would like to express our sincere thanks to the Telecommunications Advancement Organization of Japan (TAO) on behalf of the R&D project team. TAO established the Yamagata Video Archive Research Center (YRC) in 1999 for the purpose of developing an archive-based content production system. For 4 years we studied various technologies with that view in mind. In particular, through using both MPEG-7 and Content ID we ended up with some dramatic changes. A global unique identiﬁer like Content ID facilitates the management of content in an integrated fashion and the distribution of content globally. And the MPEG-7 description may bring about the worldwide use of archives because of its easy retrieval. As a result, we have successfully developed the preserving technology, visual retrieval methods, the watermarking technology, remote-editing methods, content distribution system, and so forth. Almost all the results are described in this book. I also wish to express our gratitude to the Yamagata Multimedia Development Promotion Association (YMDPA), which was established concurrently with YRC to support YRC project. YMDPA collected traditional and historical heritage material in Yamagata and acquired video material related to local events. This video material was of great value to our research and development. Haruo Hiki

Abbreviations AAF ADSL ANTS API ARF ARIB ASF AU AV AVD AVDC

Advanced Authority Format Asymmetric Digital Subscriber Line Active Node Transfer System application programming interface Art Research Foundation Association of Radio Industries and Businesses Advanced Streaming Format access unit audio-visual active video delivery AVD controller

BG BiM BML B2B B-VOP

binary group binary format of MPEG-7 Broadcast Markup Language business to business Bidirectional Video Object Plane

CBR CD CG CIC CICA CID CIDCMN cIDf CIF CM CNRI CRC

constant bit rate compact disc computer graphics Content ID Center Center for Industrial Creativity Assistance Cotent ID Content ID Center mgmt no. Content ID Forum Common Interchange Format commercial Corporation for National Research Incentives Cyclic Redundancy Check

xvi

ABBREVIATIONS

D DB DBMS DCAj DCD DCMS DCT DDL D-RAM DS DTPP DV DVCPRO DVD

descriptor database database mgmt system Digital Content Association of Japan distributed content descriptor Department for Culture, Media and Sport (UK) discrete cosine transfer description deﬁnition language Dynamic Random Memory description scheme Desk-Top Program Production Digital Video Professional Digital Video format by Panasonic digital video disc

EBU ECC EDL EJB E-SMEF

European Broadcasters Union Error Correcting Code edit decision list Enterprise Java Bean Extended SMEF

FC FFT FIAT FU FUU

Fibre Channel Fourier Function Transfer see IFTA fragment update fragment update unit

GbE GOP GUI GXF

Gigabit Ethernet group of pictures Graphical User Interface General Exchange Format

HDD HD SDI HDTV HF

hard disk drive high-deﬁnition SDI high-deﬁnition TV high frequency

IC IEC IFTA IP IPR ISAN

Integrated Circuit International Electrotechnical Commission Intl Federation of Television Archives Internet Protocol intellectual property rights International Standard Audiovisual Number

ABBREVIATIONS

ISO ITU

Intl Organization for Standardization Intl Telecommunication Union

JDAA JDBC J2EE JGN JSP

Japan Digital Archives Association Java Data Base Connectivity Java 2 Enterprise Edition Japan Gigabit Network Java Server Page

KLV

Key-Length-Value

LF LSB LSI LTC

low-frequency Least Signiﬁcant Bit large-scale integrated circuit linear time code

MD MDS ME MIL MIME MIT MJD MO MPEG MSB MXF

Mini Disc multimedia description scheme metal-evaporated music insertion list Multipurpose Internet Mail Extension Massachusetts Institute of Technology Modiﬁed Julian Date magneto-optic Moving Picture Experts Group Most Signiﬁcant Bit Material Exchange Format

NHK NTSC

Japan Broadcasting Corporation National Television Standard Committee

ODBC OS

Open Database Connectivity operating system

PAL P/FTA P/FTP PHS P-MC P/META

Phase Alternate Line Project Future Television Archives Project Future Television Production personal handyphone system Production Mgmt Committee Project Metadata Exchange Standards

RA RDBMS RTP

registration authority Relational Database Management System Real Time Protocol

xvii

xviii

ABBREVIATIONS

SCSI SDI SDTI SDTV SECAM SMEF SMPTE SNR SQL S-VHS

Small Computer Systems Interface serial digital interface Serial Data Transport Interface Standard Deﬁnition TV Sequential Couleur avec Memoire (French) Standard Media Exchange Framework Society of Motion Picture and Television Engineers signal-to-noise ratio Structured Query Language super video home system

TAO TC TeM TIFF TIL TS TUAD

Telecommunications Advancement Organization of Japan Time Code textual format of MPEG-7 tagged image ﬁle format telop insertion list Transport Stream Tohoku University of Art and Design

UMID URI

unique material identiﬁer Uniform Resource Identiﬁer

V-ISAN VITC VLAN VoD VOP VTR

version-ISAN vertical interval time code virtual local area network Video on Demand Video Object Plane videotape recorder

5W1H WMA

What, Where, Which, Who, Why and How Windows Media Audio a.wma ﬁle

xDSL XML

ADSL, HDSL, VDSL etc. in total eXtensible Markup Language

YDCC YMDPA

Yamagata Digital Content Center Yamagata Multimedia Development Promotion Association Yamagata Video Archive Res. Center

YRC

Introduction Fumio Hasegawa

I.1

SHORTAGE OF VIDEO CONTENT

The digitization of broadcast has become key to the construction of a new framework to supply digital video content. Digital technologies affect the efﬁcient use of frequency bands by keeping the picture quality high and by the provision of new services. The efﬁcient use of frequency bands has brought about more channels both from terrestrial broadcasts and from satellite broadcasts. TV viewers like to have more channels so that they can have greater choice. However, broadcasters have the task of securing enough content for broadcasting. Consequently, the demand for video content exceeds the supply. To have a readily available, large supply of video content, you may well have to consider not only an increase in the number of staff and camera units or strengthening of editing facilities but also a change in the content production method itself: namely, you will need not only to increase your ﬁlming or editing ability but also to make good use of stored materials from archives. Of course, the archive should be well prepared in advance. The archive has thus far been discussed from the viewpoint of preservation, but it will be more beneﬁcial for broadcasters to look at the archive from a content production viewpoint. Because of this, the book has been arranged so that the new content production system is based on utilization of an archive. Therefore, the supply of digital

Content Production Technologies F. Hasegawa and H. Hiki # 2004 John Wiley & Sons, Ltd ISBN: 0-470-86521-0 (HB)

2

INTRODUCTION

video content must be worked on from three viewpoints: preservation, distribution, and production. As far as these three viewpoints are concerned, R&D is now complete. In 1999 the Telecommunications Advancement Organization of Japan (TAO), under the auspices of the Ministry of Posts and Telecommunications of Japan, set up the Yamagata Video Archive Research Center (YRC) to carry out an R&D project into the production technology required to broadcast programs using a large-scale archive. YRC has carried out this task for 4 years. This book describes a largescale archive, content production from such a digital archive, related standardization, experiments for content production using Content ID and Moving Pictures Expert Group (MPEG-7) at the YRC, and the new content production and distribution environment. The R&D facilities were set up in two locations. One is at the Center for Industrial Creativity Assistance in Yamagata (CICA) and is shown in Figure I.1. The other is at Tohoku University of Art and Design (TUAD) and is shown in Figure I.2. Between these two institutions optical ﬁbers (10 km long) were put in place as communication lines.

Figure I.1

Center for Industrial Creativity Assistance in Yamagata.

I.2 PRESERVATION

3

Figure I.2 Tohoku University of Art and Design.

I.2

PRESERVATION

As far as preservation is concerned, the most important consideration is how much video material and how many ﬁnished programs can be stored in an archive without quality degradation. A vast stock will call for efﬁcient retrieval; so, ease of retrieval is important as well. This storage function is conventionally provided by archive systems, which can be described as data storage systems using a digital tape drive, since the recording efﬁciency of tape media is higher than that of other media: optical disk, hard disk, or semiconductor memory. Currently, some broadcasting stations have more than 100,000 videotape recorder (VTR) tapes in storage. The purpose of the study to bring about an archive was to realize such capacity. And generally they preserve as the tapes are. Almost all these tapes were recorded in analog and so run the risk of degradation on stored content. However, the risk of degradation can be removed by digitization. For the most important or the most precious videos, digitization will be effected by copying from analog recording to digital recording. In an effort to ﬁnd solutions to this, we constructed the archive system shown in Figure I.3.

4

INTRODUCTION

Figure I.3 Archive system constructed at the YRC.

This system saved ﬂoor space by moving vast numbers of digital videos into a large-scale archive that is fully operated by an automatic tape-feed robot. There is an easy, visual retrieval method that can ﬁnd the target video quickly and accurately. I.2.1

STORAGE SYSTEM

What is an efﬁcient storage structure for a large-scale archive system? The answer lies in a combination of different types of servers, such as low-retrieval speed and large capacity, medium-retrieval speed and medium capacity, and high-retrieval speed and small capacity. As shown in Figure I.3, our study of the archive system revealed that: low-retrieval speed and large capacity was best served by a tape server with an automatic tape-feed robot; medium-retrieval speed and medium capacity by a DVD (digital video disk) server for content distribution in MPEG-2 or MPEG-4 format; and high-retrieval speed and small capacity by a hard disk drive (HDD) server that could be quickly accessed for frequently accessed content. These servers were deployed in such a way as to make the most of their attributes in this system. As a result, we constructed an archive system that could store digital video data with a run-time as large as 15,000 hours on 7,200 tapes. The

I.3

DISTRIBUTION

5

original simulation of the tape-feed robot showed the archive system would be able to accommodate up to 100,000 tapes. I.2.2

RETRIEVAL

A conventional video database system commonly uses text data – such as title, keyword, etc. – for accessing content. However, this retrieval method requires additional work to check whether a retrieved video is in fact the correct one. So, something like a browsing function was required. For the browsing purpose we developed the following methods: automatic detection of a sequence at the cutting points of an original video, automatic extraction of a frame break point, like scene changes, and generation of a digested video sequence for browsing. In order to enhance the browsing efﬁciency each image of video was shrunk and re-synthesized into multiple images on a display (16 or 64 images per screen). I.2.3

PREVIEW

The preview function is one of the editing processes: materials in the archive can be accessed easily by the user with an appropriate preview. MPEG-4 format meets the movie preview requirement. Still images and scene explanations will be low resolution and easily accessible through the Internet – MPEG-7 meets the still-image requirement.

I.3

DISTRIBUTION

Distribution not only means distribution of ﬁnished programs on media such as CD or DVD but also the distribution of materials for content production. To promote content (including material) distribution, we believe it important that the intellectual property rights (IPR) of the copyright holder be protected ﬁrst and then the attribute information can be appropriately provided to the user. The former is required for a content creator to supply the content with peace of mind, and the latter is needed for a content user to recognize the condition of distribution and to make good use of it. We believe that Content ID, which is advocated by the Content ID Forum (cIDf) in Japan, will meet these needs. There are three levels in content management: unique code, which acts as an access key to all attributes of content; DCD (distributed content descriptor), which

6

INTRODUCTION

gives a minimum description of the content, and all the attributes of content. The unique code of Content ID can be optionally embedded in content itself as a watermark. We adopted the digital watermark technology to embed the unique code in video itself. As for content distribution, the broadcasting of programs is the most frequent method of distribution. Video materials will be distributed in future when the new content production environment is accomplished. The other method of distribution will be still pictures thanks to the application software’s ability to expand images. One example of this distribution is an electronic museum that consists of still pictures, based on HDTV (high-deﬁnition TV), stored in the archive. I.4

PRODUCTION

Many people are working together to produce new content. From planning of content to a ﬁnished program, most of the process can be networked so everything runs smoothly. Recently, content management for audio-visual data has been urgently requested by broadcasters. In particular, it is crucial for content production operators to use a network-based content management system. Thus we constructed a new framework that can have an operator edit using all manner of video materials: stored materials, newly shot materials, and CG (computer graphics). If an archive has any video material that is usable for new content, then the material will be transferred and used in an editing studio through the network. Network-based content production requires the following four elements: a global unique identiﬁer that speciﬁes the material, such as UMID (unique material identiﬁer), Content ID, etc.; an original time code that allows editing to take place anywhere and sends the EDL (edit decision list) to any editing studio; a preview in a highly compressed video format like MPEG-4 or scene explanation in the description standard like MPEG-7; IPR information – to secure originality. I.5

NEW CONTENT PRODUCTION ENVIRONMENT

As the demand for content grows, it will not be so easy to solve the shortage of video content. However, the new content production

7

I.5 NEW CONTENT PRODUCTION ENVIRONMENT

environment is expected to improve this situation by being based on digital technologies, network technologies, and international standards. In Chapter 3 some of the standards will be brieﬂy introduced: the necessary standards for use of stored materials in an archive. The more materials are stored in an archive the more easily content will be produced through the network. We studied content production using a large-scale archive from three aspects: preservation, distribution, and production. Doing many experiments and several veriﬁcation tests, we realized that the new content production environment will be similar to our experimental system at the YRC. Figure I.4 shows the new content production system from ﬁlming through to the editing studio.

Archive Orig. TC

Editing studio Local server

Panasonic

MPEG-7 descript.

IPR Orig. TC

EDL

IPR Content ID

Web server

EDL Remote terminal

Remote terminal

Internet Figure I.4

Symbolic diagram of the new content production system.

After ﬁlming, a recorded tape is played back together with an original time code. Played back data are stored in an archive while the IPR and other information are stored in an IPR database along with Content ID. The necessary data for retrieval and editing are described in MPEG-7 and stored in a Web server. Through the Internet, a user can access this Web server and retrieve materials in the archive and MPEG-7 instances of the desired materials are downloaded to a remote terminal. The user can edit those materials in the terminal and send an EDL to an editing studio. According to the EDL, the material for remote editing is transferred from the archive to the editing studio and

8

INTRODUCTION

stored in a local server. In the editing studio, automatic editing is executed by the EDL. Several remote terminals can co-exist through the Internet, and international users can work together with the same MPEG-7 instances that contain the necessary data for multi-language editing: as the proverb says, two heads are better than one.

I.6

NEW CONTENT DISTRIBUTION ENVIRONMENT

Data broadcasting or Internet broadcasting provides interactive communication and metadata distribution along with video content. This brought about the concept of an interactive commercial (CM): that is, viewers see only what they want to. The metadata of object advertisements are described in MPEG-7 format. It is expected that metadata written in MPEG-7 can be utilized effectively in other systems. Although object advertisement distribution is described in Section 5.3.1, we will brieﬂy summarize it: distributors have video content and object data. The video content is a conventionally broadcast program while object data have still pictures and metadata corresponding to the video content. These data are to be displayed on a digital TV. Another content distribution environment is active video delivery (AVD) using the active network concept. The AVD controller is capable of dynamically changing the delivery method for some speciﬁc video segments by injecting program codes into related active routers along the transmission paths. After receiving the injected program codes, the active routers judge the optimal delivery method for speciﬁc video segments by analyzing the payloads of the packets received from the video server. If only one user requests videos, then the system sends a unicast of all the segments of those chosen videos. However, if more than one request for the same videos occur, then the system sends a multicast-delivery of segments of those videos according to different users’ arrival times. In order to achieve dynamic multicast/unicast delivery, we implemented a speciﬁc protocol set that is used between the user and the video server, and between the AVD and the active routers. The details are described in Section 5.3.2. Another example is an electronic museum: a website that promotes sound distribution of content. The website utilizes content derived from archive materials with metadata that comply with Content ID. In order to meet the needs of digital material creators, the website adopts a structural description (XML, or eXtensible Markup Language) of attribute information related to content distribution. The details are described in Section 5.3.3.

1 What is a Large-scale Archive? Yoichi Ishibashi

An archive is both an old and a new word. The new word’s meaning refers to utilization of an archive. Archiving will provide a large amount of video content ready to view or to help produce other content. A library is similar to an archive, but is generally used to store goods as they are. Two different types of archives are introduced here. One is the preservation-oriented archive and the other is the new type-oriented archive.

Archives have become widely available in the age of the broadband network. The reason is because attractive content is in short supply along with rapidly growing numbers of information channels, all wanting digitized audio-visual (AV) data. The importance of an archive will be described here. An archive generally means a place where a large number of records about ofﬁcial documents are stored: that is, a place where historically valuable materials are in storage and can be referenced. The importance of a video archive had long been discussed in Europe, and the International Federation of Television Archives (FIAT, or IFTA) (http://www.fiatifta.org/aboutfiat/ whatis/) was established in 1977, principally by the BBC (UK), INA (France), and RAI (Italy). The objectives are as follows: to provide a forum for exchange of knowledge and experience between its members;

Content Production Technologies F. Hasegawa and H. Hiki # 2004 John Wiley & Sons, Ltd ISBN: 0-470-86521-0 (HB)

10

WHAT IS A LARGE-SCALE ARCHIVE?

to promote the study of any topic relevant to the development and use of AV archives; to establish international standards on key issues regarding all aspects of AV media archive management; to promote professional standards and best practices in this ﬁeld; to support the development of archives and to foster regional and inter-regional cooperation; to heighten government, broadcasters’ and users’ awareness of the responsibilities and constraints related to the preservation and use of AV media archives – their being an essential source of contemporary culture and history. The major members have a large number of video records: 1.05 million reels in the BBC (UK), 1.32 million reels in INA (France), 1 million reels in RAI (Italy), 1.5 million reels in NBC (USA), and 1.73 million reels in the NHK (Japan) (http:/www./fiatifta.org/projects/information/ fiat/digital_survey/index.html#Archive). The European Broadcasters Union (EBU) hosted the symposium about archives in which archive technologies were actively discussed. Furthermore, it promoted several projects: P/FTA (Future Television Archives), P/META (Metadata Exchange Standards), and P/FTP (Future Television Production). The work of P/FTA was organized in seven work packages: legacy television archives; near, mid and long-term migration scenarios; status and development of archival media; a roadmap to AV archival; interfaces, data formats, and protocols for archives; concepts and implementations of archives; content management. As far as the introduction of P/META is concerned, seminal work is now being carried out by SMPTE on deﬁning the Dynamic Metadata Dictionary, UMIDs (unique material identiﬁers), mapping of metadata into transports, and preparation of operational guidelines and engineering recommendations. Complementary work is proposed for the EBU to consider the adoption of a common exchange framework and

WHAT IS A LARGE-SCALE ARCHIVE?

11

format between members (and internationally) which builds on SMPTE outputs and the additional insights provided by the BBC’s Standard Media Exchange Framework. As far as the introduction of P/FTP is concerned, there is agreement among experts that the integration of new elements like compression, metadata, networking, ﬁle transfer, hierarchical storage, and content management into television operations is pivotal to meeting the challenges that broadcasters will be facing in the future. New system concepts are increasingly software-controlled and should be highly modular in order to support interoperability between systems and components. In Japan the Japan Digital Archives Association (JDAA), Art Research Foundation (ARF), and Digital Content Association of Japan (DCAj) were established. They are promoting the digital archive plan that advocates tangible and intangible cultural properties should be preserved in digital format and that such information could be dispatched through the network. According to this plan, museums should get involved in setting up archives, as should local governments. The main objectives are to support the production, distribution, and utilization of digital content including video, picture, audio, text, numerical value, and their metadata. The JDAA (http://www.jdaa.gr.jp/) develops and promotes operations based on the digital archive plan rationally and efﬁciently in cooperation with various organizations. The ARF (http://www.geiken.or.jp/) has the following remit: to investigate art-related items and exhibit them; to collect research documents about art and publish them; to hold induction courses and lecture classes. The DCAj (http://www.dcaj.org/) is involved in planning and investigation: to test and show the commercial value of content; legal environment on digital content; encouragement of future ventures; publication of the white paper on digital content. It also promotes talented individuals: training for producers and directors;

12

WHAT IS A LARGE-SCALE ARCHIVE?

a grand prix for digital content; periodical seminars. ‘Library’ is a similar word to ‘archive’. The Japan Broadcasting Corporation (NHK) deﬁnes ‘archive’ as the equipment or the operation for collection and preservation of video content that is used and re-used by the NHK itself, and ‘library’ is deﬁned as the equipment or the operation for storage of video content that is available to third parties. Both words are so common to broadcast-concerned people that the extent of their importance is not served well. Now in the digital multimedia age, the asset value of an archive is being reconsidered. This chapter describes the need for an archive in which a large amount of video content can be stored and re-used and introduces some typical examples in Japan.

1.1

NEED FOR AN ARCHIVE

The Yamagata Video Archive Research Center (YRC) was established in the northern prefecture, Yamagata, of Japan in 1999. The YRC had commenced the research and development as a part of the Telecommunications Advancement Organization under the auspices of the Ministry of Public Management, Home Affairs, Posts and Telecommunications of Japan. It is now under the management of the Yamagata Digital Content Center for Research & Promotion (YDCC). R&D at the YRC had the objective of making archive-related technologies practicable and of proposing the standardization of operating models, which were extended to personal use. The technologies include preservation, distribution, and production through the network for the beneﬁt of broadcasters and content producers. As for preservation, a large-scale archive technology was called for. The technology permitted high-speed retrieval and acquisition of desired content from a large amount of stored content, editing capability, and re-usability of acquired content. This technology has been put to practical use in electronic museums and as an electronic learning system. As the amount of stored content grows, content management becomes more complex and massive. As a result, a new archive infrastructure is required to bring about automation and a reasonable storage space. Automated video archives in practical use nowadays hold several thousands to ten of thousands of reels in videotape format, except in some particular cases when other formats are used. According to FIAT’s data, mentioned before, major broadcasting

1.1 NEED FOR AN ARCHIVE

13

stations hold more than 1 million reels, which are actually stored on shelves manually. To improve this system – say, by introducing partial automation – the balance between capacity of stored content and distribution capability should be taken into consideration. System optimization without limitation of the number of tapes is needed. The YRC developed the basic unit of a large-scale archive system (as shown in Figure 1.1) that contains 7,200 tapes for digital video including high-deﬁnition TV (HDTV).

Figure 1.1

Basic unit of a large-scale archive system.

All tapes are automatically handled in the system. Apart from the tape server, a hard disk drive (HDD) server and a digital video disk (DVD) server were systematically linked with the network. They purposely stored the data for the objectives and were examined in optimization of hybrid structure and its effectiveness by retrieval check. From the stored content the access data of an automatic tapefeed robot were taken, and it was then veriﬁed by simulation tests that this system was reasonable at the 100,000 tape level. An example of the simulation tests is shown in Figure 1.2. Retrieval for this system seems to be limited to 200,000–300,000 tapes for costefﬁciency. In addition, the simulation test included coping with a system of more than 1 million tapes. Analysis of the balance between storage capacity and distribution ability showed that the best solution could be achieved by a combination of an operative robot system and electrically powered moving shelves. It is optimum for a large-scale archive to consist of electrically powered moving shelves and basic units. Figure 1.3 shows the simulation test for 2.6 million tapes with electrically powered moving shelves.

14

WHAT IS A LARGE-SCALE ARCHIVE?

Figure 1.2 Example of the simulation tests.

Figure 1.3 shelves.

Simulation for 2.6 million tapes with electrically powered moving

Although these simulation tests were based on tapes as storage media, other media such as hard disks or semiconductor memories could be considered for real time distribution through the network. An efﬁcient system for media combination would be as follows: storage for original content – tape or DVD; storage for reference (compressed data) – HDD or semiconductor memory; storage for distribution – DVD or HDD. By using such a basic unit, the YRC promotes the reuse of stored content, the amount of which reached 3,000 titles during the 4 years of work that collected and ﬁlmed regional culture, folk entertainment, handicraft skills, sightseeing, events and so forth in order to record the history and the culture of Yamagata. This collection of present and past videos included: 1,000 titles for edited content; 2,000 titles for materials.

1.2 EXAMPLE OF A PRESERVATION-ORIENTED ARCHIVE

15

This can be further broke down into classiﬁcation by category: events – 500 titles; social and current topics – 50 titles; nature, wildlife, and environment – 900 titles; culture and heritage – 600 titles; literature – 450 titles; others – 500 titles In this manner, archive infrastructures will increase, and all digital data will be stored in the archive. Large-scale archives will be widely installed not only at broadcasting stations but also at educational institutions, art-related facilities, and local governments. As far as ordinary users are concerned, the archive system that they use to retrieve the desired content without consciousness of the access destination is preferable. In future, various scales of archive will exist and interconnect with each other effortlessly. For this purpose a total management technology that can implement the uniﬁed management of distributed archives is required. A retrieval technology that is able to ﬁnd the desired content in an archive and a management technology for protection of IPR are needed as well.

1.2

EXAMPLE OF A PRESERVATION-ORIENTED ARCHIVE

In this section an example of a preservation-oriented archive is introduced: the ‘Broadcast Library’ of the Broadcast Programming Center of Japan located in Yokohama. The Broadcast Programming Center of Japan carries out two major tasks: to provide high-quality television programs about education or culture to commercial broadcasting companies at low prices; to collect, store, and show television programs to the public – subject to the Broadcast Law. Article 53 of the Broadcast Law states, ‘The Minister of Posts and Telecommunications may appoint, upon application, a juridical person established with the objectives for promoting a sound development of broadcast in accordance with the stipulations of Article 34 of the Civil Code and deemed capable to conduct appropriately and correctly the

16

WHAT IS A LARGE-SCALE ARCHIVE?

business activities as provided for in the following article as the Broadcast Program Center (hereinafter referred to as the ‘Center’) not more than two.’ Clearly, broadcast programs are national cultural assets in which lives, customs, and culture are recorded, and as such they are methodically stored and made available to the public in the Broadcast Library. Figure 1.4 shows the Broadcast Programming Center of Japan.

Figure 1.4

1.2.1

The Broadcast Programming Center of Japan.

ESTABLISHED AS A JOINT PROJECT BY BROADCASTERS

The Broadcast Programming Center of Japan was established as a nonproﬁt organization by the NHK and all private broadcasters in Japan in March 1968. The Broadcast Library opened in October 2000 as a genuine library and one based on digital technology. 1.2.2

OUTLINE OF BROADCAST LIBRARY FACILITIES

The Broadcast Library covers about 3,000 m2 and comprises a user registration/content retrieval corner, reception, TV program viewing

1.2 EXAMPLE OF A PRESERVATION-ORIENTED ARCHIVE

17

corner, TV short program/commercial (CM) retrieval/viewing corner, and radio program-listening booths. The distribution equipment, database, authoring system, system monitoring facility, and networkrelated equipment can be found in the backyard. Figure 1.5 shows the reception area of the Broadcast Library.

Figure 1.5 Reception desk of the Broadcast Library.

1.2.3

AUDIO-VISUAL SYSTEM

TV Program Distribution Equipment This consists of a DVD cart and a video server with an HDD. The external appearance of the DVD cart is shown in Figure 1.6. The medium is DVD-RAM with 4.7 GB per side. The total number of media is 5,950 discs. A total of 79 DVD-RAM drives are connected with Small Computer Systems Interface (SCSI). The external appearance of the HDD server is shown in Figure 1.7. The HDD server plays the role of a cache to which popular programs that are frequently requested are copied from the DVD cart and then distributed from the HDD server. Nondigitized programs are distributed as video signals from super video home system

18

WHAT IS A LARGE-SCALE ARCHIVE?

Figure 1.6

Figure 1.7

External appearance of the DVD cart.

External appearance of the HDD server.

1.2 EXAMPLE OF A PRESERVATION-ORIENTED ARCHIVE

19

videotape recorders (S-VHS VTRs). Radio programs are distributed as analog signals from an MD drive. Viewer Equipment There are TV viewer booths and radio-listening booths. The TV viewer booths are shown in Figure 1.8. There are 30 booths for one person, 20 booths for two persons, 10 booths for three persons: a total of 60 booths. Apart from these booths there are 3 booths for researchers and 5 booths on remote locations.

Figure 1.8

TV viewer booths.

Authoring System TV programs that have been broadcast are supplied as tape, such as 1 inch, Digital Betacam, Betacam, etc. Before entering the authoring process, the CM is cut away and both the black signal and the television opaque projector, or telop, are inserted into the programs. Authored TV programs are recorded with Moving Pictures Expert Group (MPEG-2) TS at a constant bit rate (CBR) of 6 Mbps onto DVDRAM (4.7 GB). As one medium contains over 90 minutes, one program

20

WHAT IS A LARGE-SCALE ARCHIVE?

can in principle be recorded on one DVD-RAM. Radio programs are supplied as DAT and recorded on MD as one program in one MD. Information System The core of the information system is the program information database, which communicates with retrieval terminals, reception terminals, and work terminals. In order to input data to the program information database, several steps are needed: programs are selected according to broadcast program collection criteria; to solicit the broadcaster for providing the program to be stored; to deal with the rights; to sort out program-related information provided by the broadcaster; to input the metadata from a work terminal. Ten retrieval terminals are installed for user registration and program retrieval. They are equipped with a touch panel for easy operation. Retrieval can be executed according to program genre, broadcaster, title, producer, performer, and so on. The reception terminal has the following functions: retrieval, booth monitoring, user conﬁrmation, and re-allocation of booths. The work terminals have a number of functions: database input, program registration after authoring, and extraction of still images from programs. Network System All the facilities described above are networked at 100 Mbps using layer 3 switching hubs. The system is linked to Osaka remote terminals through the gigabit network. Movie data are transferred to these terminals at 6 Mbps/channel. There are ﬁve channels of movie data, still images, metadata, and control data that are transferred at 40 Mbps on the asynchronous transfer mode (ATM) network. 1.3

EXAMPLE OF NEW TYPE-ORIENTED ARCHIVE

The NHK has broadcast programs that have recorded scenes of every age and regional culture since the beginning of TV broadcasting in Japan in 1953. On its 50th anniversary the NHK set up the NHK

1.3 EXAMPLE OF NEW TYPE-ORIENTED ARCHIVE

Figure 1.9

21

The NHK Archives Building.

Archives Building to manage the audio and video software that the NHK has broadcast. Figure 1.9 shows the NHK Archives Building. Video content stored in the archives totals around 590,000 reels of programs and news: a veritable ‘time capsule of video’. 1.3.1

ROLES OF NHK ARCHIVES

NHK Archives will play two major roles as follows: to preserve past video content as part of national common cultural heritage and to hand it down to posterity via re-broadcasting and special programs; to give something back to society by making the assets that the NHK possesses available to the people of Japan. 1.3.2

OUTLINE OF NHK ARCHIVES FACILITIES

In the building there are storehouses, a database center, a system management room, broadcast transfer room, an ofﬁce, and an open

22

WHAT IS A LARGE-SCALE ARCHIVE?

program library. These facilities are installed on eight ﬂoors, the area of which is about 11,000 m2. To accomplish the roles mentioned above, there are functions and systems as follows: a general database that manages a massive amount of video content; a video transfer function by which video can be efﬁciently transferred from Kawaguchi (NHK Archives) to Shibuya, Tokyo (the NHK broadcast center); a video preservation function where broadcast original content can be stored over a long period of time; an open library so that users can freely view programs. 1.3.3

SCALE AND CONCEPT OF THE NHK ARCHIVES

The conceptual structure of the NHK Archives is as follows (see Figure 1.10):

Retrieval / Order / Rental

Beatus

General database Rec. for Reference Archive for test view server MPEG-4 VTR

Input

TOPICS News rights Raw tape mng.

NHK Intranet

Preservation function Archive

Archive LAN

Op. mng. server

Distribution function Rec. control Rec. VTR cart

Playback control Play VTR cart

Input server

Shared disk Retrieval server

Preservation function Key and local station Retrieval / Order / Input

Open library

Archive

Rental

Viewers

VOD Server

Figure 1.10 Conceptual structure of NHK Archives. Reproduced from A. Mori et al., ‘Facilities Outlines of NHK Archives’, Housou Gijutsu, February 2003 by permission of Kenrokukan Publishing Corporation.

1.3 EXAMPLE OF NEW TYPE-ORIENTED ARCHIVE

23

Preservation – 1.3 million reels with a life expectancy of 17 years. Stored content is currently 590,000 reels and will increase by 40,000 reels per year. storage medium – tape. The reason is that tapes are currently used in the broadcasting station and that tape cost is very low. Users are not supplied with an original tape but a copy in order to safeguard original tapes. Broadcast database – website-like retrieval. This provides a structure table with thumbnail pictures, information about IPR, and reference movies in MPEG-4 format. Transfer of ordered content – encoded video data through an exclusive Internet Protocol (IP) network. Design concept – latest technologies, high retrieval efﬁciency, copy lending, and reduction of network cost. 1.3.4

GENERAL DATABASE

The general database is the core system of NHK Archives. It consists of a database (DB) server, a Web server, and a retrieval application server. The DB server based on Relational Database Management System (RDBMS) is operable by the Structured Query Language (SQL) through Open Database Connectivity (ODBC) and Java DataBase Connector (JDBC) drivers. The DB server stores transaction data, the user’s authority data, media storage shelf data, and so forth. The Web server receives the user’s retrieval and order and sends transaction to the latter part of servers if necessary. The retrieval application server is implemented by the Java 2 Enterprise Edition (J2EE) platform, by means of servlets, Java Server Pages (JSPs), or Enterprise Java Beans (EJBs). The general DB also covers: Data input unit – per program or per news item. Programs are automatically stored in the archives and their metadata are input manually. Metadata are stored on a shared disk where they are available for retrieval. Reference movie encoding – MPEG-4 simple proﬁle at 500 kbps. B-VOP is available. Figure 1.11 shows the conceptual data ﬂow of reference movie encoding. Retrieval and ordering – certiﬁed user only. Retrieval is narrowed down by keywords, and then, by clicking the result table, the details

24

WHAT IS A LARGE-SCALE ARCHIVE?

On-air sources (clean picture) MPEG-2 …..

MPEG-2 Transcoding server

ftp

(MPEG-2 MPEG-4)

Registration instruction Material/ Content tapes D3 VTR

File sharing

Video / Audio

ftp

Cache disk for reference movie registration

TC RS422 Video input terminal

• MPEG-4 Encoding • MPEG-4 Registration • Extraction of still-image

Figure 1.11 Conceptual data ﬂow of reference movie encoding. Reproduced from A. Mori et al., ‘Facilities Outlines of NHK Archives’, Housou Gijutsu, February 2003 by permission of Kenrokukan Publishing Corporation.

Figure 1.12 Visual display of search results and reference movie.

25

1.3 EXAMPLE OF NEW TYPE-ORIENTED ARCHIVE

Cache disk for reference FC

FC

Reference server (active)

GbE GbE

Reg. server (active)

FC

GbE Reference server (Standby) Reg. server (standby) FC

FC Large-scale archive (DVD) cart system

Cache disk for registraton

Figure 1.13 Reference movie registration and distribution. Reproduced from A. Mori et al., ‘Facilities Outlines of NHK Archives’, Housou Gijutsu, February 2003 by permission of Kenrokukan Publishing Corporation.

of the content are displayed. Figure 1.12 shows a visual display of search results and reference movie. By clicking a representative stillimage, its MPEG-4 data stream down. If desired, then ordering can be done using the composition table. This composition table is connected to the program production support system called ‘Beatus’, which that will be described later. Reference movie distribution – from a reference movie server. Figure 1.13 shows reference movie registration and distribution. MPEG-4 data are stored both in a large-scale DVD cart and in a cache using Real Time Protocol (RTP). The DVD cart contains at most about 3,600 discs of DVD-RAM double-side (4.7 GByte per side). It can store about 150,000 hours of MPEG-4 data at 500 kbps. 1.3.5

VIDEO PROVISION

Ordered content can be transferred from Kawaguchi (NHK Archives) to Shibuya (NHK Broadcast Center) by IP transmission. OC192 (10 Gbps) is used for the IP network in anticipation of future expansion. A total of 15 VLANs (virtual local area networks) are set up for

26

WHAT IS A LARGE-SCALE ARCHIVE?

each function to keep the necessary bandwidth. If an ordered content is recorded on a medium in D3 or HD D5 format, the VTR cart plays it back. Other media than these are played back manually with each type of VTR. The played-back signal is encoded and then composed in General Exchange Format (GXF: SMPTE360M). Transmission includes four systems of HDTV at 60 Mbps and four systems of SDTV at 50 Mbps concurrently: VTR cart – D3, HD D5. At most eight mixed units. Tapes are managed by means of bar codes. Figure 1.14 shows the external appearance of a VTR cart.

Figure 1.14

External appearance of a VTR cart.

Sequential controller – to execute remote copying with frame accuracy. According to the transmission schedule of the general DB, it controls a VTR and a video server on a time code basis in real time. Video server – encoding and decoding capability. Four systems in SDTV or two systems in HDTV available in each video server. MPEG-2 4:2:2P@ML is used at 50 Mbps for SDTV and MPEG-2 4:2:2P@HL is used at 60 Mbps for HDTV. The group of pictures (GOP) total 15. Played-back video is encoded at Kawaguchi and transferred to Shibuya through the IP network. In Shibuya it is

1.3 EXAMPLE OF NEW TYPE-ORIENTED ARCHIVE Kawaguchi

Video audio TC

27

Shibuya

Playback site server

IP router

IP router

GXF

Record site server GXF

Video audio TC

IP Network HDTV: 60 Mbps SDTV: 50 Mbps IP router

Record site server GXF

Figure 1.15 Method of image transmission. Reproduced from A. Mori et al., ‘Facilities Outlines of NHK Archives’, Housou Gijutsu, February 2003 by permission of Kenrokukan Publishing Corporation.

decoded and recorded as a VTR. Figure 1.15 shows how an image is transmitted. 1.3.6

VIDEO PRESERVATION

NHK Archives gathered movie ﬁlms and VTR tapes that had been kept in archives at Hamamatsu and at Tsuruoka. In the storehouse, electrically powered moving shelves are used because of their costefﬁciency. People can easily handle tapes and ﬁlms with an intelligent assist function: Preservation shelves – ﬁlms and tapes are kept as they are. Figure 1.16 shows an area of preservation shelves devoted to ﬁlms and Figure 1.17 an area of preservation shelves for tapes. Each shelf is equipped with a shelf indicator to show where the ordered tape is located. The electric motors are built to withstand major earthquakes. Intelligent assist function – a handy terminal (as shown in Figure 1.18). This is linked to a DB server through a wireless LAN and indicates the store location of ﬁlms or tapes in map form. In case of an urgent order it can give that order extraction priority. 1.3.7

OPEN LIBRARY

The general public can use the open library at the NHK Archives Building. Figure 1.19 shows a viewing booth at the open library. At its opening in February 2003, there were 2,000 TV programs and 200 radio

28

WHAT IS A LARGE-SCALE ARCHIVE?

Figure 1.16

An area of preservation shelves devoted to ﬁlms.

Figure 1.17

An area of preservation shelves devoted to tapes.

1.3 EXAMPLE OF NEW TYPE-ORIENTED ARCHIVE

Figure 1.18

Figure 1.19

A handy terminal.

Viewing booth at the open library.

29

30

WHAT IS A LARGE-SCALE ARCHIVE?

programs. The plan is to increase the numbers to 5,000 TV programs and 500 radio programs by 2003. Thumbnails in MPEG-4 format enable search times to be 30 times faster than normal. Retrieval by genres or by keyword is available, too. There are 30 booths for HDTV and 50 booths for SDTV, which together can serve 120 viewers at most. The NHK has a plan to open another seven open libraries throughout Japan. Several issues still need to be addressed in order to open the programs through the Internet. At the website, you can actually search for 2,000 programs that are available at the open library, but you cannot view the program itself since the IPR of these programs are not cleared. IPR for broadcasting are naturally cleared when the program is produced; however, opening the program through the Internet requires new permission of the rights. If distribution was via a pay service, a rule for proﬁt sharing could be drawn up. It deems that major technical issues for opening programs through the Internet have been already cleared, but the protection of IPR is still very important issue. And the sharing rule of the charges from the rights may be a new issue. 1.3.8

PROGRAM PRODUCTION SUPPORT SYSTEM – ’BEATUS’

Objectives of the System In the real digital broadcast age, it will be essential that programs of high quality and various content for data broadcasting be produced efﬁciently. So far production equipment has been developed separately. It is important that this matter is resolved: the NHK has developed a new program production support system called ‘Beatus’. This system supports the whole process of program production. From ﬁlming to editing, video and its information are digitized and can be managed by a computer. Beatus not only makes program production more efﬁcient it can also cope with various services in the digital broadcast: for example, after narrowing retrieval down, users can obtain representative still-images and the composition table that manages every scene, both of which are prepared using Beatus. Figure 1.20 shows a conceptual diagram of Beatus. Major Functions: Program-related Information Can Be Handled at Cut Unit Major functions include: Information input camera – text data. Information for editing is input during ﬁlming. There is a visual function that allows every cut

31

1.4 FUTURE VIDEO ARCHIVES

Plan and Proposal

Shooting

MPEG

Composing Cut detect

Material tapes Content master

Shooting information

Content manage

Shooting manage

Material information Material manage

Editing

Content/Broadcasting

EDL Non-Linear editing

Finished content

Cut information Structure management

Scenario Retrieval/Re-use

Meta-information management database

Content MPEG

Archive Internet Distribution

Figure 1.20 Conceptual diagram of ‘Beatus’. Reproduced from ‘Total Management System for Program Production’ at http://www.nhk.or.jp/ strl/open2001/en/tenji/id23/index.html by permission of the NHK (Japan Broadcasting Corporation).

of every video to be put on the composition table and reviewed on a PC display. Network DTPP (Desk-Top Program Production) – a production environment in which many producers can ascertain the work state of each other and as a result can work in parallel. By exchanging data between a PC and editing equipment, data on the composition table can be utilized as part of the editing process and the editing result can also be utilized as a broadcast script. Retrieval and reuse – XML (eXtensible Markup Language). The semantics and structures of program information are described in XML, facilitating easy retrieval and reuse of programs. 1.4

FUTURE VIDEO ARCHIVES

The video process is complex enough: stored materials in an archive can be opened up for distribution; distributed materials can be utilized for editing; and then the edited materials (video content) can be stored in an archive for distribution. In the process, related equipment and technologies will become more reﬁned and become popular. Related equipment includes input devices such as cameras, high-density

32

WHAT IS A LARGE-SCALE ARCHIVE?

storage devices such as optical disks, and so forth. Related technologies include sensibility technologies like 3-D video, multimodality, network infrastructures such as gigabit or IP version 6, and information compression/transmission/process technologies. Fundamentally, there is a pressing need to establish a mechanism in which IPR can be protected and digital content securely distributed. The mechanism will promote digital content production that is sensibility-rich and more interesting. For that purpose it is essential that a unique ID method for stored content is popularized and that the international standardization of metadata is accessed. Future video archives should allow: everybody to use an archive and obtain content; video utilization for businesses to gain information about their own industry; educational utilization to promote learning; groundwork for content dealing with business and local industry. Utilization of an archive may facilitate production of highly valueadded goods and shift industry structures toward software dependence. Now is a critical time for large enterprises, intending to expand in a region, and for regional industries, consisting of mainly smallscale local industries, to do this: digital video archives can be very important in gaining new business.

2 Content Production from Digital Archives Haruo Hiki

The digitization of audio and visual information brought us various beneﬁts: no quality degradation in copying, high-compression capability, and easy data management. In this chapter these beneﬁts are made good use of for content production from digital archives. Content production requires information management from planning to distribution. Here the several steps involved in content production are described from the point of view of a digital archive.

Content production can be divided into three major parts: pre-production, production, and post-production. Figure 2.1 shows these three parts. In such a system new content always requires new ﬁlming or new computer graphics (CG) creations to compose the story: that is, each time new content is produced, all the necessary materials for the content should be newly shot or created. Let us look at what the three major parts entail: Pre-production – this is the intellectual and creative work undertaken at the preparation stage of the content production, such as planning, scripting, or raising funds. Production – this represents acquisition and creation and includes ﬁlming on location and in a studio, as well as CG creation. One application of this CG technology that has recently attracted a great deal of attention is the virtual studio. There are two types of virtual

Content Production Technologies F. Hasegawa and H. Hiki # 2004 John Wiley & Sons, Ltd ISBN: 0-470-86521-0 (HB)

34

CONTENT PRODUCTION FROM DIGITAL ARCHIVES

Filming on location

Video editing Planning

Filming in studio

CG creation Pre-production

Figure 2.1

Production

Audio post-production for video

Post-production

Three major parts of content production.

studios: one is where the video is produced by CG alone without ﬁlming; the other type is where ﬁlming and CG are combined on video and, as a result, a wider representation is possible than with the CG-only type. In other words, pre-production and production represent the creative part of content production. Figure 2.2 shows a simple case of ﬁlming in a studio that was set up in the Tohoku University of Art and Design (TUAD) for the Yamagata Video Archive Research Center (YRC) R&D project. Post-production – this includes linear editing, nonlinear editing, and audio post-production for video. It involves assembling and/or processing of materials to produce the content. Figures 2.3 and 2.4 show typical editing facilities in the rack and a typical editing console, respectively, both of which were set up in TUAD for the YRC R&D project. In addition to the content production mentioned above, a digital archive will bring a dramatic improvement to the content production system. The beneﬁts of a digital archive are its high-compression capability so that high-quality transfer is possible at the same bit rate or the possibility that more channels can transfer data in the same network capacity, no quality degradation in copying, and the easy management of a large number of video content. All these effects work well in a large-scale archive, enabling stored video material to be

CONTENT PRODUCTION FROM DIGITAL ARCHIVES

Figure 2.2

Figure 2.3

Filming in a studio.

Editing facilities installed in the rack.

35

36

CONTENT PRODUCTION FROM DIGITAL ARCHIVES

Figure 2.4

An editing console.

re-used for content production. The next section describes the storage method used for video material.

2.1

STORAGE OF VIDEO MATERIAL

Stored video material in large-scale archives can be used in content production to reduce cost. When using archives for production purposes, the ﬁrst step will be to search the archive for the desired scenes. Should any scene from the archive be usable for the desired scene, then new ﬁlming will not be required and cost can be reduced. A typical conceptual diagram of content production with a large-scale archive is shown in Figure 2.5. Let us say an unknown cameraman shoots a scene and stores it in the archive and an editing operator uses a known cameraman’s material and the stores it in the archive. Generally speaking, it is hard for a third person to use such video material stored in an archive. Therefore some considerations are required for content production with an archive. One example is that a camera-person who considers third-party’s usage could prepare video materials to be used well. Another is to add an appropriate explanation on the stored video materials when the materials are stored, in order that the third person could understand the video material well. As far as material (or

2.1 STORAGE OF VIDEO MATERIAL

Archive

37

Editing studio

Panasonic

Figure 2.5 archive.

Typical block diagram of content production with a large-scale

content) distribution is concerned, an important matter is to protect intellectual property rights (IPR). One solution to this issue might be to adopt a unique identiﬁcation, such as Content ID as promoted by the Content ID Forum (cIDf). This is a unique identiﬁer that should facilitate material (or content) distribution: the Content ID could be watermarked on the video itself. 2.1.1

STORAGE MEDIA

There are several digital storage media in the world and they are still being improved. Tape Tape is used both in video tape recorders (VTRs) and in digital data recorders. VTR is divided into two types: analog and digital. Examples of analog VTR are as follows: Betacam, components, 12 -inch tape; M2, components, 12 -inch tape; Examples of digital VTR are as follows: D1, ITU-R601, components 4:2:2 standard, 19-mm tape; D2, composite PAL (Phase Alternate Line, used in Europe) or NTSC (National Television Standard Committee, used in Japan and the USA), 19-mm tape; D3, composite PAL or NTSC, 12 -inch tape; D5, ITU-R601, components, 12 -inch tape;

38

CONTENT PRODUCTION FROM DIGITAL ARCHIVES

Digital Betacam, intra-ﬁeld compression, 12 -inch tape; DVCPRO, D7, intra-frame compression, 25 Mbps, 6-mm tape; DVCAM, intra-frame compression, 25 Mbps, 6-mm tape. The recording density of tape became saturated after the development of metal-evaporated (ME) tape and thin-ﬁlm layered heads. The storage lifetime of tape is fairly long (several decades), but that of VTRs is short because of constant updates. Digital data recorders are generally used for data back-up. Optical Disk Compact disc (CD) and digital video disc (DVD) are now the most familiar media. DVD-RAM is a good, rewritable medium for video archive. In future, it is expected that optical technology will make remarkable progress in recording density and optical disks will become the best media for archive purposes. In the personal computer ﬁeld, magneto-optical (MO) disks are widely used because of their capacity variety, but there is a major problem when the bit transfer rate is not high enough. Hard Disk Drive (HDD) HDDs are very popular in the computer ﬁeld and are rapidly evolving. Along with their development in the computer ﬁeld, HDDs are also being used as video servers. Their merits include high transfer bit rate and quick access; however, they are not as reliable as other storage media and, of course, they are not removable. Further advantages of HDDs include rapidly reducing cost and greatly increasing capacity. Semiconductor Memory Semiconductor memory is starting to make an impact now that the price is going down and capacity is increasing. D-RAM has the fastest access among semiconductor memory storage media. However, the weak points are high cost and loss of data at power-off. 2.1.2

CONTENT MANAGEMENT

While content management is important for the archive itself, it is deemed to be more so for archive users. Here content means both material and ﬁnished programs. Archive users want to be able to make

2.1 STORAGE OF VIDEO MATERIAL

39

good use of stored content without difﬁculty. To facilitate this, all stored content should be managed by a unique code that not only distinguishes content individually in the archive but also anywhere in the world. A global unique code is essential for content distribution through the Internet. All other information related to content is called metadata. This includes title, name of creator, created date, genre, IPR, scene explanation, and so forth. As far as scene explanation is concerned, the YRC studied how best to describe it for use in the editing process. The result was that the description could be made by 5W1H as shown in Table 2.1. This is divided into two aspects: one from the camera side and the other from the object side. Using these simple explanations, an editing operator can visualize the scene clearly. These explanations are described in Moving Pictures Expert Group (MPEG-7) format. Table 2.1

What Who How Where When Why

Scene explanation by 5W 1H

Camera side

Object side

What camera was used? Who was the cameraman? How was the camera being used? Fixed, zooming, panning Where was it ﬁlmed? When was it ﬁlmed? Why was this material created?

What was shot? Who was the actor? How was the object behaving? Fixed, moving Where was it ﬁlmed? When was it ﬁlmed? Why was this material created?

In addition, original time codes of video material should be included in content management because the material will be used in the editing process and should be protected in every frame. Unique Code A unique code is normally formed as an identiﬁer. Several identiﬁers are now standardized or under standardization in the world. Here some identiﬁers are described brieﬂy: ISAN (International Standard Audiovisual Number) – ISO 15706: 2002. This comprises 64 bits (48 bits for work root number and 16 bits for episode number) and can be used as a global unique identiﬁer for audio-visual (AV) work. V-ISAN (version identiﬁer for the International Standard Audiovisual Number) – under standardization. This comprises 96 bits (48 bits for work root number, 16 bits for episode number, and

40

CONTENT PRODUCTION FROM DIGITAL ARCHIVES

32 bits for version number) and can be used as a derivative identiﬁer from an original AV work. UMID (unique material identiﬁer) – SMPTE-330M. For details see Section 3.3. Content ID – registered in Part 3 of MPEG-21. For details see Section 3.2. Original Time Code Time codes are deﬁned in the international standard IEC 60461, ‘‘Time and control code for video tape recorders.’’ This standard deﬁnes a linear time code (LTC) and a vertical interval time code (VITC); it also explains the timing of code words relative to television signals for 525/ 60, 1125/60, and 625/50 television systems. Table 2.2 shows the VITC and LTC code word bit deﬁnitions of the standard. Table 2.2 VITC bit number 0 1 2 3 4 5 6 7 8 9

LTC bit Value number (weight)

VITC and LTC code word bit deﬁnitions Common assignment

0 1 2 3 4 5 6 7

1 0 (1) (2) (4) (8) (LSB) | | (MSB)

VITC sync bit VITC sync bit TV frame units TV frame units TV frame units TV frame units First binary group First binary group First binary group First binary group

10 11 12 13 14

8 9 10

1 0 (1) (2) FLAG

VITC sync bit VITC sync bit TV frame units TV frame units Flag

15

11

FLAG

Flag

16 17 18 19

12 13 14 15

(LSB) | | (MSB)

Second Second Second Second

binary binary binary binary

group group group group

60-ﬁeld television

50-ﬁeld television

24-frame ﬁlm

Drop frame ﬂag Colour frame ﬂag

Unused bit

Unused bit

Colour frame ﬂag

Unused bit

41

2.1 STORAGE OF VIDEO MATERIAL

Table 2.2 VITC bit number 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59

LTC bit Value number (weight)

(Continued)

Common assignment

16 17 18 19 20 21 22 23

1 0 (1) (2) (4) (8) (LSB) | | (MSB)

VITC sync bit VITC sync bit TV seconds units TV seconds units TV seconds units TV seconds units Third binary group Third binary group Third binary group Third binary group

24 25 26 27 28 29 30 31

1 0 (1) (2) (4) FLAG (LSB) | | (MSB)

VITC sync bit VITC sync bit TV seconds tens TV seconds tens TV sennconds tens Flag Fourth binary group Fourth binary group Fourth binary group Fourth binary group

32 33 34 35 36 37 38 39

1 0 (1) (2) (4) (8) (LSB) | | (MSB)

VITC sync bit VITC sync bit TV minutes units TV minutes units TV minutes units TV minutes units Fifth binary group Fifth binary group Fifth binary group Fifth binary group

40 41 42 43 44 45 46 47

1 0 (1) (2) (4) FLAG (LSB) | | (MSB)

VITC sync bit VITC sync bit TV minutes tens TV minutes tens TV minutes tens Flag Sixth binary group Sixth binary group Sixth binary group Sixth binary group

60-ﬁeld television

50-ﬁeld television

24-frame ﬁlm

Field/frame BG ﬂag 0

Phase

BG ﬂag 0

BG ﬂag 0

BG ﬂag 2

(Continues Overleaf )

42

CONTENT PRODUCTION FROM DIGITAL ARCHIVES

Table 2.2 VITC bit number 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79

LTC bit Value number (weight)

(Continued)

Common assignment

48 49 50 51 52 53 54 55

1 0 (1) (2) (4) (8) (LSB) | | (MSB)

VITC sync bit VITC sync bit TV hours units TV hours units TV hours units TV hours units Seventh binary group Seventh binary group Seventh binary group Seventh binary group

56 57 58 59 60 61 62 63

1 0 (1) (2) FLAG FLAG (LSB) | | (MSB)

VITC sync bit VITC sync bit TV hours tens TV hours tens Flag Flag Fourth binary group Fourth binary group Fourth binary group Fourth binary group

80 81 82–99

1 0 64–79

60-ﬁeld television

50-ﬁeld television

24-frame ﬁlm

BG ﬂag 1 BG ﬂag 2

BG ﬂag 1 BG ﬂag 1 Field/phase BG ﬂag 2

VITC sync bit VITC sync bit VITC CRC code LTC sync word

The author thanks the International Electrotechnical Commission (IEC) for permission to reproduce table 11 from its International Standard IEC 60461 (2001-02) ‘Time and control code for video tape recorders’. All such extracts are copyright of IEC, Geneva, Switzerland. All rights reserved. Further information on the IEC is available from www.iec. ch. IEC has no responsibility for the placement and context in which the extracts and contents are reproduced by the author; nor is IEC in any way responsible for the other content or accuracy therein.

Time codes are frequently used in editing processes as references to editing points. An original time code means a time code in the original tape. In an editing studio, an original time code is commonly used, but material from archives does not always have an original time code. If the time code in an editing system is locally generated, the edit decision list (EDL) – which arises as a result of the editing operation – is only valid for that particular editing system. It is for this reason that we advocate the use of an original time code everywhere.

2.1 STORAGE OF VIDEO MATERIAL

2.1.3

43

WHAT SHOULD BE STORED IN ARCHIVES?

Get the content you need by using material from the archive! Such an instruction brings with it some questions. One question may be, ‘Does the archive hold the material we need for new content?’ Another may be, ‘How much will it cost me to use the material?’ And yet another may be, ‘Has it been ﬁlmed in the same season as I expected?’ Anyway, it is likely that stored material will not meet your requirement: mostly, stored material in the archive does not meet users’ requirements because of the dearth of material currently stored in archives. YRC Archive Example The YRC goes against the grain of most archives by having a large amount of material in its archive, thanks to the Yamagata Multimedia Development Promotion Association (YMDPA), which was established to support R&D at the YRC by collecting video material in Yamagata. As mentioned in Chapter 1, the YMDPA has gathered around 2,000 titles of video material. Of these, 900 relate to wildlife and scenery, 600 to traditional culture, and 500 to events. The YMDPA was established not only to support R&D at the YRC but also to record local events and cultures for posterity. The latter is the principal aim of the preservation-oriented archive. Many different events, scenes, and cultures were shot and gathered. However, such an event or scene was ﬁlmed only once, despite particular events occurring annually, because the YMDPA wanted to collect as many different events or scenes as possible. Multi-purpose Material Can video material have multi-purpose usages? This question is frequently asked. And the answer is ‘no’ in most cases. The reason is that video material ﬁlmed for content A are not always suitable for content B. Is it possible to shoot scenes for content B while shooting for content A? Another frequently asked question! The reason that stored material in the archive does not normally meet your requirement is because it was shot for, say content A and not for your content. Therefore, to make such material multi-purpose, special considerations are needed at the time of ﬁlming. Figure 2.6 shows the cameraman’s task as shooting scenes as widely as possible for other people to use easily. The considerations for ﬁlming in this way are as follows:

44

CONTENT PRODUCTION FROM DIGITAL ARCHIVES

Filming

Archive

Editing studio

To shoot so widely that other people can use

Figure 2.6

The cameraman’s task.

seasons – same location or same scene in each of the four seasons; times – same location or same scene at different times of the day (morning, noon, evening, etc.); angles – high angle, low angle, and normal angle. Shots by different angles. types – wide shots, medium shots, and close-ups; zooming – as little as possible and a ﬁxed position is preferable; panning – as little as possible and a ﬁxed position is preferable. It is with these consideration in mind that a cameraman should ﬁlm video material. Thus, video material ﬁlmed in this way can be stored in the archive and will later be usable by other people. Therefore, the answer to this section’s question is in the afﬁrmative: video material ﬁlmed in the above way can be multi-purpose.

2.2 2.2.1

HOW TO SEARCH FOR SPECIFIC MATERIAL IN AN ARCHIVE RETRIEVAL METHODS FOR STORED MATERIALS

Material stored in the archive is usually retrieved through some network or the Internet. How can users recognize the material they want in the network? While handling tapes of video material, a user is able to ﬁnd the stock location, to know the physical size of the tape, to look at labels on the tapes and thereby recognize the contents. However, using a network, some different retrieval methods are required, particularly in the case of video.

2.2 HOW TO SEARCH FOR SPECIFIC MATERIAL IN AN ARCHIVE

45

First Step The ﬁrst step in the retrieval of stored material is to narrow down the amount of material. For this purpose a text search is the most popular and efﬁcient. This is done by searching for a keyword, title, identiﬁcation number, etc. Structured indexing facilitates the ﬁnding of the desired materials. This means using an index that consists of the following items: extraction of picture characteristics; visual information or sound information as key; cutting points or scene change points. Second Step The second step is to check the contents of the material that has been narrowed down. This is done simply by looking at the contents, just like browsing through still-images or the scene change points of a digested movie. Still-images are generally obtainable at the cutting points or the scene change points of video material. Digested movie can similarly be used by looking at scene change points. Third Step In this step, highly compressed video is useful since the number of materials may already be narrowed down enough. MPEG-4 compression has recently become popular and can be easily used through the Internet. New Method The Internet suits a large-scale archive system: it makes it possible for everyone to access multimedia content at anytime and from anywhere. Further, both an image and its metadata can be handled with standardized image compression formats such as MPEG-1, 2, and 4. However, even MPEG-4, which has the highest compression of the lot, puts a heavy burden on the common network. The issue is resolved by focusing on MPEG-7 as the standardized metadata format. Almost all necessary data both for retrieval and editing can be described in MPEG-7. MPEG-7 instances (ﬁles in MPEG-7 format) are small enough for transmission across the Internet. These instances can be used smoothly for retrieval and drastically decrease the editing process.

46 2.2.2

CONTENT PRODUCTION FROM DIGITAL ARCHIVES

PREVIEW OF STORED MATERIAL

During the editing process, the preview of video material is crucial to decide which points need cutting, dissolving, or wiping. The preview is usually required in frame units: that is, 25-ms accuracy in PAL or 33.3-ms accuracy in NTSC. Such an accurate preview is available in the editing studio where all editing facilities are connected and controllable in frame units. In order to preview material in an archive, however, a user has no choice but to access it through the network. A video signal requires a broadband network even if compression technologies are used. As far as compression technologies are concerned, MPEG-2, MPEG-1, and MPEG-4 have all been evaluated at the YRC. For preview through a network, there are some restrictions as follows: non-real time control – there is a poor human–machine interface and low accuracy of control; low preview accuracy – depending on the method; low picture quality – depending on the format.

MPEG-2 MPEG-2 format is used for DVDs and digital broadcasts. It gives higher picture quality but requires a fast transfer bit rate – between 6 and 15 Mbps. Currently, a cable modem and ADSL are used to provide broadband networks for the distribution of MPEG-2 streams. However, the necessary bandwidth is not guaranteed as it is shared with other users.

MPEG-1 For video CDs or movies for computer, MPEG-1 is the most popular format. The transfer bit rate is 1.5 Mbps – the same as CD. The picture quality is reasonable for movies on a TV screen.

MPEG-4 MPEG-4 is the newest of the MPEG formats. The transfer bit rate is so wide that MPEG-4 can be adopted for mobile equipment and for HDTV (high-deﬁnition TV) compression.

2.3 CONTENT PRODUCTION USING METADATA

2.3

47

CONTENT PRODUCTION USING METADATA

Figure 2.7 shows how the archive makes stored materials usable from a distance and by third persons. The archive bridges the gap between a cameraman and an editing operator in the studio. A system is needed that can describe ﬁlmed data and the necessary data for editing as accurately as possible. A good candidate for this task is the ‘Beatus’ system of the Japan Broadcasting Corporation (NHK), which plays the role of a bridge

Filming

Archive

Editing studio

To describe the shooting data and necessary data for editing

Figure 2.7

One of the archive’s tasks.

between the content production processes. In future, content distribution will be increasingly through the Internet, and so material for producing content will be more valuable than now. We studied and proposed content production using the metadata of materials, and in so doing developed our ‘metadata service’. 2.3.1

THE NHK’S ’BEATUS’ SYSTEM

The NHK has developed a content production support system called ‘Beatus’ which is now fully operational. Figure 2.8 shows its conceptual structure. Beatus consists of the smaller blocks inside the main block of this ﬁgure. In this system, video material and its metadata – from ﬁlming to editing and archives – can be completely managed by computers. The features are described below: content proposal, content information, and rights information can be input by a personal computer;

48

CONTENT PRODUCTION FROM DIGITAL ARCHIVES

Work on notebook PC! Visual composition table Making composition table and scenario program

Program notification table Program publicity information

Inquiry memo Program recording report

Data organization

NHK Archives

production starts Reserch Program and planning proposal

Composition and editing data Video material

Digitize

Nonlinear editor

VCR editing

Sound editing

Completion

Broadcasting

Figure 2.8 Conceptual structure of ‘Beatus’. Reproduced from ‘Total Management System for Program Production’ at http://www.nhk.or. jp/strl/open2001/en/tenji/id23/index.html by permission of the NHK (Japan Broadcasting Corporation).

scenarios can be visualized on a personal computer; the structural table data used in the editing process as well as the editing result can be fed back to a scenario; content metadata are available in the cutting unit; in combination with NHK Archives, the content or material can be retrieved and re-used.

2.3.2

METADATA SERVICE

Figure 2.9 shows the conceptual diagram of the metadata service. This service is based on the metadata of materials or content stored in the archive. A metadata service provider can be a separate organization from the archive and can manage all metadata as a database (DB). Sometimes it can take on the job of describing all material held in the archive.

2.3 CONTENT PRODUCTION USING METADATA

Figure 2.9

49

Conceptual diagram of virtual production.

Archive Under normal conditions, the archive retains the original tape or a copy of the highest quality of the original tape. The audio, video, and original time code of an original tape are played back and are registered with a global unique identiﬁer – Content ID, for example – in the archive as stored material. A database management system (DBMS) handles the material and its metadata, which are transferred to the metadata service provider whose task it is to promote the material. Figure 2.10 gives a sample material description of the archive. It begins with a genre dictionary to classify the material and goes on to describe the title, ﬁlm location, ﬁlm date, and Content ID of the material. The Content ID is the key that accesses the IPR DB. The Content ID is also the bridge to the metadata, such as original time code, time code at start, and its duration. It connects scene 1 to scene 2, and so on. Each scene consists of a key frame (still-image), head line, time code at start of scene, and duration of scene. These metadata will certainly make retrieval and editing easier. During the YRC project we realized that the data format of Content ID could be made relative to the data format of time code (IEC 60461). Figure 2.11 shows the relation between Content ID and time code. The intra-center number is arbitrary in length, but we decided to adopt

50

CONTENT PRODUCTION FROM DIGITAL ARCHIVES

Title

Offset to original time code

Location

Time code at start

Genre dictionary

Date

Duration

Content ID

Scene 1

Scene 2

Key frame Head line Time code at start Duration

Figure 2.10

Key frame Head line Time code at start Duration

Example of the material in the archive.

time codes here for editing purposes. As a result, 32 bits are dedicated to an identiﬁer and the other 32 bits are dedicated to the time code. These 64 bits can refer to the time code and the binary groups (BGs), which include LTC or VITC. Thus the identiﬁer and the time code can be transferred along with the LTC or VITC. Intra-center number 64 bits

Given number 16 bits

4

4

8

1

32

32

Material ID or finished ID Region number Versio number

Time code (00:00:00:00– 23:59:59:29 or 23:59:59:24)

Flag (material or finished) Center number

Material ID or finished ID = BG-1 + BG-2 + BG-3 + BG-4 + BG- 5 + BG-6 + BG-7 + BG-8

LTC or VITC

Figure 2.11 Relation between Content ID and time code.

2.3 CONTENT PRODUCTION USING METADATA

51

Metadata Service Provider A metadata service provider receives material and its metadata in compressed format and stores them in the DB. From both the compressed materials and the metadata, MPEG-7 instances for retrieval and editing are created manually or automatically, and stored in the MPEG-7 DB. Figure 2.12 shows a sample of MPEG-7 format. In the example the description is given in English, but to encourage global

. . . . . Snow monsters in Zao. . . . . .