The green circles show a limited set of processes within the media value chain – from idea over pre-production, production, post, up to the distribution and archive. Behind those, a broad set of solutions by Avid, Adobe, Sony, Blackmagic etc. does exist. To gather the metadata being created or edited in there, we create “adapters” connecting to the APIs these tools provide. And within these adapters, we also convert the data into a structure that reflects the meaning of the information, in a semantic manner (according to our Ontology).

So, all data for a certain a/v production can be queried or edited at any point in time, and the manual take-over of data drops out. Rights-management, versioning as well as the mapping to external ontologies and knowledge bases (such as DBPedia, wikidata, IMDB and others) round up the solution.

The functional components CRUD API, Adapters, Triple Store, DMO Ontology and rights-management, as depicted as the red circle and everything inside it, build up the dwerft core technology “Linked Media Data Cloud” (LMDC), and will be released Open-Source – allowing not only the big players but also SMEs to settle upon the future of media production.

Interaction with the core technology by various user groups (producers, distributors, marketing) requires a tool in which individual workflows can be defined, data import / export initiated, versions managed and interfaces managed. The “Data Tracker” allows data to be queried from the data stored in the Linked Media Data Cloud (LMDC) at any time. A semantically supported data visualization makes search queries and complex correlations in result displays intuitively usable and the stored data usable.

Via the Data Tracker, the connected components of the LMDC have to be technically addressed as well as various application scenarios for different users. In addition to the administrative and technical metadata, mapped descriptive records. For example, pre-production information about props or costumes per scene or actor can be provided for distribution or other users.

Music is one of the most emotional parts of every film. And it is one of the most complex issues in film production. Film Producers, such as the dwerft partner transfermedia, are facing three main issues regarding music during production:

1. research of complex rights of source music is a huge, manual task.
2. Information on any new music composition must be delivered by composers to the Producer and rights collecting societies.
3. Any music used within a production must be related to a certain timecode of the film to identify the music at the right place.

Usually producers are employing music rights specialists to solve all three challenges at the end of a production – right before the film is prepared for distribution. With the dwerft core technology, this complex task will be solved much easier. During production, it collects many information about the film project automatically on a scene basis and provides detailed information on what is edited in the film at which timecode.

The dwerft core technology is also directly connected to the music database of the GEMA. GEMA is an organisation based in Germany and represents the usage rights stemming from authors’ rights for the musical works of those composers, lyricists and publishers who are members in the organisation.

Producers can link a source music title used in a film directly to the database of the GEMA, providing all necessary information about a source music at the dwerft core technology. With that link it is very easy to export a full list of all used source music of a film with all rights information automatically.

In addition, newly composed music from a film will be identified through the dwerft core technology to the GEMA. This makes the GEMA aware of any new compositions and eases the workflow of providing all necessary information about this new composition.

The Linkage between the dwerft core technology as the main metadata collector of a film project with the database of a collecting society for music solves one of the most complex pains of film producers at present times. It simplifies information ways, combines unique identification between film and music and creates a huge transparency in that special field.

As already known from the film production, technical metadata covers the range of information one can retrieve from a camera, generally speaking. Trying to do this End-to-End in the post-production is still very disappointing so far, as many software solutions are not able to interact or read the complete amount of metadata from other tools involved. So even if metadata has been collected during the typical workflows, we are neither able to use it properly along the processes, nor automate the workflows. Due to the lack of metadata standards and interoperability in the post-production domain, the preparation of the delivery to the customer still needs to be done manually, by hand. Additionally, the film production remains a mammoth task for all people involved regarding collaboration – seen from the metadata perspective, there are multiple encapsulated departments, which are not able to share the complete amount of important and valuable metadata.

By making of the dwerft core technology in post-production, the dwerft partner Rotor Film is able to collect, maintain and transport metadata throughout the entire workflows. With adapters, which act like plug-ins into post-production software solutions, metadata can be read, extracted and stored within the central database of the dwerft core technology, which acts like a “global project file”. This can contain multiple project information which will be transported from all departments in the film production process. It´s the first time where the metadata can be transported from the script, through production to the post-production, and finally become used within distribution.

For the post-production, the creation of deliveries for customers is daily business. Thanks to the dwerft core technology, the information about which deliveries are required are in place, as well as all metadata related to them. This allows creating both, the metadata sets for the deliveries as well as the transcoding of the a/v files automatically, by just pressing one button. How? Since all information is being saved within the dwerft core technology continually, we know everything about the different cut versions, the different fps versions, audio versions, and all the rest we need to know. Combining all this information with the project requirements, the relevant metadata for the individual delivery will be extracted automatically. On top of that, the workflows also automatically trigger the extraction and transcoding of the a/v files according to the delivery versions required.

VoD is becoming the most important distribution channel worldwide. More and more streaming services are competing for time and consumer attention. For a long time, providers such as Netflix dominated the market with a – supposedly – full-service program.  Nowadays, other large providers are starting their own programmes, such as Disney or Apple – and they are finding their customers.

In addition, however, many specialised streaming services are emerging, especially in the arthouse sector, but also on LGBT, splatter, horror, and many other special interests. Even yoga studios, football clubs and, of course, especially now cinemas and film festivals are discovering the possibilities of their own VoD offer.

The technical basis is now provided by various providers, so that basically everyone can operate their own VoD portal, whether as a commercial offer, showroom, project for enthusiasts or even a film festival or accompanying cinema programme. This opens up previously unknown possibilities for producers to offer their films directly and without the intervention of distributors not only to major streaming services, but also to place them on VoD portals with their own community (art house, cinemas, festivals, special interests).

To cope with nowadays multi-channel distribution requirements demands for automated workflows like the ones provided by the dwerft core technology:

In addition to the video itself, information accompanying the film on the portal is indispensable for the presentation of a film, which must be stored as a – comprehensive – metadata record in the database of the VoD system. As before, this metadata record is entered manually for each film in a time-consuming manner, and individual interfaces may support even an Excel-import. Producers need to compile and deliver these metadata records from their own databases, Word and Excel documents, usually according to concrete specifications of the portal operator.

The dwerft core technology facilitates this task considerably with an automated delivery of the metadata.  All metadata aggregated during the production process can be made available for retrieval/delivery via an interface depending on the individual requirements of the VoD portal. In addition to considerably simplifying delivery, the producer can also ensure that only metadata approved by him are used on the portal.

An important aspect is also the control of whether the individual portals observe the agreed licence framework when publishing the films, e.g. if the licence expires, the film is no longer offered.

In addition to the descriptive metadata, the dwerft core technology therefore enables the automatic delivery of license data, such as type of use (TVoD, EST, SVoD and others), license period and area for direct transfer to the VoD portal database. With these automated workflows especially extended by the dwerft partner filmwerte for the license information aspects, the delivery of any number of VoD channels with all necessary metadata of the films to be placed is simple, fast, and cost-effective.

The German Broadcasting Archive (DRA) encompasses significant parts of the audio-visual heritage in Germany. By the end of 2020, it will provide access to nearly 40.000 hours of intellectually indexed and digitised footage of the former GDR television broadcasting. Due to the uniqueness and importance of the collection, the DRA is the starting point for many journalistic and scientific studies. To fulfil constantly changing user requests or the requirements of new distribution channels as well as innovative data research, visualisation and fine-grained analysis tools, the DRA aims to fit and complement its technical and descriptive metadata repositories through the use of AI-based technologies.

This is an ideal use case for the dwerft partner Interlake, which is investigating how AI algorithms can be used for content metadata indexing in media production as part of its research focus, called “Cognitive Media Framework”.

Interlake will take up with dwerft and develop use-case-specific and practical concepts how AI Services not only generate additional descriptive metadata, but also map these semantically, using dwerft’s LMDC core technology and the Ontology DMO in relation to link each other.

For the analysis of the archived assets, the DRA defined five different targets:

1) Registration and identification of the acting persons in picture or sound

2) Recognition of geographical locations, whether recognisable in the image or mentioned in the text

3) Automatic extraction of authors, producers, and contributors of a production

4) Recognition of inserted music

5) Detection of photos or still images

In a later project phase, the findings will be made available to the archivists of the DRA in a prototypical web interface with start and end time code as jump labels, so that they can assess the quality achieved and to be able to transfer it easily to  ARD Archive System (FESAD).

Even if no fully automatic indexing is the goal, AI algorithms can, as described here, make life much easier today in some areas by eliminating monotonous and redundant work, so that human experts can focus on the complex issues. In the future, it can be assumed that AI tools in combination with semantic relations will be a standard assistance instrument for search, analysis and production for any kind of media professionals.

Since the late eighty’s subtitles are integral part of public television in Germany. Interestingly the technical foundation for subtitle production has not changed very much since then. While state of the art technologies like DVB Bitmap Subtitling or customizable HbbTV subtitles are used in distribution the production of subtitles is still based on Teletext.

But the teletext technology has its limitation to meet new requirements. On the one hand for example it becomes important to enrich subtitle files with additional metadata. On the other hand, subtitle files need to be distributed on a variety of platforms but should be authored only once. Both types of requirements are coming together in a scenario where metadata enables the automatic transformation of one subtitle file in different formats that are customized for specific distribution targets and this metadata is added at the authoring stage.

The most common subtitle file format in broadcasting is EBU STL. It is designed to transport information for the distribution of teletext subtitles. Although its simple information structure has established it as an interoperable exchange format it fails short to integrate information for new target technologies that have been evolved over three decades. Furthermore, its binary structure is a barrier to establish a rich ecosystem of authoring tools.

The imscEd subtitle editor developed in the dwerft project by IRT closes this gap. It is based on the format IMSC (TTML Profiles for Internet Media Subtitles and Caption Profiles). As an XML format IMSC is very flexible to be enriched with additional metadata. IMSC also makes it possible to use and extend existing open source tools to transcode one subtitle file into different distribution specific subtitle formats. Furthermore, imscEd is designed as web-based software and is published as open source. The open source approach contributes to subtitle tool ecosystem that can respond to new requirements in a more flexible manner. The source-code of imscEd can be used by anybody to adopt it for a custom solution or to use it as reference for their own implementation. Thanks to its web-based approach it can be scaled easily. No workstation specific solution needs to be installed and maintained.

imscEd can be used for a variety of use cases ranging from the authoring of broadcast subtitles with the help of speech-to-text technologies over the production of creative subtitles with emojis or the subtitle distribution on social media platforms. One key aspect of imscEd is that subtitles can be authored once for on different distribution platforms. This is possible, amongst others, with the help of additional metadata.

imscEd is already published in an alpha-release on github.

imscED is an editor for the subtitles and caption format IMSC. It shows how imscJS from Pierre Lemieux can be used to manipulate style attributes from an parsed IMSC document and how to export it back to XML.

Note that this project is a pre-alpha release for developer review. It is a prototype in development and can change at any time. The code does not represent a public API.

Therefore it is not ready for production and use it at your own risk.

More information: https://github.com/IRT-Open-Source/imsced