Abstract
Applying machine learning (ML), and especially deep learning, to understand visual content is becoming common practice in many application areas. However, little attention has been given to its use within the multimedia creative domain. It is true that ML is already popular for content creation, but the progress achieved so far addresses essentially textual content or the identification and selection of specific types of content. A wealth of possibilities are yet to be explored by bringing the use of ML into the multimedia creative process, allowing the knowledge inferred by the former to influence automatically how new multimedia content is created. The work presented in this article provides contributions in three distinct ways towards this goal: firstly, it proposes a methodology to re-train popular neural network models in identifying new thematic concepts in static visual content and attaching meaningful annotations to the detected regions of interest; secondly, it presents varied visual digital effects and corresponding tools that can be automatically called upon to apply such effects in a previously analyzed photo; thirdly, it defines a complete automated creative workflow, from the acquisition of a photograph and corresponding contextual data, through the ML region-based annotation, to the automatic application of digital effects and generation of a semantically aware multimedia story driven by the previously derived situational and visual contextual data. Additionally, it presents a variant of this automated workflow by offering to the user the possibility of manipulating the automatic annotations in an assisted manner. The final aim is to transform a static digital photo into a short video clip, taking into account the information acquired. The final result strongly contrasts with current standard approaches of creating random movements, by implementing an intelligent content- and context-aware video.
Highlights
Multimedia content has become ubiquitous, being present in almost all aspects of our daily lives
The potential benefits of such technology are still underexplored, as their use has been essentially concentrated on automatically understanding the current interests of consumers and identifying and selecting specific types of content to be made available
The automatic identification of regions of interest (RoI) within images is achieved through computer vision approaches
Summary
Multimedia content has become ubiquitous, being present in almost all aspects of our daily lives. The potential benefits of such technology are still underexplored, as their use has been essentially concentrated on automatically understanding the current interests of consumers and identifying and selecting specific types of content to be made available This is known as keyword research and topic generation, so that media content can be automatically selected and published according to what customers are really interested in. Our vision is that it is possible to produce automatically content-aware media clips from a single photograph by contextualizing it as much as possible, including the situation where and when the photo was taken Such contextualization, in the form of metadata, can be fed into intelligent creative tools which will apply cool visual effects in an automated way, obtaining contextually and semantically aware multimedia stories. Such applications were tested by professionals in real-world conditions, demonstrating the validity of the approach
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.