Traditional travel itinerary articles, while valuable, are limited in reach due to their text format. As attention spans of audiences are dwindling, there is a need for content that is both engaging and concise. To address this, we've developed an innovative tool that transforms these text-based articles into engaging, TikTok-style videos, allowing for broader distribution on social media platforms. This automated solution not only maintains journalistic integrity by keeping the journalists involved in the creation process of the video, but also presents an opportunity for journalists’ content to reach an exponentially large new audience.

Input: Travel itinerary article HTML

Input: HTML of travel article

To use TripClip, a user will upload the html of a travel itinerary article of their choosing. With the system’s HTML article parser, all of the key content and associated images are extracted from the article. By default, TripClip is configured to work with NYT’s 36 Hours articles, but it also provides the ability for a user to upload their own article parser for different travel articles.

The system also generates a script which can then be edited freely by the user. Once the script is finalized by the user, the system extracts unique “concrete phrases” from the script using NLP. Each concrete phrase was mapped to a specific frame, and when that concrete phrase is uttered in the audio file, it acts as a signal for the frame mapped to that phrase frame to appear. Our approach took inspiration from a relevant paper, “Crosscast: Adding Visuals to Audio Travel Podcasts” by Xia, Jacobs, and Agrawala. The user has the option to either upload a recording of themself reading the script or choosing an automatically generated audio. It takes just under a minute for the system to process and consolidate everything into a aesthetic and engaging tik-tok style travel video.

After the content extraction, all information is presented back to the user and the user has the opportunity to tweak anything and also add supplemental images. Once the information is finalized, the system starts video generation. In video generation, the system automatically crops the images by detecting the most salient region of the images and cropping around it. Then, the system dynamically placed the title, caption, and icons on each image by assessing areas of lowest frequency. Map frames that provide a visual overview of the locations of activities on each day are also generated.

original image

TripClip output

TripClip demo by teammate