Python: Creating a timed image slideshow with PIL and OpenCV v2

Posted: Friday 17 July 2015

Problem

Apparently, one of the hardest video-editing task to do with a script is to create a dynamically-timed slideshow without any fancy-drag and drop GUIs.


With Adobe After Effects, you cannot dynamically load external images using an expression (they will need to be loaded into your project beforehand, and even then, you cannot load the image into a comp with an expression).

And adding hundreds of layers of images and having to go through each and every one of them to edit the expression is a fairly tedious task.

Worse of all, every change that you make - such as adding a new image to the slideshow  - will compound towards the chore of doing things manually.


Solution

Building upon my last two posts: Python: Converting from PIL to OpenCV 2 Image Formats and Python: PIL to mp4, we've reached the end.

With "Python: PIL to mp4", a simple blending transition was created using PIL and OpenCV, But the objective of this post is to introduce timings to delay the animation for numerous/multiple images.

We can extend this idea of having a primitive transition to allow for an image to be delayed from transitioning until a certain amount of time has elapsed, and to allow the transition to occur after "x" amount of seconds, hence forming a slideshow.


Process

Initialization

So to start off with, we're going to need some data to work with.
Since it's Python, you can do whatever you want to feed data in - you could use a JSON file, CSV, Pickle, whatever you're comfortable with, or perhaps, whatever arbitrary file format that you're locked into using.

But here, a basic python array will be used to indicate the timings and image file that will be fed into the slideshow, amongst other data...
songData = [
    [390, u'Fractal', u'Itvara', 'minimix', u'image1.jpg'],
    [322, u'Case & Point', u'Error Code', 'minimix', u'image2.jpg'],
    [261, u'Excision & Pegboard Nerds', u'Bring the Madness (Noisestorm Remix) [feat. Mayor Apeshit]', 'minimix', u'image3.jpg'],
    [157, u'Nitro Fun', u'Final Boss', 'minimix', u'image4.jpg'],
    [88, u'Astronaut', u'Quantum (Virtual Riot Remix)', 'minimix', u'image5.jpg'],
    [0, u'Fractal', u'Contact', 'minimix', u'image6.jpg']]

As you can see in the data above, the most relevant data is songData[][0] and songData[][4], indicating the timings (in seconds) and the image file locations, respectively.

We're going to set the FPS of the slideshow... 60FPS is the standard nowadays, so we're going to set that and process the songData above to reflect this...
FPS = 60 # Sets the FPS of the entire video
currentFrame = 0 # The animation hasn't moved yet, so we're going to leave it as zero
startFrame = 0 # The animation of the "next" image starts at "startFrame", at most
trailingSeconds = 5 # Sets the amount of time we give our last image (in seconds)
blendingDuration = 3.0 # Sets the amount of time that each transition should last for
                       # This could be more dynamic, but for now, a constant transition period is chosen
blendingStart = 10 # Sets the time in which the image starts blending before songFile

for i in songData:
    i[0] = i[0] * FPS # Makes it so that iterating frame-by-frame will result in properly timed slideshows

Now the first image is going to be loaded in by the script - as so:
im1 = Image.open(songData[-1][4]) # Load the image in
im2 = im1 # Define a second image to force a global variable to be created

current = songData[-1][4] # We're going to let the script know the location of the current image's location
previous = current # And this is to force/declare a global variable

And next up is to create the actual OpenCV video handling capability. You can have a read-up about this here: Python: PIL to mp4
height, width, layers = np.array(im1).shape # Get some stats on the image file to create the video with
video = cv2.VideoWriter("slideshow.avi",-1,60,(width,height),True)

So that was the basic initialization routine. If you don't get how it works together yet, don't worry. Just read on - as the full code with everything combined is below.

Main loop

So the strategy behind generating this slideshow is to loop through each and every frame and continuously feed that into our output video file. Sure some corners can be cut - by which you only generate the transitions (leaving the gaps to be manually filled by an external program) - but this post is looking more into automating the entire slideshow generation process with only Python, PIL and OpenCV.

We're going to have a main while loop that sets the limit on how long our slideshow should last.
while currentFrame < songData[0][0] + FPS * 60 * trailingSeconds: # RHS defines the limit of the slideshow

And this is where the nitty gritty kicks in: the actual code that makes the transition between each image within the slideshow...
    for i in songData: # Loop through each image timing
        if currentFrame >= i[0] - (blendingStart * FPS): # If the image timing happens to be for the
                                                         # current image, the continue on...
                                                         # (Notice how songData is reversed)
                                                         
            # The print statement adds some verbosity to the program
            print str(currentFrame) + " - " + str(i[0] - (blendingStart * FPS)) + " - " + i[2]
            if not current == i[4]: # Check if the image file has changed
                previous = current # We'd want the transition to start if the file has changed
                current = i[4]
                startFrame = i[0] - (blendingStart * FPS)

                # The two images in question for the blending is loaded in
                im1 = Image.open(previous)
                im2 = Image.open(current)
            break

    # See: http://blog.extramaster.net/2015/07/python-pil-to-mp4.html for the part below
    diff = Image.blend(im1, im2, min(1.0, (currentFrame - startFrame) / float(FPS) / blendingDuration))
    video.write(cv2.cvtColor(np.array(diff), cv2.COLOR_RGB2BGR))
    
    currentFrame += 1 # Next frame
The ending to this program is pretty self-explanatory...
# At this point, we'll assume that the slideshow has completed generating, and we want to close everything off to prevent a corrupted output.
video.release()


All together now!

So here's all the code required to create a timed image slideshow with PIL and OpenCV v2!

Code:
from PIL import Image
import cv2
import numpy as np

songData = [
    [390, u'Fractal', u'Itvara', 'minimix', u'image1.jpg'],
    [322, u'Case & Point', u'Error Code', 'minimix', u'image2.jpg'],
    [261, u'Excision & Pegboard Nerds', u'Bring the Madness (Noisestorm Remix) [feat. Mayor Apeshit]', 'minimix', u'image3.jpg'],
    [157, u'Nitro Fun', u'Final Boss', 'minimix', u'image4.jpg'],
    [88, u'Astronaut', u'Quantum (Virtual Riot Remix)', 'minimix', u'image5.jpg'],
    [0, u'Fractal', u'Contact', 'minimix', u'image6.jpg']]

FPS = 60 # Sets the FPS of the entire video
currentFrame = 0 # The animation hasn't moved yet, so we're going to leave it as zero
startFrame = 0 # The animation of the "next" image starts at "startFrame", at most
trailingSeconds = 5 # Sets the amount of time we give our last image (in seconds)
blendingDuration = 3.0 # Sets the amount of time that each transition should last for
                       # This could be more dynamic, but for now, a constant transition period is chosen
blendingStart = 10 # Sets the time in which the image starts blending before songFile

for i in songData:
    i[0] = i[0] * FPS # Makes it so that iterating frame-by-frame will result in properly timed slideshows

im1 = Image.open(songData[-1][4]) # Load the image in
im2 = im1 # Define a second image to force a global variable to be created

current = songData[-1][4] # We're going to let the script know the location of the current image's location
previous = current # And this is to force/declare a global variable

height, width, layers = np.array(im1).shape # Get some stats on the image file to create the video with
video = cv2.VideoWriter("slideshow.avi",-1,60,(width,height),True)

while currentFrame < songData[0][0] + FPS * 60 * trailingSeconds: # RHS defines the limit of the slideshow
    for i in songData: # Loop through each image timing
        if currentFrame >= i[0] - (blendingStart * FPS): # If the image timing happens to be for the
                                                         # current image, the continue on...
                                                         # (Notice how songData is reversed)
                                                         
            # The print statement adds some verbosity to the program
            print str(currentFrame) + " - " + str(i[0] - (blendingStart * FPS)) + " - " + i[2]
            if not current == i[4]: # Check if the image file has changed
                previous = current # We'd want the transition to start if the file has changed
                current = i[4]
                startFrame = i[0] - (blendingStart * FPS)

                # The two images in question for the blending is loaded in
                im1 = Image.open(previous)
                im2 = Image.open(current)
            break

    # See: http://blog.extramaster.net/2015/07/python-pil-to-mp4.html for the part below
    diff = Image.blend(im1, im2, min(1.0, (currentFrame - startFrame) / float(FPS) / blendingDuration))
    video.write(cv2.cvtColor(np.array(diff), cv2.COLOR_RGB2BGR))
    
    currentFrame += 1 # Next frame

# At this point, we'll assume that the slideshow has completed generating, and we want to close everything off to prevent a corrupted output.
video.release()



Sample output

So with all the code above, it begs the question, why do I need to create a slideshow using scripts?
Well, here's a little sample of what you can do with a simple little slideshow.
Note the timings from "songData",
songData = [
    [390, u'Fractal', u'Itvara', 6:30, u'image1.jpg'],
    [322, u'Case & Point', u'Error Code', 6:22, u'image2.jpg'],
    [261, u'Excision & Pegboard Nerds', u'Bring the Madness (Noisestorm Remix) [feat. Mayor Apeshit]', 4:21, u'image3.jpg'],
    [157, u'Nitro Fun', u'Final Boss', 2:37, u'image4.jpg'],
    [88, u'Astronaut', u'Quantum (Virtual Riot Remix)', 1:28, u'image5.jpg'],
    [0, u'Fractal', u'Contact', 0, u'image6.jpg']]
With this slideshow, you can really enhance the effect of audio-react "music" YouTube videos, especially Youtube Music Mixes, like this:
Direct Link: https://www.youtube.com/watch?v=XI25k5Z-t88
Direct Link: https://www.youtube.com/watch?v=XI25k5Z-t88