Movie Maker: Time gap between movie and its audio file

Processing Beginners
4

Elaborating on my previous post:

  • I used Movie Maker II with a current version of FFmpeg (see reference at the end) to create the movie (video) and add an audio file.
  • The frames came from the sketch I shared earlier:

    image
    image728×498 21 KB

  • The sketch displays frameCount and changes the background from yellow to red at the 10-second mark for a video that is 30 fps.
  • I used the following command line to offset the audio that was added by 10 seconds which then synced and started with frame 300:
    D:\Users\GLV\Documents\P4\saveFrame_sync_1_0_0\data>ffmpeg -i "test.mp4" -itsoffset 10 -i "test.mp4" -map 0:v -map 1:a -c copy "movie-audio-delayed.mp4
Command line output of above. < Click here to view! 
D:\Users\GLV\Documents\P4\saveFrame_sync_1_0_0\data>ffmpeg -i "test.mp4" -itsoffset 10 -i "test.mp4" -map 0:v -map 1:a -c copy "movie-audio-delayed.mp4
ffmpeg version 7.1.1-full_build-www.gyan.dev Copyright (c) 2000-2025 the FFmpeg developers
  built with gcc 14.2.0 (Rev1, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-lcms2 --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-libdvdnav --enable-libdvdread --enable-sdl2 --enable-libaribb24 --enable-libaribcaption --enable-libdav1d --enable-libdavs2 --enable-libopenjpeg --enable-libquirc --enable-libuavs3d --enable-libxevd --enable-libzvbi --enable-libqrencode --enable-librav1e --enable-libsvtav1 --enable-libvvenc --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxeve --enable-libxvid --enable-libaom --enable-libjxl --enable-libvpx --enable-mediafoundation --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libharfbuzz --enable-liblensfun --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-dxva2 --enable-d3d11va --enable-d3d12va --enable-ffnvcodec --enable-libvpl --enable-nvdec --enable-nvenc --enable-vaapi --enable-libshaderc --enable-vulkan --enable-libplacebo --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libcodec2 --enable-libilbc --enable-libgsm --enable-liblc3 --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
  libavutil      59. 39.100 / 59. 39.100
  libavcodec     61. 19.101 / 61. 19.101
  libavformat    61.  7.100 / 61.  7.100
  libavdevice    61.  3.100 / 61.  3.100
  libavfilter    10.  4.100 / 10.  4.100
  libswscale      8.  3.100 /  8.  3.100
  libswresample   5.  3.100 /  5.  3.100
  libpostproc    58.  3.100 / 58.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'test.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf61.7.100
  Duration: 00:00:20.00, start: 0.000000, bitrate: 112 kb/s
  Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(progressive), 300x300 [SAR 1:1 DAR 1:1], 78 kb/s, 30 fps, 30 tbr, 15360 tbn (default)
      Metadata:
        handler_name    : VideoHandler
        vendor_id       : [0][0][0][0]
        encoder         : Lavc61.19.101 libx264
  Stream #0:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 16000 Hz, mono, fltp, 75 kb/s (default)
      Metadata:
        handler_name    : SoundHandler
        vendor_id       : [0][0][0][0]
Input #1, mov,mp4,m4a,3gp,3g2,mj2, from 'test.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf61.7.100
  Duration: 00:00:20.00, start: 0.000000, bitrate: 112 kb/s
  Stream #1:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(progressive), 300x300 [SAR 1:1 DAR 1:1], 78 kb/s, 30 fps, 30 tbr, 15360 tbn (default)
      Metadata:
        handler_name    : VideoHandler
        vendor_id       : [0][0][0][0]
        encoder         : Lavc61.19.101 libx264
  Stream #1:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 16000 Hz, mono, fltp, 75 kb/s (default)
      Metadata:
        handler_name    : SoundHandler
        vendor_id       : [0][0][0][0]
File 'movie-audio-delayed.mp4' already exists. Overwrite? [y/N] y
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #1:1 -> #0:1 (copy)
Output #0, mp4, to 'movie-audio-delayed.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf61.7.100
  Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(progressive), 300x300 [SAR 1:1 DAR 1:1], q=2-31, 78 kb/s, 30 fps, 30 tbr, 15360 tbn (default)
      Metadata:
        handler_name    : VideoHandler
        vendor_id       : [0][0][0][0]
        encoder         : Lavc61.19.101 libx264
  Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 16000 Hz, mono, fltp, 75 kb/s (default)
      Metadata:
        handler_name    : SoundHandler
        vendor_id       : [0][0][0][0]
Press [q] to stop, [?] for help
[out#0/mp4 @ 00000173c684eac0] video:192KiB audio:74KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: 3.740589%
frame=  600 fps=0.0 q=-1.0 Lsize=     275KiB time=00:00:17.93 bitrate= 125.7kbits/s speed=2e+03x
  • I watched the video and the audio started at frame 300 and frame background went from yellow to red.
  • I closed my eyes and played the video. I paused it when the audio started, and it showed frame 311, about 0.367 s off at 30 fps.
    This was within normal human response time from a bit of research.
    That was fun!

I made this request to ChatGPT to explore FFmpeg commands for audio and video sync:

Provide a list of what FFmpeg can do wrt to adding audio to video. < Click here to view!
ChatGPT said:

Here’s a structured list of what FFmpeg can do regarding adding or managing audio in videos. I’ve grouped it by functionality so it’s easier to see the possibilities:

1. Adding Audio to an Existing Video

  • Add a single audio track to a video without changing the video stream.
  • Add multiple audio tracks to a video (as separate selectable tracks in players).
  • Replace an existing audio track with a new one.
  • Append audio to the end of a video as a new track.
  • Offset audio relative to video (delay or advance).

2. Audio Timing & Synchronization

  • Apply -itsoffset to delay or advance an audio track.
  • Adjust audio start/end times to sync with specific video frames.
  • Trim audio to match video length (-ss and -t options).
  • Loop audio for shorter tracks to match longer videos.

3. Audio Processing / Mixing

  • Mix multiple audio tracks into a single track (amix filter).
  • Adjust volume per track or globally (volume filter).
  • Fade-in or fade-out audio (afade filter).
  • Pan or remix audio channels (pan filter).
  • Normalize or compress audio dynamically (loudnorm or acompressor).

4. Format Conversion

  • Convert audio to a different codec when adding (e.g., MP3 → AAC).
  • Change sample rate or channel layout (mono/stereo/surround).
  • Convert audio file type while adding (e.g., WAV → MP4-compatible AAC).

5. Advanced Effects

  • Add echo, reverb, or other effects (aecho, afir, etc.).
  • Apply audio filters for equalization or noise reduction.
  • Time-stretch audio without changing pitch (atempo).
  • Change audio pitch (asetrate + aresample).

6. Stream Mapping & Control

  • Map audio from one or more sources into specific tracks.
  • Exclude video or audio streams entirely when creating output.
  • Copy audio without re-encoding for lossless speed (-c:a copy).
  • Mix streams with precise control over start times and durations.

7. Testing & Debugging

  • Extract audio from a video (-vn) for editing or analysis.
  • Analyze audio streams for codec, sample rate, and channel layout.
  • Preview audio timing vs. video using short clips.

If you want, I can also make a concise “cheat sheet of FFmpeg commands for adding or syncing audio” — basically a ready-to-use reference for practically every scenario. This is super handy if you do frequent video/audio work.

Do you want me to make that cheat sheet?

I was inspired and already thinking of the possibilities!

This ChatGPT generated FFmpeg command-line instruction creates a movie (video) from the frames and audio file without using Movie Maker:

ffmpeg -framerate 30 -start_number 1 -i "D:\Users\GLV\Documents\P4\saveFrame_sync_1_0_0\data\%06d.png" -itsoffset 10 -i "D:\Users\GLV\Music\WAVs\bcch.wav" -map 0:v -map 1:a -c:v libx264 -pix_fmt yuv420p -c:a aac "movie-from-images.mp4"

I gave ChatGPT very clear instructions and it took a few passes with corrections along the way. It is learning and a useful tool for experienced users!

Future exploration:

  • Log the frameCount at key moments where audio should be added.
  • Use the logged data as markers for audio placement in the video.
  • Create an FFmpeg script to add the audio file(s).
  • Consider automating the process, which is achievable.
  • I am already developing a personal tool for this to support my creator projects.

I have used other tools (Camtasia for example) in the past to sync\add\modify audio but that goes beyond this topic.

If the tool you are using does not work or suit you find one that does or create your own.

I don’t have much more to add, other than… where there’s a will, there’s a way.
Seek and you may find a solution.
If you don’t find one, develop one.
Sometimes the best solutions come from within.
Enjoy the ride!

References:

:)