Various Command Line Stuff
There are a few things I do regularly enough that I want to remember them but infrequently enough to remember them reliably. This is a partial list of CLI commands for doing various (somewhat random) things.
Shell Commands
To change the permissions on all directories recursively without affecting the permissions on files:
$ find . -type d -exec chmod 664 {} +
To change the permissions on all files recursively without affecting the permissions on directories:
$ find . -type f -exec chmod 775 {} +
Perl One-liners
Note: Examples use single quotes. On Windows systems, use double quotes.
Replace <old text> (a regex pattern) with <new text> in all html files in current directory. Ignore case.
$ perl -pi -e 's/<old text>/<new text>/gi' *.html
Replace ^M characters with new lines in text.txt.
$ perl -pi -e 's/\x0d\x5c/\n/g' text.txt
Here's one that changes file names from "photo-??.tif" to "photo_2_??.tif" (where ?? are two digits). Enter all on one line.
$ perl -e 'for (@ARGV) { rename($_, $n) if ( $n=$_) =~ s/photo-(\d\d)\.tif/photo_2_$1.tif/ }' *.tif
Note that the above line won't work in a Windows cmd session. That's becase Windows doesn't expand the *.tif at the end to a list of file names but simply passes it to the script as a literal string. There are a few ways to make this work and this is a pretty simple one. The command as written will rename files with a .tif extension to use the 2-f version, .tiff. Note that the Perl code is enclosed in double quotes rather than single quotes.
$ perl -e "@args = glob qq|@ARGV|;for (@args) { rename($_, $n) if ( $n=$_) =~ s/\.tif$/.tiff/ }" *.tif
This isn't quite a one liner, because it uses the perl rename.pl script (see below) to rename numbered files, reversing the order of the numbers. Since they are not always numbered starting with one, this command assumes you know the highest number used. Double that number and add one for the <max> number in the command below. Also, this command assumes four digit numbers. Adjust %04d accordingly, if not. Enter all on one line.
$ ./rename.pl 's/img_(\d+)/"A_img_".sprintf("%04d",(<max>-$1))/xe' *.jpg && ./rename.pl 's/A_//' *.jpg
The rename.pl script is easily found by searching the web, but here is the body of the script, without comments.
#!/usr/bin/perl $op = shift or die "\nUsage: rename.pl expr [files]\n\n" ; chomp(@ARGV =) unless @ARGV; for ( @ARGV ) { $was = $_ ; eval $op ; die &@ if $@ ; rename($was, $_) unless $was eq $_ ; }
ImageMagick
Cut a multipage PDF file into pieces, two per original page. This was used to take a document that was created with two pages per side to be folded into a booklet and make individual pages.
$ convert -density 600 <infile.pdf> -crop 3300x5100 +repage +adjoin <outfile_%02d.pdf>
Then, after renaming the files into the proper page order, this command will reassemble them into a single PDF:
$ convert -density 150 <outfile_??.pdf> -adjoin -compress JPEG <newfile_150.pdf>
Here's a line for converting a series of still images in jpeg format into a single animated gif. The -loop switch set to 0 (zero) causes the animation to loop continuously. The delay set to 1/30 means each frame plays for 1/30 of a second. Without the denominator, it woudl be in hundredths of a second, so -delay 10 would be 1/10 second per frame.
$ convert frame_*.jpg -delay 1/30 -size 852x480 -loop 0 animation.gif
FFMPEG
Convert a file from m4a format to mp3 format. If the file name contains spaces, it needs to be entered between quotation marks. If in doubt, rename or copy the file to something short and without spaces.
$ ffmpeg -i <infile.m4a> -acodec libmp3lame <outfile.mp3>
Convert a file from wav to mp3 format with a variable bitrate (VBR). The value for the -qscale:a parameter is an integer from 0 to 9 where a lower value is a higher quality. 0-3 will normally produce transparent results. Much more information can be found at FFmpeg MP3 Encoding Guide.
$ ffmpeg -i <infile.wav> -codec:a libmp3lame -qscale:a 3 <outfile.mp3>
Convert a MOV file from my camera to mp4 format. This requires re-encodeing the audio.
$ ffmpeg -i <infile.mov> -s hd720 -qscale:v 2 -codec:a aac -strict experimental -ab 128k <outfile.mp4>
Convert a file from mp4 format to ogv (Ogg Vorbis) format for use with the HTML5 <video> tag (found and explained here).
$ ffmpeg -i "<infile.mp4>" -acodec libvorbis -vcodec libtheora -ac 2 -ab 96k -ar 44100 -b 819200 -s 1080x720 "<outfile.ogv>"
Extract the first, third, and fifth streams (numbered 1, 2, and 4), corresponding with the video, audio, and subtitle streams in the source, and write them to a new mp4 file.
$ ffmpeg -i "<infile>" -map 0:0 -map 0:2 -codec:a aac -strict experimental -ab 128k -map 0:4 -scodec mov_text "<outfile>"
Extract a segment from one video file, leaving everything else the same. In this case, we'll be extracting starting at 4 minutes, 27 seconds into the video and taking 2 minutes, 1.5 second from that point. Note the two forms of specifying time. You may use them interchangeably. The first gives hours:minutes:seconds while the second give just seconds.
$ ffmpeg -i <infile> -vcodec copy -acodec copy -ss 00:04:27 -t 121.5 <outfile> $ ffmpeg -i <infile> -codec:v:0 copy -ss 00:04:27 -t 121.5 <outfile>
Extract a 30 second segment from a video file, starting 85 seconds into it and add a 1.5 second fade in and a 1 second fade out to the extracted segment. These two lines create mp4 and ogg versions of the original MOV file. Note that the 'st' value for the fade out is the sum of the starting offset (-ss) and the duration (-t) minus the sum of the two fades. In this example, ( 85 + 30 ) - ( 1.5 + 1.0 ) = 115 - 2.5 = 112.5
$ ffmpeg -i <infile.MOV> -ss 85 -t 30 -filter:audio afade=t=in:st=85:d=1.5,afade=t=out:st=112.5:d=1.0 -pix_fmt yuvj420p -codec:a aac -strict experimental -ab 128k <outfile.mp4> $ ffmpeg -i <infile.MOV> -ss 85 -t 30 -filter:audio afade=t=in:st=85:d=1.5,afade=t=out:st=112.5:d=1.0 -qscale:v 2 -codec:v libtheora -codec:a libvorbis <outfile.ogg>
Make jpeg images from video frames. The -s value (hd720 in this example) is the frame size to create. A table of size codes can be found on the FFmpeg-Utils documentation page). The -qscale:v value should be from 1 to 31 with 1 being the best quality and 31 the worst. This example only takes frames from the 10 seconds starting at 3 minutes 12.5 seconds into the video.
$ ffmpeg -i <infile> -s hd720 -qscale:v 2 -ss 00:03:12.5 -t 10.0 <outfile-%03d.jpg>
To go the other way, making a video from a bunch of numbered jpeg files, use this command. In this case I'm creating an avi file in mp4 format. Because you don't always want to include all images or images starting at number 0, this example uses the -start_number switch to start with image number 115 and the -vframes switch to limit the sequence to 100 frames.
$ ffmpeg -start_number 115 -i <infile-%03d.jpg> -vcodec mpeg4 -vframes 100 <outfile.avi>
I had a video that was 1920 by 1080 but the actual image was squeezed horizontally, with black bands on each end. I needed to crop the middle out and then stretch it back to the original dimensions. This command will pull out a 1438 by 1080 pixel portion of the video from 241 pixels in from the left (and 0 pixels down from the top). It will then stretch that video back to the original 1920x1080 pixel dimension. For the full list of filters, see FFmpeg Filters Documentation.
$ ffmpeg -i <infile.mkv> -vf "crop=1438:1080:241:0, scale=1920:1080" <outfile.mkv>
I had five videos that needed to be concatenated together into a single whole. This was done in a two-part process. First, each file is converted to a .ts file. Then, those five .ts files are concatenated back into an .mp4
$ ffmpeg -i ep1.mp4 -c copy -bsf h264_mp4toannexb out1.ts
$ ffmpeg -i ep2.mp4 -c copy -bsf h264_mp4toannexb out2.ts
$ ffmpeg -i ep3.mp4 -c copy -bsf h264_mp4toannexb out3.ts
$ ffmpeg -i ep4.mp4 -c copy -bsf h264_mp4toannexb out4.ts
$ ffmpeg -i ep5.mp4 -c copy -bsf h264_mp4toannexb out5.ts
$ ffmpeg -i "concat:out1.ts|out2.ts|out3.ts|out4.ts|out5.ts" -c copy
-absf aac_adtstoasc epAll.mp4
OR (for a longer list of files)
$ perl -e 'system(sprintf(qq|ffmpeg -i ep%02d.mp4 -c copy -bsf h264_mp4toannexb out%02d.ts|,$_,$_)for(1..12)'
$ perl -e '$out .= sprintf("out%02d.ts|",$_) for (1..12);chop $out;system(qq|ffmpeg -i "concat:$out" -c copy -absf aac_adtstoasc epAll.mp4|);'
Another way to concatenate files is to put the filenames in a text file like this:
file file-1.mp4 file file-2.mp4 file file-3.mp4 file file-4.mp4 file file-5.mp4 file file-6.mp4Then use the following command, where files.txt is the name of the file with the names in it.
ffmpeg -f concat -i files.txt -vcodec [whatever] -acodec [whatever] combined.mp4
To flip a file horizontally, use the hflip filter.
$ ffmpeg -i "<backwards-file.mp4>" -vf hflip -c:a copy "<forwards-file.mp4>"
MKVTools
Get the current info from and then fix the default aspect ratio for an MKV video file. In the example shown, the display-height was set wrong, making the video play conpressed from side to side. This resets it to match the pixel height (so the display size is 1280x720 (16:9). First use mkvinfo to look for lines specifying Pixel width / height and Display width / height:
$ mkvinfo ./video.mkv | more
Then, use the mkvpropedit command to change the relevant property, in this case the display height:
$ mkvpropedit ./video.mkv --edit track:1 --set display-height=720
Tesseract
Tesseract is optical character recognition (OCR) software. If you don't have it installed, binary packages can be downloaded from http://pkgs.org/download/tesseract. You will also need libpng-devel, zlib, and leptonica. I also needed libwebp. I found binaries of leptonica and libwebp, the two packages I didn't already have, on pkgs.org, also. I downloaded rpms for them and did a yum localinstall and everything worked first time.
This example is a perl one-liner that repeatedly executes a tesseract one-liner to convert a series of scanned images to text. Because Tesseract (or more precisely Leptonica) does not handle alpha-channels in TIFF files, it seems easier to save the files as PNG instead. For this example, I have subdirectories called png and txt to hold the source and target files, respectively. Note that Tesseract will add the .txt extension to the target file so you don't need to include it in the command. In this case, I am scanning 20 images named image-01.png through image-20.png.
$ perl -e 'system(sprintf(qq|tesseract ./png/image-%02d.png ./txt/text-%02d|,$_,$_))for(1..20)'
I would follow that up with another that will combine the text files into a single text file. Either of these two commands should work. While the first will probably work fine, the second will insure the files are added in numerical order, so that's my preference.
$ cat ./txt/text-??.txt > alltext.txt $ perl -e 'system(sprintf(qq|cat ./txt/text-%02d|,$_))for(1..20)' > alltext.txt