Delete links across multiple HTML files in bash

Posted on December 5, 2011

Or any type of file, really. I needed to quickly remove links from an old website that was using flat HTML files. In my linux command line, I found I could do:

perl -pi -e 's/SEARCH/REPLACE/g' *.html

To replace all instances of SEARCH with REPLACE in *.html.

Except I needed to do a fair bit of escaping, because HTML is full of characters that mean something else on the command line.

So let’s say the string I needed to remove was:

<a title="Search Engine Optimisation" href="http://superspammyseocompany.com/" target="_self"><span>Search Engine Optimisation</span></a> by <a title="Super Spammy SEO Company" href="http://superspammyseocompany.com/" target="_self">Super Spammy SEO Company</a>

I copy + pasted this into vim, and then every time these characters occur:

< , >, / and ”

I put a \ in front of each of these, which gave me:

\<a title=\"Search Engine Optimisation\" href=\"http:\/\/superspammyseocompany.com\/\" target=\"_self\"\>\<span\>Search Engine Optimisation\<\/span\>\<\/a\> by \<a title=\"Super Spammy SEO Company\" href=\"http:\/\/superspammyseocompany.com\/\" target=\"_self\">Super Spammy SEO Company\<\/a\>

Which was a bit of work, but still much more fun than manually removing the link from each file.

Note that these characters do not need to be escaped with a backslash:

= (equals), . (dot), and  _ (underscore)

So my final command was:

perl -pi -e 's\\<a title=\"Search Engine Optimisation\" href=\"http:\/\/superspammyseocompany.com\/\" target=\"_self\"\>\<span\>Search Engine Optimisation\<\/span\>\<\/a\> by \<a title=\"Super Spammy SEO Company\" href=\"http:\/\/superspammyseocompany.com\/\" target=\"_self\">Super Spammy SEO Company\<\/a\>//' *.html

I’d already initialised a git repository and committed the files so I could easily restore the files in case of a mistake. A quick look through the links showed it all worked perfectly, and it saved me so much time I thought I’d write this post about it.

Bonus: I outputted all the changed files to list.html, which had one filename per line, like:

./file1.html
./file2.html
./file3.html

Here’s the vim command to turn them all into links, for easy human checking:

:%s/^\(.*\)$/<a href="\1">\1\<\/a\>\<\/br\>

Hostmonster 500 Internal Server Error (SHTML Wrapper)

Posted on November 1, 2011

A client using Hostmonster was doing some hair-pulling over what appeared to be a .htaccess problem while installing a Symphony site I developed. We thought it might be related to the fact it was running on a subdomain.

The server would give a blank page, and on the HTML source would show something like “500 SHTML wrapper”.

It turns out this happens when group write permissions are set on the script. Via the Hostmonster Cpanel File Manager, you can right-click the file, click permissions and uncheck “write” and the script should run.

The way I worked this out was: first, rename .htaccess to something like _htaccess, so we know that’s not getting in the way.

Next try viewing a file like favicon.ico (this was served correctly). Then upload a simple PHP script like phpinfo.php, with contents:

<?php phpinfo(); ?>

When I did this I was still getting the 500 error. I found in a Hostmonster forum someone had mentioned group write permissions, which I started experimenting with until it worked.

Batch reduce resolution of images in Ubuntu Linux

Posted on February 22, 2009

Open a terminal window or shell (who needs a GUI!?)
sudo apt-get install imagemagick
cd image_folder
mkdir resized
for f in *jpg
do
convert -resize 30% $f resized/$f
done

Easier than Photoshop, I reckon!

I needed this today for some photos I was uploading. Photos from recent cameras are huge and Australian bandwidth doesn’t cut it. So this really helps.

Obviously you only need to do the first line once. I found this in the Ubuntu forums but it took me a while to find the right combination of keywords in Google. View the original thread.

I needed to change jpg to JPG, because the camera used uppercase and Linux is case-sensitive. 30% was good for me but you may need to change that.

If you don’t understand this, ask me, I’m happy to help!

Fixing Windows File Permissions in Linux

Posted on February 20, 2009

Find all files (not directories) and remove the eXecutable bit. I don’t know what the characters on the end do:
find . -type f -exec chmod a-x {} \;

Make all directories executable to user, read/writable to groups others (good for when the web server is in ‘others’):
find . -type d -exec chmod 755 {} \;

Make all files read/write/executable for user, readable for the group and others:
find . -type f -exec chmod 644 {} \;

Unix Time in MySQL and bash

Posted on February 4, 2009

A quick one today. I was working on a mysql database that used unix timestamp produced by PHP’s time() function.

I needed to be able to quickly convert this time to a human-readable format. In bash,

date -d @timestamp

is a quick way to convert.

In a terminal shell eg:

# date -d @1224992980
Sun Oct 26 14:49:40 EST 2008

In a MySQL client, you could also use

select date(from_unixtime(column_name)) from table_name;

Or if you want a little more flexibility in the output, for example outputting 27/02/09, you could do:

select date_format(from_unixtime(column_name), ‘%d/%m/%y’) from table_name;

This post is one of my most popular posts. Did you find the information you were after? Please tell me in the comments!