Richard Siggs & Network Presence

I’m an “old school” Unix user, who grew up on sed, awk & Sendmail, but many people still use sed & its syntax from the Unix or Linux command-line daily, so this is a quick note on how to use ‘sed’ when you’re working with URLs.

The ‘trick’ here (if there is one) is that most sed examples are for using it to substitute characters in a text line or file.
eg: changing http://this-site/ to http://that-site/
But the usual delimiter of the text to be changed is the ‘/’ (slash or divide-sign) character, which simply doesn’t work when the text your substituting also contains the ‘/’ (ie: URLs).

Please you don’t have to use the ‘/’ to delimit search & replace text in sed command lines, you can use (just about/within reason) any single character you like (eg: the ‘%’ (percent-sign) character), just as long as you use it consistently in your sed command line.

So if you’re using sed to substitute text in a URL, use the % to delimit the text to search for and the text to replace it with.
Here’s some examples:

Eg 1: We want to change all references of http://google.com to https://google.com in an inputfile, so a sed command line for this may be:

sed 's%http://google.com%https://google.com%g' inputfile > outputfile

Eg 2: You have a shell variable containing a URL, say $thisurl which you want to change to the contents of another variable $thaturl :

sed "s%$thisurl%$thaturl%g"

Note: the use of the double-quotes (“) character to allow shell variable expansion within the sed command.

Don’t forget that “man sed” is your friend & there’s an O’Reilly “sed” book too on it for much “sed goodness”.