Confession … I still use perl on the command line

I have a confession to make.

I still use Perl. Regularly. But mostly as a command line tool. Is that a reason? or an excuse?

When I started using Python regularly, I tried to replace my Perl usage.
Nothing against Perl, I just wanted to force myself to use Python to aid in my learning of the language.

I still have quite a few Perl habits that are hard to break.

The trouble is that some things are just so darn easy in Perl (if you’ve already learned it), and not so easy (as far as I know) in Python.

Here’s an example.

When I start writing what I know is going to be a long blog post, I start with a bit of an introduction, followed by a table of contents. The TOC is really an outline of what I want to write. I want to write it first so that the flow of the post makes sense before I start writing the content.

When I’m happy with the TOC, I want to convert it into a bunch of h2 headers.

Since I’m using markdown, it looks a bit like this:

What I want is to throw that code into something that will spit out:

Perhaps there’s an easy way to do this with Python, but I don’t know it.
I usually use Perl, because that’s what I know how to use from the command line.

My Perl solution.

I select the TOC in my editor, and copy to the copy buffer, then paste it on the command line in an echo command and pipe it to a small Perl script, like so:

I know it looks like gobbledygook.
But as I’m writing the regular expression, it makes perfect sense, and I don’t have to think about it much.

After running the command, I select the output, and copy/paste it into my editor.
Not really an efficient process, but better than doing all of the typing myself, especially for long TOCs.

My history with Perl and Python.

I started using Perl in probably 1992, I think (back when the camel book was small). I had to learn it for some university classes, used it for web cgi scripts, and then continued to use it professionally mostly as a if-its-too-complex-for-a-bash-ksh-script scripting language. I also learned to use it as a better-than-sed command line tool. I built a couple of in-house tools with Perl and tk (but… Yuck!).

I started using Python in probably 2001-ish on the job. The remote programming test scripts for the instruments I wrote code for were written in Python, with a custom in-house made test runner. My usage of Python has grown since, and it has supplanted many projects that I would have selected Perl, c++, or bash in the past. Especially if the tool needed any sort of GUI, as wxPython is reasonably painless to use.

However, Perl had almost a decade head start in my neurons, and I think some it may be hardwired now.
Perhaps it’s similar to why I use vim bindings and vim key mappings in sublime text. Really, it’s not me hitting half the keys anymore, it’s my fingers. And they learned vim a long, long time ago. (Right after I tried to write a plugin for Emacs, actually.)

Any suggestions?

I haven’t taken the time to learn to use Python as a command line tool.

Is something like the above example possible in Python?


  1. Eric says

    For something that simple, I usually use sed. As one of perl’s forebears, it’s not quite as powerful, but it’s even more commonly available.

    • says

      In this example, I would definitely agree.
      Since I have encountered so many examples in the past where the regular expression handling in sed (or the special character handling, or multi line search) isn’t enough, I just usually start with perl since its now muscle memory.
      I’m sure it’s probably my limited understanding of sed options/modifiers. But still, there it is.

      • Eric says

        In that case, use the right tool for the job and don’t feel guilty about it. I too have seen a few cases recently that wanted Perl; incrementing the number on the eighth line of a Python program from within a bash script, for example. Lately, though, I end up in the man pages every time I invoke it, so I don’t use it often enough to remember all the requisite details. Perhaps I should reach for it sooner.

    • says

      Sounds like you need a better editor, with regex find/replace support. In vi(m):

      %s /\* \[\(.*\)\](#\(.*\))/\1/

      Very similar to your perl command, but without needing to copy/paste anything. In vim, if you have ‘incsearch’ enabled, you can even test your regex as you type it to make sure you’re getting it right

      • says

        Awesome! This made me chuckle.
        Up until a week ago, I’d be right there with you. I’ve used vim for many years. However, I just switched to sublime, and to be honest, I haven’t figured out how to do this in sublime.

  2. says

    emmet (formerly known as zencoding) is an option to write markup ( html, xml, … ), you will probably find a plugin for the editor of your choice

  3. Simon Hibbs says

    I still use awk and sed too. For me Perl has been relegated to the same category – very powerful file and string manipulation tools mainly used in bash one-liners. Perl is also great for embedding inside procedures within a bash script. Python seems a step or two more removed from the bash environment, in it’s own sandbox, but I find it much easier to reason about complex problems when working in Python.

  4. cesare says

    Here is your “one liner” in python.

    echo ‘* [first section header](#anchor_to_first_section)\n* [second section](#second)’ \
    | python -c ‘import re, sys; m =”\* \[(.*)\]\(#(.*)\)”,; print “%s” % (,’

    So,perl is the right tool for this job, and for other jobs like these: oneliners with regular expressions. But use python for anything bigger that this :-)

    • says

      Wow. Kudos for figuring that one out and getting it one line.
      But… yuk. :)
      However, since I do this frequently, I think I’ll throw your code into a small script file and run that.
      So, thanks.

  5. says

    I enjoyed your post, as I’m in a very similar place at the moment (minus the sublime editor, but you’ve made me curious…). I’m currently presented with a task of renaming files across many directories, and I find myself fighting my perl habits. I’d been taking the cold-turkey approach to python conversion, which has helped me to learn a tremendous amount. Perhaps I’m finally at a level of fluency where ‘the right tool for the job’ is an option. After reading the article, and several of the responses, I must say I feel a little less dirty starting off once again with #!/usr/bin/perl

    • says

      I’m glad other people have similar experiences.
      I know, I course, that using the right tool for the job is fine. Pragmatic and practical, actually.
      However, I do sometimes have a tinge of guilt. Completely unfounded and unnecessary guilt. But it’s still there.
      And it amused me by its presence.

  6. says

    Text processing is so much easier for me in perl than Python. I use perl one-liners to do things like make substitutions in a SQL file, add or remove tags from HTML, and turn simple lists into object instantiating code for another language like C#. In Python, these simple tasks would require several lines or one very long and convoluted line. My coworkers must think I’m masochistic for using perl, but it really does just fit my regular expression thinking brain better.

  7. Gerald says

    I’m tired of this “Perl for small scripts but anything bigger ….” nonsense. Perl is used throughout the I.T. industry for some huge applications and it arguably runs faster than Python. Let’s stop perpetuating these myths about Perl. It now has full OO support via Moose as well.

    • says

      Gerald, I definitely hear what you are saying.
      I’m just giving my opinion based on my experience with both languages.

      I’m sure if you live and breath Perl all the time, it’s great.
      But I have to switch between C++, Python, PHP, javaScript, bash, … and Perl was the oddball that was, in my head, causing a ruckus.
      So, for the most part, I booted it out of my regular language options. Or tried to.

      Also, I do enjoy a spirited debate now and again.

Leave a Reply