Samuel's Tech Blog

This is something I've been thinking of for a looong long time: Creating a course covering my favourite linux commandline productivity techniques! 

The background is that during my year at UPPMAX super computing center, where we were workikng all days in the terminal, I started looking for the perfect set of easy to implement techniques, that could save me the most time and unneccessary strains in the wrists etc, without getting me drowned in complicated setups that would not work on all computers etc. So, I started thinking about gathering all these lessons in a course.

Not until this summer, when I discovered the awesome udemy platform, did I get it done though (and many months later, after I got the last bits finished, it got published)!

Guess if I was happy about the very first review I got, was a full five star! :), reading "Excellent course. I especially like the coverage of 'screen' and 'terminator'. Ben"):

Anyways, find below a special 50% off coupon just for you my blog readers (with life-time access to the course). Hope you will enjoy the course as well:

• » View the course with the 50% off blog readers coupon!

(Note: Erroneous link fixed on Feb 14, 2015. Contact me if you used the wrong link by mistake!)

Hope you will enjoy the course as much as I enjoyed creating it! :)

Somebody told me they always use US keyboard for coding, and that it really makes a difference. I typically don't care too much about such, but when I started getting a slight pain in my wrists the other day, after many days of intense bash-/vim-/screening, I decided to make a try ... and my hands and wrists are forever grateful!

Finally it starts to make sense why somebody would choose a weird character such as " for switching between windows in screen! (something you do very repeateadly sometimes), but having it just one step to the right of the natural position of your right pinkie, makes more sense than the awful crippled left-hand grip to get [shift]-[2] on the swedish keyboard.

Coders in general will save some sanity by avoid the crippled right hand-grip with AltGr+[numkey] to get to [] and {}.

The rest is maybe a little more specific to heavy bash/vim/screen usage. I'll outline a few that I noticed below:

• :, for the ubiquitous :w in vim (for saving( is very comfy placed under your right pinkie (with shift).
• The $ sign is much comfier (for selecting to end of line in vim), avoiding another crippled AltGr-grip.
• Same goes for the | sign (piping)
• /, for searching in vim, is really nicely placed one step down and right from the right pinkie.
• " is nicely placed just one step to the right of the pinkie-placement.
• Even ~, even though requiring a slightly crippled left-hand grip even in US, at least prints out directly, as opposed to the required space to force it to print, in SE.

I'm sure there are many more, but these should be enough to make it worth to try it out a little if you're spending a lot of time in bash/vim/screen. In short: Only now I can see how anyone can really love working with these vim commands!

Only thing remaining is to find a nice place to map the åäö:s.

So far, I have added these two aliases to my ~/.bash_aliases, though, for quick switching between Swedish and US:

alias kbse='setxkbmap se'
alias kbus='setxkbmap us'

I often need to develop python scripts on some remote server where I can't run graphical python IDEs like spyder.

I'm too lazy to set up an advanced vim config with full blown IDE-like features (except for some basic python support).

I have found that a much simpler solution is a GNU screen session with two vertically split screens, one for the main coding in vim, and a smalelr one below, for running ipython, to run and debug the script as it is written.

I found it useful enough to figure out how to start such a setup with one command. This is how to do it:

create a file ~/.screenrc.pydev and place the following in it:

split
screen
focus
screen
resize 20
exec ipython
focus
exec vim

then add this to the bottom of your ~/.bashrc or ~/.bash_aliases:

alias pydev='screen -mS PyDev -c ~/.screenrc.pydev'

... and source the file:

source ~/.bashrc

Now, you can start your command line python environment by:

cd some-folder-with-python-files
pydev

The result:

Oh, and to jump between the screen windows, do:

[Ctrl] + [A], [Tab]

For formatting a 3-digit integer (that is, 2 leading zeros for 1-9):

printf %03d $i

... where $i is the number in question.

Grepping for stuff in MySQL dumps is not that nice, with miles-wide lines. You could send the grep output to a command such as "cut -c 1-200", but that would still not be guaranteed to give you the actual matched content.

Enter the "fold" command, which formats output into lines with a max count of chars:

grep "stuff" sqldump.sql | fold -w 200 | grep -C 1 "stuff"

... will give you a much better view of the context of the match!

(The first grep gets the (mile-wide) line that has the match, then fold will split the mile-wide line into 200 char long lines, and "grep -C 1" will show only the one 200 char wide line where the match is + 1 line of context before and after).

GNU Screen is a nice little program, allowing you to have "terminals" that you can detach in the background, so that you can have long batch jobs started, outputting stuff to the stdout, for example, but still don't be afraid to close down your terminal by accident etc.

Unfortunately screen has, IMO, quite an awkward syntax, but I managed to learn 3 flag combinations, and two keyboard combinations (from inside screen) that seems to be what I need for basic usage of screen:

Flags

Start a new named screen session:

screen -dmS ASessionName

screen -ls

screen -r ASessionName

Shortcuts

Detach the current session in background:

Ctrl + a, Ctrl + d

Ctrl + d

My work at UPPMAX, on the Bioclipse based HPC Client i is progressing, slowly but steadily. I just screencasted an experimental version of the job configuration wizard, which loads command line tool definitions from the Galaxy workbench, and use them to generate a GUI for configuring the parameters to the command line tool in question, as well as the parameters for the Slurm Resource manager (used at UPPMAX). Have a look if you want :) :

The Wizard obviously has quite some rough edges still. My current TODO is as follows:

• Set sensible default values i widgets (i.e. when there is just 1 alt)
• Use checkboxes and radiobuttons for select fields with few options
• Use progress bar between wizard pages that takes time to load
• Decide how to take care of the cheetah #if#lse#endif syntax, available in some galaxy tool config files.
• Add validation
• Use a time widget for the job time
• Add a custom view with just a "connect" button, and showing only remote files for the configured host.
• More modular loading of modules (hierarchical etc.)
• More advanced parsing of options (i.e. allowing to omit params, rather than just saying "no" on them).

Etc ... More suggestions? :)

Install xmlstarlet in Ubuntu:

sudo apt-get install xmlstarlet

Use the formatting command:

[code that produces some xml] | xmlstarlet fo

... or, if you are generating the XML with an XSLT stylesheet, don't forget the following line, after the xsl:stylesheet tag:

<xsl:output method="xml" indent="yes" encoding="UTF-8" />