Overview

Teaching: 0 min
Exercises: 0 min

Questions

• What else is there?

Objectives

• review some content that was too important to skip but didn’t fit in our lessons

Bonus Content

Bash as a scripting language

Bash is far more than a terminal, it’s a whole language in it’s own right. What we use as our terminal is equivalent to the interpreter we interact with when we type python3 on the command line. A few key things to know about bash to get you started are:

• variables can be set like fname=myfile.csv (with no spaces around the =)
• variables can be accessed using ${fname} or the slightly more relaxed $fname
• A file that starts with #! /usr/bin/env bash and has execute permissions (chmod +x <filename>) can be run directly
• flow control uses if ... then ... else .. fi syntax
• new line continuation is accomplished by having \ as the last character on the line
• converseley, multiple commands can be used on a line, separate them with ;
• program output can be redirected to a file with > (replace) or >> (append)

For more info see this wikibook for a nice intro.

Depending on the task you are trying to accomplish, a bash script can be easier to work with than a python script.

Automation with aliases

A basic form of automation is the use of aliases. The goal here is to save you a bunch of keyboard or mouse clicks. Some common examples:

# long list all files
alias ll='ls -al'

# open ds9 and press about 20 buttons to get the interface set up just the way I like
alias ds9='~/Software/ds9/ds9 -scalelims -0.2 1 -tile -cmap cubehelix0 -lock frame wcs -lock scale yes -lock colorbar yes -lock crosshair wcs'

# run stilts and topcat and not have to bother with the java invocation
alias stilts='java -jar ~/Software/topcat/topcat-full.jar -stilts'
alias topcat='java -jar ~/Software/topcat/topcat-full.jar'

# activate my favorite python3 environment
alias py3='source ~/.py3/bin/activate'

Structured data storage

CSV files are great but can be cumbersome to work with when they become large or when you have multiple files with related data. An alternative is to store your data as tables in a database, where you can make explicit links between multiple tables, and make use of the SQL language to select, filter, order, and extract the data that you need. Some commercial (but free) solutions include PostgreSQL and MySQL, but these require a fair bit of system admin to get setup and work with. A great lightweight middle-ground is SQLite, which provides the core functionality of a relational database but without all the security, authentication, and sys admin requirements of a commercial solution. See this carpentries lesson for an introduction do databases using SQLite.

Task prioritization

In most cases you will have more tasks than time available to do said tasks. You will therefore need some way of determining which tasks should be done and which can be left undone or done at a later time. A common metric to decide on the order to do tasks is the priority matrix:

You can incorporate the above priority matrix into your work planning in a number of ways:

• Use these four categories as columns in your kanban board
• Create labels with corresponding name/colour to add to each of the tasks in your kanban board
• Create swimlanes for your kanban board that correspond to these categories (though tasks may need to change lanes!)

An alternative priority matrix which could be useful for research work is one where you consider impact (research output) and effort (time to complete). This is useful when considering which projects to take on and their goals. Successful researchers are often skilled at identifying projects that are high impact with minimal effort.

Key Points

• there is always more to learn