Month: June 2016

Bash – opinionated review

Ilya Sher1 Comment

Pros

  1. Huge immediately available “library” of external commands, providing lots of out-of-the-box functionality. You don’t even need to “import” or “require” to get it.
  2. Easy manipulation of files and processes with good syntax for these tasks
  3. Pipes make combining programs very easy
  4. Always installed (OS X comes with version 3 for some reason, version 4 is easily installable).

The pros above make bash the best language for many system tasks (and not Ruby, Perl or Python for example).

Cons

  1. Horrible syntax for general purpose programming tasks (read anything that is not a process or file manipulation), probably consequence of bash not being designed as a programming language. Language features were added with time in a backwards compatible way. The syntax looks really bad.
  2. No nested data structures
  3. No exceptions
  4. No named function parameters
  5. Subshells are forks, preventing global variables changes to be visible outside of the subshell. Combined with very “interesting” rules of what is a subshell and what is not, this behaviour can be surprising.
  6. Default behaviour is not to exit if one of the commands returns an error (use set -e to change)
  7. Default behaviour is to treat unset variables as empty strings (use set -u to change)
  8. Behaviour switches with set – Action at a distance anti-pattern
  9. Default prompt does not include critical information: exit code of the last command
  10. No semantic understanding of the output and hence no command line completion based on output of previous commands. What makes me angry the most, completing apt-get install HERE will use the whole available packages list, not the output of the apt-cache search ... you just used to find your package while most of the time completing based on the output would be the right thing.
  11. History includes the commands and sometimes timestamps but no relevant context such as working directory, relevant variables’ values, exit code, etc…
  12. Inconvenient programming

What’s being done about the numerous cons?

I don’t see much being done about these in bash. Backwards compatibility and the fear of potentially breaking huge amount of existing code is probably the reason.

There are alternative shells being/were developed. Unfortunately for compatibility and other reasons they don’t seem to address all of the cons. Some of this projects ruin the simple syntax for process and file manipulation by basing themselves on an existing general purpose programming language.

against-the-current-1356062_640

NGS, a new and completely different shell, does aim to fix all the cons while keeping the positive aspects of shell programming.

The bigger part of the language is already implemented. See how it looks like.

You are welcome to join the project.

Recommended tools – tmux

Ilya SherLeave a comment

Project URL: https://tmux.github.io/

Tmux is a newer take on screen.

tmux is a terminal multiplexer: it enables a number of terminals to be created, accessed, and controlled from a single screen. tmux may be detached from a screen and continue running in the background, then later reattached.

My usage pattern

And/or:

  1. Locally in one or more of terminal tabs. This keeps number of terminal tabs sane. I mostly use splits and occasionally tabs inside tmux.
  2. Remotely for avoiding multiple ssh connections, critical tasks, poor connection.

I’d recommend these navigation keys:

bind-key -n C-Left select-pane -L
bind-key -n C-Right select-pane -R
bind-key -n C-Up select-pane -U
bind-key -n C-Down select-pane -D

bind-key -n C-S-Left switch-client -p
bind-key -n C-S-Right switch-client -n
bind-key -n S-Left prev
bind-key -n S-Right next
bind-key -n M-j prev
bind-key -n M-k next
bind-key -n C-M-z resize-pane -Z

For a remote machine I suggest setting pane navigation keys to M-Left/Right/Up/Down. This way instead of hitting Ctrl+b C-Left or worse Ctrl+b Ctrl+b Left you just hit Alt+Left (aka Meta+Left).

Pros

  1. Zoom mode.
  2. Saner default prefix key Ctrl+b as opposed to screen’s Ctrl+a. Outside screen Ctrl+a usually jumps to beginning of a line and I use this shortcut frequently.
  3. Multiple copy buffers. See the prefix = shortcut in the manual.

Cons

  1. Mouse selection and scrolling behaves different than in plain terminal. The regular mouse selection can be done with Shift but still…
  2. I have not automated setting up remote machines’ tmux.conf automatically yet. That would be a huge win.

 

Recommended tools – VIM

Ilya Sher3 Comments

Vim is a text editor. http://www.vim.org/

vim
GVim window with horizontal split, no tabs, using darkspectrum theme, viewing NGS project.

My usage pattern

GVim window. Using Vim in it’s own window as opposed to using it inside the terminal prevents unwanted interactions with terminal shortcuts. I use splits, occasionally tabs. Two :colorschemes I use, depending on the light conditions around me are default and darkspectrum (from vim-scripts Debian package).

Notable plugins

  1. CtrlP
  2. Fugitive
  3. Syntastic
  4. Tabular

Pros

  1. Very powerful editing commands – this works well with the dot (.) command for repetition.
  2. Highly customizable
  3. Can save and load the whole layout (open files, tabs, splits) to a file. See :mksession
  4. Supports C well, including showing the warnings and errors inside the editor after you save a file.
  5. Ability to record and play macros.
  6. Many useful plugins, including the one that displays the git branch. The the bottom right part of the screenshot.
  7. Remote files editing.

Cons

  1. Vim is not an IDE. Don’t expect features such as refactoring.
  2. I haven’t found the perfect mechanism for navigation between files in a project. The Ctrl-P plugin is somewhat helpful. :BufExplorer also helps. Still doesn’t feel 100% right.

Using Vim in a terminal with tmux

As I mentioned, I don’t usually use Vim this way but some people seem to be very happy to use the terminal+tmux+vim combo.

Nice video about using Vim with tmux: https://www.youtube.com/watch?v=5r6yzFEXajQ

Why not Emacs?

Vim and Emacs are the two best editors I’ve seen. I don’t think either one is better. They are just very different. I have tried Emacs several times. Last time lasted several months. Each time it just did not feel right for me.

The sudo fraud

Ilya Sher10 Comments

Dear systems engineers,

It really amazes me how people are fine with typing sudo all the time. A kitten is denied a new toy for another day when you do this!

cat-96877_640

Typing sudo locally all the time

Is it really simpler for you to type sudo all the time rather than having one terminal tab open with a root shell? Besides, some systems even ask for a password when you run a sudo command. Be honest with yourself, are you a masochist?

Using sudo on servers

Security

Intro: each Amazon image comes with standard username for logging in. Never seen anyone changing that username.

Supposedly, the attacker would need to know the username in addition to your stolen private key. Right… and it’s not one of these: ubuntu, admin, ec2-user, centos … and looking at your ssh banner won’t give any clue as to which username is used:

SSH-2.0-OpenSSH_... Ubuntu..
SSH-2.0-OpenSSH_... Debian...
SSH-2.0-OpenSSH_6.6.1

OK, Amazon Linux did some homework but who uses it anyway. Red Hat also doesn’t say what it is. Both use ec2-user… You could customize the username but did you? So much for knowing the username.

Or maybe you think the potential attacker would not be able to run  sudo -i ?

“It stops you from doing stupid things”

Really? Do you do a mental pause and re-read your command once again when you type sudo? I’ve seen this not happening. If you are like me and you manage the servers, a big portion of commands require root, so people are just typing sudo without thinking.

Better train yourself to make a mental pause before executing destructive commands.

Conclusion

We could launch an another space mission if we would use the time people wasted on typing sudo . If you manage a server, use root. It really saves time. Do you really enjoy redoing half of your commands after “oh.. I forgot sudo … again”?

Are you afraid to kill the server? If killing one server is such a great deal, your situation might really be special but chances are that you just don’t manage your servers right. Fix that instead of typing sudo again and again.

And guys… have fun with your servers!

Update 2016-07-09 05:58 UTC

Just to be clear: Please note it’s filed under the “Rant” category. The post has some valid points but don’t take it too seriously. Use your best judgment to decide which parts of the above apply to your situation.

Bashing bash – unexceptional

Ilya SherLeave a comment

This is the third post in the “bashing bash” series. The aim of the series is to highlight problems in bash and convince systems and software engineers to help building a better alternative.

The problem

What is the output of the following script?

#!/bin/bash

set -eu

echo Here
false
echo Not here

Right, the output is

Here

Imagine that instead of false you have a lot of code. Since there are no exceptions, you have no idea where the error occurred.

inspector-160143_640

Solutions using bash:

  1. Use set -x to trace the code.
  2. Add echo something every here and there to know between which two echo‘s the error occurred.
  3. Catch the error using trap and print the line number as suggested on StackOverflow . Writing this additional catching snippet at the top of every script is not really convenient.

This problem of unclear error location is unimaginable in any normal programming language.

“There is no problem, just don’t do it”

Bash was not intended to be a “normal” programming language. Some people say it’s an abuse of bash to use it as such. Looking at the code written in Bash I can tell it really is an abuse in many cases.

The reality though is that bash is still (ab)used for programming. In some cases Bash has positive aspects which outweigh the need to use other languages. In other cases a program starts as a small Bash script and is just not rewritten in another language after the script grows.

I suggest making a better shell rather than convincing people not to abuse Bash. People will keep on doing what they are doing. Let’s make their lives easier by providing them with a better shell.

The suggested solution

Use NGS. In NGS, any failed process throws an exception. Let’s take a look at the script below

#!/usr/bin/env ngs
echo Here
false
echo Not here

What’s the output?

Here
Not here

WAT?

Well, actually false returning an exit code of 1 is not an exception, it’s normal. If any command returning non-zero code would cause an exception, you wouldn’t be able to write for example if $(test -e myfile) do_something .

Failed process is a process that returns an unexpected exit code. Here is the part of stdlib that defines what’s a fail and what’s not:

F finished_ok(p:Process) p.exit_code == 0

F finished_ok(p:Process) {
    guard p.executable.path == '/bin/false'
    p.exit_code == 1
}

F finished_ok(p:Process) {
    guard p.executable.path == '/usr/bin/test'
    p.exit_code in [0, 1]
}

Such definitions also mean that you can easily extend NGS to work properly with any other command, simply by adding another finished_ok function. (Or add it to stdlib if it’s a common command so everyone would benefit).

So where are the exceptions?

We’ll have to modify the code to get an unexpected exit code. Example:

#!/usr/bin/env ngs
echo Here
ls nosuchfile
echo Not here

Output:

Here
ls: cannot access 'nosuchfile': No such file or directory
========= Uncaught exception of type 'ProcessFailed' =========
====== Exception of type 'ProcessFailed' ======
=== [ backtrace ] ===
[Frame #0] /etc/ngs/bootstrap.ngs:158:1 - 158:10 [in <anonymous>]
[Frame #1] /etc/ngs/bootstrap.ngs:154:17 - 154:29 [in bootstrap]
[Frame #2] ./2.ngs:3:4 - 3:14 [in <anonymous>]
[Frame #3] /usr/share/ngs/stdlib.ngs:1116:11 - 1116:15 [in $()]
[Frame #4] /usr/share/ngs/stdlib.ngs:1050:29 - 1050:42 [in wait]
[Frame #5] /usr/share/ngs/stdlib.ngs:1006:7 - 1006:20 [in ProcessFailed]
=== [ dump process ] ===
(a lot of not very well formatted output with info about the process)

Please help building a better alternative

Go to https://github.com/ilyash/ngs/ and contribute some code.