bashing-bash / NGS / Programming / System administration

Why I have no favorite programming language

Ilya Sher13 Comments

TL;DR – because for me there is no good programming language.

I’m doing mostly systems engineering tasks. I manage resources in Cloud and on Linux machines mostly. I can almost hear your neurons firing half a dozen names of programming languages. I do realize that they are used by many people for systems engineering tasks:

  • Go
  • Python
  • Ruby
  • Perl
  • bash

The purpose of this post is not to diminish the value of these languages; the purpose is to share why I don’t want to use any of the languages above when I write one of my systems-engineering-task scripts. My hope is that if my points resonate with you, the reader, you might want to help spread the word about or even help with my suggested solution described towards the end.

man-2756206_640

So let’s go over and see why I don’t pick one of the languages:

Why not language X?

All languages

  • Missing smart handling of exit codes of external processes. Example in bash: if test -f my_file (file is not there, exit code 1) vs if test --f my_file (syntax error, exit code 2). If you don’t spot the syntax error with your eyes, everything behaves as if the file does not exist.
  • Missing declarative primitives libraries (for Cloud resources and local resources such as files and users). Correction: maybe found one, in Perl – (R)?ex ; unfortunately it’s not clear from the documentation how close it is to my ideas.

All languages except bash

  • Inconvenient/verbose work with files and processes. Yes, there are libraries for that but there is no syntax for that, which would be much more convenient. Never seen something that could compare to my_process > my_file or echo my_flag > my_file .

Go

  • Compiled
  • Error handling is a must. When I write a small script, it’s more important for me for it to be concise than to handle all possible failures; in many cases I prefer an exception over twice-the-size script. I do understand how mandatory and explicit error handling can be a good thing for larger programs or programs with greater stability requirements.
  • Dependencies problem seem to be unresolved issue

Python

  • Functional programming is second level citizen. In particular list/dictionary comprehension is the Pythonic way while I prefer map and filter. Yes, that’s probably one of the features that makes Python easier to learn and suggested first language. Not everything that’s optimized for beginners must be good for more experienced users. It’s OK.
  • Mixed feelings about array[slice:syntax] . It’s helpful but slice:syntax is only applicable inside [ ] , in other places you must use slice(...) function to create the same slice object

Ruby and Perl

  • The Sigils syntax does not resonate with me.

Ruby

I can’t put my finger on something specific but Ruby does not feel right for me.

Perl

  • Contexts and automatic flattening of lists in some cases make the language more complicated than it should.
  • Object orientation is an afterthought.
  • Functions that return success status. I prefer exceptions. Not the default behaviour in Perl but an afterthought: autodie.
  • Overall syntax feeling (strictly matter of personal taste).

bash

Note that bash was created in a world that was vastly different from the world today: different needs, tasks, languages to take inspiration from.

  • Missing data structures (flat arrays and hashes is not nearly enough). jq is a workaround, not a solution in my eyes.
  • Awkward error handling with default of ignoring the errors completely (proved to be bad idea)
  • Expansion of undefined variable to empty string (proved to be bad idea)
  • -e ,  -u and other action at a distance options.
  • Unchecked facts but my feelings:
    • When bash was created, it was not assumed that bash will be used for any complex scripting.
    • bash was never “designed” as a language, started with simple commands execution and other features were just bolted on as time goes by while complete redesign and rewrite were off the table, presumably for compatibility.
  • Syntax
  • No widely used libraries (except few for init scripts) and no central code repository to search for modules (Correct me if I’m wrong here. I haven’t heard of these).

My suggested solution

I would like to fill the gap. We have systems-engineering-tasks oriented language: bash. We have quite a few modern programming languages. What we don’t have is a language that is both modern and systems-engineering-tasks oriented. That’s exactly what I’m working on: Next Generation Shell. NGS is a fully fledged programming language with domain specific syntax and features. NGS tries to avoid the issues listed above.

Expected questions and my answers

People work with existing languages and tools. Why do you need something else?

  • I assume I have lower bullshit tolerance than many others. Some people might consider it to be normal to build more and more workarounds (especially around anemic DSLs) where I say “fuck this tool, I would already finish the task without it (preferably using appropriate scripting language)”. I don’t blame other people for understandable desire to work with “standard” tools. I think it’s not worth it when the solutions become too convoluted.
  • I am technically able to write a new programming language that solves my problems better than other languages.

Another programming language? Really? We have plenty already.

  • I would like to remind you that most of the programming languages were born out of dissatisfaction with existing ones.
  • Do you assume that we are at global maximum regarding the languages that we have and no better language can be made?

Feedback

Would you use NGS? Which features it must have? What’s the best way to ease the adoption? Please comment here, on Reddit (/r/bash , /r/ProgrammingLanguages) or on Hacker News.

Update: following feedback roughly of the form “Yes, I get that but many Ops tasks are done using configuration management tools and tools like CloudFormation and Terraform. How NGS compares to these tools” – there will be a blog post comparing NGS to the mentioned tools. Stay tuned!

Have a nice day!

13 thoughts on “Why I have no favorite programming language”

  1. youve described the very best reason to create a programming language. there are other good reasons, but people tend to assume otherwise.

    “dont make a programming language, there are too many already” is one of those things that people think makes them sound like a seasoned and knowledgeable coder but is absolutely baseless when it comes to logic or facts. if people stopped making programming languages, the art (yes, art) would stagnate! it evolves slowly, to be certain.

    as for features id like, please consider my codex (emulated) data type. its just syntactic sugar on dictionaries so that they (foremost) do everything that dictionaries do, but array(list)-based commands treat the dictionaries (ordered dicts) like lists.

    what?! demo here: https://i.imgur.com/YMFzxbt.png <- is this idea really good for ngs? almost certainly not!! i dont present it to you for any other reason than for you to consider whether something like it could be useful.

    i think a hybrid list/dict type has serious potential despite any perceived flaws, im not saying ngs needs it. only you would really know if it will inspire something ngs does benefit from. perhaps the obvious questions are: "does ordering slow it down?" only when used as an array, not (necessarily) when used as a dict. does anything clobber the keylist? several things, but only in a predictable and logical fashion. its a compromise, all hybrids are. but its not a huge compromise– all good ones arent. happy coding.

    Like

    Reply

      • “The hashes do preserve insertion order.”

        then the battle is practically won. i hope you blog at some point about what you would prefer to jq, that should be an interesting read. (if the phrase “interesting read” has gathered a snarky connotation over time, i mean it sincerely.)

        Like

    • Thanks for the pointer. I do regularly look at other projects and I’ve seen Ammonite. It’s even in the comparison section towards the end of readme in GitHub’s NGS repo.

      Like

      Reply

  4. Hey Ilya.
    Thanks for your insights. I have very similar feeling about programming languages and shell language in specific. That’s why I’m in process of creating one. I have a little bit different motivations, you are working on high level and I’m digging into very low level. Among others I wanna rethink very foundations of programming.
    Could you write something about what’s wrong with compiled languages? One of my views is that anything we do will translate to machine code anyway, so why would I figure out in process what to do if I can generate good machine code in the first place and then let the CPU do the stuff. How about solution from D programming language – it’s compiled, but compilation is fast enough to do it everytime you call program (so from outside it looks like it was interpreted).

    Like

    Reply

    • “I wanna rethink very foundations of programming.” – I wish you to make a breakthrough. Improvement must be made. Current situation is unlikely to be the best we can do.

      “what’s wrong with compiled languages?” – nothing. I think they are not convenient or practical for small scripts and especially for REPL. One of my attempts was to use Lisp (SBCL) for NGS. The vision was simple: NGS compiles to Lisp, SBCL compile from Lisp to machine code. Then I’ve seen a considerable (for REPL) delay after you hit enter, even for the simplest expressions. I have later seen in a discussion on NGS that same is expected if you use llvm for example.

      I did not work with D. Would it be responsive enough for REPL you think? Imagine people accustomed to bash. Will they see difference in responsiveness (after hitting enter)?

      Like

      Reply

      • > Then I’ve seen a considerable (for REPL) delay after you hit enter, even for the simplest expressions.

        Then you did something wrong. I doubt you cared enough to ask the community what it was.

        Like

    • “check out NixOS.” – anything specific?

      “I wrote this DSL for Red in a weekend which can do exactly what you are aiming for” – First, no, you haven’t. Second, it’s arrogant to claim that. Please compare more carefully.

      “when all you want is a simple dialect.” – I don’t. I want a language built ground up for the common tasks. Syntax for example is something that is not trivial to implement as a dialect.

      Like

      Reply

  6. For file and external resource handling, check out Objective-Smalltalk. Using Polymorphic Identifiers, URLs are completely integrated into the language, and any type of data source/data storage can be integrated via the URL mechanism.

    So you can write things like

    file:hello.txt := “Hello World!”.
    post := https://ilya-sher.org/2017/10/10/why-i-have-no-favorite-programming-language/
    file:{env:HOME}/InterestingPosts/proglang := post.

    And so on. Polymorphic Identifiers are a small part of the whole, and not a random feature stuck on, but rather the result of a the idea of having software architectural as the organizing principle of the programming language.

    Objective-Smalltalk the result of my very own personal “all programming languages really suck” moment(s). That doesn’t mean it isn’t going to suck, but it will solve some of the current suckage to suck in hopefully novel ways.

    Like

    Reply

Leave a comment