On Information Loss in Software

“Information Loss” is a way to look at the world. The topic is very broad. This blog post will focus on information loss during development and operation of computer software.

This post discusses why Information Loss is bad and gives some examples.

My hope is that after reading this post, you will be able to spot information loss more easily. This should help you avoiding information loss, eliminating the need for costly information recovery phase. Some examples include specific recommendations how to avoid that particular case of information loss.

Information Loss Definition

Information Loss for the purposes of this blog is the situation where information I is available and is easily accessible at point in time t1 but later, when it’s needed at point in time t2, it is either not available or not easily accessible.

The post will present various categories of information loss with examples. The list is not exhaustive; it’s not meant to be. The intention is to give some examples to help you get the feel and start looking at things from the information loss perspective.

Why Information Loss is Bad?

In many cases of Information Loss, the missing information can be recovered but that requires resources to be thrown at the issue (time and/or money). That is the situation I would like to help you to avoid.

Between the Head and the Code

When working on software, the first place the information loss occurs is when the programmer translates thoughts into code. Information loss at this stage will manifest itself as increased WTF-per-minute during code review or just code reading. Each time the code is read, there will be additional cognitive load while the reader reconstructs the programmer’s idea behind the code.

I have identified two main causes for information loss at the head-to-code stage:

  • Programmer’s fault
  • Programming language imposed

Information Loss due to Programmer’s Fault

The more a programmer is experienced, the less likely is the occurrence of information loss at this stage.

Misnamed Variable

In programmers head: number of servers running the ETL task. Name of the variable in the code: n. WTFs at code review – guaranteed.

Misnamed Function

I’m pretty sure getUser() should not update say last name of the user in database. Such naming is criminal but unfortunately I’ve seen code similar to that.

Use of Magic Numbers

if (result == 126) .... The person who wrote 126 knew what that number means. The person reading the code will need to spend time checking what that number means. One should use constants or enums instead: if (result == NOT_EXECUTABLE) ....

Missing Comments in Code

Most important comments are about why something is being done as opposed to how. If ones code is in a high-level language and of a good quality, it’s a rare occasion one needs to comment about what or how something is being done. On the other hand comments like “Working around API bug: it returns false instead of empty array” are very valuable.

Incorrect Usage of Data Types

A list of people, for example, is not just a list. It has semantic meaning. It’s much easier to understand a program when correct types are used for the data. Java has generics to convey such information, for example List<Person>. Some other languages have type systems that are powerful enough to convey such information too.

Programming Language Imposed Information Loss

Limitations of programming languages lead to less expressive code because the idea in programmer’s head can not be expressed in a straightforward manner. The readers of the code will struggle more (read waste time) to understand the code.

Unnamed Function Parameters

bash and perl5 (not sure about perl5 anymore, there was something experimental) do not have the syntax for specifying function parameter names. This makes the code less expressive. Sometimes programmers will do “the right thing”:

myfunc() {
    local target_file=$1
    ...
}

… but when they don’t, you finish with unnamed parameter, wondering what it could mean:

myfunc() {
    if [[ -f $1 ]];then
        ...
    fi
}

Is that a file to generate or a source file? You don’t know, you have to read on in myfunc hoping for the answer.

Recommendation: even if your language does not support named parameters, emulate them.

Expansion of Strings into Several Arguments (bash)

rm $x

Does that remove one file or several? What the programmer meant? You simply don’t know. It depends on the contents of x, which is typically split into arguments by spaces. You are lucky if you can deduce from the variable name whether it’s one or several files.

From today’s perspective this is just bad design. Back at the day I guess it was the most practical way to implement arrays.

Recommendation: use one of the two alternatives blow and do not use rm $x form.

  • Single file: rm "$x" (proper quoting)
  • Multiple files: rm "${my_files[@]}" (bash arrays)

Side note: this “feature” caused so much pain over the years when x would contain a spaces by accident. Even when x is meant to be used as an array, elements of that array can also contain spaces by accident.

Error Handling

In languages that do not support exceptions (bash, C, Go), the programmer is forced into one of two situations:

  • Write incorrect code that ignores the errors (on purpose or by mistake, go figure which one)
  • Write verbose code that handles the errors. When the code handles every possible error, it becomes cluttered with error handling and it takes more time to understand the code. That’s the case where information loss occurs because the reader is overwhelmed by the code.

In NGS, since typical use case is scripting, I wanted to have the option for the code to be concise. That rules out returning status code along with the result because the caller is then forced to check it. It does make more sense for NGS to have exceptions and for scripts to decide whether to catch them or let the whole script terminate with error because of an uncaught exception.

Unordered Hash/Map/dict Data Structure

Hash data structure is implemented in a non-order-preserving manner in some languages. That means that the programmer can not express the intention freely in situations where the order of key/value pairs is important. That pushes towards less readable code as the programmer fights the language by implementing his/her own ordered dictionary.

Information loss in this case is again losing the sight of programmer’s intention.

Fortunately many modern languages solved the issue by now:

Recommendation: check whether your language has the data structure you really want to use, either built-in or in a library.

Limited Data Structures (bash)

Working with data structures in bash results more or less convoluted code, depending on the data structures one need to work with. This is direct consequence of bash supporting exactly three data structures:

  • Scalar (strings which can sometimes be treated as numbers or arrays)
  • Array
  • Associative array

These data structures can not be nested.

The result is much less readable code where the original intent of the author is harder to recover as opposed to data manipulation in other popular languages (Python, Ruby, etc).

Recommendation: consider using other languages besides bash for heavy data manipulation code.

Absence of non-nullable Types

In some languages there is no straightforward way to specify non-nullable parameters. The programmers are then required to check whether each passed parameter is null. That results more boilerplate code. Let’s look at the following bit of Java code from the popular Apache Flink project:

// flink/flink-java/src/main/java/org/apache/flink/api/java/DataSet.java

protected DataSet(ExecutionEnvironment context, TypeInformation<T&gt; typeInfo) {
    if (context == null) {
        throw new NullPointerException("context is null");
    }
    if (typeInfo == null) {
        throw new NullPointerException("typeInfo is null");
    }

    this.context = context;
    this.type = typeInfo;
}

Asynchronous Computing Model (JavaScript)

In JavaScript for example, progressively more readable code uses:

Again, information loss occurs when programmer’s intention is lost in the code because the code looks like a big struggle against asynchronicity and the language.

Recommendation: prefer async/await over Promises and prefer Promises over callbacks.

Loss of semantic information (JavaScript)

console.log() vs debug('my-module')('my message') in JavaScript. When a programmer chooses to use log() instead of debug(), loss of semantic information occurs. In this case it means more effort in finding the needed information in the output as opposed to simpler turning on and off the relevant debug sections.

Recommendation: use the debug module.

Information Loss at Runtime

Information loss at runtime will manifest as harder debugging.

Empty Catch Clause

This is borderline criminal. Except for very few cases when empty catch clause is really appropriate, by placing empty catch clause in the code, you are setting up a bomb for your colleagues. They will pay with their time, tears and mental health, not to mention they will be hating you. Where is the information loss? At the time the exception is generated, there is useful information about what happened. Empty catch clause loses that information. Result: hard to find exceptions and their causes.

In NGS, there are clear ways to express that you didn’t just forgot to handle the exception (try ... catch(e) { }) but you actually don’t care (or know exactly) what happened:

  • try EXPR without the catch clause at all. If EXPR throws exception, try EXPR evaluates to null, otherwise evaluates to EXPR.
  • EXPR tor DFLT if EXPR throws an exception, evaluates to DFLT, otherwise evaluates to EXPR.

Writing to stdout Instead of stderr

stdout has semantic meaning (result of the computation) and stderr also has semantic meaning (errors description). It will make harder for any wrapper script to deal with a program that outputs errors to stdout or outputs the result to stderr. The semantic information about the text is lost and then needs to be recovered by the caller if the two outputs are mixed.

Wrong exit codes reporting

This one really hinders automation.

if ... then {
    ...
    error("error occurred")
    exit(0) # incorrect error code reported
}

Since it’s easy to forget about exit code, and the common case is that exit() means abnormal termination of the program, in NGS exit() that does not provide an exit code defaults to exit code 1.

Wrong exit codes handling

if [ -e MY_FILE ] ...

This is all over bash scripts… and it’s wrong. Which exit codes [ program/built-in returns? Zero for “yes”, one for “no”, and two for “An error occurred”. Guess what. You can’t handle three distinct cases with two if branches; “An error occurred” is causing the “false” branch of the if to be taken. If you are lucky, you will spot error message on stderr. If you are not lucky, your script will just work incorrectly in some circumstances.

At this point the tradeoff in NGS was made in favor of correctness, not simplicity. if $(test -e MY_FILE) ... in NGS can go three ways: “yes” branch, “no” branch and an exception. After any external process is finished, NGS checks the exit code. For unknown process, non-zero exit code cases an exception. For test and a few others, zero and one are not causing an exception. The exit code checking facility is extensible and one can easily “teach” NGS about new programs.

Broaden your Horizon – Extras

I’ll mention here non-strictly software development related information loss cases.

Untagged Cloud Resources (AWS)

Have you just created an EC2 instance and named it Server or maybe you haven’t tagged it at all? Congratulations, semantic information has just been lost. You colleagues will strugle to understand what is the role of instance.

Recommendation: rigorously tag the resources, have alerts for untagged or improperly tagged resources. In AWS you can also know who created the resource by looking at CloudTrail.

Side note: In Azure, any resource must belong to a “Resource Group” which makes it much easier to track the resources.

GUI

You just performed operation in GUI. The information of what happened was just lost the minute you performed the operation. Good luck reproducing or documenting it.

The plan to combat this in NGS is to have textual representation for each operation that is performed via GUI.

String Concatenation

Every time two strings are concatenated into one, there is some information loss.

Recommendation: instead of parsing unstructured text (result of concatenation) later, consider using structured data format when producing the output. (Example: JSON).


Hope that helps. Have fun!

What I did not steal from Perl 6

I’m curious about programming languages. Not because I’m creating one right now. I always was. This post is about ideas and features that I have seen in Perl 6 and found interesting. If you are curious about programming languages in general, you should take a look at these.

There are various reasons for not stealing the interesting ideas from Perl 6:

  1. I’m trying to keep number of concepts in NGS as small as possible. If I’m not seeing huge immediate value in a concept – I skip it.
  2. Not taking anything that I think can confuse me or other programmers. I’m not talking here because someone is a beginner. I’m talking about confusing concepts.
  3. Simply because I don’t have enough resources to implement it at the moment.

Here are the interesting Perl 6 features, in no particular order (except the first one). There are also my comments whether I would like the feature in NGS or why not.

  1. Syntax. Very expressive an terse. Perl6 has even more of it than Perl 5. Now that we got rid of the $ and friends in the room:
  2. Grammars. Would actually be nice to have something like that in NGS.
  3. Lots of operators. The most interesting concept is Metaoperators. I’m trying to keep the amount of syntax elements in NGS relatively low. There are already two syntaxes in NGS: commands and expressions. Not taking more syntax without serious need.
  4. How the “pointy block” syntax mixes with “for” syntax: for @list -> @element . NGS already has several syntaxes for Lambdas.
  5. Flow control
    1. when” flow control. The closest NGS has is “cond” and friends, stolen from Lisp.
    2. repeat while / repeat until . It would be nice to have something like that in NGS.
    3. once . Not sure about this one. The functionality might be needed.
  6. Slips. The behaviour is frightening me: if it does expand, how do I pass a Slip if I just want to pass it, say as an item of an array? NGS uses syntax for slips: [1, 2, *myitems, 3, 4] which I think is cleaner. You know you can’t pass it because it’s syntax.
  7. .WHAT method. I stole something similar from Ruby: the inspect method.

As a special note, I have seen a welcome change from $arr[0] to @arr[0] . I think it removes confusion. (That was Perl 5 vs Perl 6).

Please don’t be offended if you are a Perl 6 hacker and you see that there is amazing feature that I have not mentioned. It could be that I’ve seen this in several other languages already or maybe I did not find it interesting or … maybe I just missed it. Don’t hesitate to leave a comment anyway.


Happy coding, in whatever language rocks your boat! Except for bash. Coding in bash will never be happy.

JQ is a symptom

jq is a great tool. It does what bash can not – work with structured data. I use it. I would like not to use it.

In my opinion, working with structured data is such a basic thing that it makes much more sense to be handled by the language itself. I want my shell to be capable and I strongly disagree with the view that a shell “is not supposed to do that”. Shell is supposed to do whatever is needed to make my life easier. Handling structured data is one of these things.

If “shell is not supposed to do that”, by that logic, bash is not supposed to do anything except for running external commands and routing the data between them. Doesn’t it seem odd that bash does have builtin string manipulation then? Maybe bash shouldn’t have added associative arrays in version 4? … or arrays in version 2? How about if and while ? Maybe bash shouldn’t have them either?

woman-698943_640

jq is a symptom that bash can’t handle today’s reality: structured data. The world is increasingly more about APIs. APIs consume and return structured data. I do work with APIs from shell. Don’t you guys use AWS CLI or any other API that returns JSON?

The reality has changed. bash hasn’t. I’m working on bash alternative. Please help me with it. Or at least spread the word.

If you don’t like my project, join Elvish . Elvish is another shell that supports structured data.


Happy coding! Hope it’s not in bash.

Bash pitfall: if test, if [, if [[

I bet you’ve seen a lot of scripts with seemingly innocent if [ -e blah ];then ...; else ...; fi or something similar . What’s the problem? if has at most two branches while test , [ and [[ have three different exit codes. Oops.

If you make a syntax error (or any other error occurs) in the test , [ or [[ expression, it will return the exit code 2 (or above, according to man test​). if will take the else branch. If you are lucky, you will notice the error message from the test, [ or [[ commands. If not, the else branch will always be executed.

I don’t want to use bash. The pitfall above is one of the many reasons. Unfortunately, I do use bash because it’s still best tool for some tasks. I’m working on alternative to bash. It’s called NGS, the Next Generation Shell. In NGS, the situation above is solved as one would expect from a modern programming language: exit codes 2 and above throw exception.

If you also think that there should be a viable alternative to bash, you are welcome to help me working on it.


Happy coding! Hope it’s not in bash 🙂

Terraform becomes a programming language

Declarative languages failure

Approach that in my eyes failed, again and again, is to start with your own declarative language and then with time grow the language. (SQL being among notable exceptions)

Puppet is the best example. map and each, added in Puppet 4.0.0 are, in my opinion, just two in a sea of evidence that the envisioned simple format has failed to handle the needs of the real world.

Ansible’s loop looks bad as the whole idea of making top levels of programs in YAML based syntax (and the rest in Python).

In my opinion, it makes more sense to create a language first and then libraries for it, not a library and then a language around it.

My hope for Terraform

I think Terraform guys are smart. Among other things, it manifests in implementing data sources. Data sources make Terraform much more flexible. I think it’s very clever.

Terraform, which started declarative, are now inventing their own programming language. They are going the way of Puppet and Ansible. I hope they can do better, in this awkward situation: there are quite a lot of constraints on the programming language because of the existing syntax and semantics.

Happy coding, everyone!

 

How fucked is AWS CLI/API

In the 21st century, we can do significantly better than this crap AWS CLI. We just need to think from the users’ perspective for a change and not just dump on users whatever we were using internally or seen in a nightmare and then implemented.

thinking-3082823_640
Think from the users’ perspective

I’m working on a solution which is described towards the end of the post. It’s not ready yet but the (working btw) examples will convince the reader that “significantly better” is very possible… I hope.

Background

I’m building AWS library and a command line tool. Since I’m doing it for a shell-like language (NGS), the AWS library that I’m building uses AWS CLI. Frustration and anger are prominent feelings when working with AWS CLI. I will just list here some of the reasons behind that feeling.

furious-2514031_640
AWS CLI/API – WTF were they thinking?

Overall impression

Separate teams worked on different services and were not talking to each other. Then the something like this happened: “OK, we have these APIs here, let’s expose them. User experience? No, we don’t have time for that / we don’t care”.

Tags

Representing a map (key-value pairs) as an array ( "Tags": [ { "Key": "some tag name", "Value": "some tag value" }, ... ] ), as AWS CLI does is insane… but only when you think about the user (developer in this case) experience.

shocked-2681488_640
AWS Tags – no, not in this form!

Reservations

When listing EC2 instances using aws ec2 describe-instances, the data is organized as list of Reservations and each Reservation has list of instances. I’m using AWS for quite a few years and I never needed to do anything with Reservations but I did spent some time unwrapping again and again the interesting data (list of instances) from the unwanted layer of madness. It really feels like “OK, Internally we have Reservations and that’s how our API works, let’s just expose it as it is”.

annoyed-3126442_640
Who da fuck ever used Reservations?

Results data structure

AWS CLI usually (of course not always!) returns a map/hash at the top level of the result. The most interesting data would be called for example LoadBalancerDescriptions, or Vpcs, or Subnets, or … ensuring difficulty making generic tooling around AWS CLI. Thanks! Do you even use your own AWS CLI, Amazon?

Inconsistencies

beer-2370783_640
Kind of the same but not really

Security Groups

These are of course special. Think of aws ec2 create-security-group ...

  1. --group-name and --description are mandatory command line arguments. For now, I haven’t seen any other resource creation that requires both name and description.
  2. The command returns something like { "GroupId": "sg-903004f8" } which is unlike many other commands which return a hash with the properties of the newly created resource, not just the ID.

Elastic Load Balancer

Oh, I like this one. It’s unlike any other.

  1. The unique key by which you find a load balance is name, unlike other resources which use ids.
  2. Tags on load balancers work differently. When you list load balancers, you don’t get the tags with the list like you have when listing instances, subnets, vpcs, etc. You need to issue additional command: aws elb describe-tags --load-balancer-names ...
  3. aws elb describe-load-balancers – items in the returned list have field VPCId while other places usually name it VpcId .

Target groups

aws elbv2 deregister-targets --target-group-arn ...

Yes, this particular command and it’s “target group” friends use ARN, not a name (as ELB) and not an ID (like most EC2 resources).

DHCP options (sets)

(Haven’t used but considered so looked at documentation and here is what I found)

Example: aws ec2 create-dhcp-options --dhcp-configuration "Key=domain-name-servers,Values=10.2.5.1,10.2.5.2"

Yes, the create syntax is unlike other commands and looks like filter syntax. Instead of say --name-servers ns1 ns2 ... switch you have "Key=domain-name-servers,Values=" WTF?

Route 53

aws route53 list-hosted-zones does not return the tags. Here is an output of the command:

{
 "HostedZones": [
 {
 "Id": "/hostedzone/Z2V8OM9UJRMOVJ",
 "Name": "test1.com.",
 "CallerReference": "test1.com",
 "Config": {
 "PrivateZone": false
 },
 "ResourceRecordSetCount": 2
 },
 {
 "Id": "/hostedzone/Z3BM21F7GYXS7Y",
 "Name": "test2.com.",
 "CallerReference": "test2.com",
 "Config": {
 "PrivateZone": false
 },
 "ResourceRecordSetCount": 2
 }
 ]
}

Wanna get the tags? F*ck you! Here is the command: aws route53 list-tags-for-resources --resource-type hostedzone --resource-ids Z2V8OM9UJRMOVJ Z3BM21F7GYXS7Y . Get it? You are supposed to get the Id field from the list generated by list-hosted-zones, split it by slash and then use the last part as resource ids. Tagging zones also uses the rightmost part of id: aws route53 change-tags-for-resource --resource-type hostedzone --resource-id Z2V8OM9UJRMOVJ --add-tags Key=t1,Value=v11

… but that apparently was not enough differentiation 🙂 Check this out: aws elb add-tags --load-balancer-names NAME --tags TAGS vs aws route53 change-tags-for-resource --resource-type hostedzone --resource-id ID --add-tags TAGS and aws elb remove-tags --load-balancer-names NAME --tags TAGS vs aws route53 change-tags-for-resource --resource-type hostedzone --resource-id ID --remove-tag-keys TAGS . Trick question: on how many dimensions this is different? So wrong on so many levels 🙂

Here is another one: when you fetch records from a zone you use the full id, /hostedzone/Z2V8OM9UJRMOVJ , not Z2V8OM9UJRMOVJaws route53 list-resource-record-sets --hosted-zone-id /hostedzone/Z2V8OM9UJRMOVJ

 

(many more to come)

Expected comments and answers

feedback-2044700_640
Internet discussions are the best 😉

Just use Terraform

I might some day. For the cases I have in mind, for now I prefer a tool that:

  1. Is conceptually simpler (improved scripting, not something completely different)
  2. Doesn’t require a state file (I don’t want to explain to a client with two servers where the state file is and what it does)
  3. Can be easily used for ad-hoc listing and manipulation of the resources, even when these resources were not created/managed by the tool.
  4. Can stop and start instances (impossible with Terraform last time I checked a few months ago)

Just use CloudFormation

I don’t find it convenient. Also, see Terraform points above.

By the way, the phrases of the form “Just use X” are not appreciated.

You just shit on what other people do to push your tools

Yes, I reserve the right to shit on things. Here I mean specifically AWS CLI. Freedom of speech, you know… especially when:

  1. The product (AWS API) is technically subpar
  2. The creator of the product does not lack resources
  3. The product is used widely, wasting huge amounts of time of the users

If I’m investing my time to write my own tool purely out of frustration, I will definitely tell why I think the product is crap.

Is that just a rant or you have alternatives?

I’m working on it. The command line tool is called na (Ngs Aws). It’s a thin wrapper around declarative primitives library for AWS.

excited-3126450_640
There might be a solution to this madness!!!

The command line tool that I’m working on is not anywhere ready but here is a gist of what you can do with it:

# list vpcs
na vpc

# Find the default VPC
na vpc IsDefault:true

# List security groups of all vpcs which have tag "Name" "v1".
na vpc Name=v1 sg

# Idempotent operations, in this case deletion.
# No error when this group does not exist.
# Delete "test-group-name" in the given vpc(s).
na vpc Name=v1 sg GroupName:test-group-name DEL

# List all volumes which are attached to stopped instances
na i State:stopped vol

# Delete all volumes that are not attached to instances
na vol State:available DEL

# Stop all instances which are tagged as
# "role" "ocsp-proxy" and "env" "dev".
na i role=ocsp-proxy env=dev SET State:stopped
  1. Na never dumps huge amounts of data to your terminal. As a human, you will probably not be able to process it so when number of items (rows in a table actually) is above certain configurable threshold, you will see something like this:
    $ na vol 
    === Digest of 78 rows ===
    ...

    It will show how many unique values there are in each column, min and max value for each column. Thinking about displaying top 5 (or so) top and bottom values for each column.

  2. Na has concept of “related” resources so when you write something like na i State:stopped vol , it knows that the volumes you want to show are related to the instances that you mentioned. In this particular case, it means volumes attached to the instances.
  3. Note the consistency of what you see in output and arguments to CLI. If something is called “State” in the data structure returned by AWS CLI, it will be called “State” in the query, not “state” (“–state”).

I will be updating this post as things come along.

The missing link of Ops tools

It’s like we went from horse to spaceship, skipping everything in between.

Background

Let’s say you are managing your system in AWS. Amazon provides you with API to do that. What are your options for consuming that API?

Option 1: CLI or library for API access

AWS CLI let’s us access the API from the command line and bash scripts. Python/Ruby/Node.js and other languages can access the API using appropriate libraries.

Option 2: Declarative tools

You declare how the system should look like, the tool figures out dependencies and performs any API calls that are needed to achieve the declared state.

Problem with using CLI or API libraries

Accessing API using CLI or libraries is fine for one off tasks. In many cases, automation is needed and we would like to prepare scripts. Ideally, these scripts would be idempotent (can be run multiple times, converging to the desired state and not ruining it). We then quickly discover how clunky these scripts are:

# Script "original"
if resource_a exists then
  if resource_a_property_p != desired_resource_a_property_p then
    set resource_a_property_p to desired_resource_a_property_p
  end
  if resource_a_property_q != desired_resource_a_property_q then
    ...
  end
else
  # resource_a does not exist
  create resource_a
  set resource_a_property_p to desired_resource_a_property_p
  ...
end
# more chunks like the above

It’s easy to see why you wouldn’t want to write and maintain a script such as above.

How the problem was solved

What happened next: jump to “Option 2”, declarative tools such as CloudFormation, Terraform, etc.

rocket-1374248_640

Other possible solution that never happened

If you have developed any code, you probably know what refactoring is: making the code more readable, deduplicate shared code, factoring out common patterns, etc… without changing the meaning of the code. The script above is an obvious candidate for refactoring, which would be improving “Option 1” (CLI or a library for API access) above, but that never happened.

All the ifs should have been moved to a library and the script could be transformed to something like this:

# Script "refactored"
create_or_update(resource_a, {
  property_p = desired_resource_a_property_p
  property_q = desired_resource_a_property_q
})
# more chunks like the above

One might say that the “refactored” script looks pretty much like input file of the declarative tools mentioned above. Yes, it does look similar; there is a huge difference though.

Declarative tools vs declarative primitives library

By “declarative primitives library” I mean a programming language library that provides idempotent functions to create/update/delete resources. In our cases these resource are VPCs, load balancers, security groups, instances, etc…

Differences of declarative tools vs declarative primitives library

  1. Declarative tools (at least some of them) do provide dependency resolution so they can sort out in which order the resources should be created/destroyed.
  2. Complexity. The complexity of mentioned tools can not be ignored; it’s much higher than one of  declarative primitives library. Complexity means bugs and higher maintenance costs. Complexity should be considered a negative factor when picking a tool.
  3. Some declarative tools track created resources so they can easily be destroyed, which is convenient. Note that on the other hand this brings more complexity to the tool as there must be yet another chunk of code to manage the state.
  4. Interacting with existing resources. Between awkward to impossible with declarative tools; easy with correctly built declarative primitives library. Example: delete all unused load balancers (unused means no attached instances): AWS::Elb().reject(X.Instances).delete()
  5. Control. Customizing behaviour of your script that uses declarative primitives library is straightforward. It’s possible but harder with declarative tools. Trivial if in a programming language can look like count = "${length(var.public_subnets) > 0 ? 1 : 0}" (approved Terraform VPC module).
  6. Ease of onboarding has declarative tools as a clear winner – you don’t have to program and don’t even need to know a programming language, but you can get stuck without knowing it:
  7. Getting stuck. If your declarative tool does not support a property or a resource that you need, you might need to learn a new programming language because the DSL used by your tool is not the programming language of the tool itself (Terraform, Puppet, Ansible). When using declarative primitives library on the other hand you can always either extend it when/if you wish (preferable) or make your own easy workaround.
  8. Having one central place where potentially all resources are described as text (Please! don’t call that code, format is not a code!). It should be easier done with declarative tools. In practice, I think it depends more on your processes and how you work.

As you can see, it’s not black and white, so I would expect both solutions be available so that we, Ops, could choose according to our use case and our skills.

My suggestion

I don’t only suggest to have something between a horse and a spaceship; I work on a car. As part of the Next Generation Shell (a shell and a programming language for ops tasks) I work on declarative primitives library. Right now it covers some parts of AWS. Please have a look. Ideally, join the project.

Next Generation Shell – https://github.com/ilyash/ngs

Feedback

Do you agree that the jump between API and declarative tools was too big? Do you think that the middle ground, declarative primitives approach, would be useful in some cases? Comment here or on Reddit.


Have a nice day!