Category: Programming

Bashing bash – variable substitution

Ilya SherLeave a comment

This is the first post in the “bashing bash” series to highlight problems in bash and convince systems and software engineers to help building a better alternative.

Bash is a very powerful and useful tool, doing a better job than many other shells and programming languages when used for the intended tasks. Still, it’s hard to believe that writing a software decades later can not be done better.

The problem

What does the following command do?

cp $src_file $dst_file

One might think it copies the given file to the specified destination. Looking at the code we can say it was the intention. What would actually happen? It can not be known from the line above. Each $src_file and $dst_file expand to zero to N arguments so unexpected things that could happen. The correct command would be

cp "$src_file" "$dst_file"

Forgetting the quotes or leaving them out assuming  $src_file and $dst_file will always contain one bash word, expanding to exactly one argument each is dangerous.

Keeping quoting everything makes code cluttered.

The suggested solution

In NGS, $var expands to exactly one argument similar to "$var" in bash. The new syntax $*var, consistent with similar syntax in other NGS parts, would expand to zero to N arguments.

Please help building a better alternative

Go to https://github.com/ilyash/ngs/ and contribute some code.

Common Sense Movement – where is it?

Ilya SherLeave a comment

brain-605603_960_720

Let’s look at systems and software engineering.

We hear about all kinds of “cool” methodologies, ideas, frameworks: DevOps, Scrum, Agile, Kanban, Continuous Integration, Continuous Delivery, Puppet, Chef, Ansible … and combinations of these.

Why do you hear about all this cool stuff all the time? Why there are hypes all the time? Simple: money. Another hype, another opportunity to sell you something [at higher price]: products, support, certifications, guidance. You don’t necessarily need these things but they definitely would like to sell you all of it.

I’m starting Common Sense Movement

This humorous thought crossed my mind yesterday. You know, just to balance the situation a bit. Then I realized why don’t you hear much about the shiny cool “common sense” methodology: money.

You can’t sell a “common sense” certificate. That’s because if someone really has some common sense, he/she is not going to pay you for this crap and the only certificate you can sell would be “Failed Common Sense Practitioner”. Actually I doubt you can  hardly sell any crap to common sense practitioners. That’s a problem. So, there is no Common Sense Movement for you 😦

I really would like the hype waves to stop so we could just do our work in peace. Meanwhile, when you are approached by a client or a boss with yet another “We need tool X” or “We need technology Y” you should really answer with “Prove that you need it and it’s optimal usage for our money and time considering other alternatives” no matter how new,shiny or cool that thing is.

Prove your tool is the right choice

Ilya SherLeave a comment

I see two common approaches at work when it comes to choosing tools.

Photo of different tools
How do you choose?

Bad

I have heard about this new shiny cool framework/library/tool, let’s use it!

It might be a good approach when you want to learn something in your spare time, just don’t do this at work.

Good

We have this problem and found a few alternative solutions (frameworks/libraries/tools) that might solve it. We will investigate and see if any of them fit our needs.

Note that depending on your situation, it might be the case when none of the existing tools will be a good fit for your project.

It might be tempting to look at existing solutions in a positive light thinking “our problem is not unique, there are tools solving it already”. The problem might appear to be similar to the problems that these tools solve but remember to look and focus and what’s unique to your circumstances.

Suggested approach

You should prove that the points below are wrong.  Assume that the tool you are evaluating:

  1. Does not cover your needs.
  2. Has crappy/new/unstable code.
  3. Has no community so getting answers to your questions or help is a nightmare.
  4. Has none or very costly official support.
  5. It will have a steep learning curve.
  6. It will be harder to maintain than any other alternative or your own code.
  7. You will need to modify the tool and it will be tough.
  8. Provides so little that it’s not worth another external dependency.
  9. Will cause lock-in with very high switching costs. (You can affect this one to some degree with your design).
  10. Has big company behind it which
    1. had huge investment and only cares about things like ROI and market share and not you, even if they did at some point in the past.
    2. has more interesting markets than you and optimizes the product for these markets, making it not a good fit for your situation.
    3. generates hype making it all look rosy even when the product has grave flaws.
    4. makes it look like everybody uses this tool.
    5. makes it feel like the next best thing.
  11. Has a big alliance behind it with many interests of different partners with zero intersection with your interests.
  12. Most places that use this tool suffer immensely but either don’t talk about it or you don’t hear that for other reasons. Dig deeper to find such information.

Calling some friends and asking them about the tool is very good but note that your situation is different so most of the points listed above still stand.

I guess that when you start positively when evaluating a tool, you might fall into the confirmation bias trap. Beware!

The Go language TL;DR review

Ilya SherLeave a comment

I took a few hours to overview the Go language to make up my mind. Posting here because someone else might also find this useful.

Please note that I just reviewed the language. I did not write big chunks of code so my experience is fairly limited.

  • Compiled.
  • Easy to use almost as any high-level interpreted language, this is not a fiction, works as advertised. Simplicity and ease of use – goal checked.
  • Good libraries (HTTP client+server, FastCGI server, JSON, RPC, …)
  • VIM support in the code repo (syntax, indentation, …)
  • bash completion support in the code repo
  • The language as a whole makes the impression that the right choices were made.
  • It’s pragmatic.
  • It’s minimalistic for the capabilities it provides.
  • The only “big” innovation I see (besides the ease of use) is simplifying interfaces in a statically typed language. All other features resemble things I’ve seen somewhere else already. Actually “familiarity” of the language was one of the goals.
  • Has low level stuff, see http://golang.org/pkg/syscall/ for example.
  • As of this moment the Go language became my second favorite moving down JS and Python. Sorry, Lisp still remains the first. This of course might be a temporary condition till I start writing some real code. I hope it’s not.

Webkit background image wrapping bug

Ilya SherLeave a comment

Context: I am doing a mobile web project for one of my clients.

Problem screenshot

WebKit wrapping bug screenshot (zoom)
Android Browser: background image is wrapped. Red arrow shows the height of the DIV.

Problem description

When using DIV with background image that is the same height as the DIV and top pixels of the background image are transparent, with repeat-x (or no repeat) there is an observable line at the top of the div painted with the color of the bottom pixel of the image.

Suspected buggy mechanism

I suppose it’s subpixel smoothing. Because of the observed facts:

  • The additional lines is less than a pixel wide as seen when zooming
  • Pixels affect colors of neighbours pixels

Suggested solution

  • Add two pixels at the bottom of the background image: one of the same color as the last pixel and one transparent. For example, if original image is from top to bottom (transparent,…,transparent,red,green,blue) it should become (transparent,…,transparent,red,green,blue,blue,transparent).
  • Keep the DIV vertical size the same as the original background: the two new pixels should be “rendered” outside the DIV.

Initially I’ve added only the transparent pixel at the very bottom but it interacted badly with both the last visible pixel and the contents of the div (not shown in the example, for simplicity).

I have created a Gist that shows the problem and the suggested solution. The Gist also includes the half-solution of only adding one pixel. For your convenience I have also uploaded the HTML of the problem and the suggested solution to this blog.

Additional info

  • On some devices, when going to landscape mode, the line disappears and is only observable when zooming in

P.S.
I learned something the hard way but I hope you will find this post and it will save you something from 1 to 5 hours of work.

Stop thinking, start doing.

Ilya SherLeave a comment

Backgound

I could not find a “todo” software that would staisfy my needs so I decided to create one. The decision was also backed up by the fact that my good friend, Guy Egozy, was also very frustrated by the existing solutions.

The story

I started thinking about my “todo” software that I wanted to create. What it should include, how it should behave, etc… I was thinking for months. The important part is to realize when thinking becomes mental masturbation. Yes it is fun but it’s not productive. I admit I was slow to recognize the situation. When I did, I decided to start writing the code. It was a throw-away code, a protoype. When I had something that worked, I finally understood the following advantages:

  1. You learn a lot when implementing. No thinking + reading + analyzing will take you there. Of course you should but don’t over-do it.
  2. It’s much easiear to collaborate on. I can tell my friend all day about what I think but showing working application is a totally different story. Also he was finally able to participate by adding more code. Which he does till this day. (Thanks!)
  3. Makes possible to get any meaningful feedback from people, possible feature users.
  4. And of course you make steps toward the goal of having a good working version of the application (which we named Tagoholic), something you can share with the world, after few throw away versions.

Conclusion

Stop thinking, start doing — it’s way more productive.

P.S.

We are still working on Tagoholic. Collaboration (sharing) and mobile version are the two top requested features that are not there yet. We will let you know via Twitter when these are ready.

Is it so hard to get the UI right, Youtube?

Ilya Sher2 Comments

Let’s keep it short. Obvious things Youtube got wrong:

  1. Can’t drag videos into playlist – WTF? Really? How hard could that be to implement?
  2. The home page with subscriptions and etc. The user does only one thing on all sites – chooses what to click next. (Well, and watches the videos if it’s Youtube). So why the f*ck you are making it so hard? Gray out the videos that were already watched! No one should scan all the page trying to spot new videos! Unjustified cognitive load. Other pages would certainly benefit from this too but the home page is really frustrating.
  3. I wouldn’t complain if it was only this minor bug but since I’m posting anyway … “Added to queue” does not go away if you remove the video from the list. It goes away only if you reload the page after you remove the video from the queue

Editing YAML with VI

Ilya SherLeave a comment

Put the following in your .vimrc file and you are set:

au BufNewFile,BufRead *.yaml,*.yml set et ts=2 sw=2

When you will be editing YAML files, you’ll automatically have the following behaviour:

  • et – expand tabs – puts spaces whenever you use tabs
  • ts=2 – tab stop – 2 spaces per tab
  • sw=2 – shift width – 2 spaces to move with < and > commands

YAML in VI

Web programmers should know

Ilya SherLeave a comment

Here is a checklist of knowledge and abilities which I consider a must for a good web programmer.

  • HTML, w3.org standards and validator
  • AJAX
  • SQL: multiple column indexes, transactions
  • The following programming languages exist: PHP, Python, Ruby.
  • Program in at least one of the languages above
  • Program in C at least a bit
  • What is JVM and bytecode
  • What is SQL injection and how to prevent it
  • How DNS works, at least in general – what server and clients do
  • How routing works, at least in general. Tunneling. NAT. BGP is a plus
  • HTTP and HTTPS
  • Load balancing and it’s session persistence problem and how it’s usually solved
  • HTTP proxy, what it solves and how it works
  • What’s FTP
  • What’s version control and what it solves
  • How SMTP works, at least in general. POP3 and IMAP a plus
  • What’s character encoding and why UTF-8 rules

If it’s someone expected to work with Linux the following applies:

I’ll be adding to the list as soon as I remember anything important enough.
Suggestions for additional points are welcome.