An introduction to programming
(if you’re a seasoned programmer and don’t need the preface, skip on down to the lower section about paradigms) If you’re really, really seasoned, you might skip even further, to the core paradigm in this article - dataflow.
As a preface, let me say this: having programmed professionally for about three years now, there is still so much for me to learn. I will probably be learning all my life and still have not have truly mastered programming.
Programming is interesting on so many levels. I often tell people it’s like climbing a cloudy mountain: you have a general idea of where the top is, but the higher you get, the further the top seems, and the more enshrouded you become. You can see high from up there, but you still can’t see the tip-top.
That isn’t to say that it gets harder; merely that its not something that can be looked upon and digested in one sitting. It’s a long and wild ride, and if respected, it can take you into a new way of understanding the world around you, one that is deep and even unsettling.
The reason for the proliferation of programming paradigms, highly specialized domain experts, and increased complexity in technology is probably hard to pin down. It could even be argued that it hasn’t become more complex, but rather refined. Either way, we’ll leave the philosophy of it for another day.
All of this is important, but not vital to understanding some of the core principles of systems and the computations they invoke.
This is not the perfect order, nor does it claim to be the typical learning path for programmers, nor does it exactly reflect my personal experience, but we’ll use it as simply an guide on our journey.
So young Padawan, let’s just say you’ve started programming: you did some tutorials, read a book, whatever. You’ve got the basics down – functions, for loops, data structures, variables, conditional statements. You’re hot shit, and you are writing scripts like no ones business.
Time passes… now you’ve been doing a bit more, and you want to raise your skill – either by necessity or curiosity.
Now you’re learning about things like state, program composition (and decomposition), organization, naming conventions, hell, even debugging and troubleshooting!
Wow, this is really powerful, really cool stuff! And you feel totally in control. That’s an important distinction here, for reasons I’ll return to later.
Look ma, two hands!
Now things are getting interesting. You’re at a stage that most professional programmers might consider “knowing enough to get into trouble”. Not long ago I was there, and it felt really exhilarating. Nothing like a good roller-coaster, but hey, different strokes for different folks.
You might have some friends who program, or maybe you’ve posted code online in a public forum of sorts. You might get some virtual high-fives, but there’s a good chance that some people are attacking your code – what gives? Suddenly the honeymoon is starting to wear off – did I make a grave mistake?
If you’re tenacious, and you either want to make a career out of it or continue as a hobbyist, you’ll keep on keeping on. If not, you might just call it quits and chalk it up to an interesting experience. Let’s assume you’re going to stick with it. What’s next? There’s a good chance you’ll learn that people are talking a lot about “computer science”. Sounds hifalutin – what kind of science is really involved?
You’ll learn about things like data structures, about using them to store information in a way that makes your code faster, more robust, and easier to reason about. You’ll even get to the classic comp-sci stuff – Big O notation, where you learn how a program will perform as you increase the size of your dataset, and can even graph it all nice and pretty like. You’ll probably learn about the classic data structures, which are all just abstractions to what a computer really is – a big ass chunk of memory and some logic gates.
These data structures are not commonly used for most domains, as they have been abstracted into libraries and tools, but it’s good to know (sometimes) and you’ll learn it anyway. There is a certain mental masturbation involved in some of this learning, but suffice it to say that it can be useful to understand how things really work “under the hood”.
The kind I’m talking about are the classics - trees, linked lists, arrays, hashes, heaps, tries, etc…
As I said, they are abstractions – at the end of the day, it might be a tree shaped data structure (tree), it might be a list of pointers that link one section of memory to another (linked list), it might be a block of contiguous memory (array) but you’re still at the whims of the machines' guts. You’re complex program might be traversing some data, writing things to disk, posting things to another application, sending messages back and forth across continents – it’s still ones and zeros, bits and bytes, electronic switches and logic gates.
Remember I said that control was an important distinction? The reason why might be clearer now. The more you program, the more you tend to realize how much is out of your control. It’s very easy to write “one-off” scripts, but taking that mindset into a large, unwieldy code-base will get you into trouble. At this point, you are at another crossroads – specialization. You have learned quite a bit, and you’re at a level that could be arguably “professional”. You’ve got some comp sci under your belt, and you’ve done some real programming.
Engineering to the rescue!
If you’re going to continue professionally, I think it’s important to take a step back and think more in terms of engineering. If you have done extensive coding, you will have undoubtedly uncovered issues, found or pondered solutions, and become more concerned with the process of your work.
Let me also say this; the fact that software engineering and computer science are two different fields cannot be understated. I find it to be a red flag if someone says otherwise. Suffice it to say, these two categories are solving different problems. Engineering is focused on structure, maintainability, dependability and understanding of real, every day systems and programs. Computer science is intimately involved in both theory and practice, at a highly granular level.
So, now you’ve been doing some engineering. You’ve learned about design patterns, which are just common ways to write a program that are generic enough to be reused, and have been battle-tested. You’ve learned about program structure, how to organize your code, why breaking things up into small, reusable pieces is a good idea, and how important abstraction is. After all, a highly specific tool is rarely a useful one – unless you’re the kind of person who loves ‘as-seen-on-tv’ products. Understanding and applying abstraction is arguably the most important aspect of what makes a programmer, a programmer.
Maybe you’ve also learned about design constraints, working in teams, making trade-offs when choosing or developing tools, working with deadlines, understanding requirements, maybe even that weird “Scrum”, “continuous integration”, or “agile” stuff people keep talking about. It’s effectively the art of getting things done.
There’s also a chance you’ve heard or used an “API”. This means application programming interface, and while it is thrown around willy-nilly, it generally refers to the way someones code (read: library, module or tool) can be used, and what’s available for the end user.
Enter the paradigm
At this point, there’s a good chance you’ve been exposed to something called a “programming paradigm”. Once again, it might seem hifalutin, and it can even feel alien until you’ve done enough programming to fully “grok” it, but you may have heard of it. If you’ve gone through computer science, it was probably talked about, and probably centered around “Object-Oriented Programming”. Sadly, this is a stain on computer science classes in my opinion, because it is often sold as the end-all-be-all as a way of writing programs.
The reality is, there are a multitude of paradigms. In the context of programming, a paradigm is simply a way of thinking and structuring your code. All programs must run through a computer that is Turing complete, so it doesn’t matter much to the machine what paradigm you’ve used, it either works or it doesn’t.
However, programming is done by humans (for now). As such, we need ways to understand it better. It’s not exactly intuitive the first time around, though some folks are trying to change that.
Thinking with objects
So, object-oriented programming (OOP) tends to fall into two camps, the difference of which are not understood well enough by most programmers, particularly those “set in their ways”. I say this because OOP was invented by the prominent Alan Kay with his creation of the SmallTalk language, and this fact seems to escape many professional programmers. The more commonly taught form of OOP is basically the form practiced by Java programmers. Generally speaking however, OOP stresses the importance of thinking of code as objects, each of which can have their own internal state, and ways to manage that state by way of setter and getter type methods. Inheritance is also a huge component, and it is very natural way of thinking, so it’s not hard to see why it’s popular. Going with tried-and-true (yet boring) examples, OOP embraces things like a Car class, where Car is an abstract class, and different ‘makes’ and ‘models’ can inherit from it. There is a lot of rigmarole around inheritance rules and proper structure, most of which are obsessed about because of the importance it plays in maintainability and extensibility.
// This is supposed to be Java, I don't know much.
public function CarFactoryFactoryFactory { // inside joke
float speed = 0;
// "getter" function
float getCurrentSpeed() {
return speed;
}
// "setter" function
void drive(new_speed) {
speed = new_speed;
}
}
protected function Honda extends Car {}
protected function Corvette extends Car {}
var corvette = new Corvette();
corvette.drive(100);
// Later extensibility? no problem!
public function Car {
private boolean is_fast = false;
}
Another factor that is often talked about, but not distinctly part of OOP is the concept of an interface. It usually refers to a description of a formal class implementation. These are the concrete details of how a function can be used, what it can do, what it returns, etc… it is kind of an engineering principle made real as a programming construct.
You will undoubtedly be exposed by a deluge of this paradigm in your programming career. I tend to think of it as wildly useful “in the large” and wildly absurd and constricting “in the small”. I’ll get to what that means later.
Doc, are you ready for the procedure?
Another paradigm that’s even more common than OOP is Procedural programming. This is where a vast majority of programs both past and present fall into. It is the simplest form, as it has hardly any structure. The gist is this: your code is read top to bottom, and each statement finishes before continuing to the next one. Let’s see an example:
function godModule() {
// wat.
some_global_state += 1; // is this a number?
// this does a lot of stuff....!
if(this && that || that + this) {
okayWhateverDoStuff();
} else {
ahNeverMind();
}
runThat();
runThisAfterThatWasCalled(); // please god, work!
loadSomeStuff();
makeAFancyPicture('images/my_face.jpg');
var my_cat_photos = downloadCatPhotos();
if(my_cat_photos.length > 100) {
notes.write('Dear diary, I really seem to like cat photos.');
}
loadImageSlideshow(my_cat_photos);
return bananas; // undefined error;
frobnicateAllTheThings(getFrobnicationKnobs()); // unreachable
}
In the spirit of making ridiculous and weird code examples, the above is a classic procedure. It has a lot of responsibility. It excretes code smells, and it is a writhing beast. Sadly, this is quite mild compared to real code I’ve seen, and I TRIED to make it suck. Often times, people “monkey-patch” or even “duck-punch” their code to work in ways it ought not too, simply because they didn’t understand how it interacted elsewhere. You can clearly see that a lot of stuff is going on here, much of which is erroneous and unrelated.
Straw-man arguments aside, this is not uncommon, and barely scratches the surface. However, procedural code is often times necessary, and can written reasonably and elegantly.
Often times, code is not really procedural – but the program you write is, for all intents and purposes. In the above example, the code is meant to execute line by line. Generally this works, but this is another gotcha that can spring on you, depending on how far down the rabbit hole you go (network requests, latency, promises, threads, semaphores, mutexes, processes, etc…)
Listen up, I’ve got to declare something.
If, on your journey, you weren’t content with being a programming wage-earner, and decided to dig further, you’ll probably have stumbled upon another paradigm – declarative.
I just want to say that declarative is one of my personal favorites – it’s so terse and elegant! It’s also so fresh, and so clean (clean), so its got that going for it.
Oddly enough, you might have to write some procedural code just to achieve it – but the API you produce will be declarative, which is the important part. The fact that this sort of dependency exists should be a clue that declarative programming is not a one-to-one comparison, but it has some cross-cutting similarities and is still a paradigm in its own right.
var max_width = 480;
var images = getImages().resizeThem(max_width).optimizeFilesize();
// accepts an array, or a single element
postToFacebook(images);
// ... or, more functionally:
// since it accepts both array and single element,
// we can call it for each function.
images.forEach(postToFacebook);
Declarative therefore is a way to write your tools in a way that allows the user of said tools to “declare” their intentions. Rather than say, getting some value, and then checking if it’s within the bounds of another value, then doing some behavior if so, you would just say “do this thing”. You’re in effect saying – “I don’t care how it’s done, just do it”. The program should figure the rest out. The figuring the rest out can often be procedural, hence my earlier statement. Whether or not this is case depends on the limitations of the language. Some languages have these concepts built right in, others are very generic.
Function, function, what’s your composition
With the exception of OOP, the functional paradigm builds heavily on the previous ones. However, there is some blur in what functional programmings means and what it is in computing today.
So you know about functions. Those little packets of usefulness have been helping you out all over the place. You’ve been diligently organizing them into neat helper that are clean, and well documented, and (hopefully unit tested) to make your code that much more robust. Each function is well defined, with inputs and outputs that are like a “black box” – it shouldn’t matter what you put in, the same result should always come out given a specific input. These black boxes are predictable, have known, reasonable inputs, and handle edge cases, like incorrect data types and typical errors such as Range, Index and Key. They are not concerned with other parts of your program, and are therefore highly decoupled – making change easy and negotiable in the future. They don’t do too much (spaghetti code) and don’t do too little (ravioli code). It’s that damned Goldilocks effect again – it seems to crop up everywhere in the universe, doesn’t it?
Functional programming makes functions the core of everything. In fact, in a purely functional language, functions are the only construct you use. While this may not seem any different from what you’ve been doing (“I’ve been writing functions already!”) functional style stresses the absolute adherence to all functions, all the time. But it’s not just writing functions – these functions need some rules. Functional programming tries to instill things like referential transparency – the notion that functions always evaluate to the same thing, regardless of context. Calling a function in one place shouldn’t change the way it works, when it’s called somewhere else. You’d be surprised how often this phenomenon is found (and abused) in other paradigms.
Another important tenet of functional programming is the lack of state. State, being naturally mutable (able to be changed), can be useful, but comes at a cost. Mutation of state over the course of program execution means behavior that depends on that state may become unpredictable. It’s just another way that things can go haywire in a program, making it harder to reason about and debug.
State is all kinds of useful – for example, a simple for loop uses an “iterator” variable, which stores the current iteration value, so you can loop while increase or decreasing (1, 2, 3, 4…). You might also keep track of some changing thing that involves user input – a mouse position, a keyword value. The fact this seemingly fundamental property is both undesirable and unnecessary in functional languages often perplexes people, who feel it is one of the identifying traits of a computer program.
// BAD KITTY!
// store our individual values as an array.
some_array = new Array(100);
function stuff() {
// Where da arguments? Oh wait...
for(i = 0; i < some_array.length; i++) {
// so far, so good...
doStuffWithAValue(some_array[i]);
// D'oh! Now we're hosed.
some_array[i] = random(10000000);
}
}
function things() {
// no arguments either? jeez...
some_array[random(1000)] = false;
}
// ...meanwhile, in another function...
function iHaveNoIdeaWhatsGoingOn() {
if(maybe_true || probably_no_true) {
things();
}
stuff();
}
Wow, look at all that global state.
So how do we solve this problem of not having state? Simple – you just keep calling functions, passing in “updated” state as arguments. Instead of having a state that is changed and re-referenced across multiple functions (breaking the de-coupling and separation-of-concerns rules), you simply pass the new function the updated state. You get the same effect, but you can breathe easy knowing that was completely unaltered when it came into the function.
Wait… what new function? Ahh, that’s another important thing to know and remember. Functions returns things! Yes, okay, that’s elementary. But wait – functions can return new functions! And those functions can have some arguments already applied! This understanding is crucial to functional programming.
With this use comes another tenet – partial application. You see, functions may have arguments – a seemingly unlimited number that you can define and pass – but it’s advised to keep the number of arguments used to a minimum. In functional style, that minimum is preferably one – and the number of arguments (arity) is actually used to defined different functions (overloading). When you have many arguments that need to be passed in, you have to “partially apply” them – apply them one function call at a time. This can mean creating a new function with one argument applied, which can then be called again with the remaining arguments, applied over and over, until all arguments have been “used up”.
You will also find yourself intimately involved in things like function composition, and if you use a pure functional language, type signatures – ways (like interfaces) to describe how a function should be used (that are actually enforced at runtime).
I’ll leave it there for now, but there is a lot more ground that can be covered – the abstract mathematical premise of lambda calculus, for which functional programming is based on notwithstanding.
One important understanding going forward is the concept of chaining functions – combining the output of one function into the input of another, ad infinitum. This “flowing” is the keystone for concepts provided ahead.
Go with the flow (maaaaaan)
Hopefully you’ve been totally invigorated by computing and programming, and if so, you’ve probably dug further. You might just hit a gem – a shiny, even mystical gem – the gem of data-flow programming, and the crux (finally!) of this article.
Data flow programming is somewhat ambiguous, and is generally categorized as a subset of declarative.
On a conceptual, intuitive level, functional programming and data-flow programming are very similar. The concept of chaining, connecting (like Lego!) functions to each other and using one output as another input is the mainstay of data flow programming. There are merely some formalisms that define data flow specifically – a graph (data structure) of function calls (or operations) that dictate the overall flow of a program. The odd, and slightly ironic thing, is that this concept is vital to all programs – even procedural! A procedure must usually be compiled to object code (or interpreted on the fly), and that object code will have some representation of the program call graph, in the form of an abstract syntax tree. So then… the more things change, the more they stay the same.
However, this is all “in the small”. This is not necessarily what data flow is concerned with, thought it can be used to describe such examples. It is possible to switch context and think of data flow programming not as implementation details, but rather, overall structure of a larger system and code base. You see, we’re scaling up these principles of computing to the architecture of our code! Truly amazing, the dizzying complexity that arises from simple rules (which reminds me, Cellular Automata is another paradigm of it’s own, which focuses on these odd phenomenon of rules and emergent behavior.)
Data-flow example - “piping”
If you’ve programmed even a little, you’ve probably come across piping, even if you didn’t know it. The term piping comes from UNIX, where operating system commands can be piped together, filtering and transforming data as many times as necessary. For example, consider a command to search for some files, based on the directory listing, filter the results by a keyword in the file contents, and then execute that as the input to another program we made up:
ls | ag 'cat-photos' | post-to-facebook --with-pics --given-input
Okay. This is a ridiculous, nonsense example. But, we did the three things above. Each operation in the example is separated by a |, and it demonstrates both in real code, AND visually, what is happening. Those UNIX guys were pretty clever, eh? Nothing beats the speed and terseness, even today.
Data-flow example - Wolfram Language
I love Wolfram Alpha, and a lot of the what the Wolfram team has done. If you aren’t familiar, Wolfram Alpha is billed as a “computational knowledge engine”, that takes inputs (user given) and transforms it into facts, visual representations, and interactive models. After the success of WA, They’ve now released the engine behind it for public consumption – the “Wolfram Language”. I encourage you to watch the introductory video at http://www.wolfram.com/language/.
In the video, Stephen Wolfram (creator, prodigy, etc…) stresses the importance of the Wolfram Languages' ability to transform data, and continually transform data that was previously computed. This is demonstrated very powerfully in the video, with inputs and outputs being used to make more complex and exciting computations.
This is one context, though it’s a very large one – yet it’s simply using the principles of data flow programming! As you can see, the very act of using this paradigm allows for the creation of some extremely powerful tools. If the future of technology is the flow of constantly changing data (from sensors, inputs and computation), and consumption and computation of data (it is!), then this paradigm is the future of programming. I say that with extreme confidence, but feel free to mock me.
Parallel and distributed dataflows – the “future” of computing.
I can also say with extreme confidence that the future of computing is something that is, or is very similar to, parallelism. With that assumption firmly in place, we need to ask ourselves; “what does parallel programming look like”? Fortunately, this problem has also been tackled and is already pretty well established. Though some applications in computing have floundered for a while, it is highly sought after topic, particularly as it applies to artificial intelligence. Interestingly enough, this trajectory of computation is how the brain works, insofar as we understand it. The brain is a highly distributed and parallel machine. As we increase computational power, so too, we increase the mirroring of computers to brains. It is an eerie coincidence that perhaps we might give pause to.
With the exception of your typical “map, reduce” class problems, I have literally no experience in this realm, so I’ll leave this topic as merely an aside for curious readers. I bring it up because it is another piece of the bigger puzzle, and well, it’s damn cool.
Hopefully this whirlwind of topics has been very interesting, and it has at least led you on a path towards understanding the evolving themes of programming. It is amazing to see the very real “paradigm shifts” that exist in programming, and how it simple ways of thinking can have profound effects on the structure and application of your program.
Optional side note: data flow and code-as-data
So, when we write our programs to be used in larger systems, we often want to think in terms of data flow. It can often be useful when we think of code as data, where the program we write to do something also describes the actual data it is performing the computation on. This is not the best explanation, but this is some groovy mind-bending shit, once you get the grasp of it (I hardly do.) If you’re curious to learn more about this, look into “code-as-data” and “algebraic data structures”. It’s definitely next-level programming, so it might be wise to preemptively take some introductory abstract algebra courses before jumping in head first. If you don’t get it at first – don’t fret! It’s a very advanced topic.