5 minutes read | 1042 words by Ruben BerenguelSome links are affiliate links
This week is somewhat more eclectic than usual. There is nothing about any
particular programming language. There is a bit on Kubernetes, functional
programming and containers, project management, aviation history,
interviews/bios… I have started reading a few more papers, and I’ll share the
interesting ones as well. This week they are centered in data engineering.
NOTE: The themes are varied, and some links below are affiliate links.
Kubernetes, functional programming and containers, project management, aviation
history, interviews/bios… Expect a similar wide range in the future as well.
You can check all my weekly readings by checking the tag
here. You can also
get these as a weekly newsletter by subscribing
A quick overview of this paper, which essentially introduces a type checker for a subset of TLA+ and then converts it into SMT-acceptable rules. This enables checking certain properties in reasonable time (i.e. finite).
Back when I was investigating moving all our systems to Kubernetes, autoscaling (especially if I wanted to run Spark on the same cluster) was what looked thornier. From the looks of it, Clusterman by Yelp might be the answer.
A paper involving Tiark Rompf, whom you may know as involved in Flare, an optimizer for Spark that can reach 10x speed ups. The details in this article should feel as part of the reason (technically I’m horribly wrong since Flare uses only LMS, but… I have a hunch there is a lot of overlap). In short, explains how you can specialize (query) interpreters to form the basis for efficient compiled code (similar, but nope, different, than what Catalyst does). This is achieved via a Futamura projection which I may explain later (since I’d like to read the Futamura paper as well)
I’ve had what feels like a business-like week, right? TEE is a classic by Peter Drucker, and everything said in it is as valid now as it was some years ago. It applies not only to “managers”, but any knowledge worker, and yes that includes you. I’m also pretty confident I’ll re-read (well, re-listen) to it next year, as I do regularly with Getting Things Done.
So fun, it’s an IT version of The Goal, but even better as a book. It explains theory of constraints, agile methodologies and DevOps in an applied fictional setting, as a novel. By the way, the Audible edition (of the 5 year anniversary edition) is excellently narrated, I listened to it (14 hours) in day and a half.
These weekly posts are also available as a newsletter. These days (since RSS went into limbo) most of my regular information comes from several newsletters I’m subscribed to, instead of me going directly to a blog. If this is also your case, subscribe by clicking here.