Below you will find pages that utilize the taxonomy term “Scala”
Trying a cross between the old format and the new format (since there are people who like both)
This turned out a long one
Another week gone by, with a long list of readings seen pass.
Happy new year!
Christmas edition!
End of year cleanup, so a lot of goodies this time
As usual, skipping an edition means a longer collection later on.
Days of fire Kafka and thunder SSL.
This Apache Spark feature has made us scratch our heads way too much.
A week on holidays (in-between jobs), where I read more books than articles.
Next week I start a new job 😮
I have been on holidays this week, playing VR and preparing videos for an online Python event I co-organise.
I have played a ton of virtual table tennis this week.
I have read quite a bit this week, I’m also preparing a summary of the RDD paper.
I have dropped the Weekly from the title. It was about time.
We have alternated Friday’s off, so I have extra time to write this post today.
This is a project recap on writing some non-super-trivial Haskell for the first time.
I should remove the Weekly moniker of these posts and emails. They are done when they are done. Enjoy!
Another hard push at reducing my reading list. At the current pace I may not write many more of these posts.
I made a hard push to clean up my reading list, deleting a lot and reading another lot. It went from 369 items to 99 🎉
A flight last Sunday meant I was so sleepy I skipped sending last week’s newsletter… so this week is super-sized. A lot of content about distributed teams I think.
Yeah, I skipped last week. On Saturday Python Barcelona organised PyDay 2019, and I was one of the organisers aside from giving a workshop on PySpark, so I felt pretty tired on Sunday. Of course this means this is a double issue.
This week there is a lot of functional programming (mostly Scala, a bit of Haskell) and data engineering topics. Of course, there is also the usual random stuff I find interesting as well (and other engineering topics).
A mixed bag of interesting history tidbits sprinkled with Haskell code, Scala stuff, data engineering systems and practices, and how to code with your voice.
This feels like a heavy engineering edition. A lot of Haskell, Rust, Python and Scala. There’s still a bit of everything, but this will appeal hardcore developers more than usual.
Although this week I have been reading mostly Apache Cassandra documentation, I have tried to avoid an onslaught of tips, tricks and readings on it. Just one article.
I have been pretty busy lately, and although reading doesn’t stop, my writing sometimes takes a hiatus.
Data engineering, adtech, history, apple. Expect a similar wide range in the future as well. You can check all my weekly readings by checking the tag here . You can also get these as a weekly newsletter by subscribing here.
The map and problem described here were part of my presentation Mapping as a tool for thought, and mentioned in my interview with John Grant and Ben Mosior (to appear sometime soon in the Wardley Maps community youtube channel). I’m looking for ideas on how to make this map easier to understand and useful, so I posted it to the Wardley Maps Community forums requesting comments.
If you have have known me for any length of time you’ll know I write mostly Python and Scala lately (Rust is getting into the mix slowly). And you should know, I am a heavy emacs user. I have been using emacs for close to 15 years, for the past 3 my emacs of choice has been spacemacs. I used to have a very long, customised and complex .emacs and with spacemacs I get a mostly-batteries-included package.
As you may know, I’m a heavy emacs user, and a frequent Scala developer. Scala tooling for emacs was restricted to mostly ensime until recently. Although ensime is an excellent piece of software, it made my old Macbook suffer a lot (it only had 8gb of RAM). So, most of the time I just went hardcore mode developer, and worked with no autocompletion, no jump to definition, no-nothing. A pervasive use of ripgrep and good memory were sometimes enough, but I was envious of many things I could see in my colleagues using IntelliJ. Of course, switching editors was not an option.
Software engineering, Spark, history, python. Expect a similar wide range in the future as well. You can check all my weekly readings by checking the tag here . You can also get these as a weekly newsletter by subscribing here.
Testing and debugging Apache Airflow We’ve been using Airflow for almost a year (on my suggestion). I’m not super-happy with it, and testing it is one of the pain points.
You know how you slip once on a habit and everything goes crazy? Well, I’ve been 4 weeks without writing these, so here’s the accumulated reading from 4 weeks. Because, even if I don’t write it, I read a lot anyway. Also, there’s lot of interesting content this “week”.
Sorry for the delay, Sunday was my birthday (also, Elmo’s, and The Day The Music Died as well) and I spent the day without access to a computer.
Software/data engineering, languages, writing. Expect a similar wide range in the future as well. You can check all weekly readings by checking the tag here.
The year has ended, what has been going on?
This week I have been working a lot with a relatively large dataset on a Spark shell. It was a graph with 1 billion nodes and 2 billion edges that I wanted to analyse with GraphFrames (the successor of GraphX on Spark).
I ran into this problem with sbt dependency resolution around 7 weeks ago. I
was in a hurry, so I commented out the offending import (since it was not in the
subproject I was working on, so was not needed for the run I was in) sent my
commit to the heavens and CircleCI was happy.
I had always wanted to play with Processing (after leafing through The
Processing
Handbook and
Visualizing
Data
some years ago). My general dislike for Java or the JVM made me just play a
short amount of time with processing.js something around 2011 (there was a
native processing.js application for iOS, I used it for a while on my iPad and
iPod Touch).
A few months ago I stumbled into the problem of Akka logging, specifically
ClassNotFoundException when using akka.event.slf4j.Slf4jLoggingFilter, just by
following the details of the Logging - SL4J section of Akka
documentation.
Almost two months ago (time sure flies) I attended for the second time the conference Scala eXchange, one of the largest Scala conferences in the world, and which happens to be 1 tube stop from the office you can find me from time to time in London.
I am trying to make these posts a tradition (even if a few days late). I thought 2016 had been a really weird and fun year, but 2017 has beaten it easily. And I only hope 2018 will be even better in every way. For the record, when I say we, it means Laia and me unless explicitly changed.
Shading dependencies with sbt-assembly (in particular, shapeless in Spark 2.1.0)
1 minutes read | 141 words
A few weeks ago I needed to parse configuration files in Scala for a Spark project and decided to use PureConfig. It is incredibly lean to use, needing minimal boilerplate. I recommend you check it out (give also a look at CaseClassy, which I haven’t had time to test yet).
Everything seemed straightforward enough, and I got it working pretty quickly (as in, it compiled properly). The surprise? spark-submit failed with a conflict with Shapeless (lacking a witness).
Some of the links are affiliate links to Amazon. I only recommend what I use.
At last February. Finished my PhD dissertation, so can add Dr. in front of my name when ordering a Gatwick Express train ticket. Also makes for a cooler email signature. So far has been the only differences I’ve seen in the year
Work February-December Started working as a consultant in London, almost the day after delivering the PhD presentation.
October Kafka London Meetup - Jay Kreps: Distributed stream processing with Apache Kafka
2 minutes read | 263 words
This has been the first time I could attend the Apache Kafka meetup in London. Previous meetings had me in Barcelona or flying. First realisation: it is a surprisingly crowded meetup! Clearly, everyone is using Kafka, even if it is not clear from the outside. Oh, and the food was pretty good, too. Thanks to the sponsor (which I have sadly forgotten).
I really enjoyed this one. The speaker was Jay Kreps, CEO and co-founder of Confluent (and author of I Heart Logs), so, basically, a Kafka top committer himself.
More emacs configuration tweaks (multiple-cursor on click, minimap, code folding, ensime eval overlays)
3 minutes read | 472 words
At Affectv we use a wide range of editors: Sublime, Atom, Emacs, Pycharm, IntelliJ… Actually only two people use the same editor! As such, from time to time I see things in other people’s editors that I would like to have as well. So, yesterday I decided to improve on some configuration settings on Spacemacs.
Click for multiple-cursors I saw this on Jordi’s Sublime, and it is much more comfortable than using more-like-this or similar helper functions, even if I need to use the trackpad to do so.