Thoughts, lessons, and the occasional rabbit hole
28 articles
I used to know what keeping up with technology meant. In Python or data science, it meant reading release notes, trying a few libraries, following a few people whose judgment I trusted and occasionally reading a paper that seemed relevant. There was always more to learn, but the shape of the work was understandable. Most changes were incremental. The important developments were usually visible. If I fell behind for a few weeks, I could catch up.
I wanted to create a personalized children's picture book as a gift for my friends' daughter.
Together with our Staff Engineers and Tech Leads, we had mapped technical risks, organizational bottlenecks and initiatives worth addressing. We used the classic Eisenhower Matrix in a workshop and agreed on what was important, what was urgent and which topics we wanted to address first. The week afterwards, none of the topic were in my calender.
When I became a Director of Engineering, people warned me that I would miss coding. What I really underestimated was how hard it would be to stay out of the details when I knew a topic well. I often caught myself thinking: ll just have a quick look at the repo and figure out how to solve this.
Hiring is one of the most important responsibilities of an engineering leader. The people you hire shape how your organization works long after the interview is over.
One of the concepts that has influenced my leadership is psychological safety. The research is compelling: Psychological safety supports learning behaviours such as admitting mistakes, asking for help, challenging ideas, and voicing concerns. Amy Edmondson helped establish the modern, team-level concept, defining it as "a shared belief that the team is safe for interpersonal risk taking."
Before the AI era started, I had hacked together a chocolate ratings "webapp" without using classic frontend frameworks or libraries. I am definetly not a frontend engineer and it was more of an interface to a database. But with AI it should be possible to get a decent looking webapp, I thought.
One of my responsibilities as an Engineering Manager was people development. At first, that felt fairly straightforward. If someone wanted to become a better engineer, I could often help by giving them the right opportunities: a technically challenging project, ownership of a new service, the chance to lead an incident, or an opportunity to present their work. Development happened naturally as part of everyday work.
Transitioning to another SQL database? This blog post is for you. Shifting from one SQL dialect to another can be a journey full of surprises. While the basic syntax (SELECT FROM WHERE) is similar, there are important differences, that will make your queries slow, fast, fail or worse: fail silently!
Most machine learning models are only as good as the data they learn from. That sounds obvious, but it is surprisingly easy to forget once you have millions of data points and a model that performs well on your offline metrics.
You have built a great streamlit app. So far, you only ran it locally on your computer on localhost:8501. Now you would like to share your app with others, but wonder how. This blogpost introduces you to one option: Heroku. Heroku is a platform as a service that allows you to deploy your apps (not just streamlit apps, but also jvm apps, ruby apps etc.). This post will guide you through the deployment of a streamlit app on Heroku. It’s free for non-commercial apps smaller than 300 MB. You need
Search engines rely on models, which rank the matching results for a given user query. These models optimize the order of items. They learn how to rank items in a result list, therefore the name Learning-to-Rank (LTR) models.
You have kafka as your message broker up and running and you may wonder: In which format should I send my data around? Maybe the string format pops up in your mind. Why not just put all fields into a long string and separate them with a comma?
In this blogpost you will get a basic understanding about message brokers. We will look at two very popular message brokers, Kafka and RabbitMQ, and learn, how they handle messages.
This post will teach you the intuition of REST APIs and how you can use them to get interesting datasets for your data projects. First, we will look at the four components of a request. In the second part of this blogpost, we will go through one example and access the coingecko API via curl.
In many scenarios, such as a google search or a product recommendation in an online shop, we have tons of data and limited space to display it. We cannot show all the products of an online shop to the user as a possible next best offer. Neither would a user want to scroll through all the pages indexed by a search engine to find the most relevant page that matches his search keywords. The most relevant content should be on top. Learning to rank (LTR) models are supervised machine learning models that attempt to optimize the order of items. So compared to classification or regression models, they do not care about exact scores or predictions, but the relative order. LTR models are typically applied in search engines, but gained popularity in other fields such as product recommendations as well.
I was recently invited to join a panel discussion among developers to dispel the myth of the typical BS Buzzword Bingo around machine learning and AI. In this blog post, I will share some buzzwords we talked about with a little description and links. Ooops, I already used some buzzwords. So let’s start.
Humans intuitively understand the meaning of words: Which words are similar, opposites or related to each other? But our machine learning models do not have this intuition. Word embeddings are numeric vectors that represent text. These vectors are learned through neural networks. The objective when creating these embedding vectors is to capture as much “meaning” as possible: Related words should be closer together than unrelated words. Also, they should be able to preserve mathematical relationships between words such as
In this blogpost I will share some tips for working with Jupyter Notebooks. Those tips greatly improved my productivity when working with Jupyter Notebooks and I wish someone would have told me earlier. The two main topics of this post are extensions and magic commands.
Elasticsearch is often the storage engine of choice for storing and querying full text data. But writing an ElasticSearch query is pretty different compared to querying a relational database in SQL. In this blogpost, you will learn some basics you need to understand before working with ElasticSearch. In the second part, you learn how to write queries in ElasticSearch.
Searching through full text fields with regexes in relational database systems like PostgreSQL or MySQL is painful: The query latency is high and your results will be unordered, so you have no idea how relevant your query results are. Elasticsearch is often the storage engine of choice for storing and querying full text data. In ElasticSearch querying fulltext fields is among the least resource intensive tasks and your query results are ordered putting the most relevant results on top. But how does this work?
The basic idea of complex datatypes is to store multiple values in a single column. So if you are working with a Hive database and you query a column, but then you notice “This value I need is trapped in a column among other values…” you just came across a complex a.k.a. nested datatype.
Seaborn is a python library for creating plots. It is based on matplotlib and provides a high-level interface for drawing statistical graphics.
When I started working with pandas I noticed that there were so many ways how to subset, filter and join data with pandas. But I was lacking a systematic overview. How do the different approaches differ and when to use which?
So many people have recommended Git as a version control system to me. I had a look at it, but I was pretty overwhelmed. Since I did not have a technical background, everything seemed so complex! Many tutorials let me copy paste code without giving you a deeper understanding of what and why I am actually doing this. This copy pasting feels like success at first, but when I tried working with it, I could not.
This post is about how to change your R working directory. You might be wondering:
In this post, I will show you some of my best practises for formatting tables in R Markdown. We will cover
I write this blogpost for someone, who has never worked with R Markdown. After you read this post, you will