There are a bunch of webdev things I do differently now than I did last year. I'm usually keeping an eye on new technology and current support, and JS/CSS features are no different. These are some of the new features I'm starting to use.
Transitioning the height of an element from a certain size to auto has been the holy grail of CSS for a bunch of years… ever since they solved the vertical centering problem. There is finally a way of animating this height change without gross tricks like using JS or the max-height property. The key is in the following property:
This CSS property reserves space for the scrollbar, preventing layout changes as the content grows. It's like having an overflow: scroll that doesn't render that ugly disabled scrollbar when the context is not long enough:
If you have ever used SVG icons in a website, there's a good chance that you needed to make them change color when hovering. Until now, the ways to do this were not too nice:
Inlining the svg contents: using a <svg> instead of an image forces you to put the entire icon contents there, adding complexity and not letting you reuse them.
Using two separate imgs: having to maintain a set of icons that are identical with just a different fill is painful and cumbersome.
Chaining filters: this is really hacky but I liked it. You can take a white icon, apply a sepia filter, rotate its hue and then adjust its saturation and brightness.
But now, there is finally a way to do this by placing a background over an element and then using the mask-image property:
By now, everyone that works with CSS may already be used to working with media queries. If you want your CSS to apply only to large devices, you could do @media screen and (min-width: 1400px) {...}. However, sometimes you don't want your conditional CSS to depend on the overall screen width, but rather in the width of some container. Let's say you have a chatbot panel that can either be its own frontend view, or a panel on the side of a different one. You can now do the following:
This way, if your .chat container is smaller than 600px, the flex items stack vertically instead of horizontally, regardless of viewport size. You can also name containers for targeted styling with container-name: chat-container and query them with @container chat-container (...).
Next to come
As a general rule I try to avoid polyfills when possible (besides CSS prefixing) and use only features with a decent support (90%-ish browser share), so even though some of these features have been out for a while, it is now when they are fully supported. But there are other features that I'm also excited about that will likely come this year:
I've noticed that AI assistants are slowing down the adoption of new features. Since they’re trained on existing code, they tend to suggest older approaches more often. Compare that to when we used to rely on Stack Overflow, where the top-voted answer was often the modern solution (or at least had an edit saying, "As of 20XX do it this way"). It would be interesting to analyze this phenomenon with real data.
We often worry about resources we’re about to run out of: oil, helium, phosphates… but what about those we’ve already depleted? Throughout history, humanity has exhausted some natural resources and species, some with disruptive consequences and others with none at all. Most of these examples are considered to be depleted, since the world is a large place and it's pretty hard to tell there's zero of anything.
One example of this is the coelacanth. This fish with an unusual number of fins was only known from fossil records over 410 million years old. But on December 22, 1938, a South African fishing vessel caught a strange fish at ~75 meters deep while hunting sharks. The fish was remarkably strong: after being pulled up, it tried to bite the captain and stayed alive on the hot deck for almost three hours. The captain called the curator of a local museum, who recognized the specimen's significance, taxidermized it and sent a sketch to a professor who identified it as a coelacanth, marking one of the most significant zoological finds of the 20th century.
Other times, a resource is lost to history and we can't really assess how special it was. Roman concrete or Stradivarius violins are good examples where we debate if they were truly the best or we are misled by survivor bias and romanticism. One intriguing case is silphium, a herb from roman times that was considered to be worth its weight in gold. It appeared in most coins from Cyrenaica (now Libya), but its exact identity is unknown. It is believed to be related to giant fennel.
The plant resisted all attempts at cultivation, being confined to a narrow strip of land about 200×40 kilometers. Classic texts praised its versatility as a medicine and spice, mentioning uses from birth control to being one of the most fundamental food seasonings. There are various theories about its disappearance: over-harvesting, desertification, or even that it was a hybrid that couldn't be reproduced from seeds. The last known stalk was reportedly given to Emperor Nero "as a curiosity," often cited as the first recorded extinction of a species.
Most silhpium illustrations come from coins
One mineral resource hardly available anymore is low-background steel. This is, metal produced before the first nuclear weapon testing in the 40s. While modern steel works perfectly fine for most purposes, highly sensitive radiation detection equipment requires steel unexposed to nuclear fallout in our atmosphere. That's right, the air you breathe is more radioactive than before: nuclear tests added about 0.15 mSv per year to global background radiation.
Although there are specialized processes to make clean steel, they are more expensive than salvaging it from pre-nuclear shipwrecks, such as WWII sunken ships. Fortunately, the need for this material has dropped as the radioactive particles naturally break down. By 2010, the extra radiation was down to 0.005 mSv per year. Some steel experts say modern manufacturing methods have solved the problem entirely.
Due to how economy works, mineral resources become not commercially viable before fully depletion. Cryolite is the only mineral ever mined to commercial extinction. This white stone, known by the Inuit people as the ice that never melts, enabled large-scale manufacturing of aluminium for the first time in history. Added to aluminum oxide, it lowers its melting point from 2000°C to just 900°C. The sole major deposit ever found was located in Ivittuut, Greenland and operated from 1854 to 1987.
Unlike many lost resources, we found a proper replacement before running out of it. By the 1930s, we could synthesize cryolite from fluorite, with global fluorite reserves currently at 280 million tons. When the Greenland mine was finally exhausted in 1987 after producing 3.7 million tons, the industrial world barely noticed, a rare case where we had already solved the depletion problem.
The most unsettling thing about resource depletion isn't just losing stuff: it's losing the easy stuff. Take oil. In its early days, some deposits were so accessible they literally seeped out of the ground. These surface oil seeps could be collected with basic tools and refined with simple methods, exactly what humanity needed to bootstrap the Industrial Revolution.
Today's remaining oil sits deep underground, requiring advanced technology to extract. Sure, we have alternatives like solar and nuclear, but making those needs an industrial base that easy resources helped create. And it's not just oil. We've used up most of the easily-accessible copper, iron, and coal too. Think of it as using up civilization's starter pack: we better not need a reset button.
For the third year, I'm starting my advent of no-AI: uninstalling Copilot and logging out of Claude/ChatGPT/Gemini for a month. This helps me assess how dependent on it I've become, and how much rustier I am when stripped of my artificial thinking augments. Christmas is usually presented as a warm time to be with the people, and it aligns well with the personal touch of doing these things on my own.
As I write these lines, I'm already feeling the urge to paste this introduction into GPT to help me brainstorm how to follow up. But that's not going to happen. And you'll definitely find some typos in this post. The only exception I'm making to this is text translation, which is so much worse in anything that is not a transformer; in fact German-English translation was the original use case for transformers.
This might be seen as an absurd luddite move. My performance is likely going to drop over December, and there is no point in trying to refresh skills that are not so useful anymore. Also, where do I draw the line? Will I code using a text editor instead of an IDE? What takeaways am I going to have that I didn't have the last two times already?
For starters, I’ve noticed that my once-sharp googling ability, a key skill in the computer guild, has already started to atrophy. My reading attention span is also shorter than it used to, especially when using Stack Overflow. And I have tried to learn Rust in depth twice already, but the temptation to ask an AI to make things for me has been too high to learn anything valuable.
Surely having the right philosophy towards assistants would solve some of these problems. For instance, many times I find myself correcting the assistant about the same piece of code over and over, trying to make it get it right, as if I was teaching it, while I know exactly what should be done and could make it myself much faster. It is hard to remember that this is just a cold and unconscious statistical autocomplete on steroids, and after some time I just stop and think "what am I doing?".
Another thing I like about this now-tradition is how it makes me really see how the web is changing for non-AI internet travelers. I have not noticed quite enough how the quality of Stack Overflow answers is dropping at an even faster pace, mainly for topics that didn't exist before 2021. Plotting the number of Stack Exchange questions per month against the Google Trends for "ai" seems to confirm this theory.
Plotting this without AI took more time than it should.
This is concerning because so many online resources are increasingly AI-generated (just try Google Images or Medium blog posts). Without fresh human input, we risk a feedback loop AI trains on AI and the content stagnates over time, missing the diversity of thought and ingenuity that comes from human contributions.
For instance, Stack Overflow's decision to ban all AI-generated content feels like a missed opportunity to find a middle ground. Users wanting personalized answers will turn to assistants instead, while editors will secretly use it to craft their responses, like students sneaking ChatGPT in their homework. A hybrid approach could have AI generate the initial answers, and editors refine the output into quality content that transparently tells which part is done by AI. At some point we should embrace this thing as it's not getting un-invented.
Anyway, this experiment feels worthwhile, even with all its frustrations and absurdities. To all the human readers that still preserve their attention span and made it to this paragraph, Merry Christmas! And if you’re unplugging like me, don’t hesitate to drop some feedback in the comments.
Most recent language models have switched from the typical ReLU and GELU activation functions to something called SwiGLU. In trying to understand it, I realized most materials assume a lot of prior knowledge and there isn't a simple explanation of why it is good. So let's go the bottom-up approach and try to explain it from scratch.
Activation functions
One of the most basic neural network layers is the linear layer. It's called like this because it takes an input value x and applies a linear transformation, which means multiplying by a number W called the weight and adding another number b called the bias. MLP architecture is just a bunch of these Wx+b layers stacked together. However, see what happens when we stack two linear layers sequentially:
If you think of it, this is equivalent to a single linear layer in which the weight is W2W1 and the bias is W2b1+b2. Hence, stacking many linear layers has the same expressive power as having just one. This is why we need to place something in between them, and that's what we call non-linearities or activation functions.
For instance, the ReLU activation function is one of the most typical non-linearities used. It's basically ReLU(x) = max(0,x). The intuitive idea behind it is that if you place it between two linear layers, they still have expressive power, and it's an extremely simple operation at computational level. In assembly instructions would be something like:
CMP r0, #0 ; Compare x, stored at r0, with0
MOVLT r0, #0 ; If x < 0, move 0 into r0
Plus, its derivative (needed to do backpropagation) is also extremely simple: 1 if x > 0, 0 otherwise. This simplicity made it the de facto choice, but there are more alternatives with particular properties.
Sigmoid activation
In neural networks, a sigmoid activation is a function that follows the most common sigmoid curve, called the standard logistic function, which can be defined as sigmoid(x) = 1 / (1 + e⁻ˣ). It has a very particular S shape, ranging from 0 to 1 and passing through 0.5 when at x = 0.
One good thing that made sigmoid specially useful in deep learning is that any given x between (-∞, ∞) like those output by neural networks will be squeezed by sigmoid into the range (0, 1), so that they can be interpreted as probabilities. However, as the inputs become very large or very small, the output is on the flat part of the curve. This means that its derivative is almost 0, so when updating weights the learning rate becomes very low. This is known as the vanishing gradients problem.
Swish function
The Swish activation function is similar in shape to ReLU, as it's close to zero when x is negative, and close to x when x is positive. It is defined as Swish(x) = x * sigmoid(ßx), where ß is a learnable parameter. The cool thing about having a learnable parameter in the activation function is that the network learns how to better shape it:
Try to set ß to 0 and see what happens.
The closest ß is to zero, the more similar it behaves to a linear function (i.e. not having an activation function at all), and the higher ß is, the more similar the function is to ReLU. Usually the parameter is initialized with a value of 1, so that it looks similar to ReLU but it lacks the vanishing gradient problems when x is negative, because the slope of the curve is not just zero.
GLU
GLU stands for Gated Linear Unit, which is a rather complicated activation function consisting of two linear transformations, one of them activated by sigmoid, so that GLU(x) = (Wx+b)*sigmoid(Vx+c).
As you can guess, all W, b, V and c are learnable parameters. Remember how sigmoids could be used to represent probabilities? Now the left side (Wx+b) is a linear transformation over the original value, and the right sigmoid is the probability of that neuron being activated, also known as the gate. The network should learn both how the values should be transformed and when should this gate be open.
SwiGLU
As you could guess, SwiGLU is just a portmanteau of Swish and GLU, and as such, it's simply a GLU that is activated using the Swish function instead of a sigmoid, so that SwiGLU(x) = (Wx+b)*Swish(Vx+c).
We offer no explanation as to why these architectures seem to work; we attribute their success, as all else, to divine benevolence.
Playing with the parameters of SwiGLU can give you an intuition of why it may be superior to others: it offers a greater variety of shapes at the cost of adding extra parameters. For instance, the configuration in the caption can be used to make x2 and this in turn enables parameter multiplication. A paper released early this year showed that relu2(x) has similar performance with no parameters at all, so maybe that ability is the culprit of its effectiveness.
Unfortunately, as the deep learning field moves further away from its statistical roots and becomes more empirically driven, the explanation "it just works" is becoming increasingly common. In practice, SwiGLU has been shown to reduce training times by accelerating convergence, though the precise mechanism behind its effectiveness is not fully understood.
Recently my posts were getting a bit too techy and serious, so I want to make up for it. Last year, I was in charge of setting up and maintaining our 40+ camp bikes, trikes, e-bikes and scooters, so here is a list of advice I gathered for anyone who needs to do some maintenance on theirs.
Do it camp-wise
Being able to fix the main problems for one bike requires knowledge, a bunch of tools and spare parts. Being able to fix thirty bikes requires… well, the same thing. Thus, it makes sense to share all these resources and have someone dedicated to bike maintenance. In many camps (ours included), even bikes are part of the camp shared stuff that sleeps in a container from one year to the next. This is very convenient for internationals.
Keep it simple
The fewer elements a bike has, the fewer things that can will break. I prefer bikes with no gears, a back pedal brake system and no crossbar so that it's easy to hop on. All the pimping (EL wires, lumigrids, fur) will probably also break, but it's the toll you pay for the extra fanciness. It's also worth saying that the second hand stores around Reno and SF are full of playa bikes that have withstood the elements at least once, and probably will do better than some cheap Target bike.
Bikes are a reliable option but other vehicle choices are also fine
Be safe
Some parts of playa are really dark at night, and if you don't have proper rear and front lights you might run into other people or be called a darkwad. I like to use EL wires in the bike or backpack. Last year I tried wearing a headlight, but it was really annoying for people passing by. Regarding locks, you are not trying to protect yourself from thieves, but from high people taking your bike by mistake. In that sense, the cheapest combination lock (you don't want to lose a key) will do. Labeling your bike with your name and camp coordinates also helps; we retrieved two bikes last year thanks to this.
Be comfortable
Walking from the center camp to the trash fence can take around 40 minutes, while a bike at a normal pace will take just 15. For this reason, you're probably gonna spend a lot of time riding around, which is why you want to be comfortable all the way. We assigned bikes based on people's height, and had a bunch of gel pads for the most uncomfortable seats. If you are bringing your own bike, riding it before the burn is definitely a good idea. Having a basket and a cup holder also make your life easier. Adding a tennis ball to the kick stand is also a good move to avoid having to leave your bike lying on the floor.
Lube your bike properly
Got this tip from our neighbors at Camp Brewhaha last year, and it worked like a charm:
Fill a water sprayer with a mix of 3 parts vinegar, 2 parts mineral oil (the kind for kids sold in supermarkets), and 1 part water. Leave some empty space to shake.
Give it a nice shake to emulsify it.
Once done, clean the chain, bearings, and braking area of the wheels thoroughly. Make sure to have some plastic on the floor to avoid mopping.
Finally, clean the chain and bearings with a rag to avoid oily stains on your awesome outfits.
Optionally, you can apply 3-4 drops of wax-based lube on each bearing and chain to make riding smoother and provide longer-lasting lubrication. Wax-based lubricants are self-cleaning to some extent.
Bring the essentials
To be sure we're covered against the most common bike problems, this is our checklist of bike stuff (tools and spares) at the camp:
Bikes: make sure to bring this item
Bike locks and a few spares, it's easy to lose the codes so we use letter ones and set all to the same word
Bunch of spare tubes (patch kits don't work too well) for all sizes and valves we have
Bike lights (front, rear, EL) and batteries
Tennis balls
Floor bike pump: hand ones are painful when pumping too many bikes
Tire levers: metal ones, to reuse from one year to the next
Bicycle wrench: or a few monkey wrenches
Allen keys of all sizes
Large plastic tarp
Wax-based lube: 1 small bottle per each ~15 bikes
Brewhaha's magic juice: water sprayer, 3 parts vinegar, 2 parts mineral oil, 1 part water
Masking tape and sharpies to label the bikes
Work gloves
Zipties and duct tape for repairs. We buy them in multiple colors.
Once you have all these, you can be confident enough to cover for 90% of your problems and ride into the sunset.
