Ben's Blog
Posts
2024
Starting a Startup, Reloaded
My "Starting a Startup" post from a few months ago needed a refresh.
January 22, 2024
2023
Monotonic Attention
Write-up explaining an implementation of monotonic attention using a probabilistic graphical model.
November 9, 2023
Retentive Networks and RWKV
A short, hand-wavy explainer for the mathematical intuition behind faster attention mechanisms.
September 16, 2023
Diffusion verses Flow Matching
An accessible introduction to diffusion and flow matching models.
July 19, 2023
Fast Attention Implementations
A reference collection of fast attention implementations.
June 29, 2023
Starting a Startup
I left FAIR to start a startup a few weeks ago, and figured I should describe what we're actually doing.
June 27, 2023
RWKV Language Model Math
In-depth explanation of the math behind the RWKV model, with PyTorch implementations, plus a discussion of numerical stability.
June 16, 2023
Types of Gears
This post is a reference for the different types of gears that might be relevant for robotics applications.
March 6, 2023
Types of Electrical Connectors for Batteries
This post is a collection of the different types of electrical connectors that might be relevant for robotics.
March 6, 2023
2022
Possible Ideas for Robotics Startups
A list of ideas for robotics startups, focusing on things that are feasible right now and things that will likely become feasible in the next few years.
December 25, 2022
Robotics Pre-training Idea
A collection of my ideas relating to robotics pre-training.
November 1, 2022
PID Controller Explainer
Simple overview of what a PID controller is, how it works, and how to make one yourself.
September 29, 2022
Three Months of Daily Journaling
Meta-journal about journaling for more consecutive days than I initially thought I would, and whether or not it is a worthwhile habit to cultivate.
July 26, 2022
Monitoring Background GPU Usage
Python code snippit for doing background GPU monitoring using `nvidia-smi`
July 21, 2022
2021
Numpy Array Stream to Video
Short post with code snippits for creating videos from Numpy arrays in Matplotlib.
April 29, 2021
2020
Visiting the United Kingdom
Reflections on our time in the United Kingdom during the Covid-19 pandemic.
December 18, 2020
The Work-Procrastinate Cycle
The results of an experiment I've been conducting on myself.
December 2, 2020
Useful bash / zsh Profile Additions
Some functions I found useful to add to my bash and zsh profile.
May 27, 2020
Optimized Log-Sum-Exp PyTorch Function
A walkthrough of how to optimize the log-sum-exp function in PyTorch.
May 20, 2020
Torch CUDA Extension Tricks
Some tricks I found useful for writing CUDA extensions for PyTorch.
May 6, 2020
Two Students Riddle
There is a teacher and 2 students in a classroom. The students are A and B. The teacher thinks of 2 positive integers and tells the sum of those numbers to student A without student B hearing it. Then tells their product to student B without student A hearing it. After this, the teacher asks the 2 students what was the 2 numbers. First student A says: I don't know. Then student B says: I don't know either. After hearing this, student A says: Now I know. Then student B says: Now I know them too. What were the 2 numbers?
April 30, 2020
Combination Lock Riddle
A combination for a lock has 3 wheels, X, Y, and Z, each of which can be set to eight different positions. The lock is broken and when any two wheels of the lock are in the correct position, the lock will open. Thus, anyone can open the lock after 64 tries (let A and B run through all possible permutations). However, the safe can be opened in fewer tries! What is the minimum number of tries that can be guaranteed to open the lock?
April 30, 2020
HMMs and CRFs
A comparison of Hidden Markov Models and Conditional Random Fields, two kinds of probabilistic graphical models.
April 7, 2020
Recap from MIT Covid-19 Challenge
Recap and potential future directions from the MIT Covid-19 Challenge.
April 5, 2020
"The Decadent Society" Book Review
Summary and thoughts about The Decadent Society: How We Became the Victims of Our Own Success by Ross Douthat.
March 22, 2020
Coding the Viterbi Algorithm in Numpy
A demo of how to code the Viterbi algorithm in Numpy.
March 15, 2020
How Not to Productively Work From Home
Some of my thoughts about effectively controlling the elephant mind.
March 1, 2020
2017
Roaming through Japan
Exploring the Land of the Rising Sun for 19 days by train, through Tokyo, Nikko, Sendai, Sapporo, Kyoto, Osaka and Nagoya.
May 8, 2017
How I Spent Spring Break in Cuba
A field guide to traveling to Cuba, written by naive college students for naive college students, mostly as exposition.
March 4, 2017
2016
Using Gensim Word2Vec Embeddings in Keras
A short post and script regarding using Gensim Word2Vec embeddings in Keras, with example code.
August 2, 2016
Restricted Boltzmann Machines
Building on the Recurrent RBM for sequence modeling. This post relates to what I am doing for my Master's thesis.
July 18, 2016
Touring the United States and Canada
Narration from my trip through Nashville, Chicago, Toronto, Montreal, and Washington DC.
May 11, 2016
Question Answering using Keras
An in-depth introduction to using Keras for language modeling; word embedding, recurrent and convolutional neural networks, attentional RNNs, and similarity metrics for vector embeddings.
April 27, 2016
Atlanta Craft Beer Reviews
Some facts and very sarcastic opinions regarding craft beers that can be bought at the Krogers near my apartment.
March 30, 2016
A Neural Network in 28 Lines of Theano
A quick introduction to using Theano for deep learning, from the bare-bones to a full neural network.
February 23, 2016