Tokenization and Vocabulary Design
How text becomes tokens, and why a bigger vocabulary buys shorter sequences.
Research
Areas of AI and generative modeling I'm currently learning, mostly by building things from scratch.
Parameter-Efficient Fine-Tuning
LoRA and QLoRA, and how far the rank can drop before the task falls apart.
Reinforcement Learning for Style and Behavior
GRPO and reward models, including the moment a policy starts gaming the reward.
Evaluation Methodology
Why an eval set has to come from the domain you deploy into, not the one you train on.
Small Vision-Language Models
Sub-billion-parameter VLMs for narrow tasks, small enough to run on limited hardware.
Transformer Internals
Encoder versus decoder, self- versus cross-attention, and where the attention formula lands.