an interesting read.
after 2023, virtually all new research in this space used discrete diffusion, and continuous diffusion for language went extinct.
Interesting suggestion/opinion. GE tells me otherwise.
2024: Meta, Google, and Microsoft are dying
An ARC-AGI transformer is a specialized or compact transformer neural network designed to solve tasks in François Chollet’s Abstraction and Reasoning Corpus for Artificial General Intelligence (ARC-AGI) benchmark.
Unlike massive Large Language Models (LLMs) that rely on memorized internet text, an ARC-AGI transformer is built for fluid intelligence. It looks at a few visual examples of grid-based transformation puzzles (such as moving shapes, filling colors, or mirroring objects) and infers the underlying rule to solve a new test grid.
I trained a small transformer from scratch in 1.5hrs on a 5090
Beats many LLMs, and scores the same as TRM/HRM
ARC-AGI is the only AI benchmark that measures our progress towards general intelligence. Core Knowledge Priors. A principle underlying ARC-AGI’s A principle underlying ARC-AGI’s design is the need to create a fair and meaningful comparison between artificial intelligence and human intelligence. To achieve this, ARC-AGI focuses on fluid intelligence (the ability to reason, solve novel problems, and adapt to new situations) rather than crystallized intelligence, which relies on accumulated knowledge and skills. This distinction is critical because crystallized intelligence, by definition, includes cultural knowledge and learned information, which would provide an unfair advantage.
My agent skills that I use every day to do real engineering - not vibe coding.
Developing real applications is hard. Approaches like GSD, BMAD, and Spec-Kit try to help by owning the process. But while doing so, they take away your control and make bugs in the process hard to resolve.
These skills are designed to be small, easy to adapt, and composable. They work with any model. They’re based on decades of engineering experience. Hack around with them. Make them your own. Enjoy.
